Initial Data Representations

[knelson-farmbeltnorth]

We wrapped up discussion on Numeric Representations in #95. This item will track work on enumerated, flag and string representations, including bringing the current Context Item list into the new system.

I'm attaching my working spreadsheet including the finalized Numeric items and the other types to be agreed on.

DTDsForDiscussion.xlsx

[knelson-farmbeltnorth]

Updated version following 21 June 2023 meeting. Resolved items are in green and items yet to be discussed are in yellow.
DTDsForDiscussion.xlsx

[knelson-farmbeltnorth]

Having added Sensor Latency per #98
DTDsForDiscussion.xlsx

[knelson-farmbeltnorth]

Updating following 5 July discussion
-Renamed ContactType to TelecommunicationContactType.
-Added AddressContactType
-Flagged LoadType, OperationType & GNSSSourceType as partially discussed.
--@dubnemo is reviewing another list for possible additions to LoadType.
--We will resume discussion on OperationType next time
--GNSSSourceType is not a valid list. We need to define if that is really the information we want.
DTDsForDiscussion.xlsx

[dubnemo]

@knelson-farmbeltnorth reviewed the ArgoUML class diagram 20200428 Common Data Model.zargo, and recommend adding PlanterBox, Spreader, and Cart to the LoadType list. These were the permissible transfers between known containers:

These tend to be container types, aka loads in a specific container which matches 100% to Traceable Resource Unit.

We also agreed that Field Operations should be the bound context. Anything moving off the field, onto the farm (storage bin) and into the supply chain is out of scope.

[knelson-farmbeltnorth]

Updated with latest changes.
DTDsForDiscussion.xlsx

[crakerb-ship-it]

LoadType: Unknown, Tank, Field, Truck, Bale, Module is not accurate have been using bulk, discrete
ContainerUseType: Disposable, Reusable, Fixed, Mobile seems ok

After the review the team also provided the attached list of container types that was submitted a while back and should be added as another enumeration. The above values of LoadType seemed to be more a list of ContainerTypes, but incomplete compared to this list.
Container Controlled Vocabulary Submission Form - 20211117.xlsx

[crakerb-ship-it]

The Data Linking team also provided the attached CTE (Critical Tracking Event) type list that should be considered if the ADAPT Standard is planned to support CTEs and TRUs in the first release to enable traceability use cases.

Critical Tracking Event Controlled Vocabulary Submission Form - 20210307ekw.xlsx

[knelson-farmbeltnorth]

Agreement in 2 August 2023 meeting to discard Work Item Priority from the model and to make GNSS Source Type free text entry. Agreement in 9 August 2023 meeting on a list of Operation Types (see #111). Updated representations:
DTDsForDiscussion.xlsx

[dubnemo]

@knelson-farmbeltnorth is the enumerated sheet the 'approved' list of enumeration. As @crakerb-ship-it notes, we don't believe LoadType is accurate. Thank you for adding Critical Tracking Event and Container Type to the 'For Discussion' sheet. Is it that we need to show up to one of the Meeting to state our requirements?

[knelson-farmbeltnorth]

Updated sheet with bracketed unit syntax
DTDsForDiscussion.xlsx

@dubnemo Everything can still be revised, and we can work through issues here. Loads, containers etc. are a topic for our scoping discussion #117 and we can also meet on this topic as I know our normal time does not work for you.

[knelson-farmbeltnorth]

Updated sheet having removed Enumerations no longer part of the model.
DTDsForDiscussion.xlsx

[knelson-farmbeltnorth]

Following recent decisions to remove calibration factors and sensor latencies from the model as residual machine-focused data points, should we also deprecate the workstate representation? From the point of view of the field, is there ever a time when we need to know workstate of any implement that may have passed over a certain spot? The presence of a polygon containing non-zero data for the relevant rate representation will tell us where the machinery was engaged. @strhea @zwing99 thoughts?

[crakerb-ship-it]

What work states are considered, is it just working/not working? If it is binary I agree with pulling it out since as you say would be pretty straightforward. However, if we include other states there is some logic that goes into determining working/headland turning/idle/transport/off-parked that can be machine or operation specific so might make more sense to have the OEM figure that out similar to calculating the offsets etc.

[crakerb-ship-it]

Updated sheet having removed Enumerations no longer part of the model. DTDsForDiscussion.xlsx

For LoadType I thought the idea was to just use bulk or packaged. The LoadType list in the cleaned up sheet is a subset of the Container Type list in the context item tab. Or was the conclusion we need to come to consensus on the terminology between load, container, TRU and revisit this after we get consensus there?

[knelson-farmbeltnorth]

Agreement 4 Oct 2023 meeting to remove "IsActive" boolean since this is leftover from tracking machine data. The presence or absence of data reflects whether any given item was active.
DTDsForDiscussion.xlsx

Updated sheet:

[knelson-farmbeltnorth]

Updated sheet as of 11 Oct 2023.

DTDsForDiscussion.xlsx

Next steps will be to model this in a json/xml file as a release candidate.

[knelson-farmbeltnorth]

Initial json document:
DataTypeRepresentations.json

Do we need granular definition of numeric types SInt32, UInt16 etc.? I made everything a double. One or two items may be an int, but otherwise everything would be a Single or a Double.

[knelson-farmbeltnorth]

Agreement in 18 October 2023 meeting to simplify Numeric Data Type to Double and Integer.

[knelson-farmbeltnorth]

#128 opened to revisit 18 Oct data type decisions.

[knelson-farmbeltnorth]

Final list published here: https://github.com/ADAPT/Standard/blob/main/adapt-data-type-definitions.json

[dubnemo]

I still take the stance that the IDType defined is really a scheme, and IDSourceType is very confusing as both IDType and IDSourceType has GLN as an example.
A scheme defines the data representation, aka a formal descriptive way to parse and recognize the identifier, which in some case there will be logic in creating the identifer. UUID is a classic example, as GUID is the 'Microsoft way' or scheme and DCE created another scheme. There are many versions of each scheme as noted in RFC 9562. Therefore the IDSource in these examples are actually Microsoft or DCE. Just saying.

GTIN and GLN are other schemes, which the source is GS1.

I could go on an on. I bought this up the first day of the ADAPT kickoff meeting in Bloomington, citing ANSI ASC X12 as the origin of these concepts. UN/ECE spent considerable clock-cycles 'fixing this' with the notion of schemeId and schemeAgencyId, probably a LOT more human-hours spent debating this than we have spent in AgGateway.

ID Type in X12 was a business contextual name for the identifier, which is much different than a scheme since scheme is for the technical folks solving problems. The UN ended up calling this typeCode instead. I cannot remember if UN/EDIFACT also fixed the X12 submission to the UN/ECE but there were a lot of changes to address concerns like this.

Sorry I did not see this earlier.

[knelson-farmbeltnorth]

@dubnemo Can you create a new issue for this? If we want to revisit our code lists, we built in flexibility to supersede codes as I just pointed out in #155.

[dubnemo]

#156
@knelson-farmbeltnorth