Support for normalizing Period, date, and dateTime types in flattened FHIR views.

[rbrush]

This is a proposed addition to flattened FHIR views draft spec that aims to solve two distinct but related problems:

1. Normalize polymorphic date/time types to fields easily used in database queries. For instance, users could convert both FHIR dateTime and Period types to separate “start_time” and “end_time” columns that are easily used for a variety of queries. (In the FHIR dateTime case, the produced start and end times might be the same value.)
2. Provide additional date type information to the view runner if the system expects a certain type. For instance, database users may want dateTime fields to be in the database’s native dateTime type – so we need to specify that the data is a full dateTime rather than a partial date.

We attempt to solve this and similar problems with only minimal additions to the specification, although this adds some verbosity to the configuration itself as seen in the example below. This is a tradeoff where we favor a smaller specification and explicitly config.

In that spirit, we use two new constructs here:

First, we will use the ‘union’ operator proposed by Josh Mandel on another thread. This allows users to select from different date time-like types as seen below.

Second, we add a “tags” field that could be applied to any expression. “Tags” is intended to offer general hints or other information to the underlying view runner. We should tag name conventions more broadly (e.g., experimental tags could use leading underscores) but will limit scope here – and so for now we propose a single tag: “xsType” can give the runner a hint on the XML schema data type the view expects. We use xsType since it is referenced from the FHIR specification and offers more granularity, while remaining portable between technology stacks.

Here is a simple example of a view that can handle Observations that may have effective periods or effective date time types. The union keyword lets us check for both, and the “xsType” tag specifies that this use cases expects the result to be an XML date time type, so the database can use an equivalent.

{
  "name": "observation_date_time",
  "from": "Observation",
  "select": [
    { "name": "id", "expr": "id" },
    {
      "union": [{
        "from": "effective.ofType(DateTime)",
        "select": [{"name": "start_time", "expr": "$this", "tags" {"xsType": "xs:dateTime"}},
                   {"name": "end_time", "expr": "$this", "tags" {"xsType": "xs:dateTime"}}]
      }, {
        "from": "effective.ofType(Period)",
        "select": [{"name": "start_time", "expr": "start", "tags" {"xsType": "xs:dateTime"}},
                   {"name": "end_time", "expr": "end", "tags" {"xsType": "xs:dateTime"}}]
      }]
    }
  ]
}

This proposal does not specify the error handling behavior of the runner in case the incoming data does not match the expected type. This will be dependent on the runner itself.

[niquola]

May be we can allow namespaced implementation keywords and flatten it little bit:

• db/table
• pg/type
• xs/type

{
  "name": "observation_date_time",
  "from": "Observation",
  "db/table": "public.observations",
  "select": [
    { "name": "id", "expr": "id" },
    {
      "union": [{
        "from": "effective.ofType(DateTime)",
        "select": [{"name": "start_time", "expr": "$this",  "xs/type": "xs:dateTime", "pg/type": "timestamptz"},
                  {"name": "end_time", "expr": "$this",  "xs/type": "xs:dateTime", "pg/type": "timestamptz"}]
      }, {
        "from": "effective.ofType(Period)",
        "select": [{"name": "start_time", "expr": "start", "xs/type": "xs:dateTime", "pg/type": "timestamptz"},
                  {"name": "end_time", "expr": "end", "xs/type": "xs:dateTime", "pg/type": "timestamptz"}]
      }]
    }
  ]
}

[rbrush]

As discussed on Zulip and in the meeting, we'll need a way for a floorInstant and a ceilingInstant (or equivalent) so users can easily do date/time comparisons with built in SQL functions.

A couple options that seem workable to me:

1. Add a floorInstant() and ceilingInstant() as FHIRPath functions
2. Perform such transformations in the output of the view by adding tags on the fields, something like this:

{
  "union": [{
    "from": "effective.ofType(DateTime)",
    "select": [{"name": "start_time", "expr": "$this",  "tags" {"dateTimeTransform": "floorInstant"}},
              {"name": "end_time", "expr": "$this",  "tags" {"dateTimeTransform": "ceilingInstant"}}]
  }, {
    "from": "effective.ofType(Period)",
    "select": [{"name": "start_time", "expr": "start", "tags" {"dateTimeTransform": "floorInstant"}},
              {"name": "end_time", "expr": "end", "tags" {"dateTimeTransform": "ceilingInstant"}}]
  }]
}

There are likely other options we can consider as well. We could consider different names for these as well, e.g. earliestInstant and latestInstant -- although that leaves a bit of ambiguity of whether it's inclusive or exclusive.

One advantage I see to using floorInstant/ceilingInstant tags rather than new FHIRPath functions is that it keeps this more decoupled from FHIRPath...so implementations could more easily use an existing, available FHIRpath engine to compute views rather than one that supports customized functions.

[jmandel]

Asking at https://chat.fhir.org/#narrow/stream/179266-fhirpath/topic/more.20datetime.20functions.3F about interest in these functions upstream in FHIRPath (seems like a good fit to me)

[rbrush]

I went ahead and drafted a PR with the use of lowBoundary()/ highBoundary() for this, and added the tags concept as discussed as well: #82