Skip to content

Commit

Permalink
Updates for validation (#53)
Browse files Browse the repository at this point in the history
* Updates for validation

* Update synthea README to describe validation & testing against measures
  • Loading branch information
lmd59 authored Dec 13, 2024
1 parent eb42a6c commit 269d549
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 2 deletions.
15 changes: 15 additions & 0 deletions synthea/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,21 @@ The [Synthea](https://github.com/synthetichealth/synthea) project can be used to
- You may also use the flexporter standalone to map an existing exported file: `./run_flexporter -fm {mapping file location} -s {source fhir file}`.
- See the [flexporter documentation](https://github.com/synthetichealth/synthea/wiki/Flexporter) for additional information on the flexporter, mapping file, and limitations.

## Validation and Testing
1. Set up updated Synthea functionality
- Functionality from https://github.com/synthetichealth/synthea/pull/1527 is required to run the flexporter, and functionality from https://github.com/synthetichealth/synthea/pull/1540 is required for more diversity in calculations while testing. If not merged, recommend pulling one PR and cherry picking the other.
- On your local instance of synthea: in src/main/resources/synthea.properties, uncomment line 301 in order to use the terminology server, and set `exporter.fhir.us_core_version = 3.1.1`.
- Run synthea with the flexporter to generate patient files. Use -p to specify the number of patients, i.e. `./run_synthea -fm ../../bulk-export-server/synthea/qpp_qicore.yaml -p 10`

2. Use HAPI FHIR validator to validate the patients.
- Use https://github.com/hapifhir/org.hl7.fhir.validator-wrapper/releases/latest/download/validator_cli.jar
- Run with the following options `java -jar validator_cli.jar synthea/output/fhir/ -version 4.0.1 -html-output validation.html -ig http://hl7.org/fhir/us/qicore/ImplementationGuide/hl7.fhir.us.qicore%7C4.1.1 -extension any -display-issues-are-warnings -sct us -clear-tx-cache`
- Most issues are solved by these settings, but some remaining issues that don’t effect calculation may include: "Unknown code 'X9999-"* from the loinc code set

3. Use fqm-execution (or testify) to calculate output patients against measures
- Example run: `run cli -- reports -m CMS130-bundle.json --patients-directory ../synthea/output/fhir/ --trust-meta-profile true -o -s 2024-01-01 -e 2024-12-31 --report-type summary`
- This will give a summary of how many patients fall into each population. For CMS130, there is the expectation of some level of diversity in which populations synthea patients fall into. Other measures may have less diversity in calculated populations until we do further investigation into aligning synthea codes with measure codes.

## Building the Mapping File

Quality Measurement calculation requires data conformant with qicore, an expansive IG, which would require expansive effort to fully map. As such, we can piecemeal address the IG requirements by supporting requirements for individual measures. The recommended process for creating a mapping that addresses a set of measures is:
Expand Down
13 changes: 11 additions & 2 deletions synthea/qpp_qicore.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: QI Core - 130 minimal and QPP
name: QI Core - QPP
applicability: true

customValueSets:
Expand Down Expand Up @@ -119,14 +119,16 @@ actions:
profiles:
- http://hl7.org/fhir/us/qicore/StructureDefinition/qicore-devicerequest
fields:
- location: DeviceRequest.code.reference
- location: DeviceRequest.codeReference.reference
value: $findRef([Device])
- location: DeviceRequest.authoredOn
value: $getField([Device.manufactureDate]) # manufacture time is set 3 weeks before device model's start, so this is close enough
- location: DeviceRequest.status
value: completed
- location: DeviceRequest.intent
value: order
- location: DeviceRequest.subject.reference
value: $findRef([Patient])
# Creating one AdverseEvent per patient
- resourceType: AdverseEvent
based_on:
Expand All @@ -136,6 +138,10 @@ actions:
fields:
- location: AdverseEvent.event.coding
value: $randomCode([http://hl7.org/fhir/ValueSet/adverse-event-type]) # may change in future to use a smaller subset of the ValueSet
- location: AdverseEvent.subject.reference
value: $findRef([Patient])
- location: AdverseEvent.actuality
value: $randomCode([http://hl7.org/fhir/ValueSet/adverse-event-actuality,code])
# Creating one Communication per patient
- resourceType: Communication
based_on:
Expand Down Expand Up @@ -177,3 +183,6 @@ actions:
value: $randomCode([http://hl7.org/fhir/ValueSet/task-code]) # randomize based on values available
- location: Task.executionPeriod
value: $getField([Procedure.performed])
- location: Task.intent
value: $randomCode([http://hl7.org/fhir/ValueSet/task-intent,code])

0 comments on commit 269d549

Please sign in to comment.