Filter content referenced on the Composition or IG using the _lastUpdated during download

[dubdabasoduba]

Describe the feature request.

• Currently, the configurations are re-downloaded every time a user logs in with their username and password. While this ensures we fetch updated configurations, it also results in re-downloading content that hasn’t changed, impacting sync time during subsequent syncs.
• This ticket aims to introduce a lastConfigSync timestamp, which will work alongside the _lastUpdated property on FHIR resources.
  • The lastConfigSync timestamp will have the following characteristics:
  • It will be updated each time the content download is completed successfully.
  • When available, it will be included in subsequent content requests after login.
  • It will be stored in the application’s shared preferences.

Acceptance criteria

• Always save the lastConfigSync timestamp everytime the configs are complete sync successfully
• Always attach the lastConfigSync to requests for content referenced on the Composition or IG
• Correctly store the lastConfigSync on the application share preferences

Implementation plan (For Engineers)
The plan for implementing the solution e.g. via a description or a check list for the various ordered tasks that will need to be completed.
i.e. Describe how you intend to solve the problem

[dubdabasoduba]

@ellykits @pld please review this

[pld]

One issue is how to set lastConfigSync, pretty sure we'd need to set this with a time that is provided by the remote server, since I believe that is also where the _lastUpdated is set.

Is there a way that the current server time is already being set back? It might be somewhat cludgy to add that as a returned value from the remote server.

An alternative approach would be to take the largest timestamp in _lastUpdated from the synced resources and use that as the lastConfigSync timestamp. Then we'd make sure we are using the same timezone/etc the server is. If we use this approach we'd have to make sure we use a strict > comparison, as opposed to a >= comparison.

[ndegwamartin]

One issue is how to set lastConfigSync, pretty sure we'd need to set this with a time that is provided by the remote server, since I believe that is also where the _lastUpdated is set.

Is there a way that the current server time is already being set back? It might be somewhat cludgy to add that as a returned value from the remote server.

An alternative approach would be to take the largest timestamp in _lastUpdated from the synced resources and use that as the lastConfigSync timestamp. Then we'd make sure we are using the same timezone/etc the server is. If we use this approach we'd have to make sure we use a strict > comparison, as opposed to a >= comparison.

I believe this is how the FHIR SDK does it for the data FHIR resources - https://github.com/google/android-fhir/blob/master/engine/src/main/java/com/google/android/fhir/sync/download/ResourceParamsBasedDownloadWorkManager.kt#L115-L127

[pld]

cool sweet, great find, yea let's follow the exact same logic, that's the alternative approach I noted

[dubdabasoduba]

@Aleem92 I just tested the HAPI API on the current version we use.

_lastUpdated doesn't seem to work when performing GET {{base_url}}/Composition/<uuid> however it works on {{base_url}}/Composition?_lastUpdated=<date>

e.g https://{{base_url}}/fhir/Composition?_lastUpdated=gt2025-04-18

[dubdabasoduba]

@pld @ndegwamartin We might have to pause this until we test it on the latest version of HAPI. If still doesn't work then we close the ticket

[Aleem92]

@Aleem92 I just tested the HAPI API on the current version we use.

_lastUpdated doesn't seem to work when performing GET {{base_url}}/Composition/<uuid> however it works on {{base_url}}/Composition?_lastUpdated=<date>

e.g https://{{base_url}}/fhir/Composition?_lastUpdated=gt2025-04-18

Thanks for confirming this, @dubdabasoduba. I had raised this issue regarding {{base_url}}/Binary/<uuid>?_lastUpdated=<date> not working as expected. I appreciate you taking the time to test and confirm the behavior on the ticket.

[pld]

Can you clarify, it works on Composition, but does not work for Binary?

[Aleem92]

Can you clarify, it works on Composition, but does not work for Binary?

I have also tested the {{base_url}}/Composition?identifier=appid&_lastUpdated=gtdate. However, in either case, the _lastUpdated parameter does not work.

[pld]

Can we pass UUID as a search param?

[Aleem92]

Can we pass UUID as a search param?

Currently, we can retrieve any resource using a URL similar to this: {{base_url}}/Questionnaire/061e24f4-5c9b-477f-8062-249dd43bc20f.

Here, 061e24f4-5c9b-477f-8062-249dd43bc20f is the UUID and the HTTP method used is GET.

[pld]

But then it ignores last update, correct? If we use the search API but query on UUID and lastUpdated does that work?On Jan 14, 2025, at 07:25, Aleem92 ***@***.***> wrote: Can we pass UUID as a search param? Currently, we can retrieve any resource using a URL similar to this: {{base_url}}/Questionnaire/061e24f4-5c9b-477f-8062-249dd43bc20f. Here, 061e24f4-5c9b-477f-8062-249dd43bc20f is the UUID and the HTTP method used is GET. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[dubdabasoduba]

@pld I think your idea of using the id as a search param might work.

• https://unicef-mg-eusm-preview.smartregister.org/fhir/Composition?_id=726512ba-c752-4fcb-9ed1-e622143b2c3d&_lastUpdated=gt2024-11-27T13%3A18

{
    "resourceType": "Bundle",
    "id": "37c62c06-f443-4950-ba1e-49f02b9a943c",
    "meta": {
        "lastUpdated": "2025-01-14T14:04:26.882+00:00"
    },
    "type": "searchset",
    "total": 0,
    "link": [
        {
            "relation": "self",
            "url": "https://unicef-mg-eusm-preview.smartregister.org/fhir/Composition?_id=726512ba-c752-4fcb-9ed1-e622143b2c3d&_lastUpdated=gt2024-11-27T13%3A18"
        }
    ]
}

• https://unicef-mg-eusm-preview.smartregister.org/fhir/Composition?_id=726512ba-c752-4fcb-9ed1-e622143b2c3d&_lastUpdated=gt2024-11-27T13%3A00

{
  "resourceType": "Bundle",
  "id": "4b23ebc9-b127-452f-a743-2db5caead386",
  "meta": {"lastUpdated": "2025-01-14T14:05:05.683+00:00"},
  "type": "searchset",
  "total": 1,
  "link": [
    {
      "relation": "self",
      "url": "https://unicef-mg-eusm-preview.smartregister.org/fhir/Composition?_id=726512ba-c752-4fcb-9ed1-e622143b2c3d&_lastUpdated=gt2024-11-27T13%3A00"
    }
  ],
  "entry": [
    {
      "fullUrl": "https://unicef-mg-eusm-preview.smartregister.org/fhir/Composition/726512ba-c752-4fcb-9ed1-e622143b2c3d",
      "resource": {
        "resourceType": "Composition",
        "id": "726512ba-c752-4fcb-9ed1-e622143b2c3d",
        "meta": {
          "versionId"  : "22"                           ,
          "lastUpdated": "2024-11-27T13:18:17.549+00:00",
          "source"     : "#2dc85ddb5a4c85f3"
        },
        "identifier": {"use": "official", "value": "eusm"},
        "status": "final",
        "type": {
          "coding": [
            {
              "system" : "http://snomed.info/sct"                ,
              "code"   : "1156600005"                            ,
              "display": "{{ 9546a182eec392e14695c3bc527e2f8e }}"
            }
          ]
        },
        "date": "2022-07-28",
        "title": "{{ 5a2b1100d89197bf43b57475c849a73d }}",
        "confidentiality": "L",
        "section": [
          {
            "title": "Application Config",
            "focus": {
              "reference": "Binary/592f27bc-d476-4d8f-9307-07e00f1b91bb",
              "identifier": {"value": "application"}
            },
            "mode": "working"
          },
          {
            "title": "{{ d508d185313161f0e73f0c8b702a49e6 }}",
            "focus": {
              "reference": "Binary/95eb80b2-4a86-45d0-82ef-4cb597d1df1b",
              "identifier": {"value": "sync"}
            },
            "mode": "working"
          },
          {
            "title": "{{ 42df3020c4f64ab62bb1f00baf206af1 }}",
            "focus": {
              "reference": "Binary/41721d46-ab5b-4623-8da2-632936066f32",
              "identifier": {"value": "navigation"}
            },
            "mode": "working"
          },
          {
            "title": "{{ d76b6684ceb73bb1cedc0ed24f791893 }}",
            "mode": "working",
            "section": [
              {
                "title": "{{ aa07e209084c7da3d381c2c6dfcfcab8 }}",
                "focus": {
                  "reference": "Binary/c7b84635-110f-4c16-a8e2-452f4e90516b",
                  "identifier": {"value": "servicePointRegister"}
                }
              }
            ]
          },
          {
            "title": "{{ 8585e13148ea3c2729485386a365e474 }}",
            "mode": "working",
            "section": [
              {
                "title": "{{ f68cc06a57d727d070a615bc16e31ee1 }}",
                "focus": {
                  "reference": "Binary/ca3a7472-7f3a-459f-b2db-342b688e50c5",
                  "identifier": {"value": "strings"}
                }
              },
              {
                "title": "{{ f49212ddf8958bf5c61a754eab9730c7 }}",
                "focus": {
                  "reference": "Binary/08abecce-5ecd-470a-abfa-58170fdd9ca0",
                  "identifier": {"value": "stringsFr"}
                }
              }
            ]
          },
          {
            "title": "{{ 9383d610382881c5205cb6e54881c96a }}",
            "mode": "working",
            "section": [
              {
                "title": "{{ 9dd99645be1432de0a34e72421875012 }}",
                "focus": {
                  "reference": "Binary/713f3a9d-e628-4613-9173-bd24d9f130f4",
                  "identifier": {"value": "servicePointProfile"}
                }
              },
              {
                "title": "{{ bc24d98da249c756bbb426b62d4448ae }}",
                "focus": {
                  "reference": "Binary/fe973991-edf4-4321-886a-76203dc56f28",
                  "identifier": {"value": "pastVisitProfile"}
                }
              }
            ]
          },
          {
            "title": "{{ f807e86e4529410190c1d32f0d6520af }}",
            "mode": "working",
            "section": [
              {
                "title": "{{ 987df653465008a55d2b264e4edd6a90 }}",
                "focus": {
                  "reference": "Binary/71f18e80-ddb9-4ef4-a077-f7738a1ad843",
                  "identifier": {"value": "locationMap"}
                }
              }
            ]
          },
          {
            "title": "{{ 6c84d1acb017e380e18853ad8966994a }}",
            "mode": "working",
            "section": [
              {
                "title": "{{ 88bb7a4b8a41e611542419ed47d39928 }}",
                "focus": {
                  "reference": "Questionnaire/97f117a8-2a05-45c9-c49d-77d0ae5feaea",
                  "identifier": {"value": "ServicePointQuestionnaire"}
                },
                "mode": "working"
              },
              {
                "title": "{{ 787dc245bc3529570bff155fbe5eeb2f }}",
                "focus": {
                  "reference": "Questionnaire/f59f1024-3765-4982-a747-c13c8a18d4c9",
                  "identifier": {"value": "consultBeneficiariesQuestionnaire"}
                },
                "mode": "working"
              },
              {
                "title": "{{ d6d77d1d066039651ba67f6df978caa9 }}",
                "focus": {
                  "reference": "Questionnaire/ffbf0a04-1960-41c9-8ba2-396e22e8139f",
                  "identifier": {"value": "recordGPSLocationQuestionnaire"}
                },
                "mode": "working"
              },
              {
                "title": "{{ e12cc6ba435f50a3b30e11df0acd1df0 }}",
                "focus": {
                  "reference": "Questionnaire/649cd348-2d97-44e4-83c4-a8a128afba7f",
                  "identifier": {"value": "prodCheckQuestionnaire"}
                },
                "mode": "working"
              },
              {
                "title": "Warehouse check",
                "focus": {
                  "reference": "Questionnaire/b2a81660-ef28-4322-f817-f3834af0773e",
                  "identifier": {"value": "WarehouseQuestionnaire"}
                },
                "mode": "working"
              },
              {
                "title": "{{ 859eb4c1f593ef82c30bf985dbcb23fd }}",
                "focus": {
                  "reference": "Questionnaire/2e1392d4-6519-4b50-8591-b18f84814da5",
                  "identifier": {"value": "registerFilters"}
                },
                "mode": "working"
              },
              {
                "title": "{{ 1d62c00c4c2eee92e17b654e53138d60 }}",
                "focus": {
                  "reference": "Questionnaire/8f7cada7-acf5-430c-dee6-9a7cbd1dc2d1",
                  "identifier": {"value": "undo check"}
                },
                "mode": "working"
              }
            ]
          },
          {
            "title": "{{ 3395e8dc6b7e47a4b61529a0d9f5be8d }}",
            "mode": "working",
            "section": [
              {
                "title": "{{ 62ee95a86e23858d6af710ba256465e8 }}",
                "focus": {
                  "reference": "StructureMap/1e1c90d0-a4e3-485b-a5d5-5c774dff290f",
                  "identifier": {"value": "ConsultBeneficiariesStructureMap"}
                },
                "mode": "working"
              },
              {
                "title": "{{ 06fcbb01b927bc17ed0dce946d964812 }}",
                "focus": {
                  "reference": "StructureMap/d0a71261-2989-4dc7-b0b4-7f6783052217",
                  "identifier": {"value": "ServicePointCheckStructureMap"}
                },
                "mode": "working"
              },
              {
                "title": "{{ ba9b738eb6dbd2a207f3b771df47b38e }}",
                "focus": {
                  "reference": "StructureMap/cb42acd7-e9a6-4c64-94a6-45311776bebb",
                  "identifier": {"value": "recordGPSStructureMap"}
                },
                "mode": "working"
              },
              {
                "title": "{{ 911f3e87960b9a03bed54576b5d2375d }}",
                "focus": {
                  "reference": "StructureMap/02e74505-ac5e-4216-b837-76474aea99b2",
                  "identifier": {"value": "prodCheckStructureMap"}
                },
                "mode": "working"
              },
              {
                "title": "{{ d260fd4942ac96dcc831815698bd350e }}",
                "focus": {
                  "reference": "StructureMap/25716531-690e-4e33-9628-66eb3201ecef",
                  "identifier": {"value": "undoCheckStructureMap"}
                },
                "mode": "working"
              },
              {
                "title": "Warehouse check structure map",
                "focus": {
                  "reference": "StructureMap/952f3d3c-2279-43ee-95c3-ae73d90918f7",
                  "identifier": {"value": "warehouseCheckStructureMap"}
                },
                "mode": "working"
              }
            ]
          },
          {
            "title": "{{ 56333edacce0c2d2ce31f5bf7464f565 }}",
            "mode": "working",
            "section": [
              {
                "title": "{{ d41d8cd98f00b204e9800998ecf8427e }}",
                "focus": {
                  "reference": "List/d1ea7ffc-7958-11ee-b962-0242ac120002",
                  "identifier": {"value": "servicePointList"}
                },
                "mode": "working"
              }
            ]
          },
          {
            "title": "{{ 266cae2087af23c12b8c71919ae57792 }}",
            "mode": "working",
            "section": [
              {
                "title": "{{ d41d8cd98f00b204e9800998ecf8427e }}",
                "focus": {
                  "reference": "List/ea15c35a-8e8c-47ce-8122-c347cefa1b4a",
                  "identifier": {"value": "productList"}
                },
                "mode": "working"
              }
            ]
          },
          {
            "title": "{{ 3e7ecdffb97d880057b6c9463f1ff985 }}",
            "mode": "working",
            "section": [
              {
                "title": "{{ d41d8cd98f00b204e9800998ecf8427e }}",
                "focus": {
                  "reference": "List/90e91971-6a46-4948-9543-f5efb6aa48df",
                  "identifier": {"value": "inventoryList"}
                },
                "mode": "working"
              }
            ]
          }
        ]
      },
      "search": {"mode": "match"}
    }
  ]
}

[dubdabasoduba]

Just changing how we define the URLs might work.

The tests above use the current servers. The assumption is this does not change on the new server versions