fix(datastore): full sync when sync predicate changes

[lawmicha]

Issue #

#3383

Description

Perform a full sync when the sync expression for the model changes. To do this, we have to persist and retrieve it from the sync metadata table for comparison with the incoming sync predicate.

The sync expression can be made up of mulitple predicates, each correponding to a specific model. InitialSyncOperation extracts out the first sync predicate from the sync expression for the corresponding model. This QueryPredicate is then used to compare with the existing persisted sync predicate from the last sync run, stored in the ModelSyncMetadata table.

We extend the ModelSyncMetadata table to add a new column, make QueryPredicate Encodable so we can encode the sync predicate and store it as a String. The encoder has the output formatting set to .sortedKeys to make sure the same string is generated every time. We do a string comparision of the incoming sync predicate with the persisted one to see if it was changed.

We do a full sync on these scenarios:

• no previous sync predicate and now there is a sync predicate
• there is a previous sync predicate and no current sync predicate
• both exist but they do not match

When there are matching sync predicates or no previous and current sync predicates, then the sync type falls back to the previous logic to determine full or delta (based on the sync interval).

Existing app upgrade scenario

Extending the ModelSyncMetadata table means that existing apps that have already created the table will not recreate it for the new column. The most optimal way, without deleting the local data, is to add a new column if we can detect the column doesn't exist. This is in the ModelSyncMetadataMigration logic.

Verification

Detected ModelSyncMetadata table exists without syncPredicate column.
ALTER TABLE "ModelSyncMetadata" ADD COLUMN "syncPredicate" "text";
ModelSyncMetadata table altered to add syncPredicate column.

General Checklist

• Added new tests to cover change, if needed
• Build succeeds with all target using Swift Package Manager
• All unit tests pass
• All integration tests pass https://github.com/aws-amplify/amplify-swift/actions/runs/7062531087
• Security oriented best practices and standards are followed (e.g. using input sanitization, principle of least privilege, etc)
• Documentation update for the change if required
• PR title conforms to conventional commit style
• New or updated tests include Given When Then inline code documentation and are named accordingly testThing_condition_expectation()
• If breaking change, documentation/changelog update with migration instructions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[MahmoudAlsamman]

@lawmicha What is the status of this fix? is it going to be merged soon?

[codecov-commenter]

Codecov Report

Attention: 61 lines in your changes are missing coverage. Please review.

Comparison is base (66c46a7) 67.96% compared to head (83fec69) 67.91%.

Files	Patch %	Lines
...ify/Categories/DataStore/Query/QueryOperator.swift	0.00%	36 Missing ⚠️
...ePlugin/Migration/ModelSyncMetadataMigration.swift	81.81%	10 Missing ⚠️
...fy/Categories/DataStore/Query/QueryPredicate.swift	0.00%	9 Missing ⚠️
...Core/Sync/ModelSync/ModelSyncMetadata+Schema.swift	0.00%	2 Missing ⚠️
...PluginsCore/Sync/ModelSync/ModelSyncMetadata.swift	0.00%	2 Missing ⚠️
...Plugin/Sync/InitialSync/InitialSyncOperation.swift	96.22%	2 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2757      +/-   ##
==========================================
- Coverage   67.96%   67.91%   -0.05%     
==========================================
  Files        1077     1079       +2     
  Lines       35988    36127     +139     
==========================================
+ Hits        24459    24536      +77     
- Misses      11529    11591      +62     

Flag	Coverage Δ
API_plugin_unit_test	67.07% <ø> (ø)
AWSPluginsCore	64.38% <0.00%> (-0.07%)	⬇️
Amplify	47.47% <0.00%> (-0.34%)	⬇️
Analytics_plugin_unit_test	81.16% <ø> (ø)
Auth_plugin_unit_test	79.47% <ø> (-0.08%)	⬇️
CoreMLPredictions_plugin_unit_test	59.44% <ø> (ø)
DataStore_plugin_unit_test	79.88% <89.65%> (+0.16%)	⬆️
Geo_plugin_unit_test	70.75% <ø> (ø)
Logging_plugin_unit_test	63.09% <ø> (ø)
Predictions_plugin_unit_test	37.29% <ø> (ø)
PushNotifications_plugin_unit_test	87.03% <ø> (ø)
Storage_plugin_unit_test	78.00% <ø> (ø)
unit_tests	67.91% <63.03%> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lawmicha]

@lawmicha What is the status of this fix? is it going to be merged soon?

Hey @MahmoudAlsamman, I'm picking this PR back up and will provide an update soon!