Delete and Sync related issues

[super-salad]

There are quite a few delete and sync related issues coming in over the Discord channel, and submitted here on the repo. Would it be possible to create a tag or a milestone to start organizing all of these so we can get a picture of the problem and provide some context so we can start working on it?

[msprijatelj]

Created a "Data Management" milestone, though the "COMP/CON API" Milestone seemed to exist previously and covered some of these deletion/data management issues. @jarena3 If COMP/CON API is a more appropriate milestone, feel free to delete the one I created and move its issues to COMP/CON API!

[super-salad]

This is great @msprijatelj! I will start taking a look to see if there's anything that jumps out at me that I might be able to help with, but I'm a bit wary of touching any of the backend stuff just yet. But no time like the present to start learning.

[jarena3]

yes please, any help would be greatly appreciated. I thought I found the culprit like three or four times now but these issues still persist

[super-salad]

Which files have you been working on in the past? It would be helpful to have a starting point. Right now I'm not sure if it's the timing of calling the sync commands, the commands themselves, or maybe the GraphQL queries?

[msprijatelj]

First thing I'd check is how the local and cloud save dates are being compared. I'm looking at an exported pilot right now and seeing dates saved as strings:

    "lastModified": "Mon Sep 19 2022 16:31:20 GMT-0400 (Eastern Daylight Time)",
    "isDeleted": false,
    "expireTime": "",
    "deleteTime": "",
    "lastUpdate_cloud": "Mon Sep 19 2022 16:58:01 GMT-0400 (Eastern Daylight Time)",
    "lastSync": "Mon Sep 19 2022 16:58:01 GMT-0400 (Eastern Daylight Time)",
    "shareCodeExpiry": "Tue Dec 06 2022 18:40:06 GMT-0500 (Eastern Standard Time)",

My gut reaction is that if these strings are getting compared, they may get gunked up with this format unless the strings are explicitly converted into integer timestamps. E.g. "Mon" always looking "greater" than "Tue" or something.

[Hawkzed]

First thing I'd check is how the local and cloud save dates are being compared. I'm looking at an exported pilot right now and seeing dates saved as strings:

    "lastModified": "Mon Sep 19 2022 16:31:20 GMT-0400 (Eastern Daylight Time)",
    "isDeleted": false,
    "expireTime": "",
    "deleteTime": "",
    "lastUpdate_cloud": "Mon Sep 19 2022 16:58:01 GMT-0400 (Eastern Daylight Time)",
    "lastSync": "Mon Sep 19 2022 16:58:01 GMT-0400 (Eastern Daylight Time)",
    "shareCodeExpiry": "Tue Dec 06 2022 18:40:06 GMT-0500 (Eastern Standard Time)",

My gut reaction is that if these strings are getting compared, they may get gunked up with this format unless the strings are explicitly converted into integer timestamps. E.g. "Mon" always looking "greater" than "Tue" or something.

On this note I've been looking at what gets exported pilot data and noticed that the "lastModified" value doesn't seem to change in accordance with local changes.

I can't seem to map it directly to cloud save functions and the like so perhaps it's of no consequence.

This is the same file.
(Images used to show changes)

I generated these results by:

1. Creating a character then.
2. Using the home button
3. Accessing the pilot roster
4. Exporting the new pilot to clipboard
5. Editing the character
6. Using the home button
7. Accessing the pilot roster
8. Exporting the modified pilot to clipboard

---

On another note Javascript's default Date objects are known to be a bit crap. The .parse() function especially (some may call it unusable [1] links to -> [2] links to -> [3 - Section: Broken Parser – A Web Reality Issue], [4 - Also has a variety of examples]) and it is used in some of the save comparisons:

function determineLatest(cloudItem: CollectionItem, localItem: CollectionItem): string {
  if (cloudItem.lastModifiedCloud === localItem.lastModifiedLocal) return 'synced'
  if (Date.parse(cloudItem.lastModifiedCloud) > Date.parse(localItem.lastModifiedLocal)) {
    return 'cloud'
  } else {
    return 'local'
  }
}

It could be worth integrating a well known replacement such as Luxon to maybe prevent issues in the future if it ever presents a problem.

[Hawkzed]

Having done some more investigation it appears that the 'LastModified' property is sometime out of date despite local changes having occurred leading to the sync comparing the correct last updated cloud key with the incorrect local key after the stale value is collected by the ProcessItemsList function.

Not that it's much evidence but after updating a character's description I excited to the main menu and these values were compared to determine the freshness of the local data, the cloud data was preferred despite local changes and my pilot reverted to the cloud version.

EDIT:
The Last Modified key updates when saving via the SaveController.ts but this change isn't reflected in item_sync.ts

[Hawkzed]

Think I've fixed the problem along with @ptlm500.

Initially we diagnosed the problem with cloud syncing but also noticed a problem also occurred with local changes not persisting across browser refreshes. For example changing a pilot's name on refresh (with auto-sync off) forced the name to return to it's previous version.

It is related to the save() function in src\classes\components\save\SaveController.ts
Removing the || !this._isLoaded from the if statement on line 33 makes the cloud save seem to be functional as well as local changes.

The save function was being bypassed by this early return after the vast majority of changes we tested in the UI.
Certain changes did persist on changes to the pilot profile such as level ups.

public save(skip?: boolean) {
    if (skip) return  // remove || !this._isLoaded
    this.IsDirty = true
    this.LastModified = new Date().toString()
    store.dispatch(`set_${this.Parent.ItemType}_dirty`)
    // const sp = getModule(UserStore, store).UserProfile.IsSavePerformant
    // if (!sp) {
    SaveAllLocalUpdates()
    // }
  }

Not sure of the knock-on effects but when testing locally it appears to have fixed the local LastModified value as well as fixing the issues we observed with saves not persisting locally.

Happy to create a PR, let me know what you find as I'm looking forward to using this tool with my players again.