Documenting the syncing protocol

docs/SYNCING-PROTOCOL.md

@@ -4,52 +4,7 @@ This doc will try to give a breif (_as possible_) overview of the client/server
 protocol Hammer
 uses.
 
-## Goal
-
-The goal of this protocol is to synchronize the various `Entities` on each client to the server.
-
-There is no file history as in a true version control system such as `git`. The way to think about
-this protocol is more as a simpler synchronization system. However it is smart enough to detect
-conflicts: the same file that has been edited in different ways on different devices, must allow the
-user to resolve the conflict.
-
-As such, there is very little book keeping data, and none of it is actually required. When all
-actors are fully synchronized, they all contain the full set of data. Thus if the server were to die
-and lose all of its data, it wouldn't matter. Every client would contain everything necessary to
-setup on a new server.
-
-## Terminology
-
-**Entity** any individual block of data. Each entity is given a unique ID. Examples include:
-
-- Scene
-- Scene Draft
-- Timeline Event
-- Encyclopedia Entry
-- Note
-
-**Entity ID** Every Entity is given an Entity ID, which is a unique, monotonically incrementing
-integer, with the first valid ID being 1
-
-**Sync ID** This is a UUID generated by the server and passed back to the client identifying a
-particular syncing session to a particular client. The server will only allow one syncing session per
-account at a time to prevent race conditions.
-
-**Entity Update Sequence** A list of Entity IDs in a particular order determined by the server.
-The client will update these IDs in the provided order. The server will leave out IDs of Entities
-that do not need synchronization.
-
-**Re-ID** The process of taking a client side Entity and issuing it a new ID, changing any
-references to that ID in the process.
-
-**Dirty Entity** When a client edits a local Entity, the client first hashes the existing,
-pre-edited content, and saves off the **Entity ID** and this pre-edit hash of the data to a "dirty
-list". If the client and server are in sync at the time of this edit, then the saved hash in the
-dirty list will match the hash of the server's copy of the Entity.
-At syncing time this allows us to detect conflicts. If another client edits the same entity, and
-syncs with the server first.
-Thus our local "dirty list" hash will not match the hash of the server side copy, and we'll know we
-have a conflict that needs resolving.
+## Two levels of syncing
 
 **Account Sync** This synchronizes what projects the Account has, creating or deleting just the top
 level directories on the client
@@ -143,6 +98,114 @@ sequenceDiagram
 
 ## Project Sync Protocol
 
+## Goal
+
+The goal of this protocol is to synchronize the various `Entities` on the client to the server.
+
+There is no file history as in a true version control system such as `git`. This is instead a simpler synchronization system, yet still smart enough to detect `conflicts`, and prevent edits on various devices from overwriting each other on accident.
+
+As such, there is very little book keeping data, and none of it is actually required. When all actors are fully synchronized, they all contain the full set of data. Thus if the server were to die and lose all of its data, it wouldn't matter. Every client would contain everything necessary to
+setup on a new server.
+
+## Terminology
+
+**Entity**
+any individual block of data. Each entity is given a unique ID. Examples include:
+
+- Scene
+- Scene Draft
+- Timeline Event
+- Encyclopedia Entry
+- Note
+
+**Entity ID**
+Every Entity is given an Entity ID, which is a unique, monotonically incrementing
+integer, with the first valid ID being 1
+
+**Sync ID**
+This is a UUID generated by the server and passed back to the client identifying a
+particular syncing session to a particular client. The server will only allow one syncing session per
+account at a time to prevent race conditions.
+
+**Entity Update Sequence**
+A list of Entity IDs in a particular order determined by the server.
+The client will update these IDs in the provided order. The server will leave out IDs of Entities
+that do not need synchronization.
+
+**Re-ID**
+The process of taking a client side Entity and issuing it a new ID, changing any
+references to that ID in the process.
+
+**Conflicts**
+The same file that has been edited in different ways on different devices, must allow the user to resolve the conflict in order to bring them back into sync with each other.
+
+**Dirty Entity** When a client edits a local Entity, the client first hashes the existing,
+pre-edited content, and saves off the **Entity ID** and this pre-edit hash of the data to a "dirty
+list". If the client and server are in sync at the time of this edit, then the saved hash in the
+dirty list will match the hash of the server's copy of the Entity.
+At syncing time this allows us to detect conflicts. If another client edits the same entity, and
+syncs with the server first.
+Thus our local "dirty list" hash will not match the hash of the server side copy, and we'll know we
+have a conflict that needs resolving.
+
+### Network Protocol (overview)
+
+This is largely a client driven synchronization process.
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant Server
+
+    Client->>Server: POST /project/$userId/$projectName/begin_sync
+	activate Server
+	Note right of Client: ProjectID<br/>ClientState
+
+	Server -->> Client: 200 OK (Sync Began)
+	deactivate Server
+	activate Client
+	Note left of Server: ProjectSynchronizationBegan
+
+    rect rgb(74, 0, 9)
+        loop Delete Entities
+            Client->>Server: GET /project/$userId/$projectName/delete_entity/$id
+            deactivate Client
+            activate Server
+            
+            Server -->> Client: 200 OK
+            deactivate Server
+            activate Client
+            Note left of Server: DeleteIdsResponse
+        end
+    end
+
+    rect rgb(11, 0, 74)
+        loop Entity Transfer
+        Note right of Client: See breakout sectioin for details
+            Client->>Server: 
+            Server->>Client: 
+        end
+    end
+
+    Client->>Server: POST /project/$userId/$projectName/end_sync
+	deactivate Client
+	activate Server
+	Note right of Client: ProjectID<br/>SyncId
+	Server -->> Client: 200 OK (Sync Began)
+	deactivate Server
+	activate Client
+```
+
+### Network Protocol (Entity Transfer)
+TBD
+```mermaid
+```
+
+### Client Overview
+TBD
+
+### Explained (_old explaination, possibly out of date_)
+
 - Client makes a request to begin a syncing session:
 	- `/api/project/$userId/$projectName/begin_sync`
 	- The client crawls all of it's local entities, and hashes each one using `MurmurHash3`