Migrate old data

[rushsteve1]

The old versions of Observatory and the WIki need to be migrated to the new database as soon as we have a new production DB running.

Old data can be found on:

• The current rcos.io MongoDB instance
• The old Observatory-new server has the SQLite database it used and DokuWiki files
• Google Sheets documents
• And possibly elsewhere?

[Apexal]

Postgrest's bulk insert routes will come in handy here.

[rushsteve1]

Probably the easiest and best way to do this would be to transfer the old DO Droplet to the new RCOS account, then pull the data off of it. This would also preserve a backup of the old server in case we ever need it.

Actually migrating the data might be a bit of a pain though. We're probably going to have to use mongoexport or mongodump to dump the data to JSON (or CSV?), and then edit it to match the new schema before importing it to Postgres.
Something tells me this is going to be harder than it seems though.

[rushsteve1]

Attacked is a ZIP containing a script to dump files from BSON to JSON, and then a set of jq filters to transform the JSON to a more compatible schema.

data-transforms.zip