Staging deployment and Railway statistics

[jpolchlo]

We've been somewhat primitive in the way that we deploy and test different versions of OSMesa. A request arrived where we finally could not ignore the fact that a staging environment was needed, as it showed our terrible lack of sophistication in the way that we were storing and dealing with statistics. Previously, every small change to the stats would require a change in table schema, which required a complex backfill process into an existing and actively used DB table. Following which, we would need to restart a streaming update process and then infill. All because we were amending a live table that was feeding a process visible to end users.

This PR will split our deployments into staging and production environments. This considerably simplifies changes, as they can be private to a development team. Rolling out a new stat will then be accomplished by deleting the contents of any existing tables, repopulating the tables, restarting a streaming process (which can take as long as needed to catch up), and then redeploying dependent services.

This split was prompted by the need for railway statistics reporting which are also added to this PR.

The changes here are mirrored by those in azavea/osmesa-stat-server#22.

[jpolchlo]

Managed a successful ingest of the basic stats. These are currently populating the staging DB. Will take the time to see that we have identified any railway stats, but seems like the JSON stuff works, after slight modification (see last commit). I'll try and fire up the streaming job in the morning.

[jpolchlo]

Can positively report that railway stats are in the table. Also, we are now automatically adding checkpoints to the DB so that we can more smoothly start up the streaming update process

[jpolchlo]

This now depends on geotrellis/vectorpipe#109. I'll need to merge that PR and cut a new minor release prior to merging this contribution.

[mojodna]

Is this needed?

[jpolchlo]

Used here. Perhaps it's not needed? I may have just included this as a test. I don't entirely recall.

[mojodna]

When I've run things in the past, I haven't needed this, fwiw. It looks like it may be GT boilerplate for EMR.

[jpolchlo]

It's probably not needed. I was throwing whatever at the wall to get things to run to completion, so I think I can drop it now that things are all sussed out.

[mojodna]

Since this is used elsewhere, should it be outside of the oneoffs package?

[jpolchlo]

There is so much reorganization of this repo that needs to happen. I can see breaking out oneoffs into streaming, batch, and utilities.

[jpolchlo]

Seems like the long chain of discussion here has finally played out. Merging.