Skip to content

Latest commit

 

History

History
 
 

db-service

README.md

DB Service

This service is still WIP. It uses s3fs to mount an S3-compatible storage to /mnt/s3 then serve PGlite instances via the PGDATA that lives under /mnt/s3/dbs/<id>.

It also requires TLS certs, since we use SNI to reverse proxy DB connections (eg. 12345.db.example.com serves /mnt/s3/dbs/12345). These certs live under /mnt/s3/tls.

TODO

  • Containerize
  • Connect to Supabase DB to validate creds/dbs
  • DB versioning
  • PGlite upload service

Development

Without s3fs

If want to develop locally without dealing with containers or underlying storage:

  1. Generate certs that live under ./tls:

    npm run generate:certs

  2. Run the pg-gateway server:

    npm run dev

    All DBs will live under ./dbs.

  3. Connect to the server via psql:

    psql "host=localhost port=5432 user=postgres"

    or to test a real database ID, add a loopback entry to your /etc/hosts file:

     # ...

 127.0.0.1 12345.db.example.com

    and connect to that host:

    psql "host=12345.db.example.com port=5432 user=postgres"

With s3fs

To simulate an environment closer to production, you can test the service with DBs backed by s3fs using Minio and Docker.

  1. Start Minio as a local s3-compatible server:

    docker compose up -d minio

  2. Initialize test bucket:

    docker compose up minio-init

    This will run to completion then exit.

  3. Initialize local TLS certs:

    docker compose up --build tls-init

    This will build the container (if it's not cached) then run to completion and exit. Certs are stored under /mnt/s3/tls.

  4. Run the pg-gateway server:

    docker compose up --build db-service

    This will build the container (if it's not cached) then run the Node db-service. All DBs will live under /mnt/s3/dbs.

  5. Connect to the server via psql:

    psql "host=localhost port=5432 user=postgres"

    Note the very first time a DB is created will be very slow (s3fs writes are slow with that many file handles) so expect this to hang for a while. Subsequent requests will be much quicker. This is temporary anyway - in the future the DB will have to already exist in /mnt/s3/dbs/<id> in order to connect.

    or to test a real database ID, add a loopback entry to your /etc/hosts file:

     # ...

 127.0.0.1 12345.db.example.com

    and connect to that host:

    psql "host=12345.db.example.com port=5432 user=postgres"

To stop all Docker containers, run:

docker compose down