Getting Started Docker

[alexstaroselsky]

When using supabase docker and creating a client using supabase-js what additional steps need to be taken in terms of the database to get started?

• Does a database need to be created via SQL commands? If so, does it need to have a certain name? Does it need to have specific tables with specific schemas created such as one for the user management functionality? Or is createClient supposed to do this automatically?

[kiwicopple]

Hi @alexstaroselsky, the docker instructions that you linked to are only for one component of Supabase, however Supabase is actually 6 different tools (see the docs here):

We haven't yet created a docker image to pull everything into a single docker file (but we are working on it).

I'm happy to outline the steps for using the Realtime server, but I assume you want to use the full system? If this is the case, we are hoping to have it ready in about 1 month

[ctwhome]

thanks @kiwicopple, I'll give it another spin and tell you later

[Zubnix]

@kiwicopple I tried to run the docker-compose locally. It all starts up fine, however it seems the required db is not created. When connecting to postgress directly, no auth db is visible (only the internal pg one). When using the javascript example to login I get an error (because the required db doesn't exist)

auth_1      | time="2021-03-08T10:44:45Z" level=info msg="request started" component=api method=POST path=/signup referer="http://localhost:3001/" remote_addr="192.168.64.2:34854" request_id=759a6d7c-e897-41f2-a591-02383b125225
db_1        | 2021-03-08 10:44:45.018 UTC [130] ERROR:  relation "users" does not exist at character 447
db_1        | 2021-03-08 10:44:45.018 UTC [130] STATEMENT:  SELECT users.aud, users.confirmation_sent_at, users.confirmation_token, users.confirmed_at, users.created_at, users.email, users.email_change, users.email_change_sent_at, users.email_change_token, users.encrypted_password, users.id, users.instance_id, users.invited_at, users.is_super_admin, users.last_sign_in_at, users.raw_app_meta_data, users.raw_user_meta_data, users.recovery_sent_at, users.recovery_token, users.role, users.updated_at FROM users AS users WHERE instance_id = $1 and email = $2 and aud = $3 LIMIT 1
auth_1      | time="2021-03-08T10:44:45Z" level=error msg="500: Database error finding user" component=api error="error finding user: ERROR: relation \"users\" does not exist (SQLSTATE 42P01)" method=POST path=/signup referer="http://localhost:3001/" remote_addr="192.168.64.2:34854" request_id=759a6d7c-e897-41f2-a591-02383b125225
auth_1      | time="2021-03-08T10:44:45Z" level=info msg="request completed" component=api duration=971371 method=POST path=/signup referer="http://localhost:3001/" remote_addr="192.168.64.2:34854" request_id=759a6d7c-e897-41f2-a591-02383b125225 status=500

[JonDum]

@kiwicopple I really appreciate the effort to support self-hosted. From a future monetization standpoint, I think you're making a smart decision. I can't speak for others, but no self-hosting is a deal breaker and I'd rather open up the wallet to support supabase w/ self-hosting than it be quasi "open-source" without first class support for self-hosting. I'm sure many other devs feel the same — no one wants to invest time in a promising project that has no longevity or support.

That said I think you have quite a few directions you're pulling yourself in right now and this is leading to a lot of confusion. There's several threads floating around and several attempts (that I could find). To sum,

1. You created several "dev" docker images at 0.0.1 that each was responsible for running a part of the stack. It looks to me that these got abandoned. The whole "emulator" wording added to the confusion.

2. You created a "dev" docker image supabase-dev that attempted to put all the moving parts into one image for ease of use. Great idea, albeit it is not very Docker "best practices" (one container, one process) so I think people's confusion from this was if it was meant to be run in production or only for dev.

3. You made a supabase npm package that attempts to mimic a nice cli tool and create the compose stack for the developer. By no means not a bad idea and could be useful in the future for devs that like these fancy heroku-esque tools.

4. You made a docker folder in the root of the repo and made suggestions about this being the "de facto" way of running the full stack. This is probably the safest, most docker-esque route so far.

I think with a bit of consolidation and a section in the documentation about how to run locally we could put this to rest.

Personally, I think you were closest with Option 1 (standalone, prebuilt docker images). From the standpoint of a developer checking out supabase and wanting to try it out locally, they are likely already going to have Docker running their own application server via compose, so the easiest way would be to add some already pre-built images into their already-made compose file that runs their application stack. It isn't ideal to have multiple compose files so that rules out Option 4 (cloning the repo or keeping the compose file in sync with upstream sounds like a nightmare). Option 3 could be nice down the road when you have the time/resources to polish the experience, but isn't necessary now imo.

I think you're already very close, you just need to "embed" some of the initialization files into the images and publish them instead of having the compose engine build the images at runtime (aka closer back to Option 1).

The ideal docker-compose would then look like this (env/volumes omitted for brevity):

version: '3.6'
services:

    # Local images
    appserver:
      build:
        context: './myserver'

    cache:
      image: redis

    proxy:
      image: treafik-or-something 

    # Supabase images
    kong:
      image: supabase/kong:dev
      ports:
        - 8000:8000/tcp
        - 8443:8443/tcp
    auth:
      image: supabase/gotrue:dev
      ports:
        - '9999:9999'
    rest:
      image: supabase/postgrest:dev
      ports:
        - '3000:3000'
    realtime:
      image: supabase/realtime:dev
      ports:
        - '4000:4000'
    postgres:
      image: supabase/postgres:dev
      ports:
        - 5432:5432

Where each image is auto-built from the github actions/releases.

Don't want to speak for other devs, so please thumbs up/down if you agree/disagree with this approach.

[Zubnix]

@kiwicopple I tried to run the docker-compose locally. It all starts up fine, however it seems the required db is not created. When connecting to postgress directly, no auth db is visible (only the internal pg one). When using the javascript example to login I get an error (because the required db doesn't exist)

auth_1      | time="2021-03-08T10:44:45Z" level=info msg="request started" component=api method=POST path=/signup referer="http://localhost:3001/" remote_addr="192.168.64.2:34854" request_id=759a6d7c-e897-41f2-a591-02383b125225
db_1        | 2021-03-08 10:44:45.018 UTC [130] ERROR:  relation "users" does not exist at character 447
db_1        | 2021-03-08 10:44:45.018 UTC [130] STATEMENT:  SELECT users.aud, users.confirmation_sent_at, users.confirmation_token, users.confirmed_at, users.created_at, users.email, users.email_change, users.email_change_sent_at, users.email_change_token, users.encrypted_password, users.id, users.instance_id, users.invited_at, users.is_super_admin, users.last_sign_in_at, users.raw_app_meta_data, users.raw_user_meta_data, users.recovery_sent_at, users.recovery_token, users.role, users.updated_at FROM users AS users WHERE instance_id = $1 and email = $2 and aud = $3 LIMIT 1
auth_1      | time="2021-03-08T10:44:45Z" level=error msg="500: Database error finding user" component=api error="error finding user: ERROR: relation \"users\" does not exist (SQLSTATE 42P01)" method=POST path=/signup referer="http://localhost:3001/" remote_addr="192.168.64.2:34854" request_id=759a6d7c-e897-41f2-a591-02383b125225
auth_1      | time="2021-03-08T10:44:45Z" level=info msg="request completed" component=api duration=971371 method=POST path=/signup referer="http://localhost:3001/" remote_addr="192.168.64.2:34854" request_id=759a6d7c-e897-41f2-a591-02383b125225 status=500

For others facing the same issue, I fixed the issue myself and opened a pull request: #842

[kiwicopple]

Thanks for the PR @Zubnix 👍

[juzhiyuan]

Hi, after some searches, I'm not sure why we use the API Gateway Kong here 🤔 could use proxy those apis to service directly?

[kiwicopple]

proxy those apis to service directly

By exposing the services on different ports? We could do that, but also we plan to utilise some of the functionality of Kong soon - caching, ACLs, extra API keys etc.

Either way, adding a gateway like Nginx is quite a nice set up because the abstraction allows for versioning and future-proofing.

[juzhiyuan]

thanks for your explanation :) and here has one Cloud-Native API Gateway, too. Welcome to take a look and have a try 😊

[1] https://github.com/apache/apisix

[ctwhome]

I am also running the local version from the Docker folder. The apiKey eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJzdXBhYmFzZSIsImlhdCI6MTYwMzk2ODgzNCwiZXhwIjoyNTUwNjUzNjM0LCJhdWQiOiIiLCJzdWIiOiIiLCJSb2xlIjoicG9zdGdyZXMifQ.kdRWxJKxqgFOlx4BZQj-GIIOEeMILqUvdHMh8ebcn8M only works when being call from an HTTPrequest like this:

const response = await fetch(
        'http://localhost:8000/rest/v1/todos?select=*',
        {
          headers: {
            apikey:
              'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJzdXBhYmFzZSIsImlhdCI6MTYwMzk2ODgzNCwiZXhwIjoyNTUwNjUzNjM0LCJhdWQiOiIiLCJzdWIiOiIiLCJSb2xlIjoicG9zdGdyZXMifQ.kdRWxJKxqgFOlx4BZQj-GIIOEeMILqUvdHMh8ebcn8M',
          },
        }
      ).then((response) => response.json())

Does it seem the supabase-js doesn't pass correctly the apiKey? am I right?
by using the supabase way I get a 401 unauthorized error, for REST and the WS

const { data, count, error } = await supabase
      .from('todos')
      .select('*', { count: 'exact' })
      .order('created_at', { ascending: true })

Not sure how to continue from here.

[silentworks]

The docker image is still a WIP and it's probably best to use the hosted platform until an announcement has come out that the docker image is ready for use. From what @kiwicopple mentioned here #219 (reply in thread) they are trying to get the hosted platform to a stable place before they start spending more time on getting the docker image ready for general use. Something I didn't realise early on is that you are not vendor locked in on the hosted version as you are just using Postgres and you have full access to it if you decide to move away from Supabase (but I doubt you would need to).

[ctwhome]

Tho I agree with what you said I hope is helpful to mention the current problems or issues that could be addressed later. In my opinion, having the solution working seamlessly locally is one of the biggest differentiation with firebase, by all means being an open source project.

It could attract so many developers that just want to try it out by running a simple docker compose. The vendor lock comes already by the awesomeness of the project idea 😁, we will pay for the cloud service the moment it comes, but to convince the rest of the team members we need a fully local instance for development. I guess that this is me just being impatient 😅

[ctwhome]

oke, I found the error. Apparently, PostgREST needs a secret key at least 32 characters long PostgREST/postgrest#977 (comment)

edit: I made a pull request to make the real-time and calls to the DB via supabase-js working locally with this config: #872

[kiwicopple]

Amazing work with the @ctwhome - tbh, I was hoping to get to this last week but Storage + Billing have been high priority tasks, and the whole team is flat out. You've done an awesome job fixing up the docker set up 🙏

[ctwhome]

thank you for your kind words, I wish I could collaborate more. I really look forward to the pricing model and the storage. Thank you for the hard work!