supabase self-hosted: container supabase-analytics is unhealthy

[uxbxn]

Hi there,

I tried to self-host supabase on my server with docker and used their tutorial (https://supabase.com/docs/guides/self-hosting/docker). Everything worked perfectly fine until I adjusted the .env file. As described in their tutorial I adjusted

• JWT
• service key
• anon key
• postgres password
• dashboard_username
• dashboard_password

But after that I always get "container supabase-analytics is unhealthy" as a response to "docker compose up". If I use the standard .env file with the default values I can run and access my supabase environment. Can anyone help me out?

[RFlintstone]

Hi there,

I tried to self-host supabase on my server with docker and used their tutorial (https://supabase.com/docs/guides/self-hosting/docker). Everything worked perfectly fine until I adjusted the .env file. As described in their tutorial I adjusted

• JWT
• service key
• anon key
• postgres password
• dashboard_username
• dashboard_password

But after that I always get "container supabase-analytics is unhealthy" as a response to "docker compose up". If I use the standard .env file with the default values I can run and access my supabase environment. Can anyone help me out?

I solved it by first uninstalling Supabase and then executing docker compose up -d

Keep in mind that uninstalling will remove existing volumes

[tim13246879]

I think this only works because uninstalling wipes the changes in the .env file and resets them to default.

[RFlintstone]

I think this only works because uninstalling wipes the changes in the .env file and resets them to default.

That's very likely, although (at least at time of posting) it was an ok temporary solution. I left supabase alone in the hope of some promising patch notes.

[tim13246879]

Had the same problem self-hosting. After testing some things out, changing the postgres password is what causes the issue. Changing other values in the .env does NOT affect supabase analytics.

I have not yet found the root cause of this problem or how to get around it. Would be great if anyone has insights into this.

[Coddo-Python]

Had the same problem self-hosting. After testing some things out, changing the postgres password is what causes the issue. Changing other values in the .env does NOT affect supabase analytics.

I have not yet found the root cause of this problem or how to get around it. Would be great if anyone has insights into this.

What worked for me was to run this set of commands: #11957 (comment)

WARNING: This is a DESTRUCTIVE command and it WILL wipe your volume data. This should be your last resort, please take backups of your data :)

docker system prune -a
sudo rm -rf docker/volumes/db/data
docker compose up

[tim13246879]

That worked for me too, thanks!

[x-tropy]

A complete wipeout: destructive but useful!

[kerrickchan]

it will prune all unused container and volume too

[gstechno]

Changing the password for Postgres user (by logging with default password for supaabse_admin) resolves the error.

[Ziinc]

temporary fix for this bug is here
#22605 (comment)

[femans]

I have the same problem and suggestes fixes do not work. I have been trying to fix this for hours now and I don't progress.
Supabase-analytics says there is something wrong with the database. The output of supabase-db container:

PostgreSQL Database directory appears to contain a database; Skipping initialization

and the output of supabase-analytics:

13:37:46.666 [error] Could not create schema migrations table. This error usually happens due to the following:

  * The database does not exist
  * The "schema_migrations" table, which Ecto uses for managing
    migrations, was defined by another library
  * There is a deadlock while migrating (such as using concurrent
    indexes with a migration_lock)

To fix the first issue, run "mix ecto.create" for the desired MIX_ENV.

To address the second, you can run "mix ecto.drop" followed by
"mix ecto.create", both for the desired MIX_ENV. Alternatively you may
configure Ecto to use another table and/or repository for managing
migrations:

    config :logflare, Logflare.Repo,
      migration_source: "some_other_table_for_schema_migrations",
      migration_repo: AnotherRepoForSchemaMigrations

The full error report is shown below.

** (Postgrex.Error) ERROR 3F000 (invalid_schema_name) no schema has been selected to create in
    (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:913: Ecto.Adapters.SQL.raise_sql_call_error/1
    (elixir 1.14.4) lib/enum.ex:1658: Enum."-map/2-lists^map/1-0-"/2
    (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:1005: Ecto.Adapters.SQL.execute_ddl/4
    (ecto_sql 3.10.1) lib/ecto/migrator.ex:738: Ecto.Migrator.verbose_schema_migration/3
    (ecto_sql 3.10.1) lib/ecto/migrator.ex:552: Ecto.Migrator.lock_for_migrations/4
    (ecto_sql 3.10.1) lib/ecto/migrator.ex:428: Ecto.Migrator.run/4
    (ecto_sql 3.10.1) lib/ecto/migrator.ex:170: Ecto.Migrator.with_repo/3
    nofile:1: (file)

13:37:49.717 [notice]     :alarm_handler: {:set, {:system_memory_high_watermark, []}}

13:37:49.831 [info] Elixir.Logflare.SigtermHandler is being initialized...

13:37:49.913 [info] Table counters started!

13:37:49.913 [info] Rate counter table started!

13:37:49.916 [info] Going Down - {%Postgrex.Error{message: nil, postgres: %{code: :undefined_table, file: "parse_relation.c", line: "1392", message: "relation \"sources\" does not exist", pg_code: "42P01", position: "480", routine: "parserOpenTable", severity: "ERROR", unknown: "ERROR"}, connection_id: 2576, query: "SELECT s0.\"id\", s0.\"name\", s0.\"token\", s0.\"public_token\", s0.\"favorite\", s0.\"bigquery_table_ttl\", s0.\"api_quota\", s0.\"webhook_notification_url\", s0.\"slack_hook_url\", s0.\"bq_table_partition_type\", s0.\"custom_event_message_keys\", s0.\"log_events_updated_at\", s0.\"notifications_every\", s0.\"lock_schema\", s0.\"validate_schema\", s0.\"drop_lql_filters\", s0.\"drop_lql_string\", s0.\"v2_pipeline\", s0.\"suggested_keys\", s0.\"user_id\", s0.\"notifications\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"sources\" AS s0 WHERE (s0.\"log_events_updated_at\" > $1) ORDER BY s0.\"log_events_updated_at\""}, [{Ecto.Adapters.SQL, :raise_sql_call_error, 1, [file: 'lib/ecto/adapters/sql.ex', line: 913, error_info: %{module: Exception}]}, {Ecto.Adapters.SQL, :execute, 6, [file: 'lib/ecto/adapters/sql.ex', line: 828]}, {Ecto.Repo.Queryable, :execute, 4, [file: 'lib/ecto/repo/queryable.ex', line: 229]}, {Ecto.Repo.Queryable, :all, 3, [file: 'lib/ecto/repo/queryable.ex', line: 19]}, {Logflare.Source.Supervisor, :handle_continue, 2, [file: 'lib/logflare/source/supervisor.ex', line: 52]}, {:gen_server, :try_dispatch, 4, [file: 'gen_server.erl', line: 1123]}, {:gen_server, :loop, 7, [file: 'gen_server.erl', line: 865]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]} - Elixir.Logflare.Source.Supervisor

13:37:49.917 [error] GenServer Logflare.Source.Supervisor terminating
** (Postgrex.Error) ERROR 42P01 (undefined_table) relation "sources" does not exist

    query: SELECT s0."id", s0."name", s0."token", s0."public_token", s0."favorite", s0."bigquery_table_ttl", s0."api_quota", s0."webhook_notification_url", s0."slack_hook_url", s0."bq_table_partition_type", s0."custom_event_message_keys", s0."log_events_updated_at", s0."notifications_every", s0."lock_schema", s0."validate_schema", s0."drop_lql_filters", s0."drop_lql_string", s0."v2_pipeline", s0."suggested_keys", s0."user_id", s0."notifications", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."log_events_updated_at" > $1) ORDER BY s0."log_events_updated_at"
    (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:913: Ecto.Adapters.SQL.raise_sql_call_error/1
    (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:828: Ecto.Adapters.SQL.execute/6
    (ecto 3.10.3) lib/ecto/repo/queryable.ex:229: Ecto.Repo.Queryable.execute/4
    (ecto 3.10.3) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
    (logflare 1.4.0) lib/logflare/source/supervisor.ex:52: Logflare.Source.Supervisor.handle_continue/2
    (stdlib 4.3.1) gen_server.erl:1123: :gen_server.try_dispatch/4
    (stdlib 4.3.1) gen_server.erl:865: :gen_server.loop/7
    (stdlib 4.3.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: {:continue, :boot}

13:37:49.920 [info] Running LogflareWeb.Endpoint with cowboy 2.10.0 at 0.0.0.0:4000 (http)

13:37:49.923 [info] Access LogflareWeb.Endpoint at http://localhost:4000

13:37:49.924 [info] Running LogflareGrpc.Endpoint with Cowboy using http://0.0.0.0:50051

13:37:49.933 [notice] Application logflare exited: Logflare.Application.start(:normal, []) returned an error: shutdown: failed to start child: Logflare.SystemMetricsSup
    ** (EXIT) shutdown: failed to start child: Logflare.SystemMetrics.AllLogsLogged
        ** (EXIT) an exception was raised:
            ** (Postgrex.Error) ERROR 42P01 (undefined_table) relation "system_metrics" does not exist

    query: SELECT s0."id", s0."all_logs_logged", s0."node", s0."inserted_at", s0."updated_at" FROM "system_metrics" AS s0 WHERE (s0."node" = $1)
                (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:913: Ecto.Adapters.SQL.raise_sql_call_error/1
                (ecto_sql 3.10.1) lib/ecto/adapters/sql.ex:828: Ecto.Adapters.SQL.execute/6
                (ecto 3.10.3) lib/ecto/repo/queryable.ex:229: Ecto.Repo.Queryable.execute/4
                (ecto 3.10.3) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
                (ecto 3.10.3) lib/ecto/repo/queryable.ex:151: Ecto.Repo.Queryable.one/3
                (logflare 1.4.0) lib/logflare/system_metrics/all_logs_logged/all_logs_logged.ex:20: Logflare.SystemMetrics.AllLogsLogged.init/1
                (stdlib 4.3.1) gen_server.erl:851: :gen_server.init_it/2
                (stdlib 4.3.1) gen_server.erl:814: :gen_server.init_it/6

13:37:49.971 [notice]     :alarm_handler: {:clear, :system_memory_high_watermark}
{"Kernel pid terminated",application_controller,"{application_start_failure,logflare,{{shutdown,{failed_to_start_child,'Elixir.Logflare.SystemMetricsSup',{shutdown,{failed_to_start_child,'Elixir.Logflare.SystemMetrics.AllLogsLogged',{#{'__exception__' => true,'__struct__' => 'Elixir.Postgrex.Error',connection_id => 2580,message => nil,postgres => #{code => undefined_table,file => <<\"parse_relation.c\">>,line => <<\"1392\">>,message => <<\"relation \\"system_metrics\\" does not exist\">>,pg_code => <<\"42P01\">>,position => <<\"89\">>,routine => <<\"parserOpenTable\">>,severity => <<\"ERROR\">>,unknown => <<\"ERROR\">>},query => <<\"SELECT s0.\\"id\\", s0.\\"all_logs_logged\\", s0.\\"node\\", s0.\\"inserted_at\\", s0.\\"updated_at\\" FROM \\"system_metrics\\" AS s0 WHERE (s0.\\"node\\" = $1)\">>},[{'Elixir.Ecto.Adapters.SQL',raise_sql_call_error,1,[{file,\"lib/ecto/adapters/sql.ex\"},{line,913},{error_info,#{module => 'Elixir.Exception'}}]},{'Elixir.Ecto.Adapters.SQL',execute,6,[{file,\"lib/ecto/adapters/sql.ex\"},{line,828}]},{'Elixir.Ecto.Repo.Queryable',execute,4,[{file,\"lib/ecto/repo/queryable.ex\"},{line,229}]},{'Elixir.Ecto.Repo.Queryable',all,3,[{file,\"lib/ecto/repo/queryable.ex\"},{line,19}]},{'Elixir.Ecto.Repo.Queryable',one,3,[{file,\"lib/ecto/repo/queryable.ex\"},{line,151}]},{'Elixir.Logflare.SystemMetrics.AllLogsLogged',init,1,[{file,\"lib/logflare/system_metrics/all_logs_logged/all_logs_logged.ex\"},{line,20}]},{gen_server,init_it,2,[{file,\"gen_server.erl\"},{line,851}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,814}]}]}}}}},{'Elixir.Logflare.Application',start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,logflare,{{shutdown,{failed_to_start_child,'Elixir.Logflare.SystemMetricsSup',{shutdown,{failed_to_start_child,'Elixir.Logflare.SystemMetrics.AllLogsLogged',{#{'__exception__' => true,'__struct__' => 'Elixir.Postgrex.Error',connection_id => 2580,message => nil,postgres => #{code => undefined_table,file => <<"parse_relation.c">>,line => <<"1392">>,message => <<"relation \"system_metrics\" does not exist">>,pg_code => <<"42P01">>,position => <<"89">>,routine => <<"parserOpenTable">>,severity => <<"ERROR">>,unknown => <<"ERROR">>},query => <<"SELECT s0.\"id\", s0.\"all_logs_logged\", s0.\"node\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"system_metrics\" AS s0 WHERE (s0.\"node\" = $1)">>},[{'Elixir.Ecto.Adapters.SQL',raise_sql_call_error,1,[{file,"lib/ecto/adapters/sql.ex"},{line,913},{error_info,#{module => 'Elixir.Exception'}}]},{'Elixir.Ecto.Adapters.SQL',execute,6,[{file,"lib/ecto/adapters/sql.ex"},{line,828}]},{'Elixir.Ecto.Repo.

Crash dump is being written to: erl_crash.dump...done

[qxygene1]

is there any way to change password for POSTGRES and NOT make container supabase-analytics is unhealthy?

[Ziinc]

Please try to use latest logflare version v1.10.0 which contains fixes for this

[qxygene1]

Container supabase-analytics : healthy but;

Failed to retrieve tables
Error: API error happened while trying to communicate with the server.

Try refreshing your browser, but if the issue persists, please reach out to us via support.

[elisc88]

Honestly, I did the pinned solution, and I also did @Coddo-Python's complete wipeout version. Neither worked for me.

My work around was literally uninstalling everything - not ideal if you already have data and what not, but I got rid of everything. Redownloaded the Docker based CLI. Got to cp .env.example .env and edited my .env to the passwords I wanted before I performed docker compose pull. Once I started it up after composing in docker, everything was perfectly fine.

You should edit the Self-Hosted instructions to show that you must edit the fake .env vars in your .env file before you pull and start the services.

[plastic-shurikens]

Make sure to run docker compose up without -d to see what's cusing the error, could be a memory or storage allocation issue