Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix orion environment not working #4857

Merged
merged 7 commits into from
Sep 14, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 49 additions & 19 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -92,35 +92,65 @@ BALANCE_CREDIT=300000000000
BALANCE_LOCKED=300000000000

# ====================================== ORION ======================================
# Orion db config
# Overrides `DB_*` environment variables for orion services in docker-compose.yml
# We use a single postgres service for both:
# - Subsquid archive's database (squid-archive, squid-archive-gateway) and
# - Orion's database (orion_processor, orion_graphql-server).
# ORION_DB_NAME is the name of the Orion's database, while SQUID_ARCHIVE_DB_NAME
# is the name of the Subsquid archive database.
SQUID_ARCHIVE_DB_NAME=squid_archive
ORION_DB_NAME=orion
ORION_DB_USER=postgres
ORION_DB_PASS=postgres
# LOCAL DEV ENVIRONMENT

ORION_ENV=development
DEV_DISABLE_SAME_SITE=true

# Db config
ORION_DB_NAME=squid
ORION_DB_PASS=squid
DB_ADMIN_USER=admin
DB_ADMIN_PASS=admin
ORION_DB_PORT=5433

# Orion / Subsquid archive service ports
ORION_API_PORT=4350
# Processor service prometheus port
PROCESSOR_PROMETHEUS_PORT=3337
SQUID_ARCHIVE_GATEWAY_PORT=8000
# Graphql server port
GRAPHQL_API_PORT=4350
# Auth api port
AUTH_API_PORT=4074

# Other Orion configs
APP_NAME=Gleev
SUPPORT_NO_CATEGORY_VIDEOS=true
SUPPORT_NEW_CATEGORIES=true
KILL_SWITCH_ON=false
VIDEO_VIEW_PER_IP_TIME_LIMIT=30
VIDEO_RELEVANCE_VIEWS_TICK=50
RELEVANCE_WEIGHTS="[1, 0.03, 0.3, 0.5]"
# 10 seconds
VIDEO_VIEW_PER_USER_TIME_LIMIT=10
# Operator API secret
OPERATOR_SECRET=this-is-not-so-secret-change-it
# every 50 views video relevance score will be recalculated
VIDEO_RELEVANCE_VIEWS_TICK=50
# [
# newness (negative number of days since created) weight,
# views weight,
# comments weight,
# rections weights,
# [joystream creation weight, YT creation weight]
# ]
RELEVANCE_WEIGHTS="[1, 0.03, 0.3, 0.5, [7,3]]"
MAX_CACHED_ENTITIES=1000
APP_PRIVATE_KEY=this-is-not-so-secret-change-it
TRUSTED_REVERSE_PROXIES=1
SESSION_EXPIRY_AFTER_INACTIVITY_MINUTES=60
SESSION_MAX_DURATION_HOURS=720
EMAIL_CONFIRMATION_ROUTE=http://localhost:4074/api/v1/confirm-email?token={token}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obviously this will only work when opening email from the machine. But I gather we would most likely disable email confirmation and notifications in a playground setup so thats fine.

EMAIL_CONFIRMATION_TOKEN_EXPIRY_TIME_HOURS=24
EMAIL_CONFIRMATION_TOKEN_RATE_LIMIT=5
ACCOUNT_OWNERSHIP_PROOF_EXPIRY_TIME_SECONDS=300 # 5 minutes
COOKIE_SECRET=this-is-not-so-secret-change-it

TRUST_PROXY=uniquelocal

# Sendgrid API
SENDGRID_API_KEY=
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an identical SENDGRID_API_KEY variable below for the faucet.
perhaps we can prefix the one for orion with ORION_SENDGRID_API_KEY then pass that env variable in docker-compose environment section (so you don't have to modify orion code to expect a different variable name)

[email protected]

# Debug settings
SQD_DEBUG=api:*
OPENAPI_PLAYGROUND=true

ARCHIVE_GATEWAY_URL=${CUSTOM_ARCHIVE_GATEWAY_URL:-https://archive.joystream.org/graphql}

# =====================================================================================

## Telemetry
Expand Down
115 changes: 45 additions & 70 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -170,23 +170,6 @@ services:
POSTGRES_DB: ${INDEXER_DB_NAME}
PGPORT: ${DB_PORT}

orion-db:
image: postgres:14
container_name: orion-db
restart: unless-stopped
ports:
- '127.0.0.1:${ORION_DB_PORT}:${ORION_DB_PORT}'
volumes:
- orion-db-data:/var/lib/postgresql/data
env_file:
# relative to working directory where docker-compose was run from
- .env
environment:
POSTGRES_USER: ${ORION_DB_USER}
POSTGRES_PASSWORD: ${ORION_DB_PASS}
POSTGRES_DB: ${ORION_DB_NAME}
PGPORT: ${ORION_DB_PORT}

graphql-server:
image: node:18
container_name: graphql-server
Expand Down Expand Up @@ -318,92 +301,84 @@ services:
ports:
- "3002:3002"

squid-archive:
container_name: squid-archive
mnaamani marked this conversation as resolved.
Show resolved Hide resolved
image: subsquid/substrate-ingest:firesquid
depends_on:
- orion-db
# PostgerSQL database for Orion
orion-db:
image: postgres:14
container_name: orion-db
restart: unless-stopped
env_file:
- .env
environment:
- DB_NAME=${SUBSQUID_ARCHIVE_DB_NAME}
- DB_USER=${ORION_DB_USER}
- DB_PASS=${ORION_DB_PASS}
- DB_HOST=orion-db
- DB_PORT=${ORION_DB_PORT}
command: [
'-e',
'${JOYSTREAM_NODE_WS}',
'-c',
'20',
'--out',
'postgres://postgres:postgres@orion-db:${ORION_DB_PORT}/${SUBSQUID_ARCHIVE_DB_NAME}',
]
POSTGRES_PASSWORD: ${ORION_DB_PASS}
POSTGRES_DB: ${ORION_DB_NAME}
ports:
- '127.0.0.1:${ORION_DB_PORT}:${ORION_DB_PORT}'
command: ['postgres', '-c', 'config_file=/etc/postgresql/postgresql.conf', '-p', '${ORION_DB_PORT}']
# command: ['postgres', '-p', '${ORION_DB_PORT}']
shm_size: 1g
volumes:
- orion-db-data:/var/lib/postgresql/data
- ./postgres.conf:/etc/postgresql/postgresql.conf

squid-archive-gateway:
container_name: squid-archive-gateway
image: subsquid/substrate-gateway:firesquid
# Orion processor
orion-processor:
image: joystream/orion:latest
container_name: orion-processor
restart: unless-stopped
depends_on:
- orion-db
restart: unless-stopped
env_file:
- .env
environment:
- DB_NAME=${SUBSQUID_ARCHIVE_DB_NAME}
- DB_USER=${ORION_DB_USER}
- DB_NAME=${ORION_DB_NAME}
- DB_PASS=${ORION_DB_PASS}
- DB_HOST=orion-db
- DB_PORT=${ORION_DB_PORT}
- DATABASE_MAX_CONNECTIONS=5
- RUST_LOG=actix_web=info,actix_server=info
command: [
'--database-url',
'postgres://postgres:postgres@orion-db:${ORION_DB_PORT}/${SUBSQUID_ARCHIVE_DB_NAME}',
]
- DB_HOST=orion-db
- GQL_PORT=${GRAPHQL_API_PORT}
volumes:
- ./db/export:/squid/db/export
command: >
sh -c "npx squid-typeorm-migration apply && \
node lib/processor.js"
ports:
- '127.0.0.1:${SQUID_ARCHIVE_GATEWAY_PORT}:8000'
- '127.0.0.1:${PROCESSOR_PROMETHEUS_PORT}:${PROCESSOR_PROMETHEUS_PORT}'

orion-processor:
container_name: orion-processor
# Orion GraphQL API
orion-graphql-api:
container_name: orion-graphql-api
image: joystream/orion:latest
restart: unless-stopped
env_file:
- .env
depends_on:
- orion-db
env_file:
- .env
environment:
- DB_NAME=${ORION_DB_NAME}
- DB_USER=${ORION_DB_USER}
- DB_PASS=${ORION_DB_PASS}
- DB_HOST=orion-db
- DB_PORT=${ORION_DB_PORT}
- GQL_PORT=${ORION_API_PORT}
- ARCHIVE_GATEWAY_URL=http://squid-archive-gateway:8000/graphql
command: >
sh -c "npx squid-typeorm-migration apply && \
node lib/processor.js"
- DB_HOST=orion-db
- GQL_PORT=${GRAPHQL_API_PORT}
command: ['npm', 'run', 'query-node-start']
ports:
- '127.0.0.1:${PROCESSOR_PROMETHEUS_PORT}:${PROCESSOR_PROMETHEUS_PORT}'
- '127.0.0.1:${GRAPHQL_API_PORT}:${GRAPHQL_API_PORT}'

orion-api:
container_name: orion-api
# Orion Authentication API
orion-auth-api:
container_name: orion-auth-api
image: joystream/orion:latest
restart: unless-stopped
env_file:
- .env
depends_on:
- orion-db
env_file:
- .env
environment:
- DB_NAME=${ORION_DB_NAME}
- DB_USER=${ORION_DB_USER}
- DB_PASS=${ORION_DB_PASS}
- DB_HOST=orion-db
- DB_PORT=${ORION_DB_PORT}
- GQL_PORT=${ORION_API_PORT}
command: ["npm", "run", "query-node-start"]
- DB_HOST=orion-db
command: ['npm', 'run', 'auth-server-start']
ports:
- '${ORION_API_PORT}:${ORION_API_PORT}'
- '127.0.0.1:${AUTH_API_PORT}:${AUTH_API_PORT}'

volumes:
chain-data:
Expand Down
4 changes: 4 additions & 0 deletions postgres.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
listen_addresses = '*'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we rename this file to orion-postgres.conf, there is another PR where I was planning to add a postgres.conf file for the query-node db. It makes sense to have two different conf files for each db instance.

log_statement = all
autovacuum_analyze_scale_factor = 0.01
shared_buffers=1GB
5 changes: 2 additions & 3 deletions start-orion.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
set -e

docker-compose up -d orion-db
docker-compose up -d squid-archive
docker-compose up -d squid-archive-gateway
docker-compose up -d orion-processor
docker-compose up -d orion-api
docker-compose up -d orion-graphql-api
docker-compose up -d orion-auth-api
Loading