Looking for guide to self-host/deploy Dify on GCP.

[ajaman190]

Hi everyone,

I am looking to deploy Dify on Google Cloud Platform (GCP) for use by my team. After some research, I found that we can deploy Dify services on GCP as follows:

1. Cloud Run or GKE: For services such as api, worker, web, sandbox, ssrfproxy, and nginx (or alternatively, using GCP load balancer).
2. Cloud SQL: For PostgreSQL database and pgvector.
3. Memorystore: For Redis.

I attempted to deploy Dify services on Cloud Run using Terraform taking reference from Docker Compose but encountered issues that prevented a successful deployment.

Could anyone provide a detailed guide or best practices for correctly deploying Dify on GCP? Any help would be greatly appreciated.

Thanks in advance!

[dosubot[bot]]

Hey @ajaman190! 👋 I'm here to help you out with your Dify deployment on GCP. I'm a bot that can assist with bug fixes, answering questions, and becoming a contributor.

To deploy Dify on Google Cloud Platform (GCP), you can follow these steps and best practices:

Prerequisites

1. GCP Project: Ensure you have a GCP project set up.
2. Cloud SQL: Set up a PostgreSQL instance.
3. Memorystore: Set up a Redis instance.
4. Docker: Ensure Docker is installed on your local machine.

Step-by-Step Guide

1. Start Middleware Services

Start the required middleware services (PostgreSQL, Redis, and Weaviate) using Docker Compose:

cd docker
cp middleware.env.example middleware.env
docker compose -f docker-compose.middleware.yaml up -d

2. Set Up Python Environment

Create a Python 3.10 environment (recommended to use Anaconda):

conda create --name dify python=3.10
conda activate dify

3. Configure API Service

Navigate to the "api" directory and set up the environment:

cd api
cp .env.example .env
openssl rand -base64 42 | sed -i 's/SECRET_KEY=.*/SECRET_KEY=<your_value>/' .env
pip install -r requirements.txt
flask db upgrade
flask run --host 0.0.0.0 --port=5001 --debug

4. Configure Worker Service

Start the worker service for local async processing if needed:

celery -A app.celery worker -P gevent -c 1 -Q dataset,generation,mail --loglevel INFO

5. Deploy Web Service

Navigate to the "web" directory, install dependencies, and start the service:

cd web
npm install
cp .env.example .env.local
# Modify .env.local as needed
npm run build
npm run start

Cloud Run Deployment

Environment Variables

Set the following environment variables for Cloud SQL and Memorystore:

Database Configuration (Cloud SQL)

DB_USERNAME=<your-db-username>
DB_PASSWORD=<your-db-password>
DB_HOST=<your-cloud-sql-instance-ip>
DB_PORT=5432
DB_DATABASE=<your-database-name>
SQLALCHEMY_POOL_SIZE=30
SQLALCHEMY_POOL_RECYCLE=3600
SQLALCHEMY_ECHO=false

Redis Configuration (Memorystore)

REDIS_HOST=<your-memorystore-instance-ip>
REDIS_PORT=6379
REDIS_DB=0
REDIS_PASSWORD=<your-redis-password>
REDIS_USE_SSL=false

Celery Configuration

CELERY_BROKER_URL=redis://:<redis_password>@<redis_host>:<redis_port>/1
BROKER_USE_SSL=false

Enable Migrations

MIGRATION_ENABLED='true'

Best Practices for Scaling

Cloud Run

Configure autoscaling based on request count or CPU utilization for each service. Example for the api service:

api:
  image: langgenius/dify-api:0.6.12
  concurrency: 80
  autoscaling:
    max-instances: 10
  environment:
    SERVER_WORKER_AMOUNT: 4

GKE

Use Horizontal Pod Autoscaler (HPA) based on CPU or memory utilization. Example for the api service:

api:
  image: langgenius/dify-api:0.6.12
  env:
    - name: SERVER_WORKER_AMOUNT
      value: "4"
  resources:
    requests:
      cpu: "500m"
      memory: "512Mi"
    limits:
      cpu: "1"
      memory: "1Gi"
  hpa:
    minReplicas: 1
    maxReplicas: 10
    metrics:
      - type: Resource
        resource:
          name: cpu
          target:
            type: Utilization
            averageUtilization: 80

These steps and configurations should help you deploy Dify on GCP effectively. For more detailed information, you can refer to the Dify documentation and the Docker Compose setup ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.

[ajaman190]

Hey @ajaman190! 👋 I'm here to help you out with your Dify deployment on GCP. I'm a bot that can assist with bug fixes, answering questions, and becoming a contributor.

To deploy Dify on Google Cloud Platform (GCP), you can follow these steps and best practices:

Prerequisites

1. GCP Project: Ensure you have a GCP project set up.
2. Cloud SQL: Set up a PostgreSQL instance.
3. Memorystore: Set up a Redis instance.
4. Docker: Ensure Docker is installed on your local machine.

Step-by-Step Guide

1. Start Middleware Services

Start the required middleware services (PostgreSQL, Redis, and Weaviate) using Docker Compose:

cd docker
cp middleware.env.example middleware.env
docker compose -f docker-compose.middleware.yaml up -d

2. Set Up Python Environment

Create a Python 3.10 environment (recommended to use Anaconda):

conda create --name dify python=3.10
conda activate dify

3. Configure API Service

Navigate to the "api" directory and set up the environment:

cd api
cp .env.example .env
openssl rand -base64 42 | sed -i 's/SECRET_KEY=.*/SECRET_KEY=<your_value>/' .env
pip install -r requirements.txt
flask db upgrade
flask run --host 0.0.0.0 --port=5001 --debug

4. Configure Worker Service

Start the worker service for local async processing if needed:

celery -A app.celery worker -P gevent -c 1 -Q dataset,generation,mail --loglevel INFO

5. Deploy Web Service

Navigate to the "web" directory, install dependencies, and start the service:

cd web
npm install
cp .env.example .env.local
# Modify .env.local as needed
npm run build
npm run start

Cloud Run Deployment

Environment Variables

Set the following environment variables for Cloud SQL and Memorystore:

Database Configuration (Cloud SQL)

DB_USERNAME=<your-db-username>
DB_PASSWORD=<your-db-password>
DB_HOST=<your-cloud-sql-instance-ip>
DB_PORT=5432
DB_DATABASE=<your-database-name>
SQLALCHEMY_POOL_SIZE=30
SQLALCHEMY_POOL_RECYCLE=3600
SQLALCHEMY_ECHO=false

Redis Configuration (Memorystore)

REDIS_HOST=<your-memorystore-instance-ip>
REDIS_PORT=6379
REDIS_DB=0
REDIS_PASSWORD=<your-redis-password>
REDIS_USE_SSL=false

Celery Configuration

CELERY_BROKER_URL=redis://:<redis_password>@<redis_host>:<redis_port>/1
BROKER_USE_SSL=false

Enable Migrations

MIGRATION_ENABLED='true'

Best Practices for Scaling

Cloud Run

Configure autoscaling based on request count or CPU utilization for each service. Example for the api service:

api:
  image: langgenius/dify-api:0.6.12
  concurrency: 80
  autoscaling:
    max-instances: 10
  environment:
    SERVER_WORKER_AMOUNT: 4

GKE

Use Horizontal Pod Autoscaler (HPA) based on CPU or memory utilization. Example for the api service:

api:
  image: langgenius/dify-api:0.6.12
  env:
    - name: SERVER_WORKER_AMOUNT
      value: "4"
  resources:
    requests:
      cpu: "500m"
      memory: "512Mi"
    limits:
      cpu: "1"
      memory: "1Gi"
  hpa:
    minReplicas: 1
    maxReplicas: 10
    metrics:
      - type: Resource
        resource:
          name: cpu
          target:
            type: Utilization
            averageUtilization: 80

These steps and configurations should help you deploy Dify on GCP effectively. For more detailed information, you can refer to the Dify documentation and the Docker Compose setup [1][2][3][4].

To continue talking to Dosu, mention @dosu.

I want to deploy using terraform.

[dosubot[bot]]

To deploy Dify on Google Cloud Platform (GCP) using Terraform, you can follow these steps to set up services on Cloud Run or GKE, Cloud SQL for PostgreSQL with pgvector, and Memorystore for Redis. Below is a Terraform configuration example that references the Docker Compose setup provided:

1. Cloud SQL for PostgreSQL with pgvector

resource "google_sql_database_instance" "postgres" {
  name             = "dify-postgres"
  database_version = "POSTGRES_15"
  region           = "us-central1"

  settings {
    tier = "db-f1-micro"
  }
}

resource "google_sql_database" "dify_db" {
  name     = "dify"
  instance = google_sql_database_instance.postgres.name
}

resource "google_sql_user" "postgres_user" {
  name     = "postgres"
  instance = google_sql_database_instance.postgres.name
  password = "difyai123456"
}

2. Memorystore for Redis

resource "google_redis_instance" "dify_redis" {
  name           = "dify-redis"
  tier           = "STANDARD_HA"
  memory_size_gb = 1
  region         = "us-central1"
}

3. Cloud Run or GKE for Services

For Cloud Run:

resource "google_cloud_run_service" "dify_service" {
  name     = "dify-service"
  location = "us-central1"

  template {
    spec {
      containers {
        image = "gcr.io/your-project-id/dify-service:latest"
        env {
          name  = "PGUSER"
          value = "postgres"
        }
        env {
          name  = "POSTGRES_PASSWORD"
          value = "difyai123456"
        }
        env {
          name  = "POSTGRES_DB"
          value = "dify"
        }
        env {
          name  = "REDIS_HOST"
          value = google_redis_instance.dify_redis.host
        }
      }
    }
  }
}

For GKE:

resource "google_container_cluster" "primary" {
  name     = "dify-cluster"
  location = "us-central1"

  node_config {
    machine_type = "e2-medium"
  }

  initial_node_count = 1
}

resource "kubernetes_deployment" "dify_deployment" {
  metadata {
    name      = "dify-deployment"
    namespace = "default"
  }

  spec {
    replicas = 1

    selector {
      match_labels = {
        app = "dify"
      }
    }

    template {
      metadata {
        labels = {
          app = "dify"
        }
      }

      spec {
        container {
          image = "gcr.io/your-project-id/dify-service:latest"
          name  = "dify"

          env {
            name  = "PGUSER"
            value = "postgres"
          }
          env {
            name  = "POSTGRES_PASSWORD"
            value = "difyai123456"
          }
          env {
            name  = "POSTGRES_DB"
            value = "dify"
          }
          env {
            name  = "REDIS_HOST"
            value = google_redis_instance.dify_redis.host
          }
        }
      }
    }
  }
}

This Terraform configuration sets up the necessary infrastructure on GCP, including Cloud SQL for PostgreSQL with pgvector, Memorystore for Redis, and either Cloud Run or GKE for running your services. Adjust the configurations as needed for your specific requirements. The provided Docker Compose setup can be used as a reference for environment variables and service configurations, such as POSTGRES_PASSWORD, POSTGRES_DB, and Redis configurations ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[screwyforcepush]

@ajaman190 did you figure this one out? I would like to do the same

[n-guzman]

I was able to do it using this repo: https://github.com/DeNA/dify-google-cloud-terraform.