Teleport to MongoDB Replicaset Architecture

[kachi-app]

Background:
I deployed Teleport and MongoDB through Helm Charts. Here the config i used to deploy:
Teleport Kube Agent

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: teleport-kube-agent
spec:
  interval: 5m
  values:
    roles: app,db
    joinParams:
      method: "kubernetes"
      tokenName: "provision-token"
    databases:
      - name: "mongodb-main"
        protocol: "mongodb"
        uri: "mongodb://kachi.example.com:27017,kachi.example.com:27017/?replicaSet=rs0"
        description: "Proxy to MongoDB Main Service"
        tls:
          mode: insecure
        admin_user:
          name: "teleport-admin"
        static_labels:
          type: database

Teleport Role

apiVersion: resources.teleport.dev/v1
kind: TeleportRoleV7
metadata:
  name: db-admin
spec:
  options:
    # create_db_user_mode enables automatic user provisioning for matching databases
    create_db_user_mode: keep
  allow:
    db_labels:
      '*': '*'
    db_names:
      - "*"
    db_roles:
      - "{{internal.db_roles}}"
      - "{{external.db_roles}}"

proxyListenerMode: multiplex
chartMode: gcp
MongoDB: tls disabled

I am able to access the mongodb using mongosh "mongodb://kachi.example.com:27017", also through port forwarding using kubectl port-forward
However failed when proxying through teleport, by running

tsh db connect mongodb-main

or

tsh proxy db mongodb-main --tunnel --insecure --db-name kachi --port 27017
mongosh "mongodb://kachi.example.com:27017?serverSelectionTimeoutMS=60000"

Please note:
I have read the documentation below few times:

• https://goteleport.com/docs/enroll-resources/database-access/auto-user-provisioning/mongodb/
• https://goteleport.com/docs/connect-your-client/gui-clients/#mongodb-compass
• https://goteleport.com/docs/enroll-resources/database-access/enroll-self-hosted-databases/mongodb-self-hosted/

Expected behavior:
tsh proxy / tsh db connect should be able to connect to mongodb

Current behavior:

MongoServerSelectionError: your Teleport role requires automatic database user provisioning but an attempt to activate database user "kachi-app" failed due to the following error: server selection error: server selection timeout, current topology: { Type: ReplicaSetNoPrimary, Servers: [{ Addr: kachi.example.com:27017, Type: Unknown, Last error: EOF }, { Addr: kachi.example.com:27017, Type: Unknown, Last error: EOF }, ] }

Bug details:

• Teleport version: 17.0.2 CE ( Just updated, previously was 16.4.7 ), both version failed though
• MongoDB Version: docker.io/bitnami/mongodb:8.0.3-debian-12-r0
• Debug logs
  Teleport Auth

2024-11-26T03:06:17Z INFO  emitting audit event event_type:db.session.user.create fields:map[cluster_name:teleport.example.com code:TDB08W db_name:kachi db_origin:config-file db_protocol:mongodb db_service:mongodb-main db_type:self-hosted db_uri:mongodb://kachi.example.com:27017,kachi.example.com:27017/?replicaSet=rs0 db_user:kachi-app ei:0 error:server selection error: server selection timeout, current topology: { Type: ReplicaSetNoPrimary, Servers: [{ Addr: kachi.example.com:27017, Type: Unknown, Last error: EOF }, { Addr: kachi.example.com:27017, Type: Unknown, Last error: EOF }, ] } event:db.session.user.create message:server selection error: server selection timeout, current topology: { Type: ReplicaSetNoPrimary, Servers: [{ Addr: kachi.example.com:27017, Type: Unknown, Last error: EOF }, { Addr: kachi.example.com:27017, Type: Unknown, Last error: EOF }, ] } private_key_policy:none roles:<nil> sid:23751309-edfc-4a2b-b07f-1c371cac6a37 success:false time:2024-11-26T03:06:17.672Z trace.component:audit uid:c2d00669-69bd-4143-aa76-2e84d737a92e user:kachi-app user_kind:1 username:kachi-app] events/emitter.go:287