dapr Actor runtime issue in Kubernetes

[tvikure]

Ask your question here

I am trying to run a nodejs express app to create actors on demand in kubernetes. I have below services running in dapr-system namespace.

dapr-operator-559ff985fd-7s4wj
dapr-placement-server-0
dapr-scheduler-server-0
dapr-sentry-7f6966fd59-mg8jt
dapr-sidecar-injector-65fd7d94c5-qks5q

in the default namespace

my-node-app and redis is running.

When trying to create actors on demand facing the below error. Please suggest. Trying to fix this from 48 hours. this is only issue when running kubernetes cluster. When I try to run in local using dapr run it is working fine.

using below versions.

"@dapr/dapr": "^3.3.1",
"dapr-client": "^3.3.1",
"express": "^4.19.2"

Daprd failure (attached detailed log file)

time="2024-09-29T12:01:17.372807304Z" level=warning msg="Error processing operation DaprBuiltInActorNotFoundRetries. Retrying in 1s…" app_id=my-node-app instance=my-node-app-7b84959966-m4whv scope=dapr.runtime type=log ver=1.14.4

my-node-app logs

node dist/index.js
server created v6
Running API server on 3000
registered actors [ 'MyActor' ]
2024-09-29T12:00:32.803Z INFO [HTTPServer, Actors] Initializing Actors
Starting Dapr server
2024-09-29T12:00:32.805Z INFO [HTTPServer, HTTPServer] Listening on 50004
2024-09-29T12:00:32.805Z INFO [HTTPServer, HTTPServer] Registering 0 PubSub Subscriptions
2024-09-29T12:00:32.810Z INFO [HTTPClient, HTTPClient] Sidecar Started
Server started successfully
builder
actor [class MyActor extends AbstractActor]
2024-09-29T12:01:17.371Z INFO [HTTPClient, HTTPClient] Sidecar Started
error creating actors Error: {"error":"Internal Server Error","error_msg":"{"errorCode":"ERR_ACTOR_INVOKE_METHOD","message":"error invoke actor method: did not find address for actor MyActor/asset-0"}","status":500}
at HTTPClient.execute (/usr/src/app/node_modules/@dapr/dapr/implementation/Client/HTTPClient/HTTPClient.js:172:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async ActorClientHTTP.invoke (/usr/src/app/node_modules/@dapr/dapr/actors/client/ActorClient/ActorClientHTTP.js:21:24)
at async Proxy. (/usr/src/app/node_modules/@dapr/dapr/actors/client/ActorProxyBuilder.js:38:33)

import express from 'express';
import { ActorProxyBuilder, DaprServer, DaprClient, ActorId } from "@dapr/dapr";
import { MyActor, IMyActor } from './actors/MyActor';

const app = express();
app.use(express.json());

const daprHost = "127.0.0.1";
const daprPort = "3500";
const serverHost = "127.0.0.1";
const serverPort = "50004";

const server = new DaprServer({
  serverHost,
  serverPort,
  clientOptions: {
  daprHost: daprHost,
  daprPort: daprPort,
  actor: {
    actorIdleTimeout: "1h",
    actorScanInterval: "30s",
    drainRebalancedActors: true,
    reentrancy: {
      enabled: true,
      maxStackDepth: 32,
    },
    remindersStoragePartitions: 0,
  },
}});

console.log('server created');

const client = new DaprClient({ daprHost, daprPort });

async function start() { 
server.actor.registerActor(MyActor)
await new Promise(resolve => setTimeout(resolve, 5000));
const actors = await server.actor.getRegisteredActors();
console.log('registered actors ', actors);
await server.actor.init(); 

console.log("Starting Dapr server ");
await server.start();
console.log("Server started successfully");
console.log("builder  ");
}

app.post('/api/createActor', async (req, res) => {
  const { actorType } = req.body;
  const builder = new ActorProxyBuilder<IMyActor>(MyActor, client);
  const actor = builder.build(new ActorId(`asset-1`));
  console.log('actor', actor);

  await actor.start() 
});`


app.listen(3000, () => {
  console.log('Running API server on 3000');
});


start().catch(e => console.error(e));

```

Deployment yaml

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-node-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-node-app
  template:
    metadata:
      labels:
        app: my-node-app
      annotations:
        dapr.io/enabled: "true"
        dapr.io/app-id: "my-node-app"
        dapr.io/app-port: "3000"
        dapr.io/app-protocol: "http"
        dapr.io/actor-port: "50004"
    spec:
      imagePullSecrets:
        - name: myregistrykey
      containers:
        - name: my-node-app
          image: mydockerrepo/my-node-app:v1.11
          ports:
            - containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
  name: my-node-app 
spec:
  selector:
    app: my-node-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000
  type: LoadBalancer
```
  

[daprd_log.txt](https://github.com/user-attachments/files/17179948/daprd_log.txt)

[jeremylcarter]

Usually that error indicates that the actor has not been registered correctly.

Are you sure the ordering is correct?

Should you first await server.actor.init(); then call server.actor.registerActor(MyActor) afterwards?
You log out console.log('server created'); but it hasn't been started yet, its just an object at that point.

The documentation has it this way:

import { DaprServer } from "@dapr/dapr";
import ParkingSensorImpl from "./ParkingSensorImpl";

const daprHost = "127.0.0.1";
const daprPort = "50000";
const serverHost = "127.0.0.1";
const serverPort = "50001";

const server = new DaprServer({
  serverHost,
  serverPort,
  clientOptions: {
    daprHost,
    daprPort,
  },
});

await server.actor.init(); // Let the server know we need actors
server.actor.registerActor(ParkingSensorImpl); // Register the actor
await server.start(); // Start the server

// To get the registered actors, you can invoke `getRegisteredActors`:
const resRegisteredActors = await server.actor.getRegisteredActors();
console.log(`Registered Actors: ${JSON.stringify(resRegisteredActors)}`);