No clear way how to get ESC secrets in a Pulumi application

[kennethlynne]

File: themes/default/content/docs/concepts/secrets.md
I am just starting out with Pulumi and ESC.

I could not figure out from the documentation how to get a secret from Pulumi ESC and make use of it in my application.

I have created the secret and imported it in my stack, and I can successfully deploy for example a EKS cluster relying on the magic behind the scenes that sets it as environment variables, but when I want to pass something to a provider it was very hard to figure out how to get the secret correctly.

Example that I expect to work:

My environment definition:

values:
  aws:
    login:
      fn::open::aws-login:
        oidc:
          duration: 1h
          roleArn: arn:aws:iam::730335356113:role/PulumiSuperadmin
          sessionName: pulumi-environments-session
  secrets:
    fn::open::aws-secrets:
      region: eu-north-1
      login: ${aws.login}
      get:
        vercelAPIToken:
          secretId: production/platform/vercel_access_token
  environmentVariables:
    AWS_REGION: eu-north-1
    AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
    AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
    AWS_SESSION_TOKEN: ${aws.login.sessionToken}
    VERCEL_ACCESS_TOKEN: ${secrets.vercelAPIToken}

In my application (typescript):

const vercelProvider = new vercel.Provider('vercel', {
  apiToken: config.requireSecret('vercelAPIToken'),
});

But I receive the error:

    error: Missing required configuration variable 'panoptiq-infrastructure:vercelAPIToken'
        please set a value using the command `pulumi config set --secret panoptiq-infrastructure:vercelAPIToken <value>`

I was expecting to be able to use it like I do above, and I think I have misunderstood something? I struggled to find any relevant documentation.

My current workaround feels very hacky and clumsy, but it gets the job done:

if (!process.env.VERCEL_ACCESS_TOKEN) {
  throw new Error('VERCEL_ACCESS_TOKEN is required');
}

const vercelProvider = new vercel.Provider('vercel', {
  apiToken: process.env.VERCEL_ACCESS_TOKEN,
});

[toriancrane]

Hi @kennethlynne ,

Thank you for submitting this issue! We are currently in the process of enhancing the ESC docs to make topics like this a lot more clear. In the meantime, I can help you with getting this to work.

To expose values to Pulumi IaC, you will need to add them under a second-level key called "pulumiConfig". Using the environment definition you have provided, it would look something like this:

values:
  aws:
    login:
      fn::open::aws-login:
        oidc:
          duration: 1h
          roleArn: arn:aws:iam::730335356113:role/PulumiSuperadmin
          sessionName: pulumi-environments-session
    secrets:
      fn::open::aws-secrets:
        region: eu-north-1
        login: ${aws.login}
        get:
          vercelAPIToken:
            secretId: production/platform/vercel_access_token
  environmentVariables:
    AWS_REGION: eu-north-1
    AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
    AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
    AWS_SESSION_TOKEN: ${aws.login.sessionToken}
  pulumiConfig:
    vercelApiToken: ${aws.secrets.vercelAPIToken}

Also note that the secrets configuration needs to be nested under the aws second level key just like how the login configuration is as shown above. This means that the path to retrieve it is ${aws.secrets.vercelAPIToken}

Once you have added your variable under pulumiConfig, you will then need to make sure you have imported your Pulumi ESC environment into your Pulumi configuration file. If your stack is named dev, then the file would be called Pulumi.dev.yaml for example. The import will look something like this:

# Pulumi.dev.yaml
environment:
  - my-dev-environment

Make sure to replace the value of my-dev-environment with the name of your own environment.

Finally, you can now reference your ESC values as Pulumi config in the same way you would reference Pulumi config normally.

# index.ts
import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();
const apiToken = config.require("vercelApiToken");

export const yourApiToken = apiToken;

I hope this helps!

[kennethlynne]

That's perfect! Thank you! Close at will if you think it is tracked already, that solves it for me

[toriancrane]

Glad I could help!