Amplify Push: Parameters must have values for environment variables

[neuquen]

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

20.1.0

Amplify CLI Version

12.8.2

What operating system are you using?

Mac

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No

Describe the bug

We created a new lambda function on an existing project using the same steps found here.

However, with one difference. When we configured advanced settings we also included environment variables.

The issue is that after the function was created and pushed by another developer on our team, I can pull the new function using amplify pull, but when I try to push any other change using amplify push, it throws the following error:

🛑 The following resources failed to deploy:
Resource Name: customFunction (AWS::CloudFormation::Stack)
Event Type: update
Reason: Parameters: [envVarOne, envVarTwo, envVarThree] must have values
URL: ...

The developer who originally pushed the function has no issues pulling or pushing. Only I do.

The following issues look very similar to these other threads I've seen, but the main difference I see is that the environment variables are causing the issue.

#9484
#11211 (comment)

Expected behavior

Create new lambda functions with environment variables, pull in those changes, and be able to push additional changes up.

Reproduction steps

• Person 1: The same steps from here: https://docs.amplify.aws/javascript/build-a-backend/graphqlapi/connect-from-server-runtime/#utilizing-lambda-function-template-iam-authorization, but also include environment variables.
• Person 1: Amplify push function
• Person 2: Amplify pull
• Person 2: (update something else within project, like api)
• Person 2: Amplify push api
• (Error)

Project Identifier

No response

Log output

# Put your logs below this line

Additional information

No response

Before submitting, please confirm:

• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• I have removed any sensitive information from my code snippets and submission.

[ykethan]

The issue seems to have a similar behaviour as #13327.

[ykethan]

Hey @neuquen, tried reproducing the issue with the steps provided, but was not able to reproduce the issue.

1. amplify init
2. amplify add api
3. amplify add function, selected the template and in advance setting added 2 environment variables
4. amplify push
5. pulled the backend into a new empty directory
6. modified the the api model
7. amplify push api
   but did not observe any error.

Could check the issue linked in the previous comment and let us know the reproduction steps are similar in your environment.

[neuquen]

@ykethan I'll start over and do it again with more detailed steps and then get back to you.

[ykethan]

@neuquen thank you for following up. Adding the pending-response tag.

[neuquen]

We were able to replicate the issue again. I'm including more verbose output from the interactive prompts, but it honestly seems to be the same steps as before. This time around I included secrets as well as env variables.

One difference is that my colleague and I switched roles. This time I added the new function (Person 1) and he tried to make the update and push (Person 2).

PERSON 1

1. amplify add function

? Select which capability you want to add: Lambda function (serverless function)
? Provide an AWS Lambda function name: TestEnvVariables
? Choose the runtime that you want to use: NodeJS
? Choose the function template that you want to use: AppSync - GraphQL API request (with IAM)

✅ Available advanced settings:
- Resource access permissions
- Scheduled recurring invocation
- Lambda layers configuration
- Environment variables configuration
- Secret values configuration

? Do you want to configure advanced settings? Yes
? Do you want to access other resources in this project from your Lambda function? Yes
? Select the categories you want this function to have access to. api
? Api has 3 resources in this project. Select the one you would like your Lambda to access <app-name>
? Select the operations you want to permit on <app-name> Query, Mutation, Subscription

You can access the following resource attributes as environment variables from your Lambda function
        API_<app-name>_GRAPHQLAPIENDPOINTOUTPUT
        API_<app-name>_GRAPHQLAPIIDOUTPUT
        ENV
        REGION
? Do you want to invoke this function on a recurring schedule? No
? Do you want to enable Lambda layers for this function? No
? Do you want to configure environment variables for this function? Yes
? Enter the environment variable name: firstVar
? Enter the environment variable value: firstValue
? Select what you want to do with environment variables: Add new environment variable
? Enter the environment variable name: secondVar
? Enter the environment variable value: secondValue
? Select what you want to do with environment variables: Add new environment variable
? Enter the environment variable name: thirdVar
? Enter the environment variable value: thirdValue
? Select what you want to do with environment variables: I'm done

You can access the following resource attributes as environment variables from your Lambda function
        ENV
        REGION
        API_<app-name>_GRAPHQLAPIIDOUTPUT
        API_<app-name>_GRAPHQLAPIENDPOINTOUTPUT
        firstVar
        secondVar
        thirdVar
? Do you want to configure secret values this function can access? Yes
? Enter a secret name (this is the key used to look up the secret value): firstSecret
? Enter the value for firstSecret: [hidden]
? What do you want to do? Add a secret
? Enter a secret name (this is the key used to look up the secret value): secondSecret
? Enter the value for secondSecret: [hidden]
? What do you want to do? I'm done
✔ Choose the package manager that you want to use: · NPM
Use the AWS SSM GetParameter API to retrieve secrets in your Lambda function.
More information can be found here: https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParameter.html
? Do you want to edit the local lambda function now? No
✅ Successfully added resource TestEnvVariables locally.

2. amplify push function TestEnvVariables
   NOTE: I have a couple pre-push hooks that run npm install on the overall project due to project-level overrides and lambda functions with npm dependencies.

----- 🪝 pre-push execution start -----
Running npm install...

added 334 packages, and audited 369 packages in 19s

26 packages are looking for funding
  run `npm fund` for details

8 moderate severity vulnerabilities

To address issues that do not require attention, run:
  npm audit fix

To address all issues possible (including breaking changes), run:
  npm audit fix --force

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.
----- 🪝 pre-push execution end -----


----- 🪝 pre-push-function execution start -----
Running npm install...
----- 🪝 pre-push-function execution end -----


    Current Environment: dev
    
┌──────────┬──────────────────┬───────────┬───────────────────┐
│ Category │ Resource name    │ Operation │ Provider plugin   │
├──────────┼──────────────────┼───────────┼───────────────────┤
│ Function │ TestEnvVariables │ Create    │ awscloudformation │
└──────────┴──────────────────┴───────────┴───────────────────┘
✔ Are you sure you want to continue? (Y/n) · yes

Deployment completed.

PERSON 2

3. amplify pull
4. (Updated a different, existing lambda function)
5. amplify push function differentLambdaFunction

🛑 The following resources failed to deploy:
Resource Name: functionTestEnvVariables (AWS::CloudFormation::Stack)
Event Type: update
Reason: Parameters: [thirdVar, secondVar, firstVar] must have values
URL: ...

[neuquen]

Also, regarding #13327. Unfortunately, I already removed the test function, so I no longer can view function-parameters.json, but I still have the output from a git diff for the other two files.

If this isn't helpful without also seeing the function parameters file, then I can create a new lambda function again (but not until January).

team-provider-info.json:

+    },
+    "TestEnvVariables": {
+      "build": true,
+      "dependsOn": [
+        {
+          "attributes": [
+            "GraphQLAPIIdOutput",
+            "GraphQLAPIEndpointOutput"
+          ],
+          "category": "api",
+          "resourceName": "<app-name>"
+        }
+      ],
+      "providerPlugin": "awscloudformation",
+      "service": "Lambda"
+    },

...

+    "AMPLIFY_function_TestEnvVariables_firstVar": {
+      "usedBy": [
+        {
+          "category": "function",
+          "resourceName": "TestEnvVariables"
+        }
+      ]
+    },
+    "AMPLIFY_function_TestEnvVariables_s3Key": {
+      "usedBy": [
+        {
+          "category": "function",
+          "resourceName": "TestEnvVariables"
+        }
+      ]
+    },
+    "AMPLIFY_function_TestEnvVariables_secondVar": {
+      "usedBy": [
+        {
+          "category": "function",
+          "resourceName": "TestEnvVariables"
+        }
+      ]
+    },
+    "AMPLIFY_function_TestEnvVariables_secretsPathAmplifyAppId": {
+      "usedBy": [
+        {
+          "category": "function",
+          "resourceName": "TestEnvVariables"
+        }
+      ]
+    },
+    "AMPLIFY_function_TestEnvVariables_thirdVar": {
+      "usedBy": [
+        {
+          "category": "function",
+          "resourceName": "TestEnvVariables"
+        }
+      ]
+    },

backend-config.json:

"TestEnvVariables": {
+          "firstVar": "firstValue",
+          "secondVar": "secondValue",
+          "thirdVar": "thirdValue",
+          "secretsPathAmplifyAppId": "<app-id>",
+          "deploymentBucketName": "amplify-<app-name>-dev-155924-deployment",
+          "s3Key": "amplify-builds/TestEnvVariables-6b6a6443446a516c6568-build.zip"
         }

[ykethan]

Hey @neuquen, thank you for the information. Does the person 2 pull the application from git repository with the team-provider-info.json?
The function env variables are maintained in the team-provider-info.json, when sharing the project do ensure the team member does also have access to team-provider-info.json via the version control system.

[neuquen]

@ykethan, yes, we use version control and we include team-provider-info.json in our repo. Although, we actually haven't been committing our changes in git because we've been mostly testing locally. Can you give me an example of how the variables should be formatted in that file so that I can see if they show up properly in our project?

This is what was added from the previous try:

"TestEnvVariables": {
          "firstVar": "firstValue",
          "secondVar": "secondValue",
          "thirdVar": "thirdValue",
          "secretsPathAmplifyAppId": "<app-id>",
          "deploymentBucketName": "amplify-<app-name>-dev-155924-deployment",
          "s3Key": "amplify-builds/TestEnvVariables-6b6a6443446a516c6568-build.zip"
        }

I'm going to try again, but this time I will make sure we share the changes in the team-provider-infor.json as well.

[ykethan]

@neuquen the format does appear to be correct in team-provider-info.json, do let us know if this mitigates the issue

"function": {
        "tes2343f4": {
          "test": "test",
          "deploymentBucketName": "",
          "s3Key": ""
        }
      }

[neuquen]

@ykethan, thanks for all the help. The missing environment variables in team-provider-info.json ended up being the issue this whole time.

But that brings up one last question for me. Why are environment variables generated locally for the person who created/updated the lambda function, but not updated when pulled by someone else? The other values are pulled in (deploymentBucketName, s3key, secretsPathAmplifyAppId). Why wouldn't the env variable names and values also be pulled in?

[ykethan]

@neuquen Glad to hear you were able to mitigate the issue. To clarify the behavior, Amplify CLI currently does not zip team-provider-info.json to the current-cloud-backend present on the S3 bucket. https://docs.amplify.aws/react/tools/cli/teams/shared/#sharing-projects-within-the-team
On a pull the variables such deploymentBucketName, s3key, secretsPathAmplifyAppId are rehydrated from the amplify-meta.json file from the current-cloud-backend which contain the references to the resources.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.