Attempting to redeploy to a stack that is still updating fails with "UPDATE_IN_PROGRESS state and can not be updated."

[isaacrlevin]

Describe the bug

Get this error message when trying to deploy MVC Template

Unhandled exception. This is a bug. Please copy the stack trace below and file a bug at https://github.com/aws/aws-dotnet-deploy.
Source sequence doesn't contain any elements.
at System.Linq.AsyncEnumerable.g__Core|283_0[TSource](IAsyncEnumerable1 source, CancellationToken cancellationToken) in d:\a\1\s\Ix.NET\Source\System.Linq.Async\System\Linq\Operators\First.cs:line 24 at AWS.Deploy.Orchestration.Data.AWSResourceQueryer.<>c__DisplayClass26_0.<<GetDefaultVpc>b__0>d.MoveNext() in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Data/AWSResourceQueryer.cs:line 479 --- End of stack trace from previous location --- at AWS.Deploy.Orchestration.Data.AWSResourceQueryer.HandleException[T](Func1 action) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Data/AWSResourceQueryer.cs:line 887
at AWS.Deploy.Orchestration.Data.AWSResourceQueryer.GetDefaultVpc() in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Data/AWSResourceQueryer.cs:line 479
at AWS.Deploy.Orchestration.Orchestrator.ApplyAllReplacementTokens(Recommendation recommendation, String cloudApplicationName) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Orchestrator.cs:line 222
at AWS.Deploy.CLI.Commands.DeployCommand.InitializeDeployment(String cloudApplicationName, DeploymentSettings deploymentSettings, String deploymentProjectPath) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.CLI/Commands/DeployCommand.cs:line 250
at AWS.Deploy.CLI.Commands.DeployCommand.ExecuteAsync(String applicationName, String deploymentProjectPath, DeploymentSettings deploymentSettings) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.CLI/Commands/DeployCommand.cs:line 118
at AWS.Deploy.CLI.Commands.CommandFactory.b__43_0(DeployCommandHandlerInput input) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.CLI/Commands/CommandFactory.cs:line 255

Expected Behavior

Deploy MVC template to AWS

Current Behavior

Get error message

Unhandled exception. This is a bug. Please copy the stack trace below and file a bug at https://github.com/aws/aws-dotnet-deploy.
Source sequence doesn't contain any elements.
at System.Linq.AsyncEnumerable.g__Core|283_0[TSource](IAsyncEnumerable1 source, CancellationToken cancellationToken) in d:\a\1\s\Ix.NET\Source\System.Linq.Async\System\Linq\Operators\First.cs:line 24 at AWS.Deploy.Orchestration.Data.AWSResourceQueryer.<>c__DisplayClass26_0.<<GetDefaultVpc>b__0>d.MoveNext() in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Data/AWSResourceQueryer.cs:line 479 --- End of stack trace from previous location --- at AWS.Deploy.Orchestration.Data.AWSResourceQueryer.HandleException[T](Func1 action) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Data/AWSResourceQueryer.cs:line 887
at AWS.Deploy.Orchestration.Data.AWSResourceQueryer.GetDefaultVpc() in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Data/AWSResourceQueryer.cs:line 479
at AWS.Deploy.Orchestration.Orchestrator.ApplyAllReplacementTokens(Recommendation recommendation, String cloudApplicationName) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Orchestrator.cs:line 222
at AWS.Deploy.CLI.Commands.DeployCommand.InitializeDeployment(String cloudApplicationName, DeploymentSettings deploymentSettings, String deploymentProjectPath) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.CLI/Commands/DeployCommand.cs:line 250
at AWS.Deploy.CLI.Commands.DeployCommand.ExecuteAsync(String applicationName, String deploymentProjectPath, DeploymentSettings deploymentSettings) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.CLI/Commands/DeployCommand.cs:line 118
at AWS.Deploy.CLI.Commands.CommandFactory.b__43_0(DeployCommandHandlerInput input) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.CLI/Commands/CommandFactory.cs:line 255

Reproduction Steps

dotnet new web -o AWSDeploy
cd AWSDeploy 
dotnet aws deploy
AWS .NET deployment tool for deploying .NET Core applications to AWS.
Project Home: https://github.com/aws/aws-dotnet-deploy

Configuring AWS Credentials using AWS SDK credential search.
Configuring AWS region using AWS SDK region search to us-west-2.
Select an existing AWS deployment target to deploy your application to.
-----------------------------------------------------------------------
1: MVCTemplate (CloudFormation Stack) (default)
2: *** Deploy to a new Cloud Application ***
Choose deployment target (default 1):
1
Loading MVCTemplate (CloudFormation Stack) settings:
----------------------------------------------------
Elastic Beanstalk Application: MVCTemplate
Environment Name:
        Environment Name: MVCTemplate-dev
Environment Type: SingleInstance
Application IAM Role: *** Create new ***
Service IAM Role: *** Create new ***
Beanstalk Platform: arn:aws:elasticbeanstalk:us-west-2::platform/.NET Core running on 64bit Amazon Linux 2/2.3.3
Managed Platform Updates:
        Enable Managed Platform Updates: True
        Preferred Start Time: Sun:00:00
        Update Level: minor
Enable AWS X-Ray Tracing Support: False
Reverse Proxy: nginx
Environment Variables:
Virtual Private Cloud (VPC):
        Use a VPC : False
Dotnet Build Configuration: Release
Self Contained Build: False
Unhandled exception.  This is a bug.  Please copy the stack trace below and file a bug at https://github.com/aws/aws-dotnet-deploy.
Source sequence doesn't contain any elements.
   at System.Linq.AsyncEnumerable.<FirstAsync>g__Core|283_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in d:\a\1\s\Ix.NET\Source\System.Linq.Async\System\Linq\Operators\First.cs:line 24
   at AWS.Deploy.Orchestration.Data.AWSResourceQueryer.<>c__DisplayClass26_0.<<GetDefaultVpc>b__0>d.MoveNext() in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Data/AWSResourceQueryer.cs:line 479
--- End of stack trace from previous location ---
   at AWS.Deploy.Orchestration.Data.AWSResourceQueryer.HandleException[T](Func`1 action) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Data/AWSResourceQueryer.cs:line 887
   at AWS.Deploy.Orchestration.Data.AWSResourceQueryer.GetDefaultVpc() in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Data/AWSResourceQueryer.cs:line 479
   at AWS.Deploy.Orchestration.Orchestrator.ApplyAllReplacementTokens(Recommendation recommendation, String cloudApplicationName) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.Orchestration/Orchestrator.cs:line 222
   at AWS.Deploy.CLI.Commands.DeployCommand.InitializeDeployment(String cloudApplicationName, DeploymentSettings deploymentSettings, String deploymentProjectPath) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.CLI/Commands/DeployCommand.cs:line 250
   at AWS.Deploy.CLI.Commands.DeployCommand.ExecuteAsync(String applicationName, String deploymentProjectPath, DeploymentSettings deploymentSettings) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.CLI/Commands/DeployCommand.cs:line 118
   at AWS.Deploy.CLI.Commands.CommandFactory.<BuildDeployCommand>b__43_0(DeployCommandHandlerInput input) in /codebuild/output/src830051283/src/codestar-connections.us-west-2.amazonaws.com/git-http/393431854674/us-west-2/eb780f07-2f46-42e0-9c15-df6f00fe3c1c/aws/aws-dotnet-deploy/src/AWS.Deploy.CLI/Commands/CommandFactory.cs:line 255

Possible Solution

No response

Additional Information/Context

No response

Version used

1.1.15+5c57ebd489

Operating System and version

Windows 11

[philasmar]

Can you try updating the deploy tool to the latest version and try again?

[isaacrlevin]

It doesn't error but it hangs on the CDK step

[isaacrlevin]

If I close and try again I get this

dotnet aws deploy
AWS .NET deployment tool for deploying .NET Core applications to AWS.
Project Home: https://github.com/aws/aws-dotnet-deploy

Configuring AWS Credentials using AWS SDK credential search.
Configuring AWS region using AWS SDK region search to us-west-2.
Select an existing AWS deployment target to deploy your application to.

1: MVCTemplate (CloudFormation Stack) (default)
2: *** Deploy to a new Cloud Application ***
Choose deployment target (default 1):
1
Loading MVCTemplate (CloudFormation Stack) settings:

Elastic Beanstalk Application: MVCTemplate
Environment Name:
Environment Name: MVCTemplate-dev
Environment Type: SingleInstance
Application IAM Role: *** Create new ***
Service IAM Role: *** Create new ***
Beanstalk Platform: arn:aws:elasticbeanstalk:us-west-2::platform/.NET Core running on 64bit Amazon Linux 2/2.3.3
Managed Platform Updates:
Enable Managed Platform Updates: True
Preferred Start Time: Sun:00:00
Update Level: minor
Enable AWS X-Ray Tracing Support: False
Reverse Proxy: nginx
Environment Variables:
Virtual Private Cloud (VPC):
Use a VPC : False
Dotnet Build Configuration: Release
Self Contained Build: False

Current settings (select number to change its value)

1. Environment Variables:

Enter 'more' to display Advanced settings.
Or press 'Enter' to deploy:

---

Creating deployment zip bundle

Using the dotnet CLI build the project and zip the publish artifacts.

---

Creating Dotnet Publish Zip file...
MSBuild version 17.3.0+92e077650 for .NET
Determining projects to restore...
All projects are up-to-date for restore.
AWSDeploy -> D:\dev\AWSDeploy\bin\Release\net6.0\AWSDeploy.dll
AWSDeploy -> C:\Users\isaac\AppData\Local\Temp\b56715fa-93e9-4aa3-8f55-a204ff61be98\

---

Configuring AWS Cloud Development Kit (CDK)

Ensure a compatible CDK version is installed and the CDK bootstrap CloudFormation stack has been created. A CDK project to perform the deployment is generated unless an existing deployment project was selected.

---

Generating AWS Cloud Development Kit (AWS CDK) deployment project
Confirmed CDK Bootstrap CloudFormation stack already exists.

---

Deploying AWS CDK project

Use the CDK project to create or update the AWS CloudFormation stack and deploy the project to the AWS resources in the stack.

---

npm WARN config global --global, --local are deprecated. Use --location=global instead.

✨ Synthesis time: 19.49s

MVCTemplate: deploying...
[0%] start: Publishing 3e38191482508526b7142519b6af723206a51e0955f51af5b2ab43db4ba80928:563011967245-us-west-2
[0%] start: Publishing 4021cdec9e544860a835dc518dedcbd433b48d5c35e6b1ba35716b4c3250d463:563011967245-us-west-2
[50%] success: Published 4021cdec9e544860a835dc518dedcbd433b48d5c35e6b1ba35716b4c3250d463:563011967245-us-west-2
[100%] success: Published 3e38191482508526b7142519b6af723206a51e0955f51af5b2ab43db4ba80928:563011967245-us-west-2
MVCTemplate: creating CloudFormation changeset...

❌ MVCTemplate failed: Error [ValidationError]: Stack:arn:aws:cloudformation:us-west-2:563011967245:stack/MVCTemplate/8cf71b80-1513-11ed-8d1a-0689948d0e75 is in UPDATE_IN_PROGRESS state and can not be updated.
at Request.extractError (C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\protocol\query.js:50:29)
at Request.callListeners (C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\request.js:686:14)
at Request.transition (C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\state_machine.js:14:12)
at C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request. (C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\request.js:38:9)
at Request. (C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\request.js:688:12) at Request.callListeners (C:\Users\isaac\AppData\Roaming\npm\node_modules\aws-sdk\lib\sequential_executor.js:116:18) {
code: 'ValidationError',
time: 2022-08-26T00:07:32.821Z,
requestId: 'c658fd98-ed42-4b7e-b567-d1495adfd810',
statusCode: 400,
retryable: false,
retryDelay: 127.80559217350196
}

Stack:arn:aws:cloudformation:us-west-2:563011967245:stack/MVCTemplate/8cf71b80-1513-11ed-8d1a-0689948d0e75 is in UPDATE_IN_PROGRESS state and can not be updated.

We had an issue deploying your application to AWS. Check the deployment output for more details. Deployment took 25.24s.

For more information, please visit our troubleshooting guide https://aws.github.io/aws-dotnet-deploy/troubleshooting-guide/.
If you are still unable to solve this issue and believe this is an issue with the tooling, please cut a ticket https://github.com/aws/aws-dotnet-deploy/issues/new/choose.

[philasmar]

The reason why you get the error you got when you closed it and tried again is because CDK already pushed the changeset to the AWS account and the process is completing there. Even if you close the tool, the CloudFormation stack is still being updated. The tool is only showing you the update status. I propose you go to CloudFormation in your AWS account and look for the stack you are deploying. Can you report back the status of the stack as well as any errors shown in the events tab?

[isaacrlevin]

The stack failed on deployment and I had to completely delete it. Is it possible to provide a more helpful message to the user? The whole process seems to be a little opaque with what is actually going on. If we can provide a clean message with resolution steps, that would be helpful.

[ashovlin]

The original issue based on the first stack trace should be fixed in version 1.2.4, which addressed the case of a deployment failure when deploying to an account+region without any existing VPCs.

I agree we can improve detection/messaging when attempting to deploy to a stack that is still being updated, we've changed this to a feature request.

[ashishdhingra]

Closing due to lack of traction.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.