Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add provider upgrade tests #929

Closed
wants to merge 5 commits into from
Closed

Add provider upgrade tests #929

wants to merge 5 commits into from

Conversation

t0yv0
Copy link
Member

@t0yv0 t0yv0 commented Oct 19, 2023

Proposed changes

This PR adds upgrade tests that verify that taking a stack provisioned on a baseline version of the provider and upgrading it to the current version creates no replacements.

Fixes #927

Dependent PR: pulumi/providertest#19

  • providertest is not correctly reinstalling the TypeScript SDKs for the upgrade test
  • possibly as a result of that, the recorded baseline has the wrong AWS provider version listed
  • if we want to record more example programs here, GHA automation is in order to parallelize, as each takes ~20 min
  • need to ensure these tests are run in CI

Related issues (optional)

"id": "4b6db602-50de-4739-ac98-93c6c721d82b",
"type": "pulumi:providers:aws",
"inputs": {
"version": "5.31.0"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. This is not right unfortunately. The specification stated 5.42.0 - which should have been the provider that got installed in PATH. Either that's the provider really being used but the version here is taken from the SDK, or the ambient plugin installation didn't work right.

@t0yv0 t0yv0 requested review from danielrbradley, thomas11 and a team and removed request for danielrbradley October 19, 2023 01:35
Copy link
Member

@iwahbe iwahbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we are trying to run tests on go 1.19, which doesn't parse the updated go.mod.

})
}

func test(t *testing.T, dir string, opts ...providertest.Option) *providertest.ProviderTest {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see where t is used.

Suggested change
func test(t *testing.T, dir string, opts ...providertest.Option) *providertest.ProviderTest {
func test(dir string, opts ...providertest.Option) *providertest.ProviderTest {

Comment on lines +10 to +12
t.Run("cluster", func(t *testing.T) {
test(t, "../examples/cluster").Run(t)
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any opportunity for the call to test(...) to fail. We should just be able to elide the func here:

Suggested change
t.Run("cluster", func(t *testing.T) {
test(t, "../examples/cluster").Run(t)
})
t.Run("cluster", test(t, "../examples/cluster").Run)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well well, you have to leave markers for the IDE to let you click on the test to run it. I really need to test it in VS Code.

@t0yv0
Copy link
Member Author

t0yv0 commented Oct 20, 2023

I've done some more work on providertest to lean harder into pre-existing ProgramTest features to do TypeScript programs much cleaner but I didn't integrate it back here yet. Tomorrow..

@t0yv0
Copy link
Member Author

t0yv0 commented Oct 23, 2023

With providertest 0.0.2 it should be handling TypeScript SDKs correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add upgrade tests
3 participants