Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: add recursive copy #554

Merged
merged 28 commits into from
Sep 27, 2023
Merged

tests: add recursive copy #554

merged 28 commits into from
Sep 27, 2023

Conversation

jor2
Copy link
Member

@jor2 jor2 commented Aug 31, 2023

Description

Update one or more of the tests provisioning DAs in landing-zone pr_test.go to use our TestSchematicOptionsDefault function so they get tested in schematics, that is where they will most likely be run. Currently all tests in pr_test.go are using terraform cli (via terratest).

Release required?

  • No release
  • Patch release (x.x.X)
  • Minor release (x.X.x)
  • Major release (X.x.x)
Release notes content

Run the pipeline

If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.

Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:

/run pipeline

Checklist for reviewers

  • If relevant, a test for the change is included or updated with this PR.
  • If relevant, documentation for the change is included or updated with this PR.

Merge actions for mergers

  • When merging, use a relevant conventional commit message that is based on the PR contents and any release notes provided by the PR author. The commit message determines whether a new version of the module is needed, and if so, which semver increment to use (major, minor, or patch).
  • Merge by using "Squash and merge".

@jor2 jor2 self-assigned this Aug 31, 2023
@jor2
Copy link
Member Author

jor2 commented Aug 31, 2023

/run pipeline

@jor2
Copy link
Member Author

jor2 commented Sep 1, 2023

This implementation passes []string that looks like below to TarIncludePatterns:

*.tf
dynamic_values/*.tf
dynamic_values/*.yaml
dynamic_values/*.py
dynamic_values/*.tpl
dynamic_values/config_modules/*.tf
dynamic_values/config_modules/*.yaml
dynamic_values/config_modules/*.py
dynamic_values/config_modules/*.tpl
dynamic_values/config_modules/appid/*.tf
dynamic_values/config_modules/appid/*.yaml
dynamic_values/config_modules/appid/*.py
dynamic_values/config_modules/appid/*.tpl
dynamic_values/config_modules/bastion_vsi_list_to_map/*.tf
dynamic_values/config_modules/bastion_vsi_list_to_map/*.yaml
dynamic_values/config_modules/bastion_vsi_list_to_map/*.py
dynamic_values/config_modules/bastion_vsi_list_to_map/*.tpl
dynamic_values/config_modules/cluster_worker_pools/*.tf
dynamic_values/config_modules/cluster_worker_pools/*.yaml
dynamic_values/config_modules/cluster_worker_pools/*.py
dynamic_values/config_modules/cluster_worker_pools/*.tpl
dynamic_values/config_modules/clusters/*.tf
dynamic_values/config_modules/clusters/*.yaml
dynamic_values/config_modules/clusters/*.py
dynamic_values/config_modules/clusters/*.tpl
dynamic_values/config_modules/cos/*.tf
dynamic_values/config_modules/cos/*.yaml
dynamic_values/config_modules/cos/*.py
dynamic_values/config_modules/cos/*.tpl
dynamic_values/config_modules/f5/*.tf
dynamic_values/config_modules/f5/*.yaml
dynamic_values/config_modules/f5/*.py
dynamic_values/config_modules/f5/*.tpl
dynamic_values/config_modules/flow_logs/*.tf
dynamic_values/config_modules/flow_logs/*.yaml
dynamic_values/config_modules/flow_logs/*.py
dynamic_values/config_modules/flow_logs/*.tpl
dynamic_values/config_modules/get_subnets/*.tf
dynamic_values/config_modules/get_subnets/*.yaml
dynamic_values/config_modules/get_subnets/*.py
dynamic_values/config_modules/get_subnets/*.tpl
dynamic_values/config_modules/list_to_map/*.tf
dynamic_values/config_modules/list_to_map/*.yaml
dynamic_values/config_modules/list_to_map/*.py
dynamic_values/config_modules/list_to_map/*.tpl
dynamic_values/config_modules/list_to_map_value/*.tf
dynamic_values/config_modules/list_to_map_value/*.yaml
dynamic_values/config_modules/list_to_map_value/*.py
dynamic_values/config_modules/list_to_map_value/*.tpl
dynamic_values/config_modules/nested_list_to_map_and_merge/*.tf
dynamic_values/config_modules/nested_list_to_map_and_merge/*.yaml
dynamic_values/config_modules/nested_list_to_map_and_merge/*.py
dynamic_values/config_modules/nested_list_to_map_and_merge/*.tpl
dynamic_values/config_modules/service_authorizations/*.tf
dynamic_values/config_modules/service_authorizations/*.yaml
dynamic_values/config_modules/service_authorizations/*.py
dynamic_values/config_modules/service_authorizations/*.tpl
dynamic_values/config_modules/vpe/*.tf
dynamic_values/config_modules/vpe/*.yaml
dynamic_values/config_modules/vpe/*.py
dynamic_values/config_modules/vpe/*.tpl
dynamic_values/config_modules/vpn/*.tf
dynamic_values/config_modules/vpn/*.yaml
dynamic_values/config_modules/vpn/*.py
dynamic_values/config_modules/vpn/*.tpl
dynamic_values/config_modules/vsi/*.tf
dynamic_values/config_modules/vsi/*.yaml
dynamic_values/config_modules/vsi/*.py
dynamic_values/config_modules/vsi/*.tpl
f5_config/*.tf
f5_config/*.yaml
f5_config/*.py
f5_config/*.tpl
kms/*.tf
kms/*.yaml
kms/*.py
kms/*.tpl
kms/dynamic_values/*.tf
kms/dynamic_values/*.yaml
kms/dynamic_values/*.py
kms/dynamic_values/*.tpl
patterns/*.tf
patterns/*.yaml
patterns/*.py
patterns/*.tpl
patterns/diagrams/*.tf
patterns/diagrams/*.yaml
patterns/diagrams/*.py
patterns/diagrams/*.tpl
patterns/dynamic_values/*.tf
patterns/dynamic_values/*.yaml
patterns/dynamic_values/*.py
patterns/dynamic_values/*.tpl
patterns/dynamic_values/config_modules/*.tf
patterns/dynamic_values/config_modules/*.yaml
patterns/dynamic_values/config_modules/*.py
patterns/dynamic_values/config_modules/*.tpl
patterns/dynamic_values/config_modules/acl_rules/*.tf
patterns/dynamic_values/config_modules/acl_rules/*.yaml
patterns/dynamic_values/config_modules/acl_rules/*.py
patterns/dynamic_values/config_modules/acl_rules/*.tpl
patterns/dynamic_values/config_modules/bastion_gateways/*.tf
patterns/dynamic_values/config_modules/bastion_gateways/*.yaml
patterns/dynamic_values/config_modules/bastion_gateways/*.py
patterns/dynamic_values/config_modules/bastion_gateways/*.tpl
patterns/dynamic_values/config_modules/bastion_vsi/*.tf
patterns/dynamic_values/config_modules/bastion_vsi/*.yaml
patterns/dynamic_values/config_modules/bastion_vsi/*.py
patterns/dynamic_values/config_modules/bastion_vsi/*.tpl
patterns/dynamic_values/config_modules/bastion_zone_list/*.tf
patterns/dynamic_values/config_modules/bastion_zone_list/*.yaml
patterns/dynamic_values/config_modules/bastion_zone_list/*.py
patterns/dynamic_values/config_modules/bastion_zone_list/*.tpl
patterns/dynamic_values/config_modules/cloud_object_storage/*.tf
patterns/dynamic_values/config_modules/cloud_object_storage/*.yaml
patterns/dynamic_values/config_modules/cloud_object_storage/*.py
patterns/dynamic_values/config_modules/cloud_object_storage/*.tpl
patterns/dynamic_values/config_modules/default_security_group_rules/*.tf
patterns/dynamic_values/config_modules/default_security_group_rules/*.yaml
patterns/dynamic_values/config_modules/default_security_group_rules/*.py
patterns/dynamic_values/config_modules/default_security_group_rules/*.tpl
patterns/dynamic_values/config_modules/f5_deployments/*.tf
patterns/dynamic_values/config_modules/f5_deployments/*.yaml
patterns/dynamic_values/config_modules/f5_deployments/*.py
patterns/dynamic_values/config_modules/f5_deployments/*.tpl
patterns/dynamic_values/config_modules/f5_security_group_rules/*.tf
patterns/dynamic_values/config_modules/f5_security_group_rules/*.yaml
patterns/dynamic_values/config_modules/f5_security_group_rules/*.py
patterns/dynamic_values/config_modules/f5_security_group_rules/*.tpl
patterns/dynamic_values/config_modules/f5_tiers/*.tf
patterns/dynamic_values/config_modules/f5_tiers/*.yaml
patterns/dynamic_values/config_modules/f5_tiers/*.py
patterns/dynamic_values/config_modules/f5_tiers/*.tpl
patterns/dynamic_values/config_modules/key_management/*.tf
patterns/dynamic_values/config_modules/key_management/*.yaml
patterns/dynamic_values/config_modules/key_management/*.py
patterns/dynamic_values/config_modules/key_management/*.tpl
patterns/dynamic_values/config_modules/network_acls/*.tf
patterns/dynamic_values/config_modules/network_acls/*.yaml
patterns/dynamic_values/config_modules/network_acls/*.py
patterns/dynamic_values/config_modules/network_acls/*.tpl
patterns/dynamic_values/config_modules/resource_groups/*.tf
patterns/dynamic_values/config_modules/resource_groups/*.yaml
patterns/dynamic_values/config_modules/resource_groups/*.py
patterns/dynamic_values/config_modules/resource_groups/*.tpl
patterns/dynamic_values/config_modules/security_groups/*.tf
patterns/dynamic_values/config_modules/security_groups/*.yaml
patterns/dynamic_values/config_modules/security_groups/*.py
patterns/dynamic_values/config_modules/security_groups/*.tpl
patterns/dynamic_values/config_modules/subnet_cidr/*.tf
patterns/dynamic_values/config_modules/subnet_cidr/*.yaml
patterns/dynamic_values/config_modules/subnet_cidr/*.py
patterns/dynamic_values/config_modules/subnet_cidr/*.tpl
patterns/dynamic_values/config_modules/vpc/*.tf
patterns/dynamic_values/config_modules/vpc/*.yaml
patterns/dynamic_values/config_modules/vpc/*.py
patterns/dynamic_values/config_modules/vpc/*.tpl
patterns/dynamic_values/config_modules/vpc_address_prefixes/*.tf
patterns/dynamic_values/config_modules/vpc_address_prefixes/*.yaml
patterns/dynamic_values/config_modules/vpc_address_prefixes/*.py
patterns/dynamic_values/config_modules/vpc_address_prefixes/*.tpl
patterns/dynamic_values/config_modules/vpc_subnet_tiers/*.tf
patterns/dynamic_values/config_modules/vpc_subnet_tiers/*.yaml
patterns/dynamic_values/config_modules/vpc_subnet_tiers/*.py
patterns/dynamic_values/config_modules/vpc_subnet_tiers/*.tpl
patterns/mixed/*.tf
patterns/mixed/*.yaml
patterns/mixed/*.py
patterns/mixed/*.tpl
patterns/mixed/scripts/*.tf
patterns/mixed/scripts/*.yaml
patterns/mixed/scripts/*.py
patterns/mixed/scripts/*.tpl
patterns/roks/*.tf
patterns/roks/*.yaml
patterns/roks/*.py
patterns/roks/*.tpl
patterns/roks/module/*.tf
patterns/roks/module/*.yaml
patterns/roks/module/*.py
patterns/roks/module/*.tpl
patterns/roks/module/scripts/*.tf
patterns/roks/module/scripts/*.yaml
patterns/roks/module/scripts/*.py
patterns/roks/module/scripts/*.tpl
patterns/vpc/*.tf
patterns/vpc/*.yaml
patterns/vpc/*.py
patterns/vpc/*.tpl
patterns/vpc/module/*.tf
patterns/vpc/module/*.yaml
patterns/vpc/module/*.py
patterns/vpc/module/*.tpl
patterns/vpc/module/scripts/*.tf
patterns/vpc/module/scripts/*.yaml
patterns/vpc/module/scripts/*.py
patterns/vpc/module/scripts/*.tpl
patterns/vsi/*.tf
patterns/vsi/*.yaml
patterns/vsi/*.py
patterns/vsi/*.tpl
patterns/vsi/module/*.tf
patterns/vsi/module/*.yaml
patterns/vsi/module/*.py
patterns/vsi/module/*.tpl
patterns/vsi/module/scripts/*.tf
patterns/vsi/module/scripts/*.yaml
patterns/vsi/module/scripts/*.py
patterns/vsi/module/scripts/*.tpl
patterns/vsi-quickstart/*.tf
patterns/vsi-quickstart/*.yaml
patterns/vsi-quickstart/*.py
patterns/vsi-quickstart/*.tpl
ssh_key/*.tf
ssh_key/*.yaml
ssh_key/*.py
ssh_key/*.tpl
teleport_config/*.tf
teleport_config/*.yaml
teleport_config/*.py
teleport_config/*.tpl

tests/pr_test.go Outdated
@@ -75,6 +79,85 @@ func setupOptionsQuickStartPattern(t *testing.T, prefix string, dir string) *tes
return options
}

type RecursiveCopy struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct doesn't technically need to be "public" by making it capitalized, it could be "private" by having it start with lower case.

Also the word "Copy" might be misleading here, as we are not actually copying any files.

tests/pr_test.go Outdated
includeDirs []string
}

func recursiveCopy(dir string, dirsToExclude []string, fileTypesToInclude []string) []string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Word "Copy" might be misleading, as we are not actually copying any files.

This function doesn't return an error, so it will be hard for the unit test to not continue (and fail) if this function encounters an error of any kind.

tests/pr_test.go Outdated
// set up a schematics test
options := testschematic.TestSchematicOptionsDefault(&testschematic.TestSchematicOptions{
Testing: t,
TarIncludePatterns: recursiveCopy("..", excludeDirs, includeFiletypes),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioned above, the recursive routine does not return an error condition to check. Ideally you would call that function above, assign it to a variable, and also check if it errored and fail test.

Something like:
tarIncludes, recurseErr := recursiveCopy(blah blah)
if assert.NoError(t, recurseErr, "some description") {
return
}

tests/pr_test.go Outdated

tarIncludePatterns, recurseErr := getTarIncludePatternsRecursively("..", excludeDirs, includeFiletypes)

assert.NoError(t, recurseErr, "Schematic Test had unexpected error traversing directory tree...")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assert will only record the test failure, but it will still allow the test to proceed which could cause issues later.

There are two ways to handle this:

  1. instead of "assert" you can use "require", which will terminate the test if the assertion fails
  2. "assert" returns a boolean, so you can wrap the assert in an if..then and do something if false, such as "return"

Either way, you want the end result to be "if the tar include routine has an error, record it as a unit test failure and stop the test".

tests/go.mod Outdated
@@ -4,7 +4,32 @@ go 1.20

require (
github.com/stretchr/testify v1.8.4
github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper v1.10.19
github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper v1.11.9
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where are you getting 1.11.9 from? 1.11.6 seems to be the latest

Copy link
Member Author

@jor2 jor2 Sep 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those came automatically from my vscode or whilst running pre-commit hooks, but yeah weird. I'll fix it now.

@jor2
Copy link
Member Author

jor2 commented Sep 13, 2023

/run pipeline

Copy link
Member

@ocofaigh ocofaigh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't need to run TestRunQuickStartPattern as well as the new TestRunQuickStartPatternSchematics test - they both deploy the same code

@jor2
Copy link
Member Author

jor2 commented Sep 13, 2023

We probably don't need to run TestRunQuickStartPattern as well as the new TestRunQuickStartPatternSchematics test - they both deploy the same code

Wasn't sure if we want upgrade test to also be schematics?

@ocofaigh
Copy link
Member

@jor2 No the schematics test does not support an upgrade scenario, so leave the upgrade test as is

@jor2
Copy link
Member Author

jor2 commented Sep 14, 2023

/run pipeline

@jor2
Copy link
Member Author

jor2 commented Sep 22, 2023

/run pipeline

@jor2
Copy link
Member Author

jor2 commented Sep 23, 2023

/run pipeline

1 similar comment
@jor2
Copy link
Member Author

jor2 commented Sep 25, 2023

/run pipeline

@jor2
Copy link
Member Author

jor2 commented Sep 25, 2023

VSI test is the test failing due to failure to delete rg. Not my new test in schematics.

@ocofaigh
Copy link
Member

/run pipeline

tests/pr_test.go Show resolved Hide resolved
@jor2
Copy link
Member Author

jor2 commented Sep 26, 2023

/run pipeline

@jor2
Copy link
Member Author

jor2 commented Sep 26, 2023

/run pipeline

@ocofaigh ocofaigh merged commit 998c9be into main Sep 27, 2023
@ocofaigh ocofaigh deleted the schematics branch September 27, 2023 10:55
@terraform-ibm-modules-ops
Copy link
Contributor

🎉 This PR is included in version 4.12.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants