Best practices for testing local changes

[jstrunk]

I'm working on #302. In addition to adding snapshot tests, I want to do real end to end testing with a real pipeline project. For a pipeline, the new version of the package needs to be available to CodeBuild.

So far, I've had some success copying the built pipeline package including lib to the source of my test project and using relative path imports. I commit that package to the test project's got repo, so it's available in CodeBuild. When I need to make changes, I have to make them on the original source, do a build, and copy the lib directory again.

I think it would be better to use CodeArtifact or maybe even just S3 to make the package available privately where it can be installed in the CodeBuild project. Is it sufficient to use the package in pipeline's dist directory, or do I need to one for aws-prototyping-sdk? How can I version my test packages? They all have version 0.0.0. I'm nervous about running projen release which I believe is what handles versioning.

How are others testing their changes?

[cogwirrel]

I use file: dependencies for local end to end tests. This is my rough setup for testing type-safe-api (I imagine it would be pretty similar for testing the pipeline project):

const monorepo = new nx_monorepo.NxMonorepoProject({
  name: "test",
  defaultReleaseBranch: "main",
  devDeps: [
    "aws-cdk-lib",
    "projen",
    "constructs",
    "aws-prototyping-sdk",
    "@aws-prototyping-sdk/type-safe-api@file:/path/to/pdk/packages/type-safe-api/dist/js/package",
    "cdk-nag"
  ],
});

monorepo.package.addField('resolutions', {
  "**/@aws-prototyping-sdk/type-safe-api": 'file:/path/to/pdk/packages/type-safe-api/dist/js/package',
});

// Add any other projects below and dependencies on `@aws-prototyping-sdk/type-safe-api` will point to the local one

Then my dev cycle is:

• In /path/to/pdk/packages/type-safe-api run pnpm compile && pnpm package:js
• Double click the .tar.gz file in /path/to/pdk/packages/type-safe-api/dist/js to extract it to a dir called package
• In my test project (with the above .projenrc.ts) I run rm -rf node_modules && yarn && yarn build

There are probably even nicer ways with CodeArtifact as you mentioned or Verdaccio though :)

[jstrunk]

This is very useful for packages that only run on a developer's workstation. It simplifies the copy I was doing. Thanks!

The pipeline package needs to be run in CodeBuild because the pipeline is self mutating. Then the next step would be to commit the changes to the test package's git repo, so it's available in CodeBuild.

[cogwirrel]

Ooh true good call! :)

[jstrunk]

I'm using CodeArtifact to test changes before making PR #543 on the pipeline package. It's a bit complicated, but maybe if I share my steps here, we can come up with something to facilitate this kind of testing.

In the pipeline project after running pnpm build and pnpm cz in my working branch, I made a new versioned package.

I had to set the version with a git tag on the branch. It seems like projen supports prereleases, but PRERELEASE did nothing with the projen package and projen release:mainline commands. The projen release command doesn't exist in our repo.

Python requires prerelease packages to have the format 'X.Y.Z-label.sequence'.

git tag v0.19.68-beta.0

Then I had to temporarily set the version in pipeline's package.json and run projen release:mainline. Otherwise projen release:mainline failed. Also npm publish pushes the wrong version number without it.

I created a CodeArtifact Domain called pdktest and a Repository called pdkpipelinetest and logged into pip and twine easily with aws codeartifact login. Logging into npm the easy way failed, but I don't know if that has to do with my environment or this project. It worked to update .npmrc, but that wasn't needed for testing in Python.

aws codeartifact login --tool pip --repository pdkpipelinetest --domain pdktest --domain-owner <ACCOUNT_ID> --region us-east-1
aws codeartifact login --tool twine --repository pdkpipelinetest --domain pdktest --domain-owner <ACCOUNT_ID> --region us-east-1
twine upload --repository pdkpipelinetest dist/python/aws_prototyping_sdk.pipeline-0.19.68b0.tar.gz

In my test app environment, which is a PDKPipelinePyProject, I did the pip login to CodeArtifact.

I updated .projenrc.py

project = PDKPipelinePyProject(
    author_email="[email protected]",
    author_name="Jeff Strunk",
    cdk_version="2.92.0",
    dev_deps=["aws_prototyping_sdk.pipeline==0.19.68b0"],
    module_name="pdkpipelinepython",
    name="pdkpipelinepython",
    version="0.1.0",
    context={
        "repositoryName": "pdkpipelinepythontest",
        "defaultBranchName": "main",
        "artifactDomain": "pdktest",
        "artifactRepo": "pdkpipelinetest",
    },
)

I added the login command to synth_shell_partial_props when I instantiated the PDKPipeline class:

        self.pipeline = self.pipeline = PDKPipeline(
            self,
            "Pipeline",
            primary_synth_directory="cdk.out",
            repository_name=repo_name,
            default_branch_name=default_branch_name,
            publish_assets_in_parallel=False,
            docker_enabled_for_synth=True,
            synth=pipelines.ShellStep("Unused", commands=[]),
            synth_shell_step_partial_props=ShellStepProps(
                install_commands=[
                    "pip3 install awscli --upgrade --user",
                    f"aws codeartifact login --tool pip --domain {artifact_domain} --domain-owner {Stack.of(self).account} --repository {artifact_repo}",
                    "npm install -g aws-cdk",
                    "pip install -r requirements-dev.txt",
                    "pip install -r requirements.txt",
                    "projen",
                ],
                commands=["projen build"],
            ),
            code_build_defaults=CodeBuildOptions(
                role_policy=[
                    PolicyStatement(
                        effect=Effect.ALLOW,
                        actions=["sts:GetServiceBearerToken"],
                        resources=["*"],
                        conditions={
                            "StringEquals": {
                                "sts:AWSServiceName": "codeartifact.amazonaws.com"
                            }
                        },
                    ),
                    PolicyStatement(
                        effect=Effect.ALLOW,
                        actions=[
                            "codeartifact:GetAuthorizationToken",
                        ],
                        resources=[
                            Arn.format(
                                ArnComponents(
                                    service="codeartifact",
                                    resource="domain",
                                    resource_name=artifact_domain,
                                ),
                                Stack.of(self),
                            )
                        ],
                    ),
                    PolicyStatement(
                        effect=Effect.ALLOW,
                        actions=[
                            "codeartifact:GetRepositoryEndpoint",
                            "codeartifact:ReadFromRepository",
                        ],
                        resources=[
                            Arn.format(
                                ArnComponents(
                                    service="codeartifact",
                                    resource="repository",
                                    resource_name=f"{artifact_domain}/{artifact_repo}",
                                ),
                                Stack.of(self),
                            )
                        ],
                    ),
                ],
            ),
            cdk_command="cdk",
            branch_name_prefixes=PDKPipeline.ALL_BRANCHES,
        )

I had to add support for codeBuildDefaults.rolePolicy in feature_branches.ts so it could access CodeArtifact. The CodeBuild project created by CDK Pipelines already used it. I created #544 for that.