`amplify-preview`: custom action to trigger amplify previews #287

taraspos · 2024-11-06T17:06:36Z

Summary

This PR creates custom github action which:

Accepts list of Amplify App IDs
Checks all Amplify App IDs to find if branch was already connected to one of the apps
If create-branch enabled
- Connect branch to one of the AWS Amplify where limit hasn't been reached yet
- Trigger initial deployment job
If wait enabled
- Wait for deployment to succeed/fail
Post/update comment on the PR with preview URL

Context

Because of limitations with AWS Amplify, we can't use Amplify's GitHub integration in teleport or docs-website repos:

Amplify has hard limits of 50 Amplify previews per app¹, and we have more PRs in teleport app
There is no way to create PR preview programatically, to use existing status checks and PR comments:
- Programatically create PR preview aws-amplify/amplify-hosting#3963
If PR previews are created automatically there is no way to filter for which PRs to create previews, so as results previews are created even for PRs where docs were not modified which makes previews limit problem even more severe:
- Filter automatic Github PR previews based on labels, name, files modified aws-amplify/amplify-hosting#3960
Diff mechanism that exist doesn't work properly either:
- Amplify Github App posts empty preview link on the PR even if deployment was skipped because of AMPLIFY_DIFF_DEPLOY aws-amplify/amplify-hosting#3959

Testing

Tested on:

gha(amplify-preview): use custom action to generate Amplify previews for Teleport Docs teleport#48512

This is a more advanced version of workflow running in docs-website repo, once this is merged, I will configure docs-website to run this version:

gha(amplify-preview): add job to post amplify preview URL on PRs docs-website#22

Run example:

https://github.com/gravitational/teleport/actions/runs/12353860346/job/34473886441?pr=48512

PR comment looks like following:

or if latest deployment is not successful, it will show latest deployment and latest successful deployment.

https://docs.aws.amazon.com/general/latest/gr/amplify.html ↩

fheinecke

IMO this should be written as a Go or typescript tool. We've been trying to move this direction to reduce our reliance on GHA.

If you decide to go this route, there are tools in this directory for CI/CD workflows as well as local tooling.

.github/actions/amplify-preview/action.yaml

taraspos · 2024-12-16T13:42:21Z

tools/amplify-preview/go.mod

+	gopkg.in/yaml.v3 v3.0.1 // indirect
+)
+
+replace github.com/gravitational/shared-workflows/libs => ../../libs/


This is needed to use new functions added to the github package.
Can be removed once new version is released.

jimbishopp

👍 First pass.

libs/github/comment.go

jimbishopp · 2024-12-16T16:56:34Z

tools/amplify-preview/action.yaml

+        WAIT: ${{ inputs.wait }} 
+      shell: bash
+      run: |
+        pushd ./tools/amplify-preview/; go run ./; popd


Do we need shell and pushd? go run ./tools/amplify-preview

This can be removed in subsequent release PR once libs package is released and I can remove the replace rule from go.mod:

shared-workflows/tools/amplify-preview/go.mod

Line 37 in b7d7899

replace github.com/gravitational/shared-workflows/libs => ../../libs/

tools/amplify-preview/amplify.go

jimbishopp · 2024-12-16T17:03:36Z

tools/amplify-preview/main.go

+	"strings"
+	"time"
+
+	"github.com/alecthomas/kingpin/v2"


For consistency, let's use either the flag package or cobra.

E.g.

shared-workflows/bot/main.go

Lines 118 to 131 in 0f1f9de

func parseFlags() (flags, error) {

var (

workflow = flag.String("workflow", "", "specific workflow to run [assign, check, dismiss, backport]")

token = flag.String("token", "", "GitHub authentication token")

reviewers = flag.String("reviewers", "", "reviewer assignments")

local = flag.Bool("local", false, "local workflow dry run")

org = flag.String("org", "", "GitHub organization (local mode only)")

repo = flag.String("repo", "", "GitHub repository (local mode only)")

prNumber = flag.Int("pr", 0, "GitHub pull request number (local mode only)")

branch = flag.String("branch", "", "GitHub backport branch name (local mode only)")

baseStats = flag.String("base", "", "the artifact sizes as generated by binary-sizes to compare against for bloat")

buildDir = flag.String("builddir", "", "an absolute path to a build directory containing artifacts to be checked for bloat")

artifacts = flag.String("artifacts", "", "a comma separated list of compile artifacts to analyze for bloat")

)

https://github.com/gravitational/cloud/blob/master/tcctl/release/cli/cli.go

All tools in https://github.com/gravitational/shared-workflows/tree/master/tools are using github.com/alecthomas/kingpin and I followed their example for consistency 😬

Not sure why kingpin was chosen for those tools. The review bot uses the simple flag package. We previously used kingpin in cloud but changed to cobra because it has a better design.

I think it's because teleport uses kingpin everywhere:

https://github.com/search?q=repo%3Agravitational%2Fteleport%20kingpin&type=code

Changing to align with other cloud projects is up to you. Lots of big projects chose cobra (K8s, Hugo, Github CLI, Docker, Istio, Helm, etc).

I would prefer to keep it as is to avoid spending more time on this, as long term we would prefer using native integration, once Amplify get to resolve current issues.

tools/amplify-preview/main.go

Co-authored-by: Jim Bishopp <[email protected]>

libs/github/comment.go

jimbishopp · 2024-12-17T14:54:40Z

libs/github/comment.go

+	for _, c := range comments {
+		matcher := true
+		if targetComment.UserLogin != "" {
+			matcher = matcher && c.User.GetLogin() == targetComment.UserLogin
+		}
+
+		if targetComment.BodyContains != "" {
+			matcher = matcher && strings.Contains(c.GetBody(), targetComment.BodyContains)
+		}
+
+		if matcher {
+			return c, nil
+		}
+	}


Proper zero values for CommentTraits means you can remove matcher and the empty string checks. Also, see https://pkg.go.dev/slices#ContainsFunc

tools/amplify-preview/amplify.go

jimbishopp · 2024-12-17T15:07:03Z

tools/amplify-preview/main.go

+	"strings"
+	"time"
+
+	"github.com/alecthomas/kingpin/v2"


Changing to align with other cloud projects is up to you. Lots of big projects chose cobra (K8s, Hugo, Github CLI, Docker, Istio, Helm, etc).

jimbishopp · 2024-12-17T15:18:21Z

tools/amplify-preview/amplify.go

+	failedResp := aggregatedError{
+		perAppErr: map[string]error{},
+		message:   "failed to fetch branch",
+	}


Is aggregatedError needed? Looks like its value is in deduplicating error messages when the same appID is passed in via a flag. Why not deduplicate in main before reaching this point? That way this doesn't make two calls to GetBranch for the same appID and you could gather errors using a slice with errors join.

Same thing applies for CreateBranch.

The original purpose was not really to deduplicate errors but just to associate error with app where it happened, and then combine in one aggregated error.

tools/amplify-preview/github.go

tools/amplify-preview/main.go

taraspos requested a review from a team as a code owner November 6, 2024 17:06

taraspos force-pushed the taras/amplify-preview-job branch from 844b294 to 31fd385 Compare November 6, 2024 17:35

taraspos requested review from r0mant and jimbishopp November 6, 2024 18:40

fheinecke reviewed Nov 6, 2024

View reviewed changes

.github/actions/amplify-preview/action.yaml Outdated Show resolved Hide resolved

.github/actions/amplify-preview/action.yaml Outdated Show resolved Hide resolved

.github/actions/amplify-preview/action.yaml Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

taraspos requested a review from fheinecke November 12, 2024 17:07

taraspos marked this pull request as draft December 11, 2024 18:26

taraspos force-pushed the taras/amplify-preview-job branch from 5e63c93 to 31b9ce9 Compare December 11, 2024 18:29

taraspos commented Dec 11, 2024

View reviewed changes

.github/actions/amplify-preview/action.yaml Outdated Show resolved Hide resolved

taraspos added 2 commits December 11, 2024 18:32

Move action to shared repo

695a566

Create go version of amplify preview gha action

8790c1f

taraspos force-pushed the taras/amplify-preview-job branch 4 times, most recently from f89122e to ac7989e Compare December 16, 2024 12:21

add logic to wait for job completion

b1f7d99

taraspos force-pushed the taras/amplify-preview-job branch 2 times, most recently from 4578f7d to 5e7aaf5 Compare December 16, 2024 13:03

Support multiple jobs in a table

9cec7a5

taraspos force-pushed the taras/amplify-preview-job branch from 5e7aaf5 to 9cec7a5 Compare December 16, 2024 13:11

Add readme

3fcbc2c

taraspos commented Dec 16, 2024

View reviewed changes

taraspos marked this pull request as ready for review December 16, 2024 15:57

Dependency fixes

5b33a39

taraspos force-pushed the taras/amplify-preview-job branch from 92d7532 to 5b33a39 Compare December 16, 2024 16:41

Add CICD files

b7d7899

taraspos force-pushed the taras/amplify-preview-job branch from 9b2b5f0 to b7d7899 Compare December 16, 2024 17:03

jimbishopp reviewed Dec 16, 2024

View reviewed changes

Update tools/amplify-preview/amplify.go

80abb12

Co-authored-by: Jim Bishopp <[email protected]>

jimbishopp reviewed Dec 16, 2024

View reviewed changes

libs/github/comment.go Outdated Show resolved Hide resolved

taraspos force-pushed the taras/amplify-preview-job branch 4 times, most recently from 0d00076 to 877547e Compare December 17, 2024 11:59

Address PR feedback

42c033c

taraspos force-pushed the taras/amplify-preview-job branch from 877547e to 42c033c Compare December 17, 2024 12:01

taraspos requested a review from jimbishopp December 17, 2024 12:27

jimbishopp reviewed Dec 17, 2024

View reviewed changes

taraspos force-pushed the taras/amplify-preview-job branch 2 times, most recently from 8ec9db3 to ce44d5b Compare December 17, 2024 17:32

PR feedback, part 2

31cce17

taraspos force-pushed the taras/amplify-preview-job branch from ce44d5b to 31cce17 Compare December 17, 2024 17:39

jimbishopp approved these changes Dec 17, 2024

View reviewed changes

taraspos merged commit 606b252 into main Dec 18, 2024
9 checks passed

taraspos mentioned this pull request Dec 18, 2024

Release amplify-preview v0.0.1 #300

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`amplify-preview`: custom action to trigger amplify previews #287

`amplify-preview`: custom action to trigger amplify previews #287

taraspos commented Nov 6, 2024 •

edited

Loading

fheinecke left a comment

This comment was marked as outdated.

taraspos Dec 16, 2024

jimbishopp left a comment

jimbishopp Dec 16, 2024

taraspos Dec 16, 2024

jimbishopp Dec 16, 2024

taraspos Dec 16, 2024

jimbishopp Dec 16, 2024

taraspos Dec 17, 2024

jimbishopp Dec 17, 2024

taraspos Dec 17, 2024

jimbishopp Dec 17, 2024

jimbishopp Dec 17, 2024

jimbishopp Dec 17, 2024

taraspos Dec 17, 2024

	func parseFlags() (flags, error) {
	var (
	workflow = flag.String("workflow", "", "specific workflow to run [assign, check, dismiss, backport]")
	token = flag.String("token", "", "GitHub authentication token")
	reviewers = flag.String("reviewers", "", "reviewer assignments")
	local = flag.Bool("local", false, "local workflow dry run")
	org = flag.String("org", "", "GitHub organization (local mode only)")
	repo = flag.String("repo", "", "GitHub repository (local mode only)")
	prNumber = flag.Int("pr", 0, "GitHub pull request number (local mode only)")
	branch = flag.String("branch", "", "GitHub backport branch name (local mode only)")
	baseStats = flag.String("base", "", "the artifact sizes as generated by binary-sizes to compare against for bloat")
	buildDir = flag.String("builddir", "", "an absolute path to a build directory containing artifacts to be checked for bloat")
	artifacts = flag.String("artifacts", "", "a comma separated list of compile artifacts to analyze for bloat")
	)

amplify-preview: custom action to trigger amplify previews #287

amplify-preview: custom action to trigger amplify previews #287

Conversation

taraspos commented Nov 6, 2024 • edited Loading

Summary

Context

Testing

Footnotes

fheinecke left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

Choose a reason for hiding this comment

jimbishopp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

`amplify-preview`: custom action to trigger amplify previews #287

`amplify-preview`: custom action to trigger amplify previews #287

taraspos commented Nov 6, 2024 •

edited

Loading