Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed the init_test by comparing #58

Merged
merged 5 commits into from
Oct 17, 2024
Merged

Conversation

7h3-3mp7y-m4n
Copy link
Contributor

Fixed the main_init_test.go

Description

What does this PR do?

It just fixed the uncertainty error we get while testing

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • New chore (expected functionality to be implemented)

Checklist:

  • My change requires a change to the documentation.
    • I have updated the documentation accordingly.
  • I've signed off with an email address that matches the commit author.

@7h3-3mp7y-m4n
Copy link
Contributor Author

after running the or i in {1..10}; do echo "Run #$i" >> test.txt go test -timeout 30s -run ^TestInitWithProvisioners$ -count=1 github.com/score-spec/score-k8s >> test.txt 2>&1 done
it passed

@mathieu-benoit
Copy link
Contributor

mathieu-benoit commented Oct 17, 2024

Thanks, @7h3-3mp7y-m4n!

We have been seeing random issue in the CI with this unit tests, like here for example:https://github.com/score-spec/score-k8s/actions/runs/11355384862/job/31584604052:

--- FAIL: TestInitWithProvisioners (0.06s)
    main_init_test.go:153: 
        	Error Trace:	/home/runner/work/score-k[8](https://github.com/score-spec/score-k8s/actions/runs/11355384862/job/31584604052#step:5:9)s/score-k8s/main_init_test.go:153
        	Error:      	Not equal: 
        	            	expected: "template://two"
        	            	actual  : "template://one"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-template://two
        	            	+template://one
        	Test:       	TestInitWithProvisioners
    main_init_test.go:[15](https://github.com/score-spec/score-k8s/actions/runs/11355384862/job/31584604052#step:5:16)4: 
        	Error Trace:	/home/runner/work/score-k8s/score-k8s/main_init_test.go:154
        	Error:      	Not equal: 
        	            	expected: "template://one"
        	            	actual  : "template://two"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-template://one
        	            	+template://two
        	Test:       	TestInitWithProvisioners

When getting this error, just Re-run the failed jobs was successfully passing then... not very deterministic...

So if this is fixing this issue, this would be great!

I would love your review on this, @delca85, any feedback/comment on this?

@@ -150,7 +151,17 @@ func TestInitWithProvisioners(t *testing.T) {
provs, err := loader.LoadProvisionersFromDirectory(filepath.Join(td, ".score-k8s"), loader.DefaultSuffix)
assert.NoError(t, err)
if assert.Greater(t, len(provs), 2) {
assert.Equal(t, "template://two", provs[0].Uri())
assert.Equal(t, "template://one", provs[1].Uri())
expectedProvisioners := map[string]bool{
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@7h3-3mp7y-m4n sorry for being annoying, what do you think about this way to check that the expected provisioners are there without checking the order but leveraging slices functions:

expectedProvisionerUris := []string{"template://one", "template://two"}
for _, expectedUri := range expectedProvisionerUris {
	assert.True(t, slices.ContainsFunc(provs, func(p provisioners.Provisioner) bool {
		return p.Uri() == expectedUri
	}), fmt.Sprintf("Expected provisioner '%s' not found", expectedUri))
}

?

Copy link
Contributor Author

@7h3-3mp7y-m4n 7h3-3mp7y-m4n Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No no @delca85 you are not annoying, TBH I like your approach, I wrote it this way cause I was debugging at the same time. I added a map because of its fast lookup ... As we are dealing with a small number of inputs cause it's a test we can definitely use your approach, and it's more flexible and has fewer lines .. Thanks!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thank you @7h3-3mp7y-m4n !

@7h3-3mp7y-m4n
Copy link
Contributor Author

I ran it with the script and tested it 15 times and voila, no errors :)
Screenshot 2024-10-17 at 2 50 15 PM

Copy link

@delca85 delca85 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the changes!

@mathieu-benoit mathieu-benoit self-requested a review October 17, 2024 11:27
Copy link
Contributor

@mathieu-benoit mathieu-benoit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @7h3-3mp7y-m4n, for fixing this random issue we got in CI/tests!

And thanks, @delca85, as always, for your prescriptive suggestions and reviews, much appreciated!

LGTM

@mathieu-benoit mathieu-benoit merged commit e00aa70 into score-spec:main Oct 17, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants