Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easy to run a single test #178

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

raghavendra-talur
Copy link
Contributor

Describe what this PR does

This PR moves the config init code to the main function from the global init. Doing so, it allows for such config init to be skipped for tests or other scenarios.

In the second commit, we set the os environment variable for the operator namespace in the BeforeSuite() function if it is not already set. This allows users to run a specific test by using the ginkgo cli or by using the test decorators in vscode.

For example:

$ ginkgo -focus "ClientProfile Controller" -r
[1732083376] Controller Suite - 1/3 specs SS• SUCCESS! 5.797831s PASS
  Starting ceph-csi-operator suite
[1732083376] e2e suite - 0/1 specs S SUCCESS! 59.25µs PASS

Ginkgo ran 2 suites in 10.605577042s
Test Suite Passed
Screenshot 2024-11-20 at 1 24 33 AM

Provide some context for the reviewer

Is there anything that requires special attention

Earlier, the configuration was automatically inited. Now, it is being done manually in the main function.

Is the change backward compatible?
Yes

Are there concerns around backward compatibility?
No

Checklist:

  • Commit Message Formatting: Commit titles and messages follow
    guidelines in the developer
    guide
    .
  • Reviewed the developer guide on Submitting a Pull
    Request
  • Pending release
    notes

    updated with breaking and/or notable changes for the next major release.
  • Documentation has been updated, if necessary.
  • Unit tests have been added, if necessary.
  • Integration tests have been added, if necessary.

@raghavendra-talur raghavendra-talur force-pushed the rtalur-improve-tests branch 2 times, most recently from 1dd9ce4 to 7450989 Compare November 20, 2024 06:35
@leelavg
Copy link
Contributor

leelavg commented Nov 20, 2024

Isn't only second commit is enough for the required behavior, assuming the behavior is to have operator namespace to be set? First commit just delays the initialization w/o any functionality change, isn't it?

@raghavendra-talur
Copy link
Contributor Author

Isn't only second commit is enough for the required behavior, assuming the behavior is to have operator namespace to be set? First commit just delays the initialization w/o any functionality change, isn't it?

Good question. If I run the test with just the commit that sets the env var then I see this error:

$ ginkgo -focus "ClientProfile Controller" -r
panic: Required OPERATOR_NAMESPACE environment variable is either missing or empty

goroutine 1 [running]:
github.com/ceph/ceph-csi-operator/internal/controller.init.func1(...)
        /Users/rtalur/src/github.com/raghavendra-talur/ceph-csi-operator/internal/controller/defaults.go:71
github.com/ceph/ceph-csi-operator/internal/utils.Call[...](...)
        /Users/rtalur/src/github.com/raghavendra-talur/ceph-csi-operator/internal/utils/core.go:121
github.com/ceph/ceph-csi-operator/internal/controller.init()
        /Users/rtalur/src/github.com/raghavendra-talur/ceph-csi-operator/internal/controller/defaults.go:68 +0x3c8

Ginkgo ran 2 suites in 6.842060084s

There were failures detected in the following suites:
  controller ./internal/controller

Test Suite Failed

The reason is that these variables were being assigned at the global scope by calling a function.

var operatorNamespace = utils.Call(func() string {
        namespace := os.Getenv("OPERATOR_NAMESPACE")
        if namespace == "" {
                panic("Required OPERATOR_NAMESPACE environment variable is either missing or empty")
        }
        return namespace
})

This makes these function calls part of the Go Init() process. The setting of env var in suite_test.go would be run after the Init() process but that would be too late as this function panics on not finding the env var set.

@@ -66,6 +67,11 @@ var _ = BeforeSuite(func() {
fmt.Sprintf("1.29.0-%s-%s", runtime.GOOS, runtime.GOARCH)),
}

_, set := os.LookupEnv("OPERATOR_NAMESPACE")
if !set {
Expect(os.Setenv("OPERATOR_NAMESPACE", "ceph-csi-operator-system")).To(Succeed())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the AfterSuit lets unset this env if we are setting it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

)

func InitConfig() {
if operatorNamespace := os.Getenv("OPERATOR_NAMESPACE"); operatorNamespace == "" {
Copy link
Contributor

@leelavg leelavg Nov 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the walrus shawdowing global operatorNamespace var here? Similar for others as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thanks!

@leelavg
Copy link
Contributor

leelavg commented Nov 21, 2024

Possible to hold off the merge till tomorrow? pls excuse I missed checking this today.

The approach isn't sitting well w/ me for a couple of reasons, BTW open to be corrected.

  1. We are creating an indirect dependency on the pkg that something MUST be done in addition to importing it for controllers to work
  2. Amending a fundamental requirement to be able to run test, kinda inverted.

@raghavendra-talur
Copy link
Contributor Author

Possible to hold off the merge till tomorrow? pls excuse I missed checking this today.

The approach isn't sitting well w/ me for a couple of reasons, BTW open to be corrected.

  1. We are creating an indirect dependency on the pkg that something MUST be done in addition to importing it for controllers to work
  2. Amending a fundamental requirement to be able to run test, kinda inverted.

Yes, I am in no hurry to get this merged. I will reply with examples to answer the above questions.

@leelavg
Copy link
Contributor

leelavg commented Nov 22, 2024

Looked again, the fix introduces a bug in both test & runtime environments which I already left a comment.

I couldn't find any simpler alternatives, I don't use the feature (vscode & ginkgo) which requires this fix and leave the decision to maintainers.

BTW, see if t.Setenv helps in anyway which removes explicit cleanup.

@raghavendra-talur
Copy link
Contributor Author

raghavendra-talur commented Nov 22, 2024

The approach isn't sitting well w/ me for a couple of reasons, BTW open to be corrected.

  1. We are creating an indirect dependency on the pkg that something MUST be done in addition to importing it for controllers to work

This is not true. Just importing the controllers package in main.go wasn't enough even before. The SetupWithManager() has to be invoked for every controller in the package, which are 3 at this time, ClientProfileMappingReconciler, ClientProfileReconciler, and DriverReconciler.

  1. Amending a fundamental requirement to be able to run test, kinda inverted.

I don't understand which requirement is amended here?

@raghavendra-talur raghavendra-talur force-pushed the rtalur-improve-tests branch 4 times, most recently from 5fc229d to 576e374 Compare November 22, 2024 06:20
This will make it easier to setup the test framework with a different
config when required. If the configuration is built in the global init
of the controller then it can not be skipped.

Signed-off-by: Raghavendra Talur <[email protected]>
This allows one to run the a specific test using the ginkgo cli or by
using the vscode test decorators.

For example:
```
$ ginkgo -focus "ClientProfile Controller" -r
[1732083376] Controller Suite - 1/3 specs SS• SUCCESS! 5.797831s PASS
  Starting ceph-csi-operator suite
[1732083376] e2e suite - 0/1 specs S SUCCESS! 59.25µs PASS

Ginkgo ran 2 suites in 10.605577042s
Test Suite Passed
```

Signed-off-by: Raghavendra Talur <[email protected]>
@raghavendra-talur
Copy link
Contributor Author

raghavendra-talur commented Nov 22, 2024

Ok, the tests have passed but now I agree with @leelavg that it is starting to look like a big change for a project which I am not the maintainer.

@nb-ohad @Madhu-1 I will leave it up to you to merge or close the PR.

Ginkgo and VSCode both provide users a way to specify an env var file and that is one alternative option for the users.

@leelavg
Copy link
Contributor

leelavg commented Nov 22, 2024

Just importing the controllers package in main.go wasn't enough even before.

  • let me extend what I meant. Making it mandatory to run controllers.Init() for a side effect isn't exactly same as setting up controller. This creates the coupling b/n all controllers and this init as this is manipulating a var at global scope. It's kinda fine atm since this is done before setup of controllers as they can ask for concurrent reconciles.
  • with existing code one can choose to not setup a controller at all but always forced to do controller.Init()

which requirement is amended here?

  • I mean, we usually test what is expected to run and here we are changing runtime requirements to be able to test it unlike the usage of stubs, fakes, doubles, mocks etc (I'm not emphasizing on these constructs but only pointing methodologies)

having said that, I acknowledge the usage of global vars which might be used in more than one place is problematic in existing code and I feel proposed fix isn't easing it much.

@Madhu-1
Copy link
Collaborator

Madhu-1 commented Nov 26, 2024

Ginkgo and VSCode both provide users a way to specify an env var file and that is one alternative option for the users.

@raghavendra-talur we are reworking on the new framework for the testing, Lets keep this PR open for sometime and we can see if we can get this in or we need any additional changes later on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants