Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create integration tests #572

Open
hohwille opened this issue Aug 30, 2024 · 2 comments
Open

Create integration tests #572

hohwille opened this issue Aug 30, 2024 · 2 comments
Labels
enhancement New feature or request Epic something big that needs to be split into smaller stories (issues)

Comments

@hohwille
Copy link
Member

hohwille commented Aug 30, 2024

Bug #570 was a real blocker and an eye opener that we need integration tests.
I wanted to wait till we have implemented more of the required core features before we work on integration test but now that I saw how easily we have green tests for a totally broken product, I think it is time to introduce integration tests.

IMHO with minimal invest, we can introduce very simple integration tests.
Already by calling ide help or ideasy help we could have revealed this error before the merge.

Some guidance for writing integration tests with Java and Maven:

  • First of all to understand the context: Unlike a regular JUnit test, here in this context we mean with integration test that our product is already build (mvn package) and after that we run tests using the resulting package that has been build. Regular JUnit tests are executed by surefire-plugin before the package is build and if they fail, the build is aborted and no package will be build at all.
  • in general if we want to do something with JUnit on an integration test level, failsafe-plugin is the way to go. It will run JUnits using IT instead of Test as suffix.
  • as a minimal test, we could even use maven-exec-plugin to run IDEasy in a non-mocked productive way and trigger some trivial usecases like ideasy help or ideasy create demoproject. For the letter we need to prepare a real IDEasy installation and an IDE_ROOT variable in target somehow. This way we would only verify that this is running with exit code 0 and otherwise break the build. But for making real tests with assertions this is not the way to go. It just allows with minimal effort to have something that is better than nothing.
  • as a maximum test, we would include the native image build and build the full package, install it and then run tests on that. In devonfw-ide we have setup tests like but with pure bash-scripting technology. We do not want to do the same but still it might help to inspire and give a vision of what we want to archive here...
  • By default (mvn install) these new integration tests shall not be executed. They may also turn red if something external breaks but there might be no bug in our product directly. Also they will require more time and resources as well as external network activity (downloads, etc.). Therefore we should define a maven profile to activate this that is triggered in our nightly build.

If somebody can create a PR for the minimal approach in a day or less, that would be a nice first step.
Apart from that, I would expect that we create real integration tests (failsafe plugin) and create some boostrapping step before the tests are run or as setup method for each of these tests that "installs" our product somewhere in the target folder (${project.build.directory}) and can then run the test on top of this.
JUnit is not really great for testing bash and CMD executions so we would need some test-infrastructure to invoke ide commands in a given shell of that pre-setup installation. Still we can discuss if we want to write integration tests in JUnit (or bash or whatever).

Please note that here we do not want to assert lists of files to be present like software/java/lib/security/blocked.certs. Instead a test could look like this:

  • ide create maven-example-project -
  • cd maven-example-project
  • ide set-version java 21.0.4
  • ide install mvn 3.9.7
  • mvn -v
  • now we assert that the output of that last comment contains the correct Java and Maven version
@hohwille hohwille added the enhancement New feature or request label Aug 30, 2024
@github-project-automation github-project-automation bot moved this to 🆕 New in IDEasy board Aug 30, 2024
@hohwille hohwille moved this from 🆕 New to Refinement in IDEasy board Aug 30, 2024
@hohwille hohwille added the Epic something big that needs to be split into smaller stories (issues) label Sep 3, 2024
@hohwille
Copy link
Member Author

hohwille commented Sep 3, 2024

I created story #583 to get started with this epic.

@hohwille
Copy link
Member Author

hohwille commented Sep 6, 2024

For the record:
There is also https://testcontainers.com/
At least for linux there are free images of e.g. ubunut. We could run automated integration tests on linux that would work on any operating system with docker installed.
Just an idea... Still not sure if we want to do this...

@hohwille hohwille moved this from Refinement to 🆕 New in IDEasy board Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Epic something big that needs to be split into smaller stories (issues)
Projects
Status: 🆕 New
Development

No branches or pull requests

1 participant