Arquillian Governor Extension

Arquillian Governor Extension and the implementations which use its API give you the possibility to programmatically choose what test methods of your Arquillian tests are going to be executed and what are going to be skipped by putting your custom annotations on the test methods.

The resolution of the test method execution is done during the test class execution in BeforeClass phase.

Currenly, implementations which use API of the Arquillian Governor extension are Arquillian JIRA Governor extension, Arquillian GitHub Governor extension, Arquillian Redmine Governor extension and Arquillian Skipper extension.

Configuration of Arquillian Governor extension

Arquillian Governor extension is an extension on top of which all other concrete governors are built. When you provide some custom Governor implementation, this extension is automatically behind it. Arquillian Governor base implementation is configured by putting the following extension definition into arquillian.xml like this:

<extension qualifier="governor">
</extension>

Available properties for governor extension:

Configuration property	Description	Default value	Possible values
ignore	No test execution resolution will be done. All test methods, even annotated with @Governor annotation, will be executed as normally.	false	true / false
ignoreOnly	Fully qualified class name of @Governor annotation to ignore. When such annotation is put on a test method, no test execution resolution will be performed.	empty string	Fully qualified class name of an annotation you want to ignore.

Arquillian JIRA Governor Extension

In order to use JIRA Governor extension, you have to include following dependency (example for Maven):

<dependency>
    <groupId>org.arquillian.extension</groupId>
    <artifactId>arquillian-governor-jira</artifactId>
    <scope>test</scope>
</dependency>

As stated previously, JIRA Governor has the base implementation automatically bundled in it so you can configure governor extension as well.

Configuration of Arquillian JIRA Governor Extension

Arquillian JIRA Governor extension is configured by putting the following extension definition into arquillian.xml like this:

<extension qualifier="governor-jira">
</extension>

Available properties for governor-jira extension:

Configuration property	Description	Default value	Possible values
username	your JIRA user name. This user name will be used to log in to JIRA instance	this property has to be set and has to be non-empty, otherwise it fails
password	your JIRA password. This password will be used to log in to JIRA instance	this property has to be set and has to be non-empty, otherwise it fails
server	JIRA server to connect to	https://issues.jboss.org	any address which is valid URI and URL
force	Even JIRA status is Unresolved or Open, this test method will be executed anyway	false	true / false
closePassed	If you force the execution for some test method which has status as 'Unresolved' or 'Open' and this test succeeds and this property is set to true, JIRA issue on server will be automatically resolved as 'Done' and appropriate message about the issue resolving will be submitted to that JIRA issue as a comment posted by username.	false	true / false
closingMessage	Message which appears as a comment on a JIRA issue which was automatically closed.	This JIRA issue was automatically closed by %s with Arquillian Jira Governor extension. ('%s' is replaced by username property).	Any string.

The extension looks for server, username or password properties specified as system properties jira.governor.server, jira.governor.username and jira.governor.password respectively. If system properties are not defined, this extension looks for server, username or password properties in arquillian.xml for JIRA Governor.

You can also set properties force and closePassed by system properties jira.governor.force and jira.governor.closepassed. System properties take precedence over those defined in arquillian.xml.

Usage

Using Governer JIRA extension in your tests is as simple as this:

@RunWith(Arquillian.class)
@RunAsClient
public class TestCase
{
    @Test
    @Jira("ARQ-1907")
    public void test()
    {
        // this test will be run
        // because ARQ-1907 is 'Done'
        // so we assume that this test has to pass as well
    }

    @Test
    @Jira("ARQ-5000")
    public void test2()
    {
        // when this JIRA exists and its status is 'Unresolved' / 'Open'
        // this test method will be skipped because you assume that
        // if you run it, it would fail
    }

}

Automatic closing of your JIRA issues

If some JIRA issue is unresolved, normally, it would be skipped from execution. However if you force it to execute by force flag on @Jira annotation, and this test method passes with success, if you set property closePassed in your arquillian.xml, that JIRA will be automatically marked as Done on JIRA server.

@RunWith(Arquillian.class)
public class AutomaticClosingTestCase
{
    @Test
    @Jira(value = "ARQ-5000", force = true)
    public void automaticClosingTest)
    {
        // when this JIRA exists and its status is 'Unresolved' / 'Open'
        // and you have forced its execution and you set 'closePassed'
        // property in arquillian.xml to 'true',
        // when this test method succeeds, it automatically resolves
        // respective JIRA issue as 'Done'
    }
}

Detectors for environment related to JIRA issues

JIRA issues can be related to some specific environments, e.g. operation system, java implementation, timezone…​ In that case you can create your custom detectors which are necessary to reproduce the specified issue. If those detectors are not met then this issue will not force the test method to be skipped.

@RunWith(Arquillian.class)
public class IssuesWithConditionalTestCases
{
    @Test
    @Jira(value = "ARQ-1000", detector = @Detector(Windows.class))
    public void conditionalTest()
    {
        // This test case will be executed on all operation systems except for Windows.
        // If the current OS is Windows then the status of JIRA issue will be checked.
    }

    @Test
    @Jira(value = "ARQ-2000",
            detector = @Detector(value = { OpenJDK.class, Java_7.class }, strategy = And.class)
    )
    public void multipleConditionsAndStrategyTest()
    {
        // This is similar example with more detectors and merging strategy.
    }
}

Arquillian Recorder Integration with Arquillian Jira Governor Extension

How to get Jira issue, force value, detector and it’s strategy for environment in arquillian reports?

In order to see all issues & detectors in arquillian reports, you have to put arquillian-recorder-reporter dependency in your Arquillian test Maven build.

<dependency>
    <groupId>org.arquillian.extension</groupId>
    <artifactId>arquillian-recorder-reporter-impl</artifactId>
    <version>${version.arquillian.recorder}</version>
    <scope>test</scope>
</dependency>

Configure arquillian-recorder-reporter as your requirement. Follow steps from here - https://github.com/arquillian/arquillian-recorder/wiki/Reporter

You will get jira’s all issues & detector information located in target/arquillian-report.xml report.

Arquillian GitHub Governor Extension

In order to use GitHub Governor extension, you have to include this dependency:

<dependency>
    <groupId>org.arquillian.extension</groupId>
    <artifactId>arquillian-governor-github</artifactId>
    <scope>test</scope>
</dependency>

As stated previously, GitHub Governor has the base implementation automatically bundled in it so you can configure governor extension as well.

Configuration of Arquillian GitHub Governor Extension

Arquillian GitHub Governor extension is configured by putting the following extension definition into arquillian.xml like this:

<extension qualifier="governor-github">
</extension>

Possible properties for governor-github extension:

Configuration property	Description	Default value	Possible values
username	your GitHub user name. This user name will be used to log in to GitHub instance	this property has to be set and has to be non-empty, otherwise it fails if no token property set
password	your GitHub password. This password will be used to log in to GitHub instance	this property has to be set and has to be non-empty, otherwise it fails if no token property set
token	your GitHub personal token. This token will be used to log in to GitHub instance	this property has to be set and has to be non-empty, otherwise it fails if no username and password set
repositoryUser	Username part of a GitHub repository	For example in case of www.github.com/arquillian/arquillian-cube, arquillian is the repository user.
repository	GitHub repository name	For example in case of www.github.com/arquillian/arquillian-cube, arquillian-cube is the repository name.
force	Even GitHub status is Open, this test method will be executed anyway	false	true / false
closePassed	If you force the execution for some test method which has status as 'Open' and this test succeeds and this property is set to true, GitHub issue on server will be automatically resolved as 'Closed' and appropriate message about the issue resolving will be submitted to that GitHub issue as a comment posted by username.	false	true / false
closingMessage	Message which appears as a comment on a GitHub issue which was automatically closed.	This GitHub issue was automatically closed by %s with Arquillian GitHub Governor extension. ('%s' is replaced by username property). If you are using token and username is not set, as of now, it will be set to "unknown"	Any string.

The extension looks for system properties github.governor.repository, github.governor.repositoryuser, github.governor.token, github.governor.username and github.governor.password respectively. If system properties are not defined, this extension looks for repository, repositoryuser, token, username or password properties in arquillian.xml for GitHub Governor.

You can also set properties force and closePassed by system properties github.governor.force and github.governor.closepassed by setting them to "true" or "false". System properties take precedence over those defined in arquillian.xml.

Usage

Let’s have this test case:

@RunWith(Arquillian.class)
@RunAsClient
public class TestCase
{
    @Test
    @GitHub("1")
    public void test()
    {
        //...
    }

}

Automatic closing of your GitHub issues

If some GitHub issue is unresolved, normally, it would be skipped from execution. However if you force it to execute by force flag on @GitHub annotation, and this test method passes with success, if you set property closePassed in your arquillian.xml, that GitHub will be automatically marked as Closed on GitHub server.

@RunWith(Arquillian.class)
public class AutomaticClosingTestCase
{
    @Test
    @GitHub(value = "2", force = true)
    public void automaticClosingTest)
    {
        // ...
    }
}

Detectors for environment related to GitHub issues

GitHub issues can be related to some specific environments, e.g. operation system, java implementation, timezone…​ In that case you can create your custom detectors which are necessary to reproduce the specified issue. If those detectors are not met then this issue will not force the test method to be skipped.

@RunWith(Arquillian.class)
public class IssuesWithConditionalTestCases
{
    @Test
    @GitHub(value = "100", detector = @Detector(Windows.class))
    public void conditionalTest()
    {
        // This test case will be executed on all operation systems except for Windows.
        // If the current OS is Windows then the status of GitHub issue will be checked.
    }

    @Test
    @GitHub(value = "200",
            detector = @Detector(value = { OpenJDK.class, Java_7.class }, strategy = And.class)
    )
    public void multipleConditionsAndStrategyTest()
    {
        // This is similar example with more detectors and merging strategy.
    }
}

Arquillian Recorder Integration with Arquillian GitHub Governor Extension

How to get GitHub issue, force value, detector & it’s strategy for environment in arquillian reports?

In order to see all issues & detectors in arquillian reports, you have to include following dependency:

<dependency>
    <groupId>org.arquillian.extension</groupId>
    <artifactId>arquillian-recorder-reporter-impl</artifactId>
    <version>${version.arquillian.recorder}</version>
    <scope>test</scope>
</dependency>

Configure arquillian-recorder-reporter as your requirement. Follow steps from here - https://github.com/arquillian/arquillian-recorder/wiki/Reporter

You will get github’s all issues & detectors information located in target/arquillian-report.xml report.

Arquillian Redmine Governor Extension

In order to use Redmine Governor extension, you have to put this dependency into your Arquillian test Maven build:

<dependency>
    <groupId>org.arquillian.extension</groupId>
    <artifactId>arquillian-governor-redmine</artifactId>
    <scope>test</scope>
</dependency>

As stated previously, Redmine Governor has the base implementation automatically bundled in it so you can configure governor extension as well.

Configuration of Arquillian Redmine Governor Extension

To use this extension you need to enable Redmine rest api like in image below:

And also will need an apikey. The apikey can be found in your user’s account, see image below:

Arquillian Redmine Governor extension is configured by putting the following extension definition into arquillian.xml like this:

<extension qualifier="governor-redmine">
        <property name="server">http://localhost:10083/</property>
        <property name="apikey">42f0c893f65fc65cab6ddd8eaad4c5029799a7ab</property>
</extension>

Possible properties for governor-redmine extension:

Configuration property	Description	Default value	Possible values
server	The Redmine server address.	this property has to be set and has to be non-empty, otherwise it fails if no property is set	Any string.
apikey	your redmine user apikey.	this property has to be set and has to be non-empty, otherwise it fails if no property is set	Any string.
force	Even if issue status is Open, this test method will be executed anyway	false	true / false
closePassed	If you force the execution for some test method which has status as not 'closed' and this test succeeds and this property is set to true, Redmine issue on server will be automatically resolved as 'Closed' and appropriate message about the issue resolving will be submitted to that Redmine issue as a comment posted by username.	false	true / false
closingMessage	Message which appears as a comment on a Redmine issue which was automatically closed.	This Redmine issue was automatically closed by %s with Arquillian Redmine Governor extension. ('%s' is replaced by username property).	Any string.
openFailed	If this test fails and this property is set to true, Redmine issue on server will be automatically resolved as 'New' and appropriate message about the issue reopened will be submitted to that Redmine issue as a comment posted by username.	false	true / false
openingMessage	Message which appears as a comment on a Redmine issue which was automatically opened. The stacktrace will be also appended to opening message.	This Redmine issue was automatically opened by %s with Arquillian Redmine Governor extension. ('%s' is replaced by username property).	Any string.

The extension looks for system properties redmine.governor.server and redmine.governor.apikey. If system properties are not specified, this extension looks for server and apikey properties in arquillian.xml for Redmine Governor,

You can set properties force, closePassed and openFailed by system properties redmine.governor.force, redmine.governor.closepassed and redmine.governor.openFailed by setting them to "true" or "false". System properties take precedence over those defined in arquillian.xml.

Usage

In order to make your tests dependent on the statuses of corresponding issues in Redmine, simply add following annotation:

@RunWith(Arquillian.class)
public class TestCase
{
    @Test
    @Redmine("1")
    public void test()
    {
        // test code
    }

}

Automatic closing of your Redmine issues

If some Redmine issue is unresolved, normally, it would be skipped from execution. However if you force it to execute by force flag on @Redmine annotation, and this test method passes with success, if you set property closePassed in your arquillian.xml, that issue will be automatically marked as Closed on Redmine server.

@RunWith(Arquillian.class)
public class AutomaticClosingTestCase
{
    @Test
    @Redmine(value = "2", force = true)
    public void automaticClosingTest)
    {
        // ...
    }
}

Here is an issue closed by Governor Redmine:

Close order

Redmine can be configured to have a workflow for issue transitions. In some cases this workflow does not allow closing issues for some issue statuses.

Imagine you have a customised workflow for closing issues, e.g 'New' → 'Executing' → 'close', then you have to provide statuses id order so governor redmine can close your issues.

In the example above the issue have to go to 'executing'(id 2) before going to 'close'(id 5). Just add 'closeOrder' property in arquillian:

<extension qualifier="governor-redmine">
        <property name="closeOrder">2, 5</property>
</extension>

Note	You can retrieve issue status id in 'REDMINE_URL/issue_statuses.xml' address.

Automatic opening of your Redmine issues

If some Redmine issue is closed it will be executed and if your test fails nothing will be updated on Redmine server. However if you set openFailed flag on @Redmine annotation or in arquillian.xml to true, and this test method fails then that issue will be automatically marked as New on Redmine server.

@RunWith(Arquillian.class)
public class AutomaticOpeningTestCase
{
    @Test
    @Redmine(value = "3", openFailed = true)
    public void automaticOpeningTest)
    {
        // ...
    }
}

Here is an issue opened by Governor Redmine:

Setting up Redmine locally

The easiest way to run redmine locally is to use this docker image: https://github.com/sameersbn/docker-redmine

Important

before you create any project go to Administration menu and click on Load default configuration. see this issue for more details.

Arquillian Governor Skipper extension

This extension simply skips test methods which are annotated with @TestSpec. In order to use this extension, put this into pom.xml

<dependency>
    <groupId>org.arquillian.extension</groupId>
    <artifactId>arquillian-governor-skipper</artifactId>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.arquillian.extension</groupId>
    <artifactId>arquillian-recorder-reporter-impl</artifactId>
    <scope>test</scope>
</dependency>

Configuration of this extension is done via governor-skipper qualifier:

<extension qualifier="governor-skipper">
    <property name="plainAdoc">plain_table.adoc</property>
</extension>

This extension introduces just one annotation you put on your test method. Let’s see it in action:

    @Test
    @TestSpec(
        author = "Stefan Miklosovic",
        assertion = "this test should pass",
        feature = "tests if true returns true",
        issue = "ARQ-1",
        prerequisites = "have java",
        status = Status.AUTOMATED,
        steps = "some steps in order to execute this",
        test = "what does this step do"
        )
    public void someTest() {
        Assert.assertTrue(true);
    }

If status is Status.MANUAL, test method will be skipped from the execution. The generated report gives you better overview of your test cases, what they do, which methods are automated and which you have to execute manually.

If plainAdoc property is used, a report will be saved to plain asciidoc table as well. A report will be added into the tree of Arquillian Report output file in every case.

Configuration property	Description	Default value	Possible values
plainAdoc	name of file where plain asciidoc table will be stored	if this is not set, report to plain adoc will be skipped

Arquillian Recorder Integration with Arquillian Redmine Governor Extension

How to get Redmine issues,force value in arquillian reports?

In order to see all issues & force value in arquillian reports, you have to put arquillian-recorder-reporter dependency in your Arquillian test Maven build.

<dependency>
    <groupId>org.arquillian.extension</groupId>
    <artifactId>arquillian-recorder-reporter-impl</artifactId>
    <version>${version.arquillian.recorder}</version>
    <scope>test</scope>
</dependency>

Configure arquillian-recorder-reporter as your requirement. Follow steps from here - https://github.com/arquillian/arquillian-recorder/wiki/Reporter

You will get Redmine’s all issues & force value information located in target/arquillian-report.xml report.

Arquillian Governor Ignore extension

This extension simply ignores (skips) test methods which are listed in arquillian.xml. In order to use this extension, put this into pom.xml

<dependency>
    <groupId>org.arquillian.extension</groupId>
    <artifactId>arquillian-governor-ignore</artifactId>
    <scope>test</scope>
</dependency>

Configuration of this extension is done via governor-ignore qualifier:

<extension qualifier="governor-ignore">
    <property name="expression">com\.foobar\.[.]*</property>
    <property name="methods">org.acme.foo.ListTest#testStrFilter,org.bar.boo.QwertTest#testFoo</property>
    <property name="methods_1">org.arquillian.test.governor.ignore.IgnoreTest#testIgnored</property>
</extension>

The "expression" property is a regexp to which it matches simple method’s fqn (<class-name>#<method-name>).

The "methods" property contains a list of simple fqn’s, separated by comma (,), where any property starting with "methods_" is a separate simple fqn.

Of course all properties are optional. But in order to ignore something, at least one must be defined. ;-)

How do I implement my own Governor?

Glad you asked. Because of Arquillian Governor base extension, you are welcome to code your own test method execution resolver as we did with Arquillian JIRA governor extension.

The governor annotation you want to put on your test method (as we did with @Jira) have to be annotated with @Governor annotation. @Jira annotation looks like this:

@Governor // <--- you have to annotate your own annotation with this
@Retention(RetentionPolicy.RUNTIME)
@Target({ ElementType.METHOD })
@Documented
public @interface Jira
{
    String value() default "";

    boolean force() default false;
}

force field is optional. This is only specific for JIRA extension, maybe your governor will use something completely else.

Once you have your governor annotation in your extension, you have to write your own TestExecutionDecider. TestExecutionDecider implementation has to be registered as a service to your Arquillian extension like this:

public class BugzillaLoadableExtension implements LoadableExtension
{
    @Override
    public void register(ExtensionBuilder builder)
    {
        builder.service(TestExecutionDecider.class, MyTestExecutionDecider.class);
        builder.service(GovernorProvider.class, BugzillaProvider.class);
    }
}

The usage of this SPI is fully explained in official Arquillian Core 1.1.6 release blog post.

As described above, you have to register as a service your own GovernorProvider which provides the decoding annotation like this:

public class BugzillaProvider implements GovernorProvider
{
    @Override
    public Class<? extends Annotation> provides()
    {
        return Bugzilla.class;
    }
}

By doing so, you can use @Bugzilla annotations on your test methods. @Bugzilla annotation has to be itself annotated with @Governor annotation the same way as @Jira annotation is.

Finally, you have to observe ExecutionDecisionEvent where you have to decide if the annotation in this event which is put on some test method is going to be executed or not.

The example how we did this all for Jira Governor is in class JiraTestExecutionDecider.

This is the minimal scenario. Of course, your resolution can be done in a completely different way, you could resolve it against database, file, properties or what ever. It is up to you.

How do I build this extension?

$ ./mvnw clean install

Releasing new version

In order to release new version, execute following Maven command:

./mvnw clean release:prepare release:perform

Then:

1. Run test projects in tests directory and

   1. Bump version of governor dependency to the to-be-released one

   2. Bump other dependencies if needed

   3. For jira-test project provide your credentials and jira server using environment variables

2. Verify the build from staging repository

3. Make sure all JIRAs are closed

4. Release version in JIRA and create next version if not available

5. Promote the build to the stable Maven repository

6. Push commits and tag created by maven-release-plugin to the repository.