0e Development testing

Prerequisites:

Some Java IDE (Eclipse, IntelliJ...)
JUnit 4, Mockito 2 (through Maven)

Goal: get familiar with unit testing, code coverage and isolation with mocks and stubs..

1. Unit tests

In the first part of the exercise, we will analyze and test a simple project that implements a cache. This cache implementation is abstract and is also generic (AbstractCache<K,V>), thus a derived class is needed with concrete generic type parameters for testing purposes. One of the main features of the cache is the time-to-live (TTL) counter. Each element gets a predefined number that indicates its time until the removal from the cache. When an element in the cache is accessed, then its TTL is reset to the initial, while all of the other elements will lose from their TTL value. For the first part of the exercise, we are going to use it with Integers as values and Strings as keys through a simple derived class (TestingCache extends AbstractCache<String,Integer>).

The implementation outline of the previously introduced AbstractCache<K,V> is the following.

Outline of the class under test

The following project structure is available to work (hu.bme.mit.swsv.cache).

src/main/java folder contains the implementation of the cache.
src/test/java folder is dedicated to the test files.

Define at least 3 different test cases for method put(K, V, boolean) based on the previous specification and the source code found in AbstractCache.java!

CHECK: create a screenshot about a table you filled with the relevant cases.

Writing unit tests

We are going to use JUnit 4 to implement and run the test cases. See short introduction to JUnit 4 if you are not already familiar with it.

Implement all of the previously defined test cases for method put(K, V, boolean) into the src/test/java/AbstractCacheTest.java file!
The initialization phase of every test case may be very similar. Migrate the common parts into a setup method that uses the @Before annotation.
If the expected outcome is throwing an exception, then use the @Test(expected = Exception.class) annotation, where the Exception.class is the class of exception you expect.
The users are complaining about the slowness of method remove(K). Create a test case with an appropriate time limit that addresses this issue. Use the @Test(timeout = <value>) annotation for the test case.
You can disable tests with the @Ignore annotation. Use it to disable the above test, which required timeout.

CHECK: Create a screenshot of your final test code in AbstractCacheTest.java.

Parameterized tests

In the following task, we are going to analyze method get(String).

Most of the test cases for this method may only differ in the inputs and the expected outcomes. It is unnecessary to create a method for each test case.

Instead, use parameterized tests (see e.g., tutorial).
Since JUnit 4.11, it is possible to name each case when using parameterized test. Try out this feature too (see this discussion for help).
Alternatively, you can use JUnitParams, an open-source project that aims to alleviate the work with parameterized tests.

Extend the parameterized test class ParameterizedAbstractCacheTest.java found in src/test/java/ParameterizedAbstractCacheTest.java file with at least 3 new test cases. At first, you shall try to get an overview of the process of parameterized tests in JUnit.

CHECK: Create a screenshot of your final test code in ParameterizedAbstractCacheTest.java.

2. Measuring code coverage

One way to decide on the quality of a test suite is to measure the coverage it produces. Note that coverage may not indicate the real quality and bug-finding capability of a test suite, as it was already mentioned during the structure-based test design lecture. During this exercise, we are going to use JaCoCo to measure the coverage of the previously created test cases.

We are going to use the previously implemented tests for this task. The project is already configured through Maven to produce HTML coverage reports, when running the compilation.
Run the verify Maven phase: mvn verify. Note: the JaCoCo report generation goal is bound to the package phase, thus the Maven test phase will not execute it.
Check the produced coverage report (target/site/jacoco/index.html).
Think about how can you improve your different coverage statistics, including the ones JaCoCo cannot measure (e.g., C/DC, MC/DC).

Several commercial tools exist for various programming languages (e.g., CTC++), which are able to measure a wide range of coverage statistics.

CHECK: create a screenshot about the generated HTML coverage report that shows the coverage your test cases achieved.

3. Testing in isolation

One of the most important tasks during unit testing is to ensure that the dependencies of the unit is replaced and it is executed in an isolated environment. Instead of manually creating controllable and observable stubs, we are going to use the Mockito framework to automatically generate them.

During the lecture, we learned that what kind of test doubles are available. In this lab, we are going to use two of them that differ in the way the results of the tests are checked:

stub: used for checking the state or the return value of the SUT,
mock: used for checking the interaction between the SUT and its environment.

Introduction to Mockito

Mockito provides both stubs and mocks.

Checking the state:

// create a test double (there is no separate stub() call)
List mockedList = mock(List.class);
// specify how the test double should respond to calls from the SUT
when( mockedList.get(0) ).thenReturn( "first" );
// call the SUT as in any test
String r = sut.queryList(mockedList, 0);
// assert the state or the return value of the SUT to decide the outcome of the test
assertEquals("first", r);

Checking the interaction:

// create a test double, it automatically records all the calls received
List mockedList = mock(List.class);
sut.setList(mockedList);
// call the SUT as in any test
sut.add(0);
// verify the mock and not the SUT
// check that the SUT sent the required interactions
verify(mockedList).add(0);

In order to compile the previous snippets, one more line is required that imports the static methods of Mockito:

import static org.mockito.Mockito.*;

A crucial step when using Mockito is instantiating a test double using the mock call (e.g., List mockedList = mock(List.class)). It is also possible to mock an interface or a concrete class. This will cause the followings:

all methods of List can be invoked on mockedList,
returns a basic value (e.g., null, 0, empty list) for all non-predefined invocation,
stores all invocations target to mockedList, thus they can be verified later.

In addition, the following features can be useful:

Argument matcher: if the mock is not expecting equality to a specific value.

when(mockedList.get(anyInt())).thenReturn("element");

ArgumentCaptor: checking the arguments passed to the mock.
Checking a predefined number of invocations (times(n),atLeast(n),atMost(n),never()):

verify(mockedList,times(2)).add("twice");

Throwing an exception during a check.

when(mockedList.get(-1)).thenThrow(new Exception());

when rules can override each other (the last one stays active).

Further descriptions can be found in the documentation of Mockito.

Exercises

We have a skeleton that is modelling the behaviour of spaceships (hu.bme.mit.spaceship project). The task is to unit test the firing routine of a concrete spaceship of type GT4500. The definition is the following:

/**
* Tries to fire the torpedo stores of the ship.
*
* @param firingMode how many torpedo bays to fire
* SINGLE: fires only one of the bays.
* - For the first time the primary store is fired.
* - To give some cooling time to the torpedo stores,
torpedo stores are fired alternating.
* - But if the store next in line is empty the ship
tries to fire the other store.
* - If the fired store reports a failure, the ship
does not try to fire the other one.
* ALL: tries to fire both of the torpedo stores.
*
* @return whether at least one torpedo was fired successfully
*/
public boolean fireTorpedos(FiringMode firingMode)

The unit testing of class GT4500 requires the isolation of external dependencies:

Identification of module dependencies.
Can all of the dependencies be influenced during testing? Is it easy to control the unit under test? If not, what are the possible solutions?
Create unit tests for method fireTorpedos based on the description of the module. For isolation, use the Mockito framework.
Try to think through, which one is more appropriate in the current situation: checking the state or checking the interaction? Use the most suitable approach when implementing the task.

CHECK: create and annotate a screenshot about, which one you chose and where did you implement it.

Note for Eclipse: The Spaceship project uses Maven to handle external libraries (such as JUnit, Mockito). Thus, when the class GT4500 must be modified, the project shall be recompiled using Run As --> Maven Clean, then Run As --> Maven Install. When modifying the tests, this step can be omitted.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly