Smell free tests #459

jose · 2023-07-11T15:01:01Z

This pull request extends EvoSuite with new secondary criteria that optimize test smell metrics. The paper Automatic Generation of Smell-free Unit Tests published at SBFT'23 provides more details on that.

jose

João Afonso (@fc51111), I've just reviewed your changes to EvoSuite regarding the smell-free project. Any chance you could answer the questions I left? And/Or update your fork that lives in here. Thanks in advance.

jose · 2023-07-11T15:18:00Z

client/src/main/java/org/evosuite/coverage/mutation/Mutation.java

@@ -37,7 +38,9 @@
 *
 * @author Gordon Fraser
 */
-public class Mutation implements Comparable<Mutation> {
+public class Mutation implements Comparable<Mutation>, Serializable {


@fc51111, why does the Mutation class need to be Serializable?

During the test generation process, we wanted to measure the mutation score in specific time intervals, but we kept getting null pointer exceptions. I still have an old image of this error:

To solve this issue, it was necessary to make the "Mutation" class Serializable.

jose · 2023-07-11T15:18:54Z

client/src/main/java/org/evosuite/coverage/mutation/Mutation.java

-    private final BytecodeInstruction original;
+    private final transient BytecodeInstruction original;

-    private final InsnList mutation;
+    private final transient InsnList mutation;

-    private final InsnList infection;
+    private final transient InsnList infection;


Why do all these fields need to be transient?

We had to make the "Mutation" class Serializable. In turn, we made these 3 fields transient because the respective classes were not Serializable (and we decided that it didn't make sense to make them Serializable).

jose · 2023-07-11T15:23:04Z

client/src/main/java/org/evosuite/coverage/mutation/MutationTestFitness.java

-    protected transient Mutation mutation;
+    protected Mutation mutation;


Why does not the mutation field need to be transient?

We had to make the "Mutation" class Serializable, so the "mutation" field does not need to remain transient.

jose · 2023-07-11T15:23:53Z

client/src/main/java/org/evosuite/coverage/rho/RhoTestSuiteSecondaryObjective.java


+        return c1 - c2;
+
+        /*


Remove the following.

I'm afraid I can't remove this. We made this change because we decided that it would be better to normalize the secondary criteria. In our case, we wanted the smelliness of each test smell to vary between 0 and 1.
jose/smell-free-tests-evosuite@677f74d

Regarding "RhoTestSuiteSecondaryObjective.java", instead of just returning 0 or 1, we decided to calculate the smelliness of each test suite, normalize the respective values, and return the difference between the two. We calculate the difference because we considered that a test suite that is a lot less smelly than the other should have more weight than a test that is just a bit less smelly than another.

Removing this would most likely break our test smell optimization process. At the very least, there would be some unexpected results...

Is this change likely to break the rest of the code? Or are there any chances that we can keep it like this?

jose · 2023-07-11T15:24:12Z

client/src/main/java/org/evosuite/ga/Chromosome.java

-import java.util.LinkedHashMap;
-import java.util.Map;
-import java.util.Optional;
+import java.util.*;


I would rather keep the verbose imports.

jose · 2023-07-11T15:25:13Z

client/src/main/java/org/evosuite/testcase/DefaultTestCase.java

@@ -75,7 +75,7 @@ public class DefaultTestCase implements TestCase, Serializable {
    /**
     * Coverage goals this test covers
     */
-    private transient Set<TestFitnessFunction> coveredGoals = new LinkedHashSet<>();
+    private final Set<TestFitnessFunction> coveredGoals = new LinkedHashSet<>();


Why does not the coveredGoals need to be transient?

"TestRedundancy.java" (i.e., one of the implemented test smells) uses the method "getCoveredGoals()" of "DefaultTestCase.java":
Set<TestFitnessFunction> testGoals = testChromosome.getTestCase().getCoveredGoals();

We were having problems because "testGoals" was null. In turn, this was happening because "coveredGoals" (a field in "DefaultTestCase.java") was transient, hence the change.

jose · 2023-07-11T15:25:43Z

client/src/main/java/org/evosuite/testcase/ExecutableChromosome.java

@@ -34,7 +34,8 @@ public abstract class ExecutableChromosome<E extends ExecutableChromosome<E>> ex

    private static final long serialVersionUID = 1L;

-    protected transient ExecutionResult lastExecutionResult = null;
+    //protected transient ExecutionResult lastExecutionResult = null;
+    protected ExecutionResult lastExecutionResult = null;


Why does not the lastExecutionResult need to be transient?

The same explanation as "ExecutionResult.java"

jose · 2023-07-11T15:27:32Z

client/src/main/java/org/evosuite/testcase/execution/ExecutionResult.java

 import java.util.*;
 import java.util.stream.IntStream;

-public class ExecutionResult implements Cloneable {
+public class ExecutionResult implements Cloneable, Serializable {


Why does the ExecutionResult class need to be Serializable?

The class "RottenGreenTests" (i.e., one of the implemented test smells) has the following line of code:
ExecutionResult lastExecutionResult = chromosome.getLastExecutionResult();

where "chromosome" is of type "TestChromosome". Originally, in "ExecutableChromosome.java" (where the "getLastExecutionResult()" method is implemented), we had the following field:
protected transient ExecutionResult lastExecutionResult = null;

We were having problems because the variable "lastExecutionResult" in "RottenGreenTests.java" was null. Therefore, it was necessary to make 2 changes:

1 - In "ExecutableChromosome.java", remove "transient" from the field:
protected ExecutionResult lastExecutionResult = null;

2 - Make ExecutionResult Serializable

fc51111 added 30 commits November 24, 2021 23:56

Test smells as secondary criterion V1

49723bb

Test smells secondary criterion structure

61a7904

Added Indirect Testing smell

bc7b95d

Added ObscureInlineSetup smell

c463940

Added EmptyTest smell

92ee33e

Added SlowTests smell

62bd4c5

DummySystemTest

65544fb

AssertionRoulette and UnknownTest

b108890

BrittleAssertion and UnusedInputs

ff0f809

DuplicateAssert and RedundantAssertion

6efd28e

Update to include test suites

7419c7f

EagerTest and IndirectTesting fix

16ce6c9

Lack of Cohesion of Methods

7ae1011

Count test suite smells in abstract

7a138a6

MysteryGuest and LazyTest

47bba20

SensitiveEquality

b8d3254

LikelyIneffectiveObjectComparison

d3b3a8e

Write test smell info to statistics file

5dcaa72

Set smell name inside respective constructor

9e0fbec

Experimental unit test for test smells

a65fb39

Requested changes

86d8009

Change test smell names

2237b86

Test Smell Timeline Variables

70367b0

The first completed unit test

8a6a8bf

Overreferencing & Resource Optimism

9fa2c81

Testing classes

c6aca39

Update EagerTest and respective unit test

3fdd80e

Indirect Testing correction

c80838e

Chromosomes manage the respective smell metrics

6061077

TestCodeDuplication & RottenGreenTests

7eb2500

fc51111 and others added 29 commits May 2, 2022 23:50

Update delta of asserts & description update

7bca87f

Include verbose with other smells

4e73029

Combine test smells (unstable)

5f12a1b

Removed option: "TEST_SMELL" secondary criteria

f8ee30c

Secondary criteria normalized

677f74d

Smell values clone correction

b05930f

Added serialVersion & getSmellType secondary objective

17b6db4

Updated several smell descriptions

17af78d

Update smell computation descriptions V1

9f0408e

Update all smells computation descriptions

b53a5bd

Print smells before minimization

73f9b73

Improved version of the dummy test

8ce65f7

Runtime variables update & corrections

55c23fb

BrittleAssertion & UnusedInputs description update

af2e3cc

Print smells before post-processing

da292b8

Android (Paper): Update smell descriptions

c979bf9

Small updates: smell descriptions

2d6d197

LazyTest update

e8d7938

Smells before post-processing (inside method)

e55ff88

Update smell descriptions (avoid conflicts)

3894143

Added the tiebreaker for the secondary criteria

78a009a

Print smells for each test case

65fe996

Print smells for each test case V2

e5fc0a7

Removed tiebreakers

8b74a5a

Updates before RQ0

5849c63

Random secondary criterion

5ae200c

Use Randomness instead of Random

c82e320

Removed some commented out code

df982fb

Merge remote-tracking branch 'origin/master' into smell-free-tests

3b4fa71

jose commented Jul 11, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Smell free tests #459

Smell free tests #459

jose commented Jul 11, 2023

jose left a comment

jose Jul 11, 2023

fc51111 Oct 22, 2023

jose Jul 11, 2023

fc51111 Oct 22, 2023

jose Jul 11, 2023

fc51111 Oct 22, 2023

jose Jul 11, 2023

fc51111 Oct 23, 2023

jose Jul 11, 2023

fc51111 Oct 23, 2023

jose Jul 11, 2023

fc51111 Oct 22, 2023

jose Jul 11, 2023

fc51111 Oct 22, 2023

jose Jul 11, 2023

fc51111 Oct 22, 2023

		protected transient Mutation mutation;
		protected Mutation mutation;

Smell free tests #459

Are you sure you want to change the base?

Smell free tests #459

Conversation

jose commented Jul 11, 2023

jose left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment