Heuristic jesse (gousiosg#13)

* Started commit for heuristic processing

* Update configs

* Add heuristic class GetBest with integration to the Test step for JCallGraph

* Update GetBest to update color values, add hash for compare, and update algo used for path sum (to avoid infinite loop with previous all paths algo).

* Update mph-table.yaml

* Started commit for heuristic processing

* Update configs

* Add heuristic class GetBest with integration to the Test step for JCallGraph

* Update GetBest to update color values, add hash for compare, and update algo used for path sum (to avoid infinite loop with previous all paths algo).

* Update mph-table.yaml

* Added preliminary unit tests

* Failsafe plugin added to pom file

* Added jetty plugin to start server for integration testing

* Added RepoTool integration test, modified POM to reflect int-test stage

* Set 'skipTests' parameter to false to run int tests

* Updated Integration Tests

* Added Convex Integration Test

* Added 'runOrder' parameter in POM for Int Tests, Updated Integration tests to sequentially run and added assert statements

* Removed unnecessary test files on local branch

* Update with patched items based on feedback

* Add updates to GetBest score calculation and fix issue with output directory

* Update patch/yaml names to have correct extensions

* Add runall.sh and buildsvg.sh

* Add runall.sh and buildsvg.sh

* Updated MphTableIT to run without being platform specific

* Resolved ordering issue

* Started commit for heuristic processing

* Update configs

* Add heuristic class GetBest with integration to the Test step for JCallGraph

* Update GetBest to update color values, add hash for compare, and update algo used for path sum (to avoid infinite loop with previous all paths algo).

* Update mph-table.yaml

* Update with patched items based on feedback

* Add updates to GetBest score calculation and fix issue with output directory

* Update patch/yaml names to have correct extensions

* Add runall.sh and buildsvg.sh

* Add runall.sh and buildsvg.sh

* Started commit for heuristic processing

* Update configs

* Add heuristic class GetBest with integration to the Test step for JCallGraph

* Update GetBest to update color values, add hash for compare, and update algo used for path sum (to avoid infinite loop with previous all paths algo).

* Update mph-table.yaml

* Started commit for heuristic processing

* Update configs

* Add heuristic class GetBest with integration to the Test step for JCallGraph

* Update GetBest to update color values, add hash for compare, and update algo used for path sum (to avoid infinite loop with previous all paths algo).

* Update mph-table.yaml

* Update with patched items based on feedback

* Add updates to GetBest score calculation and fix issue with output directory

* Update patch/yaml names to have correct extensions

* Add runall.sh and buildsvg.sh

* Add runall.sh and buildsvg.sh

* Started commit for heuristic processing

* Update configs

* Add heuristic class GetBest with integration to the Test step for JCallGraph

* Update GetBest to update color values, add hash for compare, and update algo used for path sum (to avoid infinite loop with previous all paths algo).

* Update mph-table.yaml

* Update with patched items based on feedback

* Add updates to GetBest score calculation and fix issue with output directory

* Update patch/yaml names to have correct extensions

* Move fixed to new "project" directory (e.g. convex-fixed).

* Add green/yellow/red arrows for paths.

* Update so buildyaml looks for @Property attributes instead of just junit Test attribute

* Add basic documentation to the read me

* Rename files

* commit runAll

* Update yaml patch names for fixed

* Remove unneeded configs

Co-authored-by: ameka4 <[email protected]>
Co-authored-by: alekh <[email protected]>
Co-authored-by: Alekh Meka <[email protected]>

README.md

@@ -76,6 +76,12 @@ There are command line options that can be used:
 | jcg.includeExceptionBasicBlocks | Set to "true" to included exception basic blocks, otherwise these will be excluded by default | -Djcg.includeExceptionBasicBlocks=true |
 
 
+#### Heuristic
+The "test" phase has been updated to calculate and append a heuristic value to each node. The heuristic value represents the likelihood that a property could be improved 
+through altering the generator and/or test case.  A higher value indicates greater chance of improvement, so the paths with the top three values should be explored first. 
+The "test" phase will generate another dot file that has the heuristic value prepended to the name of each node.
+
+
 ## Known Restrictions
 
 * The static call graph generator does not account for methods invoked via reflection.
@@ -84,6 +90,7 @@ There are command line options that can be used:
 
 Georgios Gousios <[email protected]>  
 Will Cygan <[email protected]>
+Jesse Coultas <[email protected]>
 
 ### License
 

artifacts/configs/convex-fixed/convex-fixed.patch

@@ -0,0 +1,156 @@
+diff --git a/convex-core/pom.xml b/convex-core/pom.xml
+index 471f81bf..3f83db91 100644
+--- a/convex-core/pom.xml
++++ b/convex-core/pom.xml
+@@ -32,6 +32,59 @@
+ 					</execution>
+ 				</executions>
+ 			</plugin>
++			<plugin>
++				<groupId>org.jacoco</groupId>
++				<artifactId>jacoco-maven-plugin</artifactId>
++				<version>0.8.6</version>
++				<executions>
++					<execution>
++						<id>default-prepare-agent</id>
++						<goals>
++							<goal>prepare-agent</goal>
++						</goals>
++					</execution>
++					<execution>
++						<id>jacoco-report</id>
++						<phase>test</phase>
++						<goals>
++							<goal>report</goal>
++						</goals>
++					</execution>
++				</executions>
++			</plugin>
++			<plugin>
++				<groupId>org.apache.maven.plugins</groupId>
++				<artifactId>maven-assembly-plugin</artifactId>
++				<version>3.3.0</version>
++
++				<configuration>
++					<descriptorRefs>
++						<descriptorRef>jar-with-dependencies</descriptorRef>
++					</descriptorRefs>
++				</configuration>
++
++				<executions>
++					<execution>
++						<id>make-assembly</id>
++						<phase>package</phase>
++						<goals>
++							<goal>single</goal>
++						</goals>
++					</execution>
++				</executions>
++			</plugin>
++			<plugin>
++				<groupId>org.apache.maven.plugins</groupId>
++				<artifactId>maven-jar-plugin</artifactId>
++				<version>3.2.0</version>
++				<executions>
++					<execution>
++						<goals>
++							<goal>test-jar</goal>
++						</goals>
++					</execution>
++				</executions>
++			</plugin>
+ 		</plugins>
+ 		<resources>
+ 			<resource>
+@@ -44,6 +97,22 @@
+ 		</resources>
+ 	</build>
+
++	<reporting>
++		<plugins>
++			<plugin>
++				<groupId>org.jacoco</groupId>
++				<artifactId>jacoco-maven-plugin</artifactId>
++				<reportSets>
++					<reportSet>
++						<reports>
++							<report>report</report>
++						</reports>
++					</reportSet>
++				</reportSets>
++			</plugin>
++		</plugins>
++	</reporting>
++
+ 	<dependencies>
+ 		<dependency>
+ 			<groupId>org.bouncycastle</groupId>
+diff --git a/convex-core/src/test/java/convex/test/generators/PrimitiveGen.java b/convex-core/src/test/java/convex/test/generators/PrimitiveGen.java
+index 1eba23e1..045f17d6 100644
+--- a/convex-core/src/test/java/convex/test/generators/PrimitiveGen.java
++++ b/convex-core/src/test/java/convex/test/generators/PrimitiveGen.java
+@@ -5,6 +5,7 @@ import com.pholser.junit.quickcheck.generator.Generator;
+ import com.pholser.junit.quickcheck.random.SourceOfRandomness;
+
+ import convex.core.data.ACell;
++import convex.core.data.Blob;
+ import convex.core.data.prim.CVMBool;
+ import convex.core.data.prim.CVMByte;
+ import convex.core.data.prim.CVMChar;
+@@ -16,6 +17,8 @@ import convex.core.data.prim.CVMLong;
+  */
+ public class PrimitiveGen extends Generator<ACell> {
+ 	public final static PrimitiveGen INSTANCE = new PrimitiveGen();
++	private static final int ONE_KB = 1024;
++	private static final int ONE_MB = ONE_KB * 1024;
+
+ 	// public final Generator<Byte> BYTE = gen().type(byte.class);
+
+@@ -25,7 +28,7 @@ public class PrimitiveGen extends Generator<ACell> {
+
+ 	@Override
+ 	public ACell generate(SourceOfRandomness r, GenerationStatus status) {
+-		int type = r.nextInt(6);
++		int type = r.nextInt(7);
+ 		switch (type) {
+ 		case 0:
+ 			return null;
+@@ -39,8 +42,38 @@ public class PrimitiveGen extends Generator<ACell> {
+ 			return CVMDouble.create(r.nextDouble());
+ 		case 5:
+ 			return CVMBool.create(r.nextBoolean());
++		case 6:
++			return Blob.create(r.nextBytes(getByteSize(r)));
+ 		default:
+ 			throw new Error("Unexpected type: " + type);
+ 		}
+ 	}
++
++	private static int getByteSize(SourceOfRandomness r) {
++		int rnd = r.nextInt(1, 100);
++
++		// 1% change of getting number 0
++		if (rnd == 1) {
++			return 0;
++		}
++
++		// 1% change of getting number 1
++		if (rnd == 2) {
++			return 1;
++		}
++
++		// 15% chance of getting a "larger" size
++		if (rnd >= 3 && rnd <= 17) {
++			return r.nextInt(ONE_KB * 4, ONE_KB * 100);
++		}
++
++		// 5% chance of getting a "huge" size
++		if (rnd >= 18 && rnd <= 22) {
++			return r.nextInt(ONE_KB * 100, ONE_MB);
++		}
++
++		// 78% - normalish size
++		return r.nextInt(2, ONE_KB * 4);
++	}
++
+ }

artifacts/configs/convex-fixed/convex-fixed.yaml

@@ -0,0 +1,14 @@
+name: convex-fixed
+URL: https://github.com/Convex-Dev/convex.git
+checkoutID: e6db05a611cd4a1fb51f959e20d246637bb7744a
+patchName: artifacts/configs/convex-fixed/convex-fixed.patch
+subProject: convex-core
+#mvnOptions: -DfailIfNoTests=false -Dmaven.surefire.debug="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000"
+mvnOptions: -DfailIfNoTests=false
+properties:
+  - name: GenTestFormat#messageRoundTrip
+    entryPoint: "convex.comms.GenTestFormat.messageRoundTrip(Ljava/lang/String;)V"
+  - name: GenTestFormat#primitiveRoundTrip
+    entryPoint: "convex.comms.GenTestFormat.primitiveRoundTrip(Lconvex/core/data/ACell;)V"
+  - name: GenTestFormat#dataRoundTrip
+    entryPoint: "convex.comms.GenTestFormat.dataRoundTrip(Lconvex/core/data/ACell;)V"

artifacts/configs/convex/convex.yaml

@@ -6,9 +6,9 @@ subProject: convex-core
 #mvnOptions: -DfailIfNoTests=false -Dmaven.surefire.debug="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000"
 mvnOptions: -DfailIfNoTests=false
 properties:
-#  - name: GenTestFormat#messageRoundTrip
-#    entryPoint: "convex.comms.GenTestFormat.messageRoundTrip(Ljava/lang/String;)V"
+  - name: GenTestFormat#messageRoundTrip
+    entryPoint: "convex.comms.GenTestFormat.messageRoundTrip(Ljava/lang/String;)V"
   - name: GenTestFormat#primitiveRoundTrip
     entryPoint: "convex.comms.GenTestFormat.primitiveRoundTrip(Lconvex/core/data/ACell;)V"
-#  - name: GenTestFormat#dataRoundTrip
-#    entryPoint: "convex.comms.GenTestFormat.dataRoundTrip(Lconvex/core/data/ACell;)V"
+  - name: GenTestFormat#dataRoundTrip
+    entryPoint: "convex.comms.GenTestFormat.dataRoundTrip(Lconvex/core/data/ACell;)V"