diff --git a/README.md b/README.md index 521416e8..449743f0 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,21 @@ # Native image integration tests -Builds Native image executables for small applications based on some Native image capable runtimes such -as Quarkus and Helidon. It also provides a convenient way of building small test apps for Native image backbox testing. +Builds Native image executables for small applications based on some Native image capable +runtimes such as Quarkus and Helidon. It also provides a convenient way of building small test +apps for Native image blackbox testing. ## Prerequisites -The test suite (TS) expects you run Java 11+ and have ```ps``` program available on your Linux/Mac -and ```wmic``` (by default present) on your Windows system. +The test suite (TS) expects you run Java 11+ and have ```ps``` program available on your +Linux/Mac and ```wmic``` (by default present) on your Windows system. + +Some tests also require ```lsof``` and ```perf``` commands to be available on Linux. + +Depending on the test suite configuration, you might need to have Podman or Docker installed. +For gdb tests, you need to have ```gdb``` installed. Attaching gdb in a container scenario might require sudo privileges. + +Some JFR or performance sensitive tests might ask for sudo to e.g. clear OS cache or +to switch on/off Intel Turbo boost. + Native image builds also require you have the following packages installed: * glibc-devel * zlib-devel @@ -45,7 +55,8 @@ exercises some rudimentary business logic of selected libraries. ### Collect results: -There are several log files archived in `testsuite/target/archived-logs/` after runtimes test execution. e.g.: +There are several log files archived in `testsuite/target/archived-logs/` after runtimes test +execution. e.g.: ``` org.graalvm.tests.integration.RuntimesSmokeTest/quarkusFullMicroProfile/report.md @@ -53,16 +64,19 @@ org.graalvm.tests.integration.RuntimesSmokeTest/quarkusFullMicroProfile/build-an org.graalvm.tests.integration.RuntimesSmokeTest/quarkusFullMicroProfile/measurements.csv ``` -`report.md` is human readable description of what commands were executed and how much time and memory was spent to -get the expected output form the runtime's web server. `measurements.csv` contains the same data in a machine friendly form. -Last but not least, `build-and-run.log` journals the whole history of all commands and their output. +`report.md` is human-readable description of what commands were executed and how much time and +memory was spent to get the expected output form the runtime's web server. +`measurements.csv` contains the same data in a machine friendly form. +Last but not least, `build-and-run.log` journals the whole history of all commands +and their output. -There is also an aggregated `testsuite/target/archived-logs/aggregated-report.md` describing all the testsuite did. +There is also an aggregated `testsuite/target/archived-logs/aggregated-report.md` describing all +the testsuite did. ## AppReproducersTest -This part of the test suite runs smaller apps that are not expected to offer a web server, so just their stdout/stderr -is checked. No measurements are made as to the used memory at the time of writing, but it could be easily changed. +This part of the test suite runs smaller apps that are not expected to offer a web server, +so just their stdout/stderr is checked. e.g. a current example: ``` @@ -105,24 +119,75 @@ INFO [o.g.t.i.AppReproducersTest] (randomNumbersReinit) [UUID: 323a8f36-e196-4f ## Logs and Whitelist -Logs are checked for error and warning messages. Expected error messages can be whitelisted in [WhitelistLogLines.java](./testsuite/src/it/java/org/graalvm/tests/integration/utils/WhitelistLogLines.java). +Logs are checked for error and warning messages. Expected error messages can be whitelisted +in [WhitelistLogLines.java](./testsuite/src/it/java/org/graalvm/tests/integration/utils/WhitelistLogLines.java). + +## Thresholds properties -## Thresholds +We need to switch on and off certain tests depending on native-image versions used, +on host vs. container and most importantly, based on Quarkus version. + +We use method annotations for that, e.g. `@IfQuarkusVersion(min = "3.6.0")`, +or `@IfMandrelVersion(min = "23.0.0", inContainer = true)`. + +We also use properties in text files to validate whether particular thresholds were crossed, e.g. whether the app used more RAM than expected or whether the measured time delta between JVM HotSpot mode and native-image mode was bigger than expected. + +e.g. + +``` +linux.jvm.time.to.finish.threshold.ms=6924 +linux.native.time.to.finish.threshold.ms=14525 +``` -The test suite works with ```threshold.properties```, e.g. `./apps/quarkus-full-microprofile/threshold.properties`: +The current `.conf` format enhances `.properties` format with the power of using the +annotation strings, see: ``` -linux.time.to.first.ok.request.threshold.ms=50 -linux.RSS.threshold.kB=120000 -windows.time.to.first.ok.request.threshold.ms=80 -windows.RSS.threshold.kB=120000 +# Comments and empty lines are ignored +linux.jvm.time.to.finish.threshold.ms=6000 +linux.native.time.to.finish.threshold.ms=14000 +linux.executable.size.threshold.kB=79000 + +@IfQuarkusVersion(min ="2.7.0", max="3.0.0") +linux.jvm.time.to.finish.threshold.ms=5000 +linux.native.time.to.finish.threshold.ms=13000 +linux.executable.size.threshold.kB=75000 + +@IfQuarkusVersion(min ="3.5.0", max="3.5.999") +@IfMandrelVersion(min = "23.1.2", minJDK = "21.0.1" ) +linux.jvm.time.to.finish.threshold.ms=6924 +linux.native.time.to.finish.threshold.ms=14525 +linux.executable.size.threshold.kB=79000 + +@IfQuarkusVersion(min ="3.6.0") +linux.jvm.time.to.finish.threshold.ms=6924 +linux.native.time.to.finish.threshold.ms=14525 +linux.executable.size.threshold.kB=79000 + +@IfMandrelVersion(min = "24", minJDK = "21.0.1" ) +linux.executable.size.threshold.kB=90000 ``` -**THIS IS NOT A PERFORMANCE TEST** The thresholds are in place only as a sanity check to make sure -an update to Native image did not make the application runtime to run way over the expected benevolent values. +Properties are being added to a map top to bottom, overwriting their previous values +unless an `@If` constraint fails. If a condition fails, the following properties are +not added to the map until the next `@If` constraint is met. + +If two `@If` constraints follow immediately one after the other, they both MUST be true +to process the following properties. + +Take a look at [ThresholdsTest.java](./ThresholdsTest.java) and its `threshold-*.conf` test [files](../../../../../../../../test/resources/) for a comprehensive overview. + +The parsing logic is compatible with plain `.properties` files as we have been using before, +i.e. any key-value pair where the value is interpreted as the long type. + + +**THIS IS NOT A PERFORMANCE TEST** The thresholds are in place only as a sanity check to make +sure an update to Native image did not make the application runtime to run way over the +expected benevolent values. -The measured values are simply compared to be less or equal to the set threshold. One can overwrite the `threshold.properties` -by using env variables or system properties (in this order). All letter are capitalized and dot is replaced with underscore, e.g. +The measured values are simply compared to be less or equal to the set threshold. +One can overwrite the `threshold.conf` by using env variables or system properties +(in this order). All letter are capitalized and dot is replaced with underscore, e.g. ``` APPS_QUARKUS_FULL_MICROPROFILE_LINUX_TIME_TO_FIRST_OK_REQUEST_THRESHOLD_MS=35 mvn clean verify -Ptestsuite diff --git a/apps/debug-symbols-smoke/threshold.properties b/apps/debug-symbols-smoke/threshold.conf similarity index 100% rename from apps/debug-symbols-smoke/threshold.properties rename to apps/debug-symbols-smoke/threshold.conf diff --git a/apps/helidon-quickstart-se/threshold.properties b/apps/helidon-quickstart-se/threshold.conf similarity index 100% rename from apps/helidon-quickstart-se/threshold.properties rename to apps/helidon-quickstart-se/threshold.conf diff --git a/apps/jfr-native-image-performance/threshold.properties b/apps/jfr-native-image-performance/threshold.conf similarity index 100% rename from apps/jfr-native-image-performance/threshold.properties rename to apps/jfr-native-image-performance/threshold.conf diff --git a/apps/quarkus-full-microprofile/threshold.properties b/apps/quarkus-full-microprofile/threshold.conf similarity index 100% rename from apps/quarkus-full-microprofile/threshold.properties rename to apps/quarkus-full-microprofile/threshold.conf diff --git "a/apps/quarkus-sp\303\266klik-encoding/threshold.properties" "b/apps/quarkus-sp\303\266klik-encoding/threshold.conf" similarity index 100% rename from "apps/quarkus-sp\303\266klik-encoding/threshold.properties" rename to "apps/quarkus-sp\303\266klik-encoding/threshold.conf" diff --git a/testsuite/src/it/java/org/graalvm/tests/integration/utils/Apps.java b/testsuite/src/it/java/org/graalvm/tests/integration/utils/Apps.java index 5358e353..8b5f3cc1 100755 --- a/testsuite/src/it/java/org/graalvm/tests/integration/utils/Apps.java +++ b/testsuite/src/it/java/org/graalvm/tests/integration/utils/Apps.java @@ -25,11 +25,14 @@ import java.io.FileInputStream; import java.io.IOException; import java.io.InputStream; +import java.nio.file.Files; +import java.nio.file.Path; import java.util.HashMap; import java.util.Map; import java.util.Properties; import static org.graalvm.tests.integration.RuntimesSmokeTest.BASE_DIR; +import static org.graalvm.tests.integration.utils.thresholds.Thresholds.parseProperties; import static org.junit.jupiter.api.Assertions.fail; /** @@ -178,34 +181,31 @@ public enum Apps { this.whitelistLogLines = whitelistLogLines; this.buildAndRunCmds = buildAndRunCmds; this.runtimeContainer = runtimeContainer; - File tpFile = new File(BASE_DIR + File.separator + dir + File.separator + "threshold.properties"); - // Some apps don't have threshold.properties - if (tpFile.exists()) { - String appDirNormalized = dir.toUpperCase() - .replace(File.separator, "_") - .replace('-', '_') - + "_"; - try (InputStream input = new FileInputStream(tpFile)) { - Properties props = new Properties(); - props.load(input); - for (String pn : props.stringPropertyNames()) { - String normPn = pn.toUpperCase().replace('.', '_'); - String env = System.getenv().get(appDirNormalized + normPn); + + // Some apps don't have threshold.conf + final Path tcFile = Path.of(BASE_DIR, dir, "threshold.conf"); + if (Files.exists(tcFile)) { + final String appDirNormalized = dir.toUpperCase().replace(File.separator, "_").replace('-', '_') + "_"; + try { + final Map props = parseProperties(tcFile); + for (String pn : props.keySet()) { + final String normPn = pn.toUpperCase().replace('.', '_'); + final String env = System.getenv().get(appDirNormalized + normPn); if (StringUtils.isNotBlank(env)) { - props.replace(pn, env); + props.replace(pn, Long.parseLong(env)); } - String sys = System.getProperty(appDirNormalized + normPn); + final String sys = System.getProperty(appDirNormalized + normPn); if (StringUtils.isNotBlank(sys)) { - props.replace(pn, sys); + props.replace(pn, Long.parseLong(sys)); } - thresholdProperties.put(pn, Long.parseLong(props.getProperty(pn))); + thresholdProperties.put(pn, props.get(pn)); } } catch (NumberFormatException e) { - fail("Check threshold.properties and Sys and Env variables " + + fail("Check threshold.conf and Sys and Env variables " + "(upper case, underscores instead of dots). " + - "All values are expected to be of type long."); + "All values are expected to be of type long.", e); } catch (IOException e) { - fail("Couldn't find " + tpFile.getAbsolutePath()); + fail("Couldn't find " + tcFile, e); } } } diff --git a/testsuite/src/it/java/org/graalvm/tests/integration/utils/Logs.java b/testsuite/src/it/java/org/graalvm/tests/integration/utils/Logs.java index fdcdc628..f9870fbc 100644 --- a/testsuite/src/it/java/org/graalvm/tests/integration/utils/Logs.java +++ b/testsuite/src/it/java/org/graalvm/tests/integration/utils/Logs.java @@ -56,7 +56,7 @@ public static void checkLog(String testClass, String testMethod, Apps app, File System.arraycopy(app.whitelistLogLines.get(inContainer), 0, whitelistPatterns, 0, app.whitelistLogLines.get(inContainer).length); System.arraycopy(WhitelistLogLines.ALL.get(inContainer), 0, whitelistPatterns, app.whitelistLogLines.get(inContainer).length, WhitelistLogLines.ALL.get(inContainer).length); try (Scanner sc = new Scanner(log, UTF_8)) { - Set offendingLines = new HashSet<>(); + final Set offendingLines = new HashSet<>(); while (sc.hasNextLine()) { final String line = sc.nextLine(); final boolean error = WARN_ERROR_DETECTION_PATTERN.matcher(line).matches(); @@ -95,18 +95,18 @@ public String toString() { } public static void checkThreshold(Apps app, Mode mode, long executableSizeKb, long rssKb, long timeToFirstOKRequest, - long timeToFinishMs, long mean, long p50, long p90) { + long timeToFinishMs, long mean, long p50, long p90) { + final Path properties = Path.of(BASE_DIR, app.dir, "threshold.conf"); if (app.thresholdProperties.isEmpty() && (executableSizeKb != SKIP || rssKb != SKIP || timeToFirstOKRequest != SKIP || timeToFinishMs != SKIP)) { - LOGGER.warn("It seem there is no " + Path.of(BASE_DIR, app.dir, "threshold.properties. ") + - "Skipping checking thresholds."); + LOGGER.warn("It seem there is no " +properties + + ". Skipping checking thresholds."); return; } final String propPrefix = (IS_THIS_WINDOWS ? "windows" : "linux") + ((app.runtimeContainer != ContainerNames.NONE) ? ".container" : "") + ((mode != Mode.NONE) ? "." + mode : ""); - final Path properties = Path.of(BASE_DIR, app.dir, "threshold.properties"); final List failures = new ArrayList<>(); if (executableSizeKb != SKIP) { diff --git a/testsuite/src/it/java/org/graalvm/tests/integration/utils/thresholds/Thresholds.java b/testsuite/src/it/java/org/graalvm/tests/integration/utils/thresholds/Thresholds.java new file mode 100644 index 00000000..15cb7ba9 --- /dev/null +++ b/testsuite/src/it/java/org/graalvm/tests/integration/utils/thresholds/Thresholds.java @@ -0,0 +1,167 @@ +/* + * Copyright (c) 2023, Red Hat Inc. All rights reserved. + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * You may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package org.graalvm.tests.integration.utils.thresholds; + +import org.apache.logging.log4j.util.Strings; +import org.graalvm.home.Version; +import org.graalvm.tests.integration.utils.versions.QuarkusVersion; +import org.graalvm.tests.integration.utils.versions.UsedVersion; +import org.jboss.logging.Logger; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.stream.Collectors; + +import static org.graalvm.tests.integration.utils.versions.MandrelVersionCondition.jdkConstraintSatisfied; +import static org.graalvm.tests.integration.utils.versions.MandrelVersionCondition.mandrelConstraintSatisfied; +import static org.graalvm.tests.integration.utils.versions.QuarkusVersionCondition.quarkusConstraintSatisfied; + +/** + * See @README.md + * + * @author Michal Karm Babacek + */ +public class Thresholds { + + private static final Logger LOGGER = Logger.getLogger(Thresholds.class.getName()); + + //@formatter:off + public static final Pattern MVERSION_PATTERN = Pattern.compile( + "\\s*@IfMandrelVersion\\s*\\((\\s*" + + "(?:min\\s*=\\s*\"(?[^\"]+?)\"\\s*,?\\s*)|\\s*" + + "(?:max\\s*=\\s*\"(?[^\"]+?)\"\\s*,?\\s*)|\\s*" + + "(?:minJDK\\s*=\\s*\"(?[^\"]+?)\"\\s*,?\\s*)|\\s*" + + "(?:maxJDK\\s*=\\s*\"(?[^\"]+?)\"\\s*,?\\s*)|\\s*" + + "(?:inContainer\\s*=\\s*(?(?:true|false)?)\\s*,?\\s*))+\\s*\\)\\s*"); + public static final Pattern QVERSION_PATTERN = Pattern.compile( + "\\s*@IfQuarkusVersion\\s*\\((\\s*" + + "(?:min\\s*=\\s*\"(?[^\"]+?)\"\\s*,?\\s*)|\\s*" + + "(?:max\\s*=\\s*\"(?[^\"]+?)\"\\s*,?\\s*))+\\s*\\)\\s*"); + public static final Pattern PROP_PATTERN = Pattern.compile( + "\\s*(?[^=]+?)\\s*=\\s*(?[0-9]+?)\\s*"); + //@formatter:on + private static final String QMARK = "@IfQ"; + private static final String MMARK = "@IfM"; + + public static Map parseProperties(final Path conf) throws IOException { + final Map props = new HashMap<>(); + // Ignore empty lines, leading, trailing spaces, comments + //@formatter:off + final List lines = Files.readAllLines(conf).stream() + .map(String::trim) + .filter(line -> !Strings.isBlank(line) && !line.startsWith("#")) + .collect(Collectors.toList()); + //@formatter:on + boolean useProp = true; + for (int i = 0; i < lines.size(); i++) { + final String line = lines.get(i); + // Mandrel's annotation first + if (line.startsWith(MMARK)) { + final Boolean ifMandrel = ifMandrel(line); + if (ifMandrel == null) { + continue; + } + useProp = ifMandrel; + if (i + 1 < lines.size()) { + final String lookAhead = lines.get(i + 1); + if (lookAhead.startsWith(QMARK)) { + final Boolean ifQuarkus = ifQuarkus(lookAhead); + if (ifQuarkus != null) { + // Both must be satisfied. + useProp = ifMandrel && ifQuarkus; + } + // We do not re-evaluate the IfQuarkus line. + i++; + } + } + continue; + } + // Quarkus' annotation first + if (line.startsWith(QMARK)) { + final Boolean ifQuarkus = ifQuarkus(line); + if (ifQuarkus == null) { + continue; + } + useProp = ifQuarkus; + if (i + 1 < lines.size()) { + final String lookAhead = lines.get(i + 1); + if (lookAhead.startsWith(MMARK)) { + final Boolean ifMandrel = ifMandrel(lookAhead); + if (ifMandrel != null) { + // Both must be satisfied. + useProp = ifMandrel && ifQuarkus; + } + // We do not re-evaluate the IfQuarkus line. + i++; + } + } + continue; + } + final Matcher propMatch = PROP_PATTERN.matcher(line); + if (propMatch.matches()) { + final String key = propMatch.group("key"); + final String value = propMatch.group("value"); + if (useProp) { + props.put(key, Long.parseLong(value)); + } + } else { + LOGGER.error("Line '" + line + "' does not match the pattern '" + PROP_PATTERN.pattern() + "'. Ignoring."); + } + } + return props; + } + + public static Boolean ifMandrel(final String line) { + final Matcher mVerMatch = MVERSION_PATTERN.matcher(line); + if (mVerMatch.matches()) { + final String min = mVerMatch.group("min"); + final String max = mVerMatch.group("max"); + final String minJDK = mVerMatch.group("minJDK"); + final String maxJDK = mVerMatch.group("maxJDK"); + final boolean inContainer = Boolean.parseBoolean(mVerMatch.group("inContainer")); + Version version = UsedVersion.getVersion(inContainer); + if (inContainer && version == null) { + LOGGER.error("There is probably no container runtime configured and thus no way to " + + "determine the version of Mandrel for `inContainer = true`. Trying to fall back to the host runtime."); + version = UsedVersion.getVersion(false); + } + return (jdkConstraintSatisfied(minJDK, maxJDK, inContainer) && mandrelConstraintSatisfied(version, min, max)); + } + LOGGER.error("Line '" + line + "' does not match the pattern '" + MVERSION_PATTERN.pattern() + "' although it starts with '@IfMandrel.*'. Ignoring."); + return null; + } + + public static Boolean ifQuarkus(final String line) { + final Matcher qVerMatch = QVERSION_PATTERN.matcher(line); + if (qVerMatch.matches()) { + final String min = qVerMatch.group("min"); + final String max = qVerMatch.group("max"); + return (quarkusConstraintSatisfied(new QuarkusVersion(), min, max)); + } + LOGGER.error("Line '" + line + "' does not match the pattern '" + QVERSION_PATTERN.pattern() + "' although it starts with '@IfQuarkus.*'. Ignoring."); + return null; + } +} diff --git a/testsuite/src/it/java/org/graalvm/tests/integration/utils/thresholds/ThresholdsTest.java b/testsuite/src/it/java/org/graalvm/tests/integration/utils/thresholds/ThresholdsTest.java new file mode 100644 index 00000000..cb833661 --- /dev/null +++ b/testsuite/src/it/java/org/graalvm/tests/integration/utils/thresholds/ThresholdsTest.java @@ -0,0 +1,249 @@ +package org.graalvm.tests.integration.utils.thresholds; +/* + * Copyright (c) 2023, Red Hat Inc. All rights reserved. + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * You may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +import org.graalvm.tests.integration.utils.versions.QuarkusVersion; +import org.graalvm.tests.integration.utils.versions.UsedVersion; +import org.junit.jupiter.api.Tag; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.CsvSource; + +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.StandardOpenOption; +import java.nio.file.attribute.PosixFilePermissions; +import java.util.Map; +import java.util.regex.Matcher; + +import static java.lang.System.clearProperty; +import static java.lang.System.lineSeparator; +import static java.lang.System.setProperty; +import static org.graalvm.tests.integration.RuntimesSmokeTest.BASE_DIR; +import static org.graalvm.tests.integration.utils.Commands.IS_THIS_WINDOWS; +import static org.graalvm.tests.integration.utils.Commands.getProperty; +import static org.graalvm.tests.integration.utils.thresholds.Thresholds.MVERSION_PATTERN; +import static org.graalvm.tests.integration.utils.thresholds.Thresholds.QVERSION_PATTERN; +import static org.graalvm.tests.integration.utils.thresholds.Thresholds.parseProperties; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Test thresholds config values + * + * @author Michal Karm Babacek + */ +@Tag("testing-testsuite") +public class ThresholdsTest { + + @ParameterizedTest + //@formatter:off + @CsvSource(value = { + "@IfMandrelVersion(min = \"23\", max = \"23.3\" , minJDK = \"17.0.1\", maxJDK = \"20.0.0\") | 23 | 23.3 | 17.0.1 | 20.0.0 | N/A ", + "@IfMandrelVersion(maxJDK = \"20.0.0\", min = \"23\", max = \"23.3\", minJDK = \"17.0.1\", inContainer =true) | 23 | 23.3 | 17.0.1 | 20.0.0 | true ", + "@IfMandrelVersion(maxJDK = \"20.0.0\", min = \"23\", max = \"23.3\", minJDK = \"17.0.1\") | 23 | 23.3 | 17.0.1 | 20.0.0 | N/A ", + " @IfMandrelVersion(max = \"23.3\", maxJDK = \"20.0.0\", min = \"23\", minJDK = \"17.0.1\") | 23 | 23.3 | 17.0.1 | 20.0.0 | N/A ", + "@IfMandrelVersion(min = \"24\", minJDK = \"21.0.1\") | 24 | N/A | 21.0.1 | N/A | N/A ", + "@IfMandrelVersion(inContainer = true,min = \"24\", minJDK = \"21.0.1\") | 24 | N/A | 21.0.1 | N/A | true ", + "@IfMandrelVersion( min = \"22\", max = \"22.3.999\") | 22 | 22.3.999 | N/A | N/A | N/A ", + "@IfMandrelVersion(max= \"22.3.999\" , min = \"22\") | 22 | 22.3.999 | N/A | N/A | N/A ", + "@IfMandrelVersion(max = \"22.3.999\" ,minJDK = \"17.0.1\", min = \"22\") | 22 | 22.3.999 | 17.0.1 | N/A | N/A ", + "@IfMandrelVersion(min = \"23\") | 23 | N/A | N/A | N/A | N/A ", + "@IfMandrelVersion (max = \"21\") | N/A | 21 | N/A | N/A | N/A ", + "@IfMandrelVersion(maxJDK = \"23\", minJDK=\"17\") | N/A | N/A | 17 | 23 | N/A ", + "@IfMandrelVersion(maxJDK = \"23\", inContainer =false, minJDK=\"17\") | N/A | N/A | 17 | 23 | false ", + "@IfMandrelVersion(maxJDK = \"23\") | N/A | N/A | N/A | 23 | N/A ", + "@IfMandrelVersion( minJDK=\"17\" ) | N/A | N/A | 17 | N/A | N/A ", + "@IfMandrelVersion( minJDK=\"17\" ,inContainer =true ) | N/A | N/A | 17 | N/A | true ", + }, delimiter = '|') + //@formatter:on + public void testMandrelVersion(String expression, String min, String max, String minJDK, String maxJDK, String inContainer) { + final Matcher m = MVERSION_PATTERN.matcher(expression); + assertTrue(m.matches(), "Expression '" + expression + "' MUST match the pattern '" + MVERSION_PATTERN.pattern() + "'"); + if ("N/A".equals(min)) { + assertNull(m.group("min")); + } else { + assertEquals(min, m.group("min")); + } + if ("N/A".equals(max)) { + assertNull(m.group("max")); + } else { + assertEquals(max, m.group("max")); + } + if ("N/A".equals(minJDK)) { + assertNull(m.group("minJDK")); + } else { + assertEquals(minJDK, m.group("minJDK")); + } + if ("N/A".equals(maxJDK)) { + assertNull(m.group("maxJDK")); + } else { + assertEquals(maxJDK, m.group("maxJDK")); + } + if ("N/A".equals(inContainer)) { + assertNull(m.group("inContainer")); + } else { + assertEquals(inContainer, m.group("inContainer")); + } + } + + @ParameterizedTest + //@formatter:off + @CsvSource(value = { + "@IfQuarkusVersion(min = \"2.7.0.Final\", max = \"3\" ) | 2.7.0.Final | 3 ", + "@IfQuarkusVersion(max = \"3\",min = \"2.7.0.Final\" ) | 2.7.0.Final | 3 ", + "@IfQuarkusVersion(min=\"3.6.0\") | 3.6.0 | N/A ", + "@IfQuarkusVersion( max = \"3.2\" ) | N/A | 3.2 ", + " @IfQuarkusVersion ( max = \"3\", min = \"2.7.0.Final\" ) | 2.7.0.Final | 3 ", + }, delimiter = '|') + //@formatter:on + public void testQuarkusVersion(String expression, String min, String max) { + final Matcher m = QVERSION_PATTERN.matcher(expression); + assertTrue(m.matches(), "Expression '" + expression + "' MUST match the pattern '" + MVERSION_PATTERN.pattern() + "'"); + if ("N/A".equals(min)) { + assertNull(m.group("min")); + } else { + assertEquals(min, m.group("min")); + } + if ("N/A".equals(max)) { + assertNull(m.group("max")); + } else { + assertEquals(max, m.group("max")); + } + } + + @ParameterizedTest + //@formatter:off + @CsvSource(value = { + // Native-image version string | Config file | Q version | a | b | c + "native-image 21.0.1 2023-10-17\\n" + + "OpenJDK Runtime Environment Mandrel-23.1.1.0-Final (build 21.0.1+12-LTS)\\n" + + "OpenJDK 64-Bit Server VM Mandrel-23.1.1.0-Final (build 21.0.1+12-LTS, mixed mode) | threshold-1.conf | 3.3.3.Final | 150 | 250 | 350 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 21.0.1 2023-10-17\\n" + + "GraalVM Runtime Environment GraalVM CE 21.0.1-dev+12.1 (build 21.0.1+12-jvmci-23.1-b22)\\n" + + "Substrate VM GraalVM CE 21.0.1-dev+12.1 (build 21.0.1+12, serial gc) | threshold-1.conf | 3.1.9.Final | 160 | 260 | 360 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 23 2026-09-01\\n" + + "GraalVM Runtime Environment GraalVM CE 23+35.1 (build 23+35-jvmci-24.1-b99)\\n" + + "Substrate VM GraalVM CE 23+35.1 (build 23+35, serial gc) | threshold-1.conf | 3.7.0 | 170 | 270 | 100 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 21.3.6.0-Final Mandrel Distribution (Java Version 17.0.7+7) | threshold-1.conf | 3.6.0 | N/A | 90 | 100 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 22.3.4.0-Final Mandrel Distribution (Java Version 17.0.9+9) | threshold-2.conf | 2.7.9.Final | 200 | 300 | 400 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 17.0.9 2023-10-17\\n" + + "OpenJDK Runtime Environment Mandrel-23.0.2.1-Final (build 17.0.9+9)\\n" + + "OpenJDK 64-Bit Server VM Mandrel-23.0.2.1-Final (build 17.0.9+9, mixed mode) | threshold-3.conf | 3.6.0 | 100 | 200 | 300 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 20.0.2 2023-07-18\\n" + + "OpenJDK Runtime Environment Mandrel-23.0.1.2-Final (build 20.0.2+9)\\n" + + "OpenJDK 64-Bit Server VM Mandrel-23.0.1.2-Final (build 20.0.2+9, mixed mode) | threshold-4.conf | 3.6.0 | 100 | 200 | 300 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 20.0.2 2023-07-18\\n" + + "OpenJDK Runtime Environment Mandrel-23.0.1.2-Final (build 20.0.2+9)\\n" + + "OpenJDK 64-Bit Server VM Mandrel-23.0.1.2-Final (build 20.0.2+9, mixed mode) | threshold-4.conf | 3.8.0 | 300 | 400 | 100 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 20.0.8 2023-07-18\\n" + + "OpenJDK Runtime Environment Mandrel-23.0.1.2-Final (build 20.0.8+9)\\n" + + "OpenJDK 64-Bit Server VM Mandrel-23.0.1.2-Final (build 20.0.8+9, mixed mode) | threshold-4.conf | 3.0.0 | 191 | 192 | 193 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 23 2026-09-01\\n" + + "GraalVM Runtime Environment GraalVM CE 23+35.1 (build 23+35-jvmci-24.1-b99)\\n" + + "Substrate VM GraalVM CE 23+35.1 (build 23+35, serial gc) | threshold-5.conf | 3.5.5 | 110 | 210 | 310 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 23 2026-09-01\\n" + + "GraalVM Runtime Environment GraalVM CE 23+35.1 (build 23+35-jvmci-24.1-b99)\\n" + + "Substrate VM GraalVM CE 23+35.1 (build 23+35, serial gc) | threshold-5.conf | 3.6.0 | 100 | 200 | 300 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 20.0.1 2023-07-18\\n" + + "OpenJDK Runtime Environment Mandrel-23.0.0.0-Final (build 20.0.1+9)\\n" + + "OpenJDK 64-Bit Server VM Mandrel-23.0.0.0-Final (build 20.0.1+9, mixed mode) | threshold-6.conf | 3.6.0 | 130 | 230 | 330 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 20.0.1 2023-07-18\\n" + + "OpenJDK Runtime Environment Mandrel-23.0.0.0-Final (build 20.0.1+9)\\n" + + "OpenJDK 64-Bit Server VM Mandrel-23.0.0.0-Final (build 20.0.1+9, mixed mode) | threshold-7.conf | 3.6.0 | 100 | 200 | 300 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + "native-image 20.0.1 2023-07-18\\n" + + "OpenJDK Runtime Environment Mandrel-23.0.0.0-Final (build 20.0.1+9)\\n" + + "OpenJDK 64-Bit Server VM Mandrel-23.0.0.0-Final (build 20.0.1+9, mixed mode) | threshold-8.conf | 3.6.0 | 100 | 200 | 300 ", + // ----------------------------------------------------------------------------------------------------------------------------------------- + }, delimiter = '|') + //@formatter:on + public void testThreshold(String nativeImageVersion, String conf, String quarkusVersion, String a, String b, String c) throws IOException { + final Path config = Path.of(BASE_DIR, "testsuite", "src", "test", "resources", conf); + assertTrue(Files.exists(config), "Config file '" + config + "' MUST exist."); + final Path tmpDir = Files.createTempDirectory(ThresholdsTest.class.getSimpleName()); + final Path nativeImage = tmpDir.resolve(Path.of(IS_THIS_WINDOWS ? "native-image.cmd" : "native-image")); + final String quarkusVersionTmp = getProperty("QUARKUS_VERSION", QuarkusVersion.DEFAULT_VERSION); + setProperty("FAKE_NATIVE_IMAGE_DIR", tmpDir.toAbsolutePath() + File.separator); + setProperty("QUARKUS_VERSION", quarkusVersion); + final String nativeImageText = createFakeNativeImageFile(nativeImage, nativeImageVersion); + try { + // Reset parsed instances to avoid side effects on other tests + // Note that container instance isn't being used, so isn't reset. + UsedVersion.Locally.resetInstance(); + + final Map thresholds = parseProperties(config); + final String[] expected = { a, b, c }; + final String[] propNames = { "a", "b", "c" }; + for (int i = 0; i < expected.length; i++) { + final String k = "some.property." + propNames[i]; + assertEquals(expected[i], thresholds.containsKey(k) ? thresholds.get(k).toString() : "N/A", + String.format("Expected value for some.property.%s is %s, but got %s,\n " + + "Native-image: %s\n" + + "Quarkus: %s\n" + + "Conf: %s\n", propNames[i], expected[i], thresholds.get(k), nativeImageText, quarkusVersion, config)); + } + } finally { + clearProperty("FAKE_NATIVE_IMAGE_DIR"); + setProperty("QUARKUS_VERSION", quarkusVersionTmp); + Files.deleteIfExists(nativeImage); + Files.deleteIfExists(tmpDir); + } + } + + /** + * @param file path to the fake native-image file + * @param contents

if multiline, all newlines must be escaped as \\n (double backslash), the reason is that JUnit's @CsvSource doesn't like \n in its parameters.

+ * @return the contents of the fake native-image file + */ + public static String createFakeNativeImageFile(final Path file, final String contents) throws IOException { + // Why this \\\\n? @CsvSource does not like \n in its values. + final String nativeImageText; + if (IS_THIS_WINDOWS) { + nativeImageText = "@echo off" + lineSeparator() + + "setlocal EnableDelayedExpansion" + lineSeparator() + + "set LINE=^&echo." + lineSeparator() + + "echo " + contents.replaceAll("\\\\n", "%LINE%"); + } else { + nativeImageText = "#!/bin/sh" + lineSeparator() + + "echo '" + contents.replaceAll("\\\\n", lineSeparator()) + "'"; + } + Files.writeString(file, nativeImageText, StandardCharsets.UTF_8, StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING, StandardOpenOption.WRITE); + if (!IS_THIS_WINDOWS) { + Files.setPosixFilePermissions(file, PosixFilePermissions.fromString("rwxr-xr-x")); + } + return nativeImageText; + } +} diff --git a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/MandrelVersionCondition.java b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/MandrelVersionCondition.java index a3b4e9cb..7c65f43e 100644 --- a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/MandrelVersionCondition.java +++ b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/MandrelVersionCondition.java @@ -19,6 +19,7 @@ */ package org.graalvm.tests.integration.utils.versions; +import org.apache.logging.log4j.util.Strings; import org.graalvm.home.Version; import org.junit.jupiter.api.extension.ConditionEvaluationResult; import org.junit.jupiter.api.extension.ExecutionCondition; @@ -42,6 +43,8 @@ public class MandrelVersionCondition implements ExecutionCondition { private static final ConditionEvaluationResult ENABLED_BY_DEFAULT = enabled("@IfMandrelVersion is not present"); + private static final Pattern JDK_PATTERN = Pattern.compile("(?[0-9]+)(\\.(?[0-9]*)\\.(?[0-9]*))?"); + @Override public ConditionEvaluationResult evaluateExecutionCondition( ExtensionContext context) { @@ -54,22 +57,9 @@ public ConditionEvaluationResult evaluateExecutionCondition( } private ConditionEvaluationResult disableIfVersionMismatch(IfMandrelVersion annotation, AnnotatedElement element) { + final boolean jdkConstraintSatisfied = jdkConstraintSatisfied(annotation.minJDK(), annotation.maxJDK(), annotation.inContainer()); final Version usedVersion = UsedVersion.getVersion(annotation.inContainer()); - boolean jdkConstraintSatisfied = true; - if (!annotation.minJDK().isBlank() || !annotation.maxJDK().isBlank()) { - final int[] jdkVersion = new int[]{ - UsedVersion.jdkFeature(annotation.inContainer()), - UsedVersion.jdkInterim(annotation.inContainer()), - UsedVersion.jdkUpdate(annotation.inContainer()) - }; - final Pattern p = Pattern.compile("(?[0-9]+)(\\.(?[0-9]*)\\.(?[0-9]*))?"); - final int[] min = featureInterimUpdate(p, annotation.minJDK(), Integer.MIN_VALUE); - final int[] max = featureInterimUpdate(p, annotation.maxJDK(), Integer.MAX_VALUE); - jdkConstraintSatisfied = compareJDKVersion(jdkVersion, min) >= 0 && compareJDKVersion(jdkVersion, max) <= 0; - } - final boolean mandrelConstraintSatisfied = - (annotation.min().isBlank() || usedVersion.compareTo(Version.parse(annotation.min())) >= 0) && - (annotation.max().isBlank() || usedVersion.compareTo(Version.parse(annotation.max())) <= 0); + final boolean mandrelConstraintSatisfied = mandrelConstraintSatisfied(usedVersion, annotation.min(), annotation.max()); if (mandrelConstraintSatisfied && jdkConstraintSatisfied) { return enabled(format( "%s is enabled as Mandrel version %s does satisfy constraints: min: %s, max: %s, minJDK: %s, maxJDK: %s", @@ -79,4 +69,24 @@ private ConditionEvaluationResult disableIfVersionMismatch(IfMandrelVersion anno "%s is disabled as Mandrel version %s does not satisfy constraints: min: %s, max: %s, minJDK: %s, maxJDK: %s", element, usedVersion, annotation.min(), annotation.max(), annotation.minJDK(), annotation.maxJDK())); } + + public static boolean jdkConstraintSatisfied(final String minJDK, final String maxJDK, final boolean inContainer) { + if (!Strings.isBlank(minJDK) || !Strings.isBlank(maxJDK)) { + final int[] jdkVersion = new int[] { + UsedVersion.jdkFeature(inContainer), + UsedVersion.jdkInterim(inContainer), + UsedVersion.jdkUpdate(inContainer) + }; + final int[] min = featureInterimUpdate(JDK_PATTERN, minJDK, Integer.MIN_VALUE); + final int[] max = featureInterimUpdate(JDK_PATTERN, maxJDK, Integer.MAX_VALUE); + return compareJDKVersion(jdkVersion, min) >= 0 && compareJDKVersion(jdkVersion, max) <= 0; + } else { + return true; + } + } + + public static boolean mandrelConstraintSatisfied(final Version usedVersion, final String min, final String max) { + return (Strings.isBlank(min) || usedVersion.compareTo(Version.parse(min)) >= 0) && + (Strings.isBlank(max) || usedVersion.compareTo(Version.parse(max)) <= 0); + } } diff --git a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/QuarkusVersion.java b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/QuarkusVersion.java index 2579376f..3d6eba77 100644 --- a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/QuarkusVersion.java +++ b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/QuarkusVersion.java @@ -31,8 +31,10 @@ public class QuarkusVersion implements Comparable { private final boolean snapshot; private final String gitSHA; + public static final String DEFAULT_VERSION = "2.13.9.Final"; + public QuarkusVersion() { - this(getProperty("QUARKUS_VERSION", "2.13.9.Final")); + this(getProperty("QUARKUS_VERSION", DEFAULT_VERSION)); } public QuarkusVersion(String version) { @@ -46,8 +48,8 @@ public QuarkusVersion(String version) { } else { final String[] split = version.split("\\."); this.major = Integer.parseInt(split[0]); - this.minor = Integer.parseInt(split[1]); - this.patch = Integer.parseInt(split[2]); + this.minor = split.length > 1 ? Integer.parseInt(split[1]) : 0; + this.patch = split.length > 2 ? Integer.parseInt(split[2]) : 0; this.gitSHA = null; } } @@ -97,12 +99,6 @@ public String getGitSHA() { @Override public String toString() { - return "QuarkusVersion{" + - "version='" + version + '\'' + - ", major=" + major + - ", minor=" + minor + - ", patch=" + patch + - ", snapshot=" + snapshot + - '}'; + return "QuarkusVersion{" + "version='" + version + '\'' + ", major=" + major + ", minor=" + minor + ", patch=" + patch + ", snapshot=" + snapshot + '}'; } } diff --git a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/QuarkusVersionCondition.java b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/QuarkusVersionCondition.java index b6623d75..51879f43 100644 --- a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/QuarkusVersionCondition.java +++ b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/QuarkusVersionCondition.java @@ -19,7 +19,7 @@ */ package org.graalvm.tests.integration.utils.versions; -import org.graalvm.tests.integration.utils.Commands; +import org.apache.logging.log4j.util.Strings; import org.junit.jupiter.api.extension.ConditionEvaluationResult; import org.junit.jupiter.api.extension.ExecutionCondition; import org.junit.jupiter.api.extension.ExtensionContext; @@ -39,7 +39,7 @@ public class QuarkusVersionCondition implements ExecutionCondition { @Override public ConditionEvaluationResult evaluateExecutionCondition( ExtensionContext context) { - AnnotatedElement element = context + final AnnotatedElement element = context .getElement() .orElseThrow(IllegalStateException::new); return findAnnotation(element, IfQuarkusVersion.class) @@ -48,11 +48,11 @@ public ConditionEvaluationResult evaluateExecutionCondition( } private ConditionEvaluationResult disableIfVersionMismatch(IfQuarkusVersion annotation, AnnotatedElement element) { - QuarkusVersion usedVersion = Commands.QUARKUS_VERSION; - final boolean quarkusConstraintSatisfied = - (annotation.min().isBlank() || usedVersion.compareTo(new QuarkusVersion(annotation.min())) >= 0) && - (annotation.max().isBlank() || usedVersion.compareTo(new QuarkusVersion(annotation.max())) <= 0); - if (quarkusConstraintSatisfied) { + // `new QuarkusVersion()` is used instead of `Commands.QUARKUS_VERSION` for the + // sake of testability via `System.setProperty("QUARKUS_VERSION", "3.6.0")` at the cost + // of re-evaluation the property with each IfQuarkusVersion evaluation. + final QuarkusVersion usedVersion = new QuarkusVersion(); + if (quarkusConstraintSatisfied(usedVersion, annotation.min(), annotation.max())) { return enabled(format( "%s is enabled as Quarkus version %s does satisfy constraints: min: %s, max: %s", element, usedVersion, annotation.min(), annotation.max())); @@ -61,4 +61,9 @@ private ConditionEvaluationResult disableIfVersionMismatch(IfQuarkusVersion anno "%s is disabled as Quarkus version %s does not satisfy constraints: min: %s, max: %s", element, usedVersion, annotation.min(), annotation.max())); } + + public static boolean quarkusConstraintSatisfied(final QuarkusVersion usedVersion, final String min, final String max) { + return (Strings.isBlank(min) || usedVersion.compareTo(new QuarkusVersion(min)) >= 0) && + (Strings.isBlank(max) || usedVersion.compareTo(new QuarkusVersion(max)) <= 0); + } } diff --git a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/UsedVersion.java b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/UsedVersion.java index fb3977f0..df7e9d3a 100644 --- a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/UsedVersion.java +++ b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/UsedVersion.java @@ -104,11 +104,12 @@ public static int jdkUpdate(boolean inContainer) { // Implements version parsing after https://github.com/oracle/graal/pull/6302 static final class VersionParseHelper { + //@formatter:off private static final Map GRAAL_MAPPING = Map.of(22, "24.0", 23, "24.1", 24, "25.0", 25, "25.1"); - + //@formatter:on private static final String JVMCI_BUILD_PREFIX = "jvmci-"; private static final String MANDREL_VERS_PREFIX = "Mandrel-"; @@ -142,11 +143,11 @@ static final class VersionParseHelper { private static final Pattern VERSION_PATTERN = Pattern.compile(VERS_FORMAT); static MVersion parse(List lines) { - Matcher firstMatcher = FIRST_PATTERN.matcher(lines.get(0)); - Matcher secondMatcher = SECOND_PATTERN.matcher(lines.get(1)); - Matcher thirdMatcher = THIRD_PATTERN.matcher(lines.get(2)); + final Matcher firstMatcher = FIRST_PATTERN.matcher(lines.get(0)); + final Matcher secondMatcher = SECOND_PATTERN.matcher(lines.get(1)); + final Matcher thirdMatcher = THIRD_PATTERN.matcher(lines.get(2)); if (firstMatcher.find() && secondMatcher.find() && thirdMatcher.find()) { - String javaVersion = firstMatcher.group(VNUM_GROUP); + final String javaVersion = firstMatcher.group(VNUM_GROUP); java.lang.Runtime.Version v = null; try { v = java.lang.Runtime.Version.parse(javaVersion); @@ -154,15 +155,15 @@ static MVersion parse(List lines) { return MVersion.UNKNOWN_VERSION; } - String vendorVersion = secondMatcher.group(VENDOR_VERSION_GROUP); + final String vendorVersion = secondMatcher.group(VENDOR_VERSION_GROUP); - String buildInfo = secondMatcher.group(BUILD_INFO_GROUP); - String graalVersion = graalVersion(buildInfo, v.feature()); - String mandrelVersion = mandrelVersion(vendorVersion); - String versNum = (isMandrel(vendorVersion) ? mandrelVersion : graalVersion); - Version vers = versionParse(versNum); + final String buildInfo = secondMatcher.group(BUILD_INFO_GROUP); + final String graalVersion = graalVersion(buildInfo, v.feature()); + final String mandrelVersion = mandrelVersion(vendorVersion); + final String versNum = (isMandrel(vendorVersion) ? mandrelVersion : graalVersion); + final Version vers = versionParse(versNum); final String lastLine = lines.get(lines.size() - 1).trim(); - VersionBuilder builder = new VersionBuilder(); + final VersionBuilder builder = new VersionBuilder(); return builder .jdkUsesSysLibs(lastLine.contains("-LTS")) .beta("" /* not implemented */) @@ -196,7 +197,7 @@ private static String mandrelVersion(String vendorVersion) { } private static String matchVersion(String version) { - Matcher versMatcher = VERSION_PATTERN.matcher(version); + final Matcher versMatcher = VERSION_PATTERN.matcher(version); if (versMatcher.find()) { return versMatcher.group(VERSION_GROUP); } @@ -206,16 +207,16 @@ private static String matchVersion(String version) { private static String graalVersion(String buildInfo, int jdkFeatureVers) { if (jdkFeatureVers >= 22) { // short-circuit new version scheme with a mapping - return GRAAL_MAPPING.get(Integer.valueOf(jdkFeatureVers)); + return GRAAL_MAPPING.get(jdkFeatureVers); } if (buildInfo == null) { return null; } - int idx = buildInfo.indexOf(JVMCI_BUILD_PREFIX); + final int idx = buildInfo.indexOf(JVMCI_BUILD_PREFIX); if (idx < 0) { return null; } - String version = buildInfo.substring(idx + JVMCI_BUILD_PREFIX.length()); + final String version = buildInfo.substring(idx + JVMCI_BUILD_PREFIX.length()); return matchVersion(version); } @@ -255,23 +256,28 @@ private MVersion(Version version, boolean jdkUsesSysLibs, int jdkFeature, int jd } public static MVersion of(boolean inContainer) { - List lines = runNativeImageVersion(inContainer); - MVersion mandrelVersion; + final List lines = runNativeImageVersion(inContainer); + final MVersion mandrelVersion; if (lines.size() == 1) { mandrelVersion = parseOldVersion(lines, inContainer); } else if (lines.size() == 3) { mandrelVersion = VersionParseHelper.parse(lines); } else { mandrelVersion = UNKNOWN_VERSION; + LOGGER.warn("Failed to correctly parse native-image version command output. " + + "Is it on PATH? Unknown version format? " + + "Output reads in " + lines.size() + " lines, see them in an array: " + lines); } LOGGER.infof("The test suite runs with Mandrel version %s %s, JDK %d.%d.%d%s.", - mandrelVersion.version.toString(), inContainer ? "in container" : "installed locally on PATH", mandrelVersion.jdkFeature, mandrelVersion.jdkInterim, mandrelVersion.jdkUpdate, mandrelVersion.betaBits); + mandrelVersion.version == null ? "UNKNOWN" : mandrelVersion.version.toString(), + inContainer ? "in container" : "installed locally on PATH", + mandrelVersion.jdkFeature, mandrelVersion.jdkInterim, mandrelVersion.jdkUpdate, mandrelVersion.betaBits); return mandrelVersion; } private static MVersion parseOldVersion(List lines, boolean inContainer) { final String lastLine = lines.get(lines.size() - 1).trim(); - VersionBuilder builder = new VersionBuilder(); + final VersionBuilder builder = new VersionBuilder(); builder.jdkUsesSysLibs(lastLine.contains("-LTS")); if (inContainer && !lastLine.contains("Mandrel")) { LOGGER.warn("You are probably running GraalVM and not Mandrel container. " + @@ -303,7 +309,7 @@ private static MVersion parseOldVersion(List lines, boolean inContainer) builder.jdkInterim(jInterim == null ? UNDEFINED : Integer.parseInt(jInterim)); builder.jdkUpdate(jUpdate == null ? UNDEFINED : Integer.parseInt(jUpdate)); } - MVersion mandrelVersion = builder.build(); + final MVersion mandrelVersion = builder.build(); if (mandrelVersion.jdkFeature == UNDEFINED) { LOGGER.warn("Failed to correctly parse Java feature (major) version from native-image version command output. " + "JDK version constraints in tests won't work reliably."); @@ -397,23 +403,23 @@ static void resetInstance() { // used in tests } } - static class Locally { + public static class Locally { private static volatile MVersion mVersion = MVersion.of(false); - static void resetInstance() { // used in tests + public static void resetInstance() { // used in tests mVersion = MVersion.of(false); } } public static int[] featureInterimUpdate(Pattern pattern, String version, int defaultValue) { - final Matcher m = pattern.matcher(version); - if (!m.matches()) { - return new int[]{defaultValue, defaultValue, defaultValue}; + final Matcher m; + if (version == null || !(m = pattern.matcher(version)).matches()) { + return new int[] { defaultValue, defaultValue, defaultValue }; } final String jFeature = m.group("jfeature"); final String jInterim = m.group("jinterim"); final String jUpdate = m.group("jupdate"); - return new int[]{ + return new int[] { jFeature == null ? defaultValue : Integer.parseInt(jFeature), jInterim == null ? defaultValue : Integer.parseInt(jInterim), jUpdate == null ? defaultValue : Integer.parseInt(jUpdate) diff --git a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/VersionsTest.java b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/VersionsTest.java index 1488de93..50377a72 100644 --- a/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/VersionsTest.java +++ b/testsuite/src/it/java/org/graalvm/tests/integration/utils/versions/VersionsTest.java @@ -33,9 +33,10 @@ import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.StandardOpenOption; -import java.nio.file.attribute.PosixFilePermissions; import static org.graalvm.tests.integration.utils.Commands.IS_THIS_WINDOWS; +import static org.graalvm.tests.integration.utils.Commands.getProperty; +import static org.graalvm.tests.integration.utils.thresholds.ThresholdsTest.createFakeNativeImageFile; import static org.junit.jupiter.api.Assertions.assertEquals; /** @@ -48,11 +49,15 @@ * their executing into a file. * After all tests are done, the file is examined. If it contains any superfluous entries * or if it's missing anything, the test lass fails. + * + * @author Michal Karm Babacek */ @Tag("testing-testsuite") public class VersionsTest { - static final StandardOpenOption[] LOG_FILE_OPS = new StandardOpenOption[]{StandardOpenOption.CREATE, StandardOpenOption.APPEND, StandardOpenOption.WRITE}; + static final StandardOpenOption[] LOG_FILE_OPS = new StandardOpenOption[] { StandardOpenOption.CREATE, StandardOpenOption.APPEND, StandardOpenOption.WRITE }; static final String VERSION = "native-image 22.2.0-devdb26f5c4fbe Mandrel Distribution (Java Version 17.0.3-beta+5-202203292328)"; + static final String QUARKUS_VERSION = "3.6.0"; + static final String QUARKUS_VERSION_TMP; static final Path TEMP_DIR; static final Path LOG; static final Path NATIVE_IMAGE; @@ -62,6 +67,7 @@ public class VersionsTest { TEMP_DIR = Files.createTempDirectory(VersionsTest.class.getSimpleName()); LOG = TEMP_DIR.resolve(Path.of("versions-log")); NATIVE_IMAGE = TEMP_DIR.resolve(Path.of(IS_THIS_WINDOWS ? "native-image.cmd" : "native-image")); + QUARKUS_VERSION_TMP = getProperty("QUARKUS_VERSION", QuarkusVersion.DEFAULT_VERSION); } catch (IOException e) { throw new RuntimeException("Failed setting up temp directory for test"); } @@ -69,18 +75,11 @@ public class VersionsTest { @BeforeAll public static void setup() throws IOException { - System.setProperty("FAKE_NATIVE_IMAGE_DIR", TEMP_DIR.toAbsolutePath().toString() + File.separator); - Files.writeString(NATIVE_IMAGE, IS_THIS_WINDOWS ? - "@echo off" + System.lineSeparator() + - "echo " + VERSION + System.lineSeparator() : - "#!/bin/sh" + System.lineSeparator() + - "echo '" + VERSION + "'" + System.lineSeparator(), - StandardCharsets.UTF_8, StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING, StandardOpenOption.WRITE); - if (!IS_THIS_WINDOWS) { - Files.setPosixFilePermissions(NATIVE_IMAGE, PosixFilePermissions.fromString("rwxr-xr-x")); - } + System.setProperty("FAKE_NATIVE_IMAGE_DIR", TEMP_DIR.toAbsolutePath() + File.separator); + System.setProperty("QUARKUS_VERSION", QUARKUS_VERSION); + createFakeNativeImageFile(NATIVE_IMAGE, VERSION); Files.deleteIfExists(LOG); - // Reset parsed instances so as to avoid side-effects on other tests + // Reset parsed instances to avoid side effects on other tests // Note that container instance isn't being used, so isn't reset. UsedVersion.Locally.resetInstance(); } @@ -88,11 +87,13 @@ public static void setup() throws IOException { @AfterAll public static void teardown() throws IOException { System.clearProperty("FAKE_NATIVE_IMAGE_DIR"); + System.setProperty("QUARKUS_VERSION", QUARKUS_VERSION_TMP); Files.deleteIfExists(NATIVE_IMAGE); final String testlog = Files.readString(LOG, StandardCharsets.UTF_8); try { assertEquals( - "Running test jdkVersionCheckA\n" + + "Running test quarkusVersionCheckA\n" + + "Running test jdkVersionCheckA\n" + "Running test jdkVersionCheckB\n" + "Running test jdkVersionCheckC\n" + "Running test jdkVersionCheckD\n" + @@ -104,67 +105,95 @@ public static void teardown() throws IOException { } } + private static void test(TestInfo testInfo) throws IOException { + Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + } + @Test @Order(1) @IfMandrelVersion(min = "21.3.1") public void jdkVersionCheckA(TestInfo testInfo) throws IOException { - Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + test(testInfo); } @Test @Order(2) @IfMandrelVersion(min = "21.3.1", minJDK = "17") public void jdkVersionCheckB(TestInfo testInfo) throws IOException { - Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + test(testInfo); } @Test @Order(3) @IfMandrelVersion(min = "22", minJDK = "17") public void jdkVersionCheckC(TestInfo testInfo) throws IOException { - Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + test(testInfo); } @Test @Order(4) @IfMandrelVersion(min = "22", minJDK = "17.0.2") public void jdkVersionCheckD(TestInfo testInfo) throws IOException { - Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + test(testInfo); } @Test @Order(5) @IfMandrelVersion(min = "22", minJDK = "17", maxJDK = "17.0.2") public void jdkVersionCheckE(TestInfo testInfo) throws IOException { - Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + test(testInfo); } @Test @Order(6) @IfMandrelVersion(min = "21", minJDK = "11.0.12", maxJDK = "17.0.1") public void jdkVersionCheckF(TestInfo testInfo) throws IOException { - Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + test(testInfo); } @Test @Order(7) @IfMandrelVersion(min = "21", maxJDK = "11") public void jdkVersionCheckG(TestInfo testInfo) throws IOException { - Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + test(testInfo); } @Test @Order(8) @IfMandrelVersion(min = "21.2", minJDK = "18.0.0") public void jdkVersionCheckH(TestInfo testInfo) throws IOException { - Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + test(testInfo); } @Test @Order(9) @IfMandrelVersion(min = "22", max = "22.2", minJDK = "17.0.1", maxJDK = "17.0.3") public void jdkVersionCheckI(TestInfo testInfo) throws IOException { - Files.writeString(LOG, "Running test " + testInfo.getTestMethod().get().getName() + "\n", StandardCharsets.UTF_8, LOG_FILE_OPS); + test(testInfo); + } + + @Test + @Order(10) + @IfMandrelVersion(min = "22") + @IfQuarkusVersion(min = "3.6.0") + public void quarkusVersionCheckA(TestInfo testInfo) throws IOException { + test(testInfo); + } + + @Test + @Order(11) + @IfMandrelVersion(min = "22") + @IfQuarkusVersion(max = "3.5.0") + public void quarkusVersionCheckB(TestInfo testInfo) throws IOException { + test(testInfo); + } + + @Test + @Order(12) + @IfMandrelVersion(min = "22") + @IfQuarkusVersion(max = "3") + public void quarkusVersionCheckC(TestInfo testInfo) throws IOException { + test(testInfo); } } diff --git a/testsuite/src/test/resources/threshold-1.conf b/testsuite/src/test/resources/threshold-1.conf new file mode 100644 index 00000000..25991d9f --- /dev/null +++ b/testsuite/src/test/resources/threshold-1.conf @@ -0,0 +1,37 @@ +# This is a comment +some.property.c=100 + +# Won't be read, maxJDK excludes JDK 21 +@IfMandrelVersion(min = "23", max = "23.3", minJDK = "17.0.1" , maxJDK = "20.0.0") +@IfQuarkusVersion(min ="3.2.0", max="3.6.1") +some.property.a=110 +some.property.b=210 +some.property.c=310 + +# This wll be read, JDK 21 is included +@IfMandrelVersion(min = "23", max = "23.3", minJDK = "20") + +@IfQuarkusVersion(min ="3.2.0", max="3.6.1") +some.property.a=150 +some.property.b=250 +some.property.c=350 +@IfQuarkusVersion( max="3.1.999", min = "2.7.0") +@IfMandrelVersion(min = "23", minJDK = "21.0.1" ) +some.property.a=160 +some.property.b=260 +some.property.c=360 +@IfQuarkusVersion(min ="3.7" ) +some.property.a=170 +@IfMandrelVersion(min = "24.1") +some.property.b=270 +# This is a comment # This is a comment # This is a comment + +# + +@IfMandrelVersion(min = "21", max = "21.999") +some.property.b=90 + +# + + +@IfQuarkusVersion(min ="3.7" ) diff --git a/testsuite/src/test/resources/threshold-2.conf b/testsuite/src/test/resources/threshold-2.conf new file mode 100644 index 00000000..d8d7b9bf --- /dev/null +++ b/testsuite/src/test/resources/threshold-2.conf @@ -0,0 +1,9 @@ +@IfMandrelVersion(min = "22", max = "22.3.999") + some.property.a=100 +some.property.b =200 +some.property.c=300 + @IfMandrelVersion (min = "22.3.4") +some.property.a =200 +some.property.b= 300 +some.property.c=400 + diff --git a/testsuite/src/test/resources/threshold-3.conf b/testsuite/src/test/resources/threshold-3.conf new file mode 100644 index 00000000..b96daae1 --- /dev/null +++ b/testsuite/src/test/resources/threshold-3.conf @@ -0,0 +1,9 @@ + +some.property.a=100 +some.property.b=200 +some.property.c=300 + + + + + diff --git a/testsuite/src/test/resources/threshold-4.conf b/testsuite/src/test/resources/threshold-4.conf new file mode 100644 index 00000000..c08492a7 --- /dev/null +++ b/testsuite/src/test/resources/threshold-4.conf @@ -0,0 +1,19 @@ +# Common for all Quarkus versions +some.property.c=100 +@IfMandrelVersion(min = "23", max = "23.3", minJDK = "17.0.1" , maxJDK = "20.0.2") +@IfQuarkusVersion(min ="3.2.0", max="3.6.1") +some.property.a=100 +some.property.b=200 +some.property.c=300 +@IfQuarkusVersion( max="3.1.999", min = "2.7.0") +@IfMandrelVersion(min = "23.0.1.2", minJDK = "20.0.8" ) +some.property.a=191 +some.property.b =192 +some.property.c=193 +@IfQuarkusVersion(min ="3.7" ) +some.property.a=300 +some.property.b=400 +# This is a comment # This is a comment # This is a comment + +# + diff --git a/testsuite/src/test/resources/threshold-5.conf b/testsuite/src/test/resources/threshold-5.conf new file mode 100644 index 00000000..11af85c4 --- /dev/null +++ b/testsuite/src/test/resources/threshold-5.conf @@ -0,0 +1,13 @@ +# Here is some interesting comment as to +# why this is needed for this version. +@IfQuarkusVersion( max="3.5.99" ) +some.property.a=110 +some.property.b=210 +some.property.c=310 + +# Things got much better since 3.6.0 ! + @IfQuarkusVersion(min="3.6") + some.property.a=100 + some.property.b=200 + some.property.c=300 + diff --git a/testsuite/src/test/resources/threshold-6.conf b/testsuite/src/test/resources/threshold-6.conf new file mode 100644 index 00000000..38e8bd4c --- /dev/null +++ b/testsuite/src/test/resources/threshold-6.conf @@ -0,0 +1,23 @@ +# Nope, Q version won't match +@IfQuarkusVersion(max="3.1.999", min = "2.7.0") +@IfMandrelVersion(min = "23", minJDK = "20" ) +some.property.a=110 +some.property.b=210 +some.property.c=310 +# Nope, Mandrel version won't match +@IfQuarkusVersion( min = "3.5.0") +@IfMandrelVersion(min = "23", minJDK = "20.0.2" ) +some.property.a=120 +some.property.b=220 +some.property.c=320 +# Yeap, both match +@IfQuarkusVersion(min = "3.5.0") +@IfMandrelVersion(min = "23", minJDK = "20") +some.property.a=130 +some.property.b=230 +some.property.c=330 +# And here is a property that does not match value long and will be ignored: +some.property.c=This is not a long type, error message expected +# This will also match, but there are no properties to read after that. +@IfQuarkusVersion(min = "3.5.0") +@IfMandrelVersion(min = "23", minJDK = "20") diff --git a/testsuite/src/test/resources/threshold-7.conf b/testsuite/src/test/resources/threshold-7.conf new file mode 100644 index 00000000..eef463a3 --- /dev/null +++ b/testsuite/src/test/resources/threshold-7.conf @@ -0,0 +1,9 @@ +some.property.a=100 +some.property.b=200 +some.property.c=300 +# Nope, Q version won't match +@IfQuarkusVersion(max="3.1.999", min = "2.7.0") +@IfMandrelVersion(min = "23", minJDK = "20" ) +some.property.a=110 +some.property.b=210 +some.property.c=310 diff --git a/testsuite/src/test/resources/threshold-8.conf b/testsuite/src/test/resources/threshold-8.conf new file mode 100644 index 00000000..c31e9532 --- /dev/null +++ b/testsuite/src/test/resources/threshold-8.conf @@ -0,0 +1,9 @@ +some.property.a=100 +some.property.b=200 +some.property.c=300 +# Nope, Mandrel version won't match +@IfMandrelVersion(min = "23", minJDK = "20.0.2" ) +@IfQuarkusVersion( min = "3.5.0") +some.property.a=120 +some.property.b=220 +some.property.c=320