PRODUCT_NAME: Content of the env var PRODUCT_NAME, only added if present.PRODUCT_VERSION: Content of env var PRODUCT_VERSION, only added if present.TOTAL_EXECUTION_TIME: Full run time of the build in seconds.java.lang.Objectcall(java.lang.Object log = false, java.lang.Object jobName = '', int buildNumber = 0, java.lang.Object customAttributes = [:], java.lang.Object customConstants = [:])calculateErrorTime(java.lang.Object executionTestSteps)java.lang.ObjectcalculateTime(java.lang.Object executionTestSteps, java.lang.Object build)java.lang.Objectcall(java.lang.Object log = false, java.lang.Object jobName = '', int buildNumber = 0, java.lang.Object customAttributes = [:], java.lang.Object customConstants = [:])java.lang.ObjectconvertTimeValueToDouble(java.lang.Object value)java.lang.ObjectcrawlRows(java.lang.Object row, java.lang.Object appendLogs, java.lang.Object insideStage = false)java.lang.ObjectcreateTestStep(java.lang.Object row, java.lang.Object appendLogs, java.lang.Object skipped = false)java.lang.ObjectcreateTestStepFolder(java.lang.Object row, java.lang.Object appendLogs)java.lang.ObjectfindNonPassedVerdict(java.lang.Object steps, java.lang.Object accumulatedTime)java.lang.ObjectgenerateJsonReport(java.lang.Object build, java.lang.Object attributes, java.lang.Object constants, java.lang.Object executionTestSteps, java.lang.Object logFile)generateJsonReport(java.lang.Object build, java.lang.Object attributes, java.lang.Object constants, java.lang.Object executionTestSteps, java.lang.Object parameters, java.lang.Object logFile)java.lang.ObjectgetBuildAttributes(java.lang.Object build, java.lang.Object customAttributes)getBuildAttributes(java.lang.Object build, java.lang.Object customAttributes)PRODUCT_NAME: Content of the env var PRODUCT_NAME, only added if present.java.lang.ObjectgetBuildConstants(java.lang.Object build, java.lang.Object customConstants)getBuildConstants(java.lang.Object build, java.lang.Object customConstants)PRODUCT_VERSION: Content of env var PRODUCT_VERSION, only added if present.java.lang.ObjectgetBuildMetrics(java.lang.Object build, java.lang.Object testExecutionSteps)TOTAL_EXECUTION_TIME: Full run time of the build in seconds.java.lang.ObjectgetConsoleLog(java.lang.Object build)java.lang.ObjectgetCurrentPhase(java.lang.Object stageName, java.lang.Object currentPhase)java.lang.Objectjava.lang.ObjectgetTimeFromCommitToStart(java.lang.Object build)nullstatic voidmain(java.lang.String[] args)static java.lang.ObjectresultToATXVerdict(java.lang.Object result)Determines the time until an error within the execution phase occurs. +
executionTestSteps
+ - the stages of the pipeline buildCalculates the duration metrics and their percentages. +
executionTestSteps
+ - the stages of the pipeline buildbuild
+ - the pipeline buildGenerates a test.guide compatible JSON report of a pipeline build including logs and stage meta data. +
Generates a test.guide compatible JSON report of a pipeline build including logs, metrics, and stage meta data. The report is compressed as a zip file within the build workspace and can be uploaded using the JSON2ATX converter. This method can be called downstream or within a running build. @@ -332,7 +390,19 @@
true: each step log is passed to the test step description
false: only the log file will be archived (optional and default)jobName
- the name of the pipeline job (optional)buildNumber
- - the number of the pipeline build (optional)customAttributes
+ - the custom attributes for the pipline build (optional)customConstants
+ - the custom constants for the pipeline build (optional)Converts a none null time value to a double with one decimal place. +
value
+ - the time value to be convertedScans the verdicts value while traversing the stages of the pipeline build +
steps
+ - the steps within a stageaccumulatedTime
+ - accumulated duration values until an error occurredGenerates a test.guide compatible JSON report of the pipeline build.
build
- the pipeline buildattributes
- the collected attributesconstants
- the collected constantsexecutionTestSteps
- - the stages of the pipeline buildlogFile
+ - the stages of the pipeline buildparameters
+ - the collected parameterslogFile
- the log file name if per-step logging is enabledCollects all relevant build information and parameter as a key-value-map: - - PRODUCT_NAME (content of the env var PRODUCT_NAME, only added if present) - - GIT_URL (content of env var GIT_URL, only added if present) - - JENKINS_PIPELINE (name of the current Jenkins build job) - - JENKINS_URL (url to the current Jenkins build job) - - JENKINS_WORKSPACE (path to the Jenkins pipeline workspace) - - TEST_LEVEL (content of env var TEST_LEVEL, only added if present) - - customAttributes overrides build attributes (when adding maps the second map will override values present in both) +
Collects all relevant build information and parameters as a key-value map: +
PRODUCT_NAME: Content of the env var PRODUCT_NAME, only added if present.GIT_URL: Content of env var GIT_URL, only added if present.JENKINS_PIPELINE: Name of the current Jenkins build job.JENKINS_URL: URL to the current Jenkins build job.JENKINS_WORKSPACE: Path to the Jenkins pipeline workspace.TEST_LEVEL: Content of env var TEST_LEVEL, only added if present.build
- the pipeline raw buildCollects all relevant build information and parameter as a key-value-map: - - PRODUCT_VERSION (content of env var PRODUCT_VERSION, only added if present) - - GIT_COMMIT (content of env var GIT_COMMIT, only added if present) - - JENKINS_BUILD_ID (number of current Jenkins build) - - JENKINS_EXECUTOR_NUMBER (number of currecnt Jenkins executor) - - JENKINS_NODE_NAME (name of current node the current build is running on) +
Collects all relevant build information and parameters as a key-value map: +
PRODUCT_VERSION: Content of env var PRODUCT_VERSION, only added if present.GIT_COMMIT: Content of env var GIT_COMMIT, only added if present.JENKINS_BUILD_ID: Number of current Jenkins build.JENKINS_EXECUTOR_NUMBER: Number of current Jenkins executor.JENKINS_NODE_NAME: Name of current node the current build is running on.build
- the pipeline raw buildCollects all relevant build metrics information and parameters as a key-value map: +
TOTAL_EXECUTION_TIME: Full run time of the build in seconds.SETUP_TIME: Full run time of the setup phase of the build in seconds.EXECUTION_TIME: Full run time of the execution phase of the build in seconds.TEARDOWN_TIME: Full run time of the teardown phase of the build in seconds.SETUP_PERCENTAGE: Percentage of the setup phase in relation to the TOTAL_EXECUTION_TIME.EXECUTION_PERCENTAGE: Percentage of the execution phase in relation to the TOTAL_EXECUTION_TIME.TEARDOWN_PERCENTAGE: Percentage of the teardown phase in relation to the TOTAL_EXECUTION_TIME.QUEUE_TIME: Time until the build starts in seconds.COMMIT_TO_START_TIME: Time from commit to start of the build in seconds, only for push-based executions.TIME_TO_ERROR: Time until an error occurred in seconds, only for stages in the execution phase.TIME_TO_ERROR, will be filtered out before collection
+ build
+ - the pipeline raw buildtestExecutionSteps
+ - the stages of the pipeline buildDetermines the respective phase of the stage being examined. +
stageName
+ - the name of the current stagecurrentPhase
+ - the corresponding phase in which the current stage is locatedDetermines the time from commit to start of the build +
build
+ - the pipeline buildPRODUCT_NAME: Content of the env var PRODUCT_NAME, only added if present.GIT_URL: Content of env var GIT_URL, only added if present.JENKINS_PIPELINE: Name of the current Jenkins build job.JENKINS_URL: URL to the current Jenkins build job.JENKINS_WORKSPACE: Path to the Jenkins pipeline workspace.TEST_LEVEL: Content of env var TEST_LEVEL, only added if present.PRODUCT_VERSION: Content of env var PRODUCT_VERSION, only added if present.GIT_COMMIT: Content of env var GIT_COMMIT, only added if present.JENKINS_BUILD_ID: Number of current Jenkins build.JENKINS_EXECUTOR_NUMBER: Number of current Jenkins executor.JENKINS_NODE_NAME: Name of current node the current build is running on.TOTAL_EXECUTION_TIME: Full run time of the build in seconds.SETUP_TIME: Full run time of the setup phase of the build in seconds.EXECUTION_TIME: Full run time of the execution phase of the build in seconds.TEARDOWN_TIME: Full run time of the teardown phase of the build in seconds.SETUP_PERCENTAGE: Percentage of the setup phase in relation to the TOTAL_EXECUTION_TIME.EXECUTION_PERCENTAGE: Percentage of the execution phase in relation to the TOTAL_EXECUTION_TIME.TEARDOWN_PERCENTAGE: Percentage of the teardown phase in relation to the TOTAL_EXECUTION_TIME.QUEUE_TIME: Time until the build starts in seconds.COMMIT_TO_START_TIME: Time from commit to start of the build in seconds, only for push-based executions.TIME_TO_ERROR: Time until an error occurred in seconds, only for stages in the execution phase.TIME_TO_ERROR, will be filtered out before collection
+ * @param build
+ * the pipeline raw build
+ * @param testExecutionSteps
+ * the stages of the pipeline build
+ * @return the collected build information and parameters in ATX parameters format
+ */
+def getBuildMetrics(build, testExecutionSteps) {
+ def timeValues = calculateTime(testExecutionSteps, build)
+ def metrics = [
+ [name: "TOTAL_EXECUTION_TIME", direction: "OUT", value: timeValues.totalDuration],
+ [name: "SETUP_TIME", direction: "OUT", value: timeValues.setupDuration],
+ [name: "EXECUTION_TIME", direction: "OUT", value: timeValues.executionDuration],
+ [name: "TEARDOWN_TIME", direction: "OUT", value: timeValues.teardownDuration],
+ [name: "SETUP_PERCENTAGE", direction: "OUT", value: timeValues.setupPercentage],
+ [name: "EXECUTION_PERCENTAGE", direction: "OUT", value: timeValues.executionPercentage],
+ [name: "TEARDOWN_PERCENTAGE", direction: "OUT", value: timeValues.teardownPercentage],
+ [name: "QUEUE_TIME", direction: "OUT", value: timeValues.queueDuration],
+ [name: "COMMIT_TO_START_TIME", direction: "OUT", value: timeValues.fromCommitToStartTime],
+ [name: "TIME_TO_ERROR", direction: "OUT", value: timeValues.errorTime]
+ ]
+ return metrics.findAll { param -> param.value != null }
+}
+
/**
* Generates a test.guide compatible JSON report of the pipeline build.
*
@@ -155,11 +203,13 @@ def getBuildConstants(build, customConstants) {
* the collected constants
* @param executionTestSteps
* the stages of the pipeline build
+ * @param parameters
+ * the collected parameters
* @param logFile
* the log file name if per-step logging is enabled
* @return the formatted JSON report
*/
-def generateJsonReport(build, attributes, constants, executionTestSteps, logFile) {
+def generateJsonReport(build, attributes, constants, executionTestSteps, parameters, logFile) {
Map testcase = [:]
testcase.put("@type", "testcase")
@@ -174,6 +224,7 @@ def generateJsonReport(build, attributes, constants, executionTestSteps, logFile
}
testcase.put("constants", constants)
testcase.put("executionTestSteps", executionTestSteps)
+ testcase.put("parameters", parameters)
def testCases = [testcase]
JsonBuilder jsonBuilder = new JsonBuilder([name : 'JenkinsPipeline',
@@ -183,6 +234,189 @@ def generateJsonReport(build, attributes, constants, executionTestSteps, logFile
return JsonOutput.prettyPrint(jsonString)
}
+/**
+ * Calculates the duration metrics and their percentages.
+ *
+ * @param executionTestSteps
+ * the stages of the pipeline build
+ * @param build
+ * the pipeline build
+ * @return a map containing the calculated durations in seconds and percentages
+ */
+def calculateTime(executionTestSteps, build) {
+ def currentPhase = 'setup'
+ def setupDuration = 0.0
+ def executionDuration = 0.0
+ def teardownDuration = 0.0
+ def queueDuration = (build.getStartTimeInMillis() - build.getTimeInMillis()) / 1000.0
+ def errorTime = calculateErrorTime(executionTestSteps)
+ def fromCommitToStartTime = currentBuild.getBuildCauses('jenkins.branch.BranchEventCause').isEmpty() ? null : getTimeFromCommitToStart(build)
+
+ executionTestSteps.each { stage ->
+ def stageName = stage.name
+ // Update the current phase based on the stage name
+ currentPhase = getCurrentPhase(stageName, currentPhase)
+
+ if (currentPhase == 'teardown') {
+ teardownDuration += stage.duration
+ } else if (currentPhase == 'setup') {
+ setupDuration += stage.duration
+ } else if (currentPhase == 'execution') {
+ executionDuration += stage.duration
+ }
+ }
+
+ def totalDuration = queueDuration + setupDuration + executionDuration + teardownDuration
+
+ def setupPercentage = 0.0
+ def executionPercentage = 0.0
+ def teardownPercentage = 0.0
+
+ if (totalDuration > 0) {
+ setupPercentage = (setupDuration / totalDuration) * 100
+ executionPercentage = (executionDuration / totalDuration) * 100
+ teardownPercentage = (teardownDuration / totalDuration) * 100
+ }
+
+ return [
+ setupDuration: convertTimeValueToDouble(setupDuration),
+ setupPercentage: convertTimeValueToDouble(setupPercentage),
+ executionDuration: convertTimeValueToDouble(executionDuration),
+ executionPercentage: convertTimeValueToDouble(executionPercentage),
+ teardownDuration: convertTimeValueToDouble(teardownDuration),
+ teardownPercentage: convertTimeValueToDouble(teardownPercentage),
+ queueDuration: convertTimeValueToDouble(queueDuration),
+ totalDuration: convertTimeValueToDouble(totalDuration),
+ fromCommitToStartTime: fromCommitToStartTime != null ? convertTimeValueToDouble(fromCommitToStartTime) : null,
+ errorTime: errorTime != null ? convertTimeValueToDouble(setupDuration + queueDuration + errorTime) : null]
+}
+
+/**
+ * Converts a none null time value to a double with one decimal place.
+ * @param value
+ * the time value to be converted
+ * @return the time value converted as double
+ */
+def convertTimeValueToDouble(def value) {
+ return String.format("%.1f", value).replace(',', '.').toDouble()
+}
+
+/**
+ * Determines the time until an error within the execution phase occurs.
+ *
+ * @param executionTestSteps
+ * the stages of the pipeline build
+ * @return the duration value in seconds or null in case of a error free build
+ */
+def calculateErrorTime(executionTestSteps) {
+ def errorTime = null
+ def accumulatedTime = 0
+ def currentPhase = 'setup'
+
+ executionTestSteps.each { stage ->
+ if (stage == null) {
+ return
+ }
+
+ def stageName = stage.get('name')
+ def stageDuration = stage.get('duration', 0)
+
+ currentPhase = getCurrentPhase(stageName, currentPhase)
+
+ // Only process stages in the execution phase
+ if (currentPhase == 'execution') {
+ def teststeps = stage.get('teststeps', [])
+ def stageErrorTime = findNonPassedVerdict(teststeps, stageDuration)
+ if (stageErrorTime != null) {
+ errorTime = accumulatedTime + stageErrorTime
+ return errorTime
+ }
+ accumulatedTime += stageDuration
+ }
+ }
+ return errorTime
+}
+
+/**
+ * Determines the respective phase of the stage being examined.
+ *
+ * @param stageName
+ * the name of the current stage
+ * @param currentPhase
+ * the corresponding phase in which the current stage is located
+ * @return the current phase
+ */
+def getCurrentPhase(stageName, currentPhase) {
+ if (stageName.contains("stage (Declarative: Post")) {
+ return 'teardown'
+ } else if (!stageName.contains("stage (Declarative") && currentPhase == 'setup') {
+ return 'execution'
+ }
+ return currentPhase
+}
+
+/**
+ * Scans the verdicts value while traversing the stages of the pipeline build
+ *
+ * @param steps
+ * the steps within a stage
+ * @param accumulatedTime
+ * accumulated duration values until an error occurred
+ * @return the duration value in seconds or null in case of a error free build
+ */
+def findNonPassedVerdict(steps, accumulatedTime) {
+ def errorTime = null
+ steps.each { step ->
+ if (step == null) {
+ return
+ }
+
+ def stepType = step.get('@type')
+ if (stepType == 'teststep') {
+ def verdict = step.get('verdict')
+ if (verdict == 'FAILED' || verdict == 'ERROR') {
+ errorTime = accumulatedTime
+ return errorTime
+ }
+ } else if (stepType == 'teststepfolder') {
+ def nestedSteps = step.get('teststeps', [])
+ errorTime = findNonPassedVerdict(nestedSteps, accumulatedTime)
+ if (errorTime != null) {
+ return errorTime
+ }
+ }
+ }
+ return errorTime
+}
+
+/**
+ * Determines the time from commit to start of the build
+ *
+ * @param build
+ * the pipeline build
+ * @return the duration value in seconds
+ */
+@NonCPS
+def getTimeFromCommitToStart(build) {
+ def commitTime = null
+ def startTime = build.getStartTimeInMillis() / 1000
+ try {
+ def process = ["git", "-C", env.WORKSPACE, "show", "-s", "--format=%ct", env.GIT_COMMIT].execute()
+ def output = new StringBuffer()
+ def error = new StringBuffer()
+ process.consumeProcessOutput(output, error)
+ process.waitFor()
+
+ if (process.exitValue() == 0) {
+ def commitTimeStamp = output.toString().trim().toLong()
+ commitTime = startTime - commitTimeStamp
+ }
+ } catch (Exception e) {
+ println("Exception occurred: ${e.message}")
+ }
+ return commitTime
+}
+
/**
* Gets the current build result.
*
@@ -398,6 +632,7 @@ def createTestStep(row, appendLogs, skipped = false) {
*/
@NonCPS
def createTestStepFolder(row, appendLogs) {
+ def duration = row.getDurationMillis() / 1000
Map testStepFolder = [:]
def name = getTestStepName(row)
@@ -430,6 +665,7 @@ def createTestStepFolder(row, appendLogs) {
testStepFolder.put("@type", "teststepfolder")
testStepFolder.put("name", name)
+ testStepFolder.put("duration", duration)
testStepFolder.put("teststeps", testSteps)
if (appendLogs) {