-
Notifications
You must be signed in to change notification settings - Fork 86
Tutorials
- Requirements For Tutorials
- Quick Start
- Triggering Webhook Scans with CxFlow
- GitHub Webhook Tutorial
- GitLab Webhook Tutorial
- GitLab CI/CD
- Azure DevOps Webhook Tutorial
- Azure DevOps Pipeline
- Bitbucket Cloud Webhook Tutorial
- CircleCI
- CxFlow CLI & JIRA Tutorial
- CxFlow Batch Mode Tutorial
- GitHub Actions with JIRA Integration
- CxFlow IAST Integration Tutorial
- CxFlow SonarQube Integration
- CxSAST Branching Project
- Create a folder on the C:\ drive called CxFlow
- Into this folder, download the latest CxFlow .jar for JDK8
https://github.com/checkmarx-ltd/cx-flow/releases
The Java 11 version will have -java11 at the end of the file name
Note This guide is using CxFlow version 1.6.12, if you download another version, input your version in the command below - Download the appropriate version of the example application.yml file for your CxSAST version from https://github.com/checkmarx-ts/cicd-examples/tree/master/cxflow/YML-templates
Under the Checkmarx heading, you should enter your service account's username, password, and confirm the base-url. Under the Source Control heading please enter your token and web-hook token if you have entered a value for the web-token different from this guide's value of 12345. Finally, enter another port if you are using a port other than 8982. Note Each lesson will walk through creating a personal access token in the source control.
- Once the .yml is completely filled out including the personal access token, start CxFlow in webhook mode by opening CMD prompt/shell, navigate to your CxFlow directory (created above) and entering the following, after updating the path\to\CxFlow folder:
cd C:\CxFlow
java -jar cx-flow-1.6.19.jar --spring.config.location="<path\to>\CxFlow\application.yml" --web
Note The client-secret value included here is the correct value for CxSAST and is not actually a secret value. It is the OIDC client secret used for API login to Checkmarx.
- If following this guide for demo purposes, you can use ngrok to generate a resolvable address for your CxFlow. This guide includes ngrok in its examples
- Download ngrok from https://ngrok.com/download and unzip to the CxFlow folder
- Start ngrok on port 8982 by opening CMD and entering the following command:
cd C:\CxFlow
ngrok http 8982
This quick start guide describes how to trigger a CxSAST scan on a Pull Request and a Push to a protected GitHub branch. Pushes to a protected branch will create GitHub Issues from the scan results.
Requirements:
- Create a folder called CxFlow
- Into this folder, download the latest CxFlow .jar for JDK8
https://github.com/checkmarx-ltd/cx-flow/releases
Note This guide is using CxFlow version 1.5.4, if you download another version, input your version in the command below - In the folder create a file titled application.yml
- Add the text below to the application.yml file replacing any values enclosed in ###<>### with your appropriate value
Under the Checkmarx heading, you should enter your service account's username, password, and confirm the base-url. Under the GitHub heading please enter your GitHub token and web-hook token if you have entered a value for the web-token different from this guide's value of 12345. Finally, enter another port if you are using a port other than 8982.
Note This .yml file is for CxSAST version 8.9. For later versions, navigate to the 9.0 link on the side bar
Note The client-secret value included here is the correct value for CxSAST and is not actually a secret value. It is the OIDC client secret used for API login to Checkmarx.
server:
port: 8982
logging:
file:
name: flow.log
cxflow:
bug-tracker: GitHub
bug-tracker-impl:
- GitHub
branches:
- main
filter-severity:
filter-category:
- SQL_Injection
- Stored_XSS
- Reflected_XSS_All_Clients
filter-cwe:
filter-status:
#mitre-url: https://cwe.mitre.org/data/definitions/%s.html
#wiki-url: https://custodela.atlassian.net/wiki/spaces/AS/pages/79462432/Remediation+Guidance
checkmarx:
username: ###<username>###
password: ###<password>###
client-secret: 014DF517-39D1-4453-B7B3-9930C563627C
base-url: ###<CxSAST url or http://localhost>###
team: \CxServer\SP\Company
url: ${checkmarx.base-url}/cxrestapi
#WSDL Config
portal-url: ${checkmarx.base-url}/cxwebinterface/Portal/CxWebService.asmx
sdk-url: ${checkmarx.base-url}/cxwebinterface/SDK/CxSDKWebService.asmx
portal-wsdl: ${checkmarx.base-url}/Portal/CxWebService.asmx?wsdl
sdk-wsdl: ${checkmarx.base-url}/SDK/CxSDKWebService.asmx?wsdl
github:
webhook-token: 12345
token: ###<githubtoken>###
url: https://github.com
api-url: https://api.github.com/repos/
false-positive-label: false-positive
block-merge: true
- If following this guide for demo purposes, you can use ngrok to generate a resolvable address for your CxFlow. This guide includes ngrok in its examples
- Download ngrok from https://ngrok.com/download and unzip to the CxFlow folder
- Start ngrok on port 8982 by opening CMD and entering the following command:
ngrok http 8982
- Create an account at www.github.com
- Create a public repository titled CxFlowGitHub
- Import code from your favorite small demo codebase on github. This guide will use
https://github.com/psiinon/bodgeit - Create a token by clicking on your profile in upper right corner > settings
- Click Developer settings > Personal Access Tokens > Generate New Token
- Give the token a name, for example cxFlow-minimal, and both repo:status and public_repo scopes.
- Ensure "Issues" are enabled on the project Settings > Options > Features > Issues
- Copy this token and keep it safe. It should be posted into the token <> of the application.yml
- Once the .yml is completely filled out, start CxFlow in webhook mode by opening CMD prompt/shell, navigate to your CxFlow directory (created above) and entering the following, after updating the path\to\CxFlow folder:
java -jar cx-flow-1.6.19.jar --spring.config.location="<path\to>\CxFlow\application.yml" --web
- Create a webhook by selecting your profile and selecting the repo you just created
- Navigate to Settings > Webhooks > Add Webhook and fill in the details
- Payload URL: ngrok example: http://xxxxx.ngrok.io
- Content type: application/json
- Secret: Webhook token from .yml file, in this example 12345
- Select Events: Pull Requests, Pushes
- Click Add Webhook, there should be a checkmarx next to the hook name now
- Navigate to Settings > Webhooks > Add Webhook and fill in the details
- Open your IDE of choice. This demo will use IntelliJ
- Check out code using Check out from Version Control, input the URL for your repo example:
https://github.com/<username>/CxFlowGitHub - Open README.md and add a line, example: CxFlowMasterPush-Test1
- Commit to local git repo and push to origin with comments by clicking the following: VCS > Git > Commit File enter a message like CxFlow push to a protected branch
- Click commit and push
- Click Push and enter GitHub credentials on popup. Username is your username, password is the token you created above.
- Check out code using Check out from Version Control, input the URL for your repo example:
- Navigate to the Checkmarx web portal. You should see a new scan in the CxSAST queue
Notice the project name is the RepoName-Branch
Notice the team is the GitHub organization. This is set by the team line in the .yml file. It auto creates a team if it does not exist. This can be overridden in the config file with the multi-tenant setting. Please see the CxFlow configuration page for more information. - When the scan finishes, you should see issues on the Issue tab of your GitHub repo
https://github.com/<username>/CxFlowGitHub/issues - Examine the following issue CX SQL_Injection @ roost/basket.jsp [main]
- Open the Checkmarx link and examine the finding
- We will now trigger CxFlow from a Pull Request to a protected branch, from branch security-fix to main
- Open IntelliJ and create a new local branch called security-fix VCS > Git > Branches > New Branch
- Type security-fix and click ok
- Open basket.jsp under the root folder and replace lines 53-55 with the following
//Statement stmt = conn.createStatement();
//Security Fix
PreparedStatement preparedStatement = con.prepareStatement(sql);
try {
//ResultSet rs = stmt.executeQuery("SELECT * FROM Baskets WHERE basketid = " + basketId);
String sql = "SELECT * FROM Baskets WHERE basketid =?");
preparedStatement.setString(1, basketId);
ResetSet rs = preparedStatement.executeQuery();
- add the following to line 7 to import the correct package
<%@ page import="java.sql.PreparedStatement" %>
- Save the file, commit to the local repo and push to origin:
- File > Save All
- VCS > Git > Commit File and add commit message like added prepared statement on line 55
- Click Commit and Push, then Push
- Navigate to GitHub
- Navigate to Pull Requests
- Click Compare and Pull Request > Create Pull Request
- Alternatively you can create the pull request through the IDE. In IntelliJ click VCS > Create Pull Request
- In GitHub there will be some checks that have not finished yet - Checkmarx Scan
- In the Checkmarx web portal there will be a new CxSAST scan where the project name is RepoName-Branch
- Once the scan finished you can see the post in the GitHub merge pull request comments with all the vulnerabilities found
- The basket.jsp SQLi is gone
- Click Merge Pull Request > Confirm Merge to accept the risk CxSAST has posted in the comments
- After confirming the pull request, there will be a new CxSAST scan in the Checkmarx web portal for the main branch
- In GitHub Issues there will be one fewer vulnerability
- In the Checkmarx web portal, the CxFlowGitHub-main project will now have both solved and recurrent issues.
- Open your IDE of choice. This tutorial will use IntelliJ
- Check out code using Check out from Version Control, input the URL for your repo example:
https://github.com/<username>/CxFlowBodgeit - Open README.md and add a line, example: CxFlowMasterPush-Test1
- Commit to local git repo and push to origin with comments by clicking the following: VCS > Git > Commit File enter a message like CxFlow push to a protected branch
- Click commit and push
- Click Push and enter the source control credentials on popup. Username is your username, password is the personal access token you created.
- Check out code using Check out from Version Control, input the URL for your repo example:
- Navigate to the Checkmarx web portal. You should see a new scan in the CxSAST queue
Notice the project name is the RepoName-Branch
Notice the team is the organization. This is set by the team line in the .yml file. It auto creates a team if it does not exist. This can be overridden in the config file with the multi-tenant setting. Please see the CxFlow configuration page for more information. - When the scan finishes, you should see issues on the Issue tab of your repo
https://github.com/<username>/CxFlowBodgeit/issues - Examine the following issue CX SQL_Injection @ root/basket.jsp [main]
- Open the Checkmarx link and examine the finding
- We will now trigger CxFlow from a Pull Request to a protected branch, from the branch security-fix to main
- Open IntelliJ and create a new local branch called security-fix VCS > Git > Branches > New Branch
- Type security-fix and click ok
- Open basket.jsp under the root folder and replace lines 53-55 with the following
//Statement stmt = conn.createStatement();
//Security Fix
PreparedStatement preparedStatement = con.prepareStatement(sql);
try {
//ResultSet rs = stmt.executeQuery("SELECT * FROM Baskets WHERE basketid = " + basketId);
String sql = "SELECT * FROM Baskets WHERE basketid =?");
preparedStatement.setString(1, basketId);
ResetSet rs = preparedStatement.executeQuery();
- add the following to line 7 to import the correct package
<%@ page import="java.sql.PreparedStatement" %>
- Save the file, commit to the local repo and push to origin:
- File > Save All
- VCS > Git > Commit File and add commit message like added prepared statement on line 55
- Click Commit and Push, then Push
- Navigate to the source control in your browser
- Navigate to Pull Requests
- Click Compare and Pull Request > Create Pull Request
- Alternatively you can create the pull request through the IDE. In IntelliJ click VCS > Create Pull Request
- In the pull request there will be some checks that have not finished yet - Checkmarx Scan
- In the Checkmarx web portal there will be a new CxSAST scan where the project name is RepoName-Branch
- Once the scan is finished you can see the post in the pull request comments with all the vulnerabilities found
- The basket.jsp SQLi is gone
- Click Merge Pull Request > Confirm Merge to accept the risk CxSAST has posted in the comments
- After confirming the pull request, there will be a new CxSAST scan in the Checkmarx web portal for the master branch
- In Issues section of the source control there will be one fewer vulnerability
- In the Checkmarx web portal, the CxFlowBodgeit-main project will now have both solved and recurrent issues.
This tutorial is designed to teach the following topics:
- How to scan on a Pull Request to a Protected Branch
- How to scan on a Push to Protected Branch
- GitHub Issue Creation on Push to Protected Branch
- Update the bugtracker section of the application.yml file with the following
bug-tracker: GitHub
bug-tracker-impl:
- GitHub
-
Create an account at www.github.com
-
Create a public repository titled CxFlowBodgeit
-
Import code from your favorite small demo codebase on github. This guide will use
https://github.com/psiinon/bodgeit -
Create a token by clicking on your profile in upper right corner > settings
- Click Developer settings > Personal Access Tokens > Generate New Token
- Give the token a name, for example cxFlow-minimal, and both repo:status and public_repo scopes.
- Ensure "Issues" are enabled on the project Settings > Options > Features > Issues
- Copy this token and keep it safe. It should be posted into the token <> of the application.yml
- After .YML file is completely filled out and saved, start CxFlow in webhook mode by opening a CMD prompt
-
Create a webhook by selecting your profile and selecting the repo you just created
- Navigate to Settings > Webhooks > Add Webhook and fill in the details
- Payload URL: ngrok example: https://xxxx.ngrok.io
- Content type: application/json
- Secret: Webhook token from .yml file, in this example 12345
- Select Events: Pull Requests, Pushes, Branch or tag deletion
- Click Add Webhook, there should be a checkmarx next to the hook name now
- Navigate to Settings > Webhooks > Add Webhook and fill in the details
-
Continue to Triggering Webhook Scans with CxFlow
This tutorial is designed to teach the following topics:
- How to scan on a Merge Request to a Protected Branch
- How to scan on a Push to Protected Branch
- GitLab Issue Creation on a Push to Protected Branch
- Update the bugtracker section of the application.yml file with the following
bug-tracker: GitLab
bug-tracker-impl:
- GitLab
-
Create an account at http://www.gitlab.com
-
Create a new private group called <yourname>-checkmarx
-
Create a new subgroup called GitLab CxFlow
-
Create a new private project called CxFlowGitLab
- Import code from your favorite small demo codebase
- This tutorial will use https://github.com/psiion/bodgeit
-
Click Import Project > Repo By URL
- Git Repository URL https://github.com/psiion/bodgeit
- Project Name = CxFlowGitLab
- Ensure the project's status is set to private and click Create Project
-
Create a token by clicking your profile in upper right corner >settings
- Click Access Tokens & add a personal access token
- Give the token api, read_user, write_repository, read_registry scopes
- Copy this token and keep safe - it should be pasted into the token: <> of the application.yml
-
After .YML file is completely filled out and saved, start CxFlow in webhook mode by opening a CMD prompt and typing the following
-
Create a webhook by selecting Projects>Your Projects and select the repo you just created
-
Click Settings>Webhooks and fill in details
- URL = ngrok location of CxFlow that is running - example: https://xxxx.ngrok.io
- Secret = webhook-token: from .yml file - example: 12345
- Trigger = Push events, Merge request events
- Click Add Webhook
-
Continue to Triggering Webhook Scans with CxFlow
Back to Tutorials Table of Contents
There are several ways of integrating Checkmarx security scans into GitLab’s ecosystem. This document specifically outlines how to integrate GitLab with Checkmarx’s Containerized CxFlow CLI. Checkmarx integrates with GitLab, enabling the identification of new security vulnerabilities with proximity to their creation. GitLab integration triggers Checkmarx scans as defined by the GitLab CI/CD pipeline. Once a scan is completed, both scan summary information and a link to the Checkmarx Scan Results will be provided. Both CxSAST and CxSCA are supported within the GitLab integration.
The following steps represent the containerized CxFlow CLI integration flow:
- GitLab’s CI/CD pipeline is triggered (as defined in the .gitlab-ci.yml file)
- During the test stage of GitLab’s CI/CD pipeline, Checkmarx’s containerized CxFlow CLI is invoked
- CxFlow CLI triggers a security scan via the Checkmarx Scan Manager
- Results can be configured to be displayed with GitLab’s ecosystem or a supported bug tracker via CxFlow YAML configuration
- a. Results will be within Checkmarx Scan Results within the Checkmarx Manager Server
- b. Results can be accessed within GitLab’s Merge Request Overview (if the scan was initiated during a Merge Request)
- c. Results can be accessed within GitLab’s Issues if configured (or can be filtered into external bug tracker tools)
- d. Results can be accessed within GitLab’s security dashboard, if you have access to it (Gold/Ultimate packages or if your project is public)
! Within GitLab, CxFlow CLI will zip the source directory of the repository and send it to the Checkmarx Scan Manager to perform the security scan
GitLab can access a running Checkmarx CxSAST Server with an up-to-date Checkmarx license If performing CxSCA scans, you must have a valid CxSCA license and GitLab must be able to access the CxSCA tenant To review scan results within GitLab’s Security Dashboard, you need the Gold/Ultimate tier or the GitLab project must be public
- To review results in the issue management of your choice (i.e. JIRA) configuration is needed in the CxFlow YAML file, please refer to Bug Tracker documentation
To allow for easy configuration, it is necessary to create environment variables with GitLab to run the integration. For more information on GitLab CI/CD variables, visit here: GitLab: CI/CD - Environment Variables Edit the CI/CD variables under Settings → CI / CD → Variables and add the following variables for a CxSAST and/or CxSCA scan:
Variable/ Inputs | Value |
---|---|
GITLAB_TOKEN | API token to create Merge Request Overview entries, should have “api” privileges. |
CX_FLOW_BUG_TRACKER (Type: Variable) | Type of bug tracking ('GitLabDashboard' or ‘GitLab’). For vulnerabilities to be exported to GitLab’s Dashboard, use ‘GitLabDashboard’ and for vulnerabilities to be added to GitLab’s Issues, use ‘GitLab’ For more details on complete list of Bug Trackers, please refer to CxFlow Configuration |
CX_FLOW_ENABLED_VULNERABILITY_SCANNERS | Vulnerability Scanners (sast, sca, ast, cxgo). Multiple comma separated values allowed. |
CHECKMARX_PASSWORD (Type: Variable) | Password for CxSAST |
CHECKMARX_SERVER (Type: Variable) | The base URL of CxSAST Manager Server (i.e. https://checkmarx.company.com) |
CHECKMARX_USERNAME (Type: Variable) | User Name for the CxSAST Manager. User must have ‘SAST Scanner’ privileges. For more information on CxSAST roles, please refer to CxSAST / CxOSA Roles and Permissions |
CHECKMARX_TEAM (Type: Variable) | Checkmarx Team Name (i.e. /CxServer/teamname) |
CHECKMARX_CLIENT_SECRET | Checkmarx OIDC Client Secret |
SCA_TENANT (Type: Variable) | The name of the CxSCA Account (i.e. SCA-CompanyName). Only needed if you have a valid license for CxSCA |
SCA_USERNAME (Type: Variable) | The username of the CxSCA Account. Only needed if you have a valid license for CxSCA |
SCA_PASSWORD (Type: Variable) | The password of the CxSCA Account. Only needed if you have a valid license for CxSCA |
CXGO_CLIENT_SECRET | Client-Secret needed for AST Cloud (CxGo). |
AST_API_URL | API URL for AST scan |
AST_WEBAPPURL | WebApp URL for AST scan |
AST_CLIENT_ID | Client-ID configured within AST. |
AST_CLIENT_SECRET | Client-Secret within AST. |
PARAMS | Any additional parameters for CxFlow. For a full list of all the parameters, check here |
The gitlab configuration file is stored at a remote location within the cxflow repo. It can be directly used as a template using the following syntax.
include: 'https://raw.githubusercontent.com/checkmarx-ltd/cx-flow/master/templates/gitlab/v1/Checkmarx.gitlab-ci.yml'
Also, you can add/override CI/CD variables like this
variables:
CX_FLOW_ENABLED_VULNERABILITY_SCANNERS: "sast,sca"
SCA_appUrl: https://eu.sca.checkmarx.net
SCA_apiUrl: https://eu.api-sca.checkmarx.net
SCA_accessControlUrl: https://eu.platform.checkmarx.net
The GitLab CI/CD pipeline is controlled by a file named ‘.gitlab-ci.yml’ located in the root directory of the project. Please refer to GitLab: CI YAML for more info.
! It is suggested not to over-pollute your companies already existing '.gitlab-ci.yml' file. Instead, create a new YAML file in the root directory named ‘.checkmarx.yml’ and include it in ‘.gitlab-ci.yml’
# Note that image is a docker container maintained by Checkmarx
include: 'https://raw.githubusercontent.com/checkmarx-ltd/cx-flow/master/templates/gitlab/v1/Checkmarx.gitlab-ci.yml'
variables:
CX_FLOW_ENABLED_VULNERABILITY_SCANNERS: sast
CX_TEAM: "/CxServer/MP"
CHECKMARX_USERNAME: $CX_USERNAME
CHECKMARX_PASSWORD: $CX_PASSWORD
CHECKMARX_BASE_URL: $CHECKMARX_SERVER
CHECKMARX_CLIENT_SECRET: $CHECKMARX_CLIENT_SECRET
stages:
- scan
include: '.checkmarx.yml'
stages:
- scan
####Run pipeline To run a Checkmarx scan, you need to trigger the pipeline. The trigger is based on the .gitlab-ci.yml and in the provided sample above, it will be triggered on Merge Requests and on changes to the main branch
- For information on triggering a pipeline scan, please refer to GitLab: triggering a pipeline
- For information on Merge Requests, please refer to GitLab: Merge Requests
While the scan results will always be available in the Checkmarx UI, users can also access results within the GitLab ecosystem. Currently there are three different ways to review results from the scan:
- Merge Request Overview
- GitLab Issues
- Security Dashboard
When you have configured the .gitlab-ci.yml file to scan on merge_requests issues (please refer to GitLab: Pipelines for Merge Requests), a high level report of the Checkmarx scan will be displayed within GitLab Merge Request Overview.
An example of a Merge Request with a Checkmarx scan report can be found in the below image.
When you have configured the BUG_TRACKER variable to use “GitLab”, CxSAST and CxSCA issues found in Checkmarx will be opened within GitLab Issues
For more information on GitLab issues, please refer to GitLab: Issues
An example of Issues created can be found in the below image.
To integrate GitLab’s Security Dashboard with CxFlow, you need to configure CxFlow to use GitLab as the bug tracker. This setup will allow CxFlow to create and manage security issues directly in GitLab, leveraging the Security Dashboard to display results from SAST (Static Application Security Testing) and SCA (Software Composition Analysis) scans.
- You need to configure the
butracker
andbug-tracker-impl
settings in your CxFlow YAML configuration file to use GitLabDashboard.
- Ensure that either SAST or SCA scans are enabled in your project settings. This will generate the necessary artifacts (
gl-sast-report
andgl-sca-report
).
Here’s an example of how you might configure your *.ci.yml
file:
An example of vulnerabilities displayed in the Security Dashboard can be found in the below image.
cxflow:
zip-exclude: projects/[^c].*,src/.*,\.angular/.*,\.vscode/.*,\.git/.*,node_modules/.*,apps/.*,dist/.*,coverage/.*,tmp/.*,out-tsc/.*,e2e/.*,server/.*,test/.*,\.gitlab/.*,\.husky/.*,deploy/.*,dev-cluster/.*,ng-serve/.*,proxy-configs/.*,re-docs/.*,schema/.*,scripts/.*,projects/schema/.*,projects/volterra-clients/.*,projects/stellar-testing/.*,package.json,package-lock.json,tsconfig.json,tsconfig.*.json,README.md,.*\.spec\.ts,.*\.scss,.*\.type\.ts
bug-tracker: GitLabDashboard
branchProtectionEnabled: false
bug-tracker-impl:
# - Azure
# - Csv
# - CxXml
# - CxXml
# - GitHub
# - GitLab
- GitLabDashboard
# - GitLab
# - Rally
# - Json
# - PDF
# - JIRA
# - Sarif
# - SonarQube
# - GITHUBPULL
# - BITBUCKETCOMMIT
branches:
- gl-sast-report: This artifact contains the results of the SAST scan. It includes details about any vulnerabilities found in the source code.
- gl-sca-report: This artifact contains the results of the SCA scan. It includes details about any vulnerabilities found in the dependencies and third-party libraries used by the project.
- Centralized Security Management: By integrating CxFlow with GitLab’s Security Dashboard, you can manage all security issues in one place.
- Automated Issue Creation: CxFlow automatically creates and updates issues in GitLab based on the results of SAST and SCA scans.
- Enhanced Visibility: The Security Dashboard provides a comprehensive view of the security posture of your project, making it easier to track and address vulnerabilities.
- GitLab config for AST
- GitLab config for SAST and SCA combined
- GitLab config for AST Cloud
- GitLab config for SCA
Back to Tutorials Table of Contents
This tutorial is designed to teach the following topics:
- How to scan on a Merge Request to a Protected Branch
- How to scan on a Push to Protected Branch
- Azure Work Item creation on a Push to Protected Branch
- Update the bugtracker section of the application.yml file with the following
bug-tracker: Azure
bug-tracker-impl:
- Azure
- Create an account at https://azure.microsoft.com/en-us/services/devops/
- Create a new organization if one does not already exist
- Create a new private project called CxFlowBodgeit
- Make sure repo type is Git under Advanced
- Click Repos & Import code from your favorite small demo codebase on GitHub
- This tutorial will use https://github.com/psiinon/bodgeit
- Create a token by clicking your profile in upper right corner > Personal Access Tokens
- Give the token a name and change Expiration to Custom defined and set to a year
- Give the token full access to Work Items, Code, Build, Release
- Copy this token and keep safe - it should be pasted into the token: <> of the application-azure.yml
- After .YML file is completely filled out and saved, start CxFlow in webhook mode
- Create a webhook by selecting in the upper left corner Azure DevOps & select the new repo you just created
- Create a Webhook for Merge Requests
- Click Project Settings > Service hooks > Create subscription and fill in details
- Click Web Hooks then Next
- Change drop down to Pull request created
- Repository = CxFlowADO
- Branch = main
- URL = https://<cxflow>/ado/pull
- Note <cxflow> is https ngrok location of CxFlow that is running
- Example: https://xxxxx.ngrok.io/ado/pull
- Basic authentication username = webhook-token: left side of : from .yml file - example: cxflow
- Basic authentication password = webhook-token: right side of : from .yml file - example: 12345
- Click Test and a green check should appear, then click Finish
- Create a Webhook for Push to Master
- Click Project Settings > Service hooks > Create subscription and fill in details
- Click Web Hooks then Next
- Change drop down to Code pushed
- Repository = CxFlowADO
- Branch = main
- URL = https://<cxflow>/ado/push
- Note <cxflow> is https ngrok location of cxflow that is running
- Example: https://xxxxx.ngrok.io/ado/push
- Basic authentication username = webhook-token: left side of : from .yml file - example: cxflow
- Basic authentication password = webhook-token: right side of : from .yml file - example: 12345
- Click Add Webhook
- Continue to Triggering Webhook Scans with CxFlow
- Windows Agents
- Docker Container
- Configuration
- Upgrading to CxSAST v9.0 and Above
- Environment Variables
- Scripts
- Building
- Wget Script
Back to Tutorials Table of Contents
This documentation is to help organizations create and run CxFlow in Azure DevOps (ADO) Pipelines.
The key features of doing this are:
- Utilize CxFlow as a Stage/Task in ADO Pipelines
- Automatically determine matching variables between the Azure Pipeline and Checkmarx
- Variables can optionally be statically set by the developer team
- Automatically generating work items from the pipeline if required
- Cross platform Azure DevOps Agent support
- Docker image for cross organisation updating
- Updating the image will update all projects configurations
- Ability to create custom workflows for pipelines to run via the endpoint script
- Run multi-stage CxFlow jobs
Below are examples of Azure DevOps Pipeline YAML files that use CxFlow to scan the code and create Work Items with vulnerabilities. CxFlow is invoked with custom workflow(s) that an organization might require.
- Run multi-stage CxFlow jobs
This Windows based script is called entrypoint.ps1 which is the Powershell script that allows developers to run a wrapper around CxFlow. This can be distributed to all (security focused) Agents in the environment along with the application.yml and the Java archive of CxFlow.
Auto-downloader
The Powershell script has the ability to download automatically the current release of CxFlow as a Jar off the GitHub Releases. This feature can be disabled in environments that do not allow out-bound connections to the internet or downloading of binaries.
trigger:
- main
pool:
name: Agents
vmImage: 'CxAgent'
stages:
- stage: Security
jobs:
- job: CxFlow
steps:
# This will have to be present on the agent
- task: PowerShell@2
inputs:
# Full or Relative path to Powershell script
filePath: '.\entrypoint.ps1'
The docker container version of CxFlow runs the exact same code as the Linux based Agents do. The only primary difference is that you can create a Docker image (container all the code and configuration) in a single binary which is immutable and can be distributed by using Docker Registries.
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Security
jobs:
- job: CxFlow
steps:
# This step might not be needed if Docker is pre installed
- task: DockerInstaller@0
inputs:
dockerVersion: '17.09.0-ce'
# Run CxFlow
- bash: docker run -v `pwd`:/code --rm --env-file <(env) organisation/cxflow-azure:latest
env:
# Required Settings
AZURE_ACCESS_TOKEN: $(System.AccessToken)
CHECKMARX_PASSWORD: $(CHECKMARX_PASSWORD)
The application.yml is where most of the static settings are stored that do not change. These can be configured per organisation and nothing sensitive should be stored in this file unless encrypted (encrypt them using Jasypt).
# ...
checkmarx:
username: ${CHECKMARX_USER}
password: ${CHECKMARX_PASSWORD}
client-secret: 014DF517-39D1-4453-B7B3-9930C563627C
base-url: ${CHECKMARX_URI}
multi-tenant: false
scan-preset: ${CHECKMARX_PRESET:Checkmarx Default}
configuration: Default Configuration
team: ${CHECKMARX_TEAM:\CxServer\SP\Company}
preserve-xml: true
incremental: false
azure:
token: ${AZURE_ACCESS_TOKEN}
url: ${AZURE_URL}
api-version: 5.0
issue-type: issue
closed-status: Closed
open-status: Active
false-positive-label: false-positive
block-merge: true
When updating to CxSAST version 9.0 or above, the REST API changes so CxFlow needs to swap to version 9.0 support and some configuration changes need to be done. This requires the following changes:
More information can be found on the CxSAST Version 9.0 page
# ...
checkmarx:
version: 9.0
client-id: resource_owner_client
client-secret: 014DF517-39D1-4453-B7B3-9930C563627C
scope: access_control_api sast_rest_api
# ...
team: /CxServer/Checkmarx/CxFlow
The Team syntax changes from version 8.9 to 9.0. Originally back-slashes are now forward-slashes.
Here is a list the different variables that can be passed into the Docker environment or the endpoint.sh script.
Name | Required? | Description |
---|---|---|
AZURE_ACCESS_TOKEN | Yes | This is the token that is used to clone the repository and open/edit/close Work Items. You can use the Azure System.AccessToken |
AZURE_URL | No | This is the URL to the organisation in Azure. Default is System.TeamFoundationCollectionUri |
CHECKMARX_URI | Yes | The URL/URI of where Checkmarx is hosted. This can be built in by default by editing the application.yml. |
CHECKMARX_USERNAME | Yes | Username of the Checkmarx user (typically a service scanner account) |
CHECKMARX_PASSWORD | Yes | Password for the Checkmarx user. This should be a Azure Pipeline Secrets or encrypted using Jasypt (see CXFLOW_KEY section). |
CHECKMARX_PROJECT | No | Project name in Checkmarx. The Default is the Azure Project name (Build.Repository.Name). This can work along side the project-script feature of CxFlow. |
CHECKMARX_TEAM | No | Project Team that the project should be under. Default \CxServer\SP\Company. This can be built in by default by editing the application.yml. This can work along side the team-script feature of CxFlow. |
CHECKMARX_PRESET | No | Project Preset to use. Default is Checkmarx Default |
CXFLOW_KEY | No | This key is used for decryption of the tokens or sensitive data using Jasypt. By Default, the application will not decrpt anything. |
CXFLOW_KEY_ALGORITHM | No | Custom algorithm you want to use with Jasypt. The default value is PBEWITHHMACSHA512ANDAES_256. |
These scripts are used on an Azure DevOps Agent as part of a Pipeline. They provide a wrapper around CxFlow to automatically pull out various built-in Azure Pipeline variables to provide a seamless experience for organizations. Many of the variables are dictated based on environment variables passed into the Docker container at run time or the application.yml.
These can be updated to your requirements and can be different from organization-to-organization.
The entrypoint.sh script is to support both Linux based agents and it’s the entry point for the Docker image.
Docker Image
We recommend that organizations create a git repository of these files to track changes and easily deploy the images for all pipelines in the organisation in a private registry.
Note: This Docker image can be used for any pipelines as long as the ADO variables being supplied are updated to corresponding build systems/bug tracking systems.
Command Line Interface
In the working directory of the source code, run the following commands:
# Building the Docker image
docker build -t organisation/cxflow .
# Pushing image to registry
docker push private-registry:5000/organisation/cxflow
Feel free to change the name of the image to anything but make sure that the pipelines match the container name.
Build CxFlow using an Azure Pipeline
If you have created a separate repository in Azure DevOps and use this simple pipeline to build and push the Docker image into an internal registry. This allows for organisations to automatically make updates to CxFlow, commit the changes, build the Docker container and push them to a globally accessible directory.
# This Azure Pipeline is for building Docker images using Azure
pool:
vmImage: 'ubuntu-latest'
variables:
imageName: 'organisation/cxflow-azure'
steps:
- task: Docker@2
displayName: Login
inputs:
command: login
containerRegistry: dockerRegistryServiceConnection1
- task: Docker@2
displayName: Build and Push Image
inputs:
repository: $(imageName)
command: 'buildAndPush'
Dockerfile: '**/Dockerfile'
Alternatively, you can download the CxFlow jar directly from the GitHub release page and run using the shell command below to scan the workspace and open work items.
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Security
jobs:
- job: CxFlow
steps:
- task: CmdLine@2
inputs:
script: |
wget -O cxflow.jar https://github.com/checkmarx-ltd/cx-flow/releases/download/1.6.19/cx-flow-1.6.19.jar
java -version
whoami
pwd
java -jar cxflow.jar --spring.config.location="./application.yml" --project --cx-project="InsertCheckmarxProjectNameHere" --alt-project="InsertADOProjectNameHere" --namespace="$(System.TeamProject)" --repo-name="$(Build.Repository.Name)" --branch="$(Build.SourceBranchName)"
This tutorial is designed to teach the following topics:
- How to scan on a Merge Request to a Protected Branch
- How to scan on a Push to Protected Branch which opens tickets in JIRA
- Login or sign up for an account at https://www.bitbucket.org
- Note Use the same email address you used or will use to setup JIRA
- Ensure JIRA & the application.yml are setup according to CxFlow CLI & JIRA
- Connect JIRA and Bitbucket
- Create a new private repository named CxFlowBodgeit by clicking the + button on the sidebar
- Click Import repository to import code from your favorite small demo codebase on GitHub
- This tutorial will use https://github.com/psiinon/bodgeit
- Create an app password by clicking your profile in lower-left corner & Personal settings
- Click App Passwords & Create app password
- Create a Label (i.e. CxFlow)
- Give the app password all Read/Write access to Pull requests & Webhooks
- Copy this app password and keep safe - it should be pasted into the token: <> of the application.yml
- The app password in the YML file should follow the format
<userid>:<app password>
- Once the YAML file is completely filled out and saved, start CxFlow in webhook mode
- In Bitbucket, create a webhook by selecting Repositories & select the new repo you just created
- Click Repository settings>Webhooks>Add Webhook and fill in details
- Title = CxFlow
- URL = ngrok location of cxflow that is running + ?token=webtoken from yml file - example: https://xxxxx.ngrok.io?token=12345
- Choose from a full list of triggers = Push, Pull Request Created
- Click Save
- Continue to Triggering Webhook Scans with CxFlow
Back to Table of Contents
CircleCI can be configured with CxFlow.
Checkmarx CxFlow Orb can be used to simplify your configuration.
Checkmarx CxFlow Orb for executing Checkmarx Scans and Publishing results to various feedback channels.
Additional information regarding Checkmarx can be found here: https://checkmarx.atlassian.net/wiki/spaces/KC/overview
Additional information regarding CxFlow can be found here: https://github.com/checkmarx-ltd/cx-flow/wiki
The following Environment Variables must be set within your CircleCI project for this orb to function:
CHECKMARX_URL: High level dns entry for the Checkmarx Instance including protocol/port (i.e. https://cxsast.example.com)
CHECKMARX_USERNAME: Service Account within Checkmarx that will be used for triggering scans and retrieving results
CHECKMARX_PASSWORD: Password of the Service Account.
CHECKMARX_CLIENT_SECRET: Client secret key associated with your Checkmarx SAST account
AST_CLIENT_ID: Service account Client ID for Checkmarx AST
AST_CLIENT_SECRET: Client secret key associated with your Checkmarx AST account
SCA_USERNAME: Service Account within Checkmarx SCA that will be used for triggering scans and retrieving results
SCA_PASSWORD: Password of the Checkmarx SCA Service Account
SCA_TENANT: Tenant information of the Checkmarx SCA account
CXGO_CLIENT_SECRET: Client secret key associated with your Checkmarx CxGo account
Below is sample config.yml for CircleCI using Checkmarx CxFlow Orb
jobs:
cx-scan:
executor: cxflow/default
steps:
- checkout
- cxflow/scan:
preset: Checkmarx Express
report-file: checkmarx.json
scanners: 'sast'
team: /CxServer
version: '9.4'
incremental: false
params: '--cx-flow.thresholds.High=0 --cx-flow.thresholds.Medium=0'
- store_artifacts:
path: checkmarx.json
orbs:
cxflow: checkmarx-ts/[email protected]
version: 2.1
workflows:
sast-scan:
jobs:
- cx-scan:
filters:
branches:
only: master
version: 2
As shown in above sample file, additional parameters can be passed in cxflow using params attribute. Thresholds for High Issue is passed as '--cx-flow.thresholds.High=0' inside 'params' attribute in sample config.yml.
-
Checkmarx CxFlow Orb Readme
https://github.com/checkmarx-ts/checkmarx-cxflow-orb#readme -
Checkmarx CxFlow Orb Documentation
https://circleci.com/developer/orbs/orb/checkmarx-ts/cxflow -
Orb Introduction
https://circleci.com/docs/2.0/orb-intro/?section=configuration
This tutorial is designed to teach the following topics:
- How to configure a Jira Cloud project for CxFlow
- Automated ticket creation using CxFlow CLI
- Scanning via CxFlow CLI
- Sign up for free Atlassian Cloud account at https://www.atlassian.com/try/cloud/signup?bundle=jira-software&edition=free
-
Note If your company email is already associated with an Atlassian account, to follow this guide:
- Open an incognito browser window
- Navigate to https://www.atlassian.com/try/cloud/signup?bundle=jira-software&edition=free
- Use a personal or other email account to create an account
- During the auto-setup choose the following options
- I am experienced with Jira
- My team is experienced with agile methodologies
- We spend our time working on fixing bugs
- We have a flexible schedule to finish our work
- Create a new project & choose a Kanban project
- Project Name = APPSEC
- Project Key = APPSEC
- Note The 'Jira Project' field in the .yml file corresponds to this 'Project Key' and is case sensitive
- Create an API token from your Atlassian account:
- Log in to https://id.atlassian.com/manage-profile/security/api-tokens
- Click Create API token.
- From the dialog that appears, enter ‘CxFlow’ and click Create.
- Click Copy to clipboard, then paste the token to your script, or elsewhere to save: it should be pasted into the token: <> of the application.yml
- Create a custom field for this project and issue type screen by clicking the settings wheel in the top right corner
- Click Issues > Custom Fields > Create Custom Field
- Click Tutorials and give it a name “Application”
- Description = CxSAST Project
- Select the checkboxes next to APPSEC: Kanban Bug Screen & APPSEC: Kanban Default Issue Screen
- Click Update
- Create another custom field for Category
- Name = Category
- Description = CxSAST Vulnerability Type
- Select the checkboxes next to APPSEC: Kanban Bug Screen & APPSEC: Kanban Default Issue Screen
- Click Update Note : Jira's credentials configuration differs for on-premises and cloud environments, Please refer to Bug Trackers and Feedback Channels chapter for more details
- Update the bugtracker section of the application.yml file with the following
bug-tracker: JIRA
#bug-tracker-impl:
- After the .YML file is completely filled out and saved
- The following command clones a GitHub repo, creates a CxSAST scan for the cloned repo, and creates tickets according to the .yml file
cd C:\CxFlow
git clone https://github.com/ethicalhack3r/DVWA.git
cd C:\CxFlow
java -jar cx-flow-1.6.19.jar --spring.config.location="C:\CxFlow\application.yml" --scan --f="./DVWA" --cx-team="CxServer\SP\Company" --cx-project="DVWA" --app="DVWA"
- Note The url for the jira section of the .yml file should be the one assigned to you when you first start your Jira account, for example
url: https://<username>.atlassian.net/
- The following command opens tickets for a CxSAST project’s last finished scan according to the .yml file
cd C:\CxFlow
java -jar cx-flow-1.6.19.jar --spring.config.location="C:\CxFlow\application.yml" --project --cx-team="CxServer\SP\Company" --cx-project="DVWA" --app="DVWA"
- Open the APPSEC project in Jira and note the vulnerabilities that have been opened
- You can kick off batch mode ticket creation in any Linux pipeline by supplying the application.yml file and using the following code to download CxFlow and run
- Note Replace cx-project and app flags with environment variables relevant to the pipeline. wget can also be used instead of curl
apk add --update curl
curl -O -k https://github.com/checkmarx-ltd/cx-flow/releases/download/1.6.19/cx-flow-1.6.19.jar
java -jar cx-flow-1.6.19.jar --spring.config.location="./application.yml" --scan --f=. --cx-team="CxServer" --cx-project="Bodgeit" --app="Bodgeit"
Back to Tutorials Table of Contents
This tutorial is designed to teach the following topics:
- Run CxFlow CLI with XML results output
- Automate email notifications on Proposed Not Exploitable Vulnerabilities using the EmailPNEVulns.ps1 script below
- Create a folder on the C:\ drive called CxFlow
- Into this folder, download the latest CxFlow .jar for JDK8
https://github.com/checkmarx-ltd/cx-flow/releases
The Java 11 version will have -java11 at the end of the file name
Note This guide is using CxFlow version 1.6.12, if you download another version, input your version in the command below - Create a new file called EmailPNEVulns.ps1 in C:\Flow with the text at the bottom of the page and replace any values surrounded in ###<>### with your appropriate values, see SMTP Server Prep steps below
- In the same folder create a file titled application-email.yml
- Add the text below to the application-email.yml file replacing any values enclosed in ###<>### with your appropriate value
Under the Checkmarx heading, you should enter your service account's username, password, and confirm the base-url. Under the GitHub heading please enter your GitHub token and web-hook token if you have entered a value for the web-token different from this guide's value of 12345. Finally, enter another port if you are using a port other than 8982.
Note This .yml file is for CxSAST version 8.9. For later versions, navigate to the 9.0 link on the side bar
Note The client-secret value included here is the correct value for CxSAST and is not actually a secret value. It is the OIDC client secret used for API login to Checkmarx.
server:
port: 8982
logging:
file:
name: flow.log
cxflow:
bug-tracker: CxXml
bug-tracker-impl:
- CxXml
filter-severity:
filter-category:
- SQL_Injection
- Stored_XSS
- Reflected_XSS_All_Clients
filter-cwe:
filter-status:
- Proposed Not Exploitable
checkmarx:
username: ###<cxsast username>###
password: ###<cxsast password>###
client-secret: 014DF517-39D1-4453-B7B3-9930C563627C
base-url: ###<CxSAST url or http://localhost>###
team: \CxServer\SP\Company
url: ${checkmarx.base-url}/cxrestapi
#WSDL Config
portal-url: ${checkmarx.base-url}/cxwebinterface/Portal/CxWebService.asmx
sdk-url: ${checkmarx.base-url}/cxwebinterface/SDK/CxSDKWebService.asmx
portal-wsdl: ${checkmarx.base-url}/Portal/CxWebService.asmx?wsdl
sdk-wsdl: ${checkmarx.base-url}/SDK/CxSDKWebService.asmx?wsdl
preserve-xml: true
cx-xml:
file-name-format: "xmlresults.xml"
data-folder: "./"
- Open a google chrome browser window & signup for SendGrid using a personal or fake email
- https://signup.sendgrid.com/
- Note After April 6, 2020, Single Sender Verification may be required, see:
- https://sendgrid.com/docs/ui/sending-email/sender-verification#adding-a-sender
- Confirm your email address
- Click Email API > Integration Guide on the left sidebar
- Choose SMTP Relay
- My First API Key Name - CxSAST
- Click Create Key and add to the application-email.yml file
- Scan the following GitHub project using source control scan in CxSAST under the following team
- URL = https://github.com/ethicalhack3r/DVWA.git
- Team = CxServer\SP\Company
- Project Name = DVWA
- After the scan completes, open the CxViewer & mark all SQL_Injection as Proposed Not Exploitable
- After the .YML file and .PS1 file are completely filled out and saved
- Run CxFlow in batch mode & the email PowerShell script by opening a Powershell prompt and typing the following
cd C:\CxFlow
java -jar cx-flow-1.6.19.jar --spring.config.location="C:\CxFlow\application-email.yml" --project --cx-team="CxServer\SP\Company" --cx-project="DVWA" --app="DVWA"
.\EmailPNEVulns.ps1 -results_xml .\xmlresults.xml -email <youremail>
- Open your email & verify that the Proposed Not Exploitable results have been emailed.
- The email might be in your junk folder.
You can use Windows Task Scheduler to call the above commands/scripts & run this every evening.
# Usage: EmailPNEVulns.ps1 -results_xml <path-to-xml-results> -email_to <comma-separated-email-addresses>
param (
[Parameter(Mandatory = $true)][string]$results_xml,
[Parameter(Mandatory = $true)][string]$email_to
)
#=======================================
# **************************************
#
# Modify SMTP Settings below before use
#
# **************************************
#=======================================
[string] $smtpServer = "smtp.sendgrid.net"
[int] $smtpPort = 587
[string] $smtpUser = "apikey"
[string] $smtpPass = <###your send grid API key###>
[string] $smtpFrom = "[email protected]"
#=======================================
[string[]] $recipients = $email_to -split ","
#Parse XML file into an object
try {
[xml]$results = get-content $results_xml
}
catch {
Write-Output "Error parsing " + $results_xml
Write-Output "Exception: $($_.Exception.Message)"
}
[string]$project_name = $results.CxXMLResults.ProjectName
[string]$message = "<font size='4'>Checkmarx CxSAST found New Vulnerabilities in Project : <b>" + $project_name + "</b>"
$message += "<br>Detailed results are available at : <a href='" + $results.CxXMLResults.DeepLink + "'>" + $results.CxXMLResults.DeepLink + "</a>"
$message += "<br><b>New vulnerabilities found : " + $results.CxXMLResults.ScanStart + "</b></font><br>"
Write-Output $message
#Find all NEW vulnerabilities
[Object []]$Vulns = $results.CxXMLResults.Query.Result | Where-Object {$_.state -eq "4"}
$nv = "Found " + $Vulns.Count + " vulnerabilities"
Write-Output $nv
# NOP if no vulnerabilities match above filter
if ($Vulns.Count -eq 0) {
exit
}
[int]$issue_count = 0
#For each vulnerability, update the message
Foreach ($Vuln in $Vulns) {
$message += "<br>" + "<b>New [" + $Vuln.Severity + "] Severity [" + $Vuln.ParentNode.name + "] vulnerability</b>"
#For data flows with more than one node, provide filename, line number, and code snippet for source and sink
#Otherwise, provide the filename, line number, and code snippet of the single node
[Object]$origin = $Vuln.Path.FirstChild
if ($Vuln.Path.ChildNodes.Count -gt 1) {
$originCode = $origin.Snippet.Line.Code -replace '\t', ' '
$destinationCode = $destination.Snippet.Line.Code -replace '\t', ' '
$message += "<br>Source: [" + $origin.FileName + " : " + $origin.Line + "]. Code: <i>" + $originCode + "</i>"
[Object]$destination = $Vuln.Path.LastChild
$message += "<br>Sink: [" + $destination.FileName + " : " + $destination.Line + "]. Code: <i>" + $destinationCode + "</i>"
}
elseif ($Vuln.Path.ChildNodes.Count -eq 1) {
$originCode = $origin.Snippet.Line.Code -replace '\t', ' '
$message += "<br>File [" + $origin.FileName + " : " + $origin.Line + "]. Code: <i>" + $originCode + "</i>"
}
#Deep link to the vulnerability information in the portal
$message += "<br>View result on CxSAST : <a href='" + $Vuln.DeepLink + "'>" + $Vuln.DeepLink + "</a><br>"
#Escape characters
#$message = $message.Replace("\", "\\").Replace("`"", "\`"")
Write-Debug $message
$issue_count++
}
$message += "<br><b>Total New Vulnerabilities : " + $issue_count + "</b>"
$subject = "Checkmarx found $issue_count NEW vulnerabilities in [$project_name]"
Write-Output "Sending notifications to $recipients"
# Notify email list
try {
$secP = ConvertTo-SecureString $smtpPass -AsPlainText -Force
$smtpCreds = New-Object System.Management.Automation.PSCredential $smtpUser,$secP
$smtpMessage = @{
SmtpServer = $smtpServer
Port = $smtpPort
UseSsl = $true
Credential = $smtpCreds
From = $smtpFrom
To = $recipients
Subject = $subject
Body = $message
}
Send-MailMessage @smtpMessage -BodyAsHtml
$message = "Found and notified [$email_to] about " + $issue_count + " NEW vulnerabilities."
Remove-Item -path .\xmlresults.xml
Write-Output $message
}
catch {
$exception = $_.Exception
Write-Output "Failed to send out notification emails."
Write-Output "Reason: $($exception.Message)"
}
Pre-requisites:
Have a JIRA project ready with the Application and Category custom fields (see previous tutorials)
Have a private GitHub repo (no webhook required)
Be familiar with config-as-code overrides (see previous tutorials)
Goal:
The goal of this workshop is to set up a GitHub Action workflow to leverage the CxFlow GitHub Action in a private GitHub repository to launch a CxSAST scan on code pushes and pull requests on the master branch and create/update issues in JIRA.
GitHub Actions:
“GitHub Actions help you automate your software development workflows in the same place you store code and collaborate on pull requests and issues. You can write individual tasks, called actions, and combine them to create a custom workflow.”
How does the CxFlow GitHub Action work ?
The action is available on the GitHub marketplace. The source is available here:
GitHub actions rely on a .yml workflow definition file stored under /.github/workflows.
The action runs CxFlow in a container based on the standard checkmarx/cx-flow available from Docker Hub and uses a basic application.yml file. There are different tags of the action available depending on your version of CxSAST.
By default, the output of the action is a Sarif file for integration into GitHub’s CodeQL (See https://github.com/marketplace/actions/checkmarx-cxflow-action).
It is possible to override the default application.yml file from the workflow yml file using the params: field and via the usual cx.config file in the repository.
Step 1: Create a config-as-code override file in your GitHub repo
Since the application.yml provided by the CxFlow GitHub Action doesn’t contain a complete jira section, we have to use a configuration override.
Create a new file named cx.config at the root of your repository (main branch) containing the following (adapt the values with your specific environment details).
{ "application": "DSVW", "branches": ["develop", "main"], "bugTracker": "JIRA", "jira": { "project": "DSVW", "issue_type": "Bug", "opened_status": ["Open","Reopen"], "closed_status": ["Closed","Done"], "open_transition": "Reopen Issue", "close_transition": "Close Issue", "close_transition_field": "resolution", "close_transition_value": "Done", "priorities": { "High": "High", "Medium": "Medium", "Low": "Low" }, "fields": [ { "type": "result", "name": "application", "jira_field_name": "Application", "jira_field_type": "label" }, { "type": "result", "name": "category", "jira_field_name": "Category", "jira_field_type": "label" } ] } }
Step 2: Create a workflow
In your GitHub repository, click on “Actions”, then “set up a workflow yourself”
In the Marketplace panel, search for “cxflow”
Click on “Checkmarx CxFlow Action”
Select your version (only version v1.0-9.x supports CxSAST 9.x at the moment) and click the icon on the right to copy the action code to your clipboard.
Then on the left panel, replace this section with it:
Correct the indentation. Then enter your details or use GitHub Secrets (setup in your repository’s settings). You must provide more details (including the JIRA connection) via the params: field.
params: --bug-tracker=jira --config=cx.config --repo-name=DSVW --namespace=jbruinaud --branch=main --jira.url=${{secrets.JIRA_URL}} --jira.username=${{secrets.JIRA_USER}} --jira.token=${{secrets.JIRA_TOKEN}
Here is a complete main.yml working example with GitHub Secrets. Notice the top section with the name of the workflow and the triggers configuration and also the bottom parameters.
- Have a JIRA project ready with the Application and Category custom fields (see previous tutorials)
- Have a private GitHub repo (no webhook required)
- Be familiar with config-as-code overrides
The goal of this workshop is to set up a GitHub Action workflow to leverage the CxFlow GitHub Action in a private GitHub repository to launch a CxSAST scan on code pushes and pull requests on the master branch and create/update issues in JIRA.
“GitHub Actions help you automate your software development workflows in the same place you store code and collaborate on pull requests and issues. You can write individual tasks, called actions, and combine them to create a custom workflow.”
The action is available on the GitHub marketplace. The source is available here.
GitHub actions rely on a .yml workflow definition file stored under /.github/workflows.
The action runs CxFlow in a container based on the standard checkmarx/cx-flow available from Docker Hub and uses a basic application.yml file. There are different tags of the action available depending on your version of CxSAST.
By default, the output of the action is a Sarif file for integration into GitHub’s CodeQL.
It is possible to override the default application.yml file from the workflow yml file using the params: field and via the usual cx.config file in the repository.
Since the application.yml provided by the CxFlow GitHub Action doesn’t contain a complete jira section, we have to use a configuration override.
Create a new file named cx.config at the root of your repository (main branch) containing the following (adapt the values with your specific environment details).
{
"application": "DSVW",
"branches": ["develop", "main"],
"bugTracker": "JIRA",
"jira": {
"project": "DSVW",
"issue_type": "Bug",
"opened_status": ["Open","Reopen"],
"closed_status": ["Closed","Done"],
"open_transition": "Reopen Issue",
"close_transition": "Close Issue",
"close_transition_field": "resolution",
"close_transition_value": "Done",
"priorities": {
"High": "High",
"Medium": "Medium",
"Low": "Low"
},
"fields": [
{
"type": "result",
"name": "application",
"jira_field_name": "Application",
"jira_field_type": "label"
},
{
"type": "result",
"name": "category",
"jira_field_name": "Category",
"jira_field_type": "label"
}
]
}
}
In your GitHub repository, click on “Actions”, then “set up a workflow yourself”
In the Marketplace panel, search for “cxflow”
Click on “Checkmarx CxFlow Action”
Select your version (only version v1.0-9.x supports CxSAST 9.x at the moment) and click the icon on the right to copy the action code to your clipboard.
Then on the left panel, replace this section with it:
Correct the indentation. Then enter your details or use GitHub Secrets (setup in your repository’s settings). You must provide more details (including the JIRA connection) via the params: field.
params: --bug-tracker=jira --config=cx.config --repo-name=DSVW --namespace=jbruinaud --branch=main --jira.url=${{secrets.JIRA_URL}} --jira.username=${{secrets.JIRA_USER}} --jira.token=${{secrets.JIRA_TOKEN}
Here is a complete main.yml working example with GitHub Secrets. Notice the top section with the name of the workflow and the triggers configuration and also the bottom parameters.
# This is a basic workflow to help you get started with Actions
name: CxFlow
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the main branch
on:
push:
branches: [ master, main ]
pull_request:
branches: [ master, main ]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
# Cxflow Action
- name: Checkmarx CxFlow Action
# You may pin to the exact commit or the version.
uses: checkmarx-ts/[email protected]
with:
# Provide Checkmarx URL
checkmarx_url: ${{secrets.CHECKMARX_URL}}
# Provide team
team: /CxServer/SP/EMEA/UK
# Provide Checkmarx Username
checkmarx_username: ${{secrets.CHECKMARX_USERNAME}}
# Provide Checkmarx Password
checkmarx_password: ${{secrets.CHECKMARX_PASSWORD}}
# Provide Checkmarx Client Secret
checkmarx_client_secret: ${{secrets.CHECKMARX_CLIENT_SECRET}}
# Select a Checkmarx Project
project: DSVW-GitHub-action
# Select an Application Name used by downstream bug tracker systems
app: DSVW
# Select a Checkmarx Preset
#preset: # optional, default is Checkmarx Default
# Break build based on Checkmarx findings?
#break_build: # optional
# Incremental Scans?
incremental: true
# GitHub API Token (note: you don't have to create secrets.GITHUB_TOKEN, it is created automatically and will not appear in your repo's custom secrets)
github_token: ${{secrets.GITHUB_TOKEN}}
# extra parameters
params: --bug-tracker=jira --config=cx.config --repo-name=DSVW --namespace=jbruinaud --branch=main --jira.url=${{secrets.JIRA_URL}} --jira.username=${{secrets.JIRA_USER}} --jira.token=${{secrets.JIRA_TOKEN}}
Click “Start commit” then “Commit new file” to complete the process. This will trigger the workflow automatically since we are committing a new file to the main branch.
Click on “Actions”. You will see your workflow execution details, in yellow (in execution), green (succeeded) or red (failed). Click on it.
Then click on “build”
You now can see the execution logs.
Additional parameters (passed via the params: field) can be found in the Configuration Definitions section.
For example, only process Urgent and Confirmed results by adding this parameter:
--cx-flow.filter-state=Confirmed,Urgent
Configure Jira Parameter in workflow yml file instead of cx.config.
Here is a complete main.yml working example with GitHub Secrets. Notice the top section with the name of the workflow and the triggers configuration and also the bottom parameters.
# This workflow is to automate Checkmarx SAST scans. It runs on a push to the main branch.
#
# The following GitHub Secrets must be first defined:
# - CHECKMARX_URL
# - CHECKMARX_USER
# - CHECKMARX_PASSWORD
# - CHECKMARX_CLIENT_SECRET
#
# The following variables must be inserted below:
# - <ProjectName>
#
# Update the 'team' field to reflect the team name used in Checkmarx.
#
# For full documentation, including a list of all inputs, please refer to the README https://github.com/checkmarx-ts/checkmarx-cxflow-github-action
name: Checkmarx SAST Scan
on:
push:
branches:
- main
- master
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Checkmarx CxFlow Action
uses: checkmarx-ts/[email protected] #Github Action version
with:
project: ${{ secrets.CHECKMARX_PROJECT }} # <-- Insert Checkmarx SAST Project Name
team: ${{ secrets.CHECKMARX_TEAMS }}
checkmarx_url: ${{ secrets.CHECKMARX_URL }} # To be stored in GitHub Secrets.
checkmarx_username: ${{ secrets.CHECKMARX_USER }} # To be stored in GitHub Secrets.
checkmarx_password: ${{ secrets.CHECKMARX_PASSWORD }} # To be stored in GitHub Secrets.
checkmarx_client_secret: ${{ secrets.CHECKMARX_CLIENT_SECRET }} # To be stored in GitHub Secrets.
break_build: false
scanners: sast
bug_tracker: JIRA
jira_url: ${{ secrets.JIRA_URL }}
jira_username: ${{ secrets.JIRA_USERNAME }}
jira_token: ${{ secrets.JIRA_TOKEN }}
jira_project: ${{ secrets.JIRA_PROJECT }}
jira_issue_type: 'Application Security Bug'
jira_open_transition: 'In Progress'
jira_close_transition: 'Done'
jira_open_status: 'Backlog,Selected for Development,In Progress'
jira_closed_status: 'Done'
params: --namespace=${{ github.repository_owner }} --repo-name=${{ github.event.repository.name }} --branch=${{ github.ref }} --cx-flow.filterSeverity --cx-flow.filterCategory --jira.priorities.High=High --jira.priorities.Medium=Medium --jira.priorities.Low=Low --jira.priorities.Informational=Lowest
The following must be set up:
- CxIAST Management Server (refer to CxIAST Setup Guide and Installing the CxIAST Management Server)
- The application under test (AUT) (refer to Configuring the AUT Environment)
- Jenkins server (refer to Installing Jenkins)
- Bug Tracker (refer to Bug Tracker)
Create a pipeline in Jenkins that will perform the following:
- Start CxFlow.
- Create a scan tag so that CxFLow can identify the scan.
- Start the AUT with a CxIAST agent.
- Start an E2E test suite on the AUT.
- Stop the CxIAST scan and open tickets
Following is a description of a sample Jenkins declarative pipeline written in Groovy.
// 1. Get CxFlow - download CxFlow .jar and save it in a "cx-flow" folder
stage('Build CxFlow Jar'){
steps {
script{
dir("cx-flow"){
sh "curl https://github.com/checkmarx-ltd/cx-flow/releases/download/${CX_FLOW_VERSION}/cx-flow-${CX_FLOW_VERSION}.jar --output cx-flow.jar"
}
}
}
}
// CxFlow can be used in server mode, or in CLI mode.
// If used in CLI mode - skip this step.
// If used in server mode, run it (needs to happen before running the AUT):
stage('Run CxFlow in server mode'){
steps{
script{
dir('cx-flow'){
sh """
JENKINS_NODE_COOKIE=dontKillMe nohup java -jar cx-flow*.jar --spring.config.location=/var/jenkins/CxFlow/application.yml --web&>/tmp/start_app.log &
"""
sleep 30
}
}
}
}
// 2. Create a scan tag so that CxFlow can identify the scan.
// CxIAST scan is started with the generated scan tag so CxFlow can stop it after tests are finished.
stage('Generate Scan Tag'){
steps{
script{
def tag = ""
// CxFlow can be used in server mode, or in CLI mode
// If CxFlow is in server mode, get a generated scan tag from CxFlow:
def response = sh(script: "curl --header \"Content-Type:application/json\" \
--header \"X-Atlassian-Token:xxxx\" \
--request POST \
--data '{\"success\":\"true\",\"message\":\"ok\"}' \
http://CXFLOW-URL/iast/generate-tag",returnStdout: true)
def jsonSlurper = new JsonSlurper(type: JsonParserType.INDEX_OVERLAY)
def object = jsonSlurper.parseText(response)
tag = object.message
// If CxFlow is in CLI mode, generate a scan tag (make sure it's unique)
def now = new Date()
tag = now.format("yyMMddHHmmSS", TimeZone.getTimeZone('UTC'))
// Used in following steps
APP_TAG = tag
}
}
}
// 3. Start the AUT with a CxIAST agent.
// This example downloads a Java agent from CxIAST manager, extracts it, and runs WebGoat8 with an agent attached
stage('Run The App'){
steps {
script {
sh "curl http://CXIAST-URL/iast/compilation/download/JAVA --output javaagent.zip"
sh "unzip javaagent.zip -d javaagent"
sh "JENKINS_NODE_COOKIE=dontKillMe nohup java -javaagent:javaagent/cx-launcher.jar -Dcx.appName=WebGoat -DcxScanTag=${APP_TAG} -jar /var/jenkins/webgoat-server-8.0.0.M21.jar --server.address=0.0.0.0 &>/tmp/run_webgoat.log &"
}
}
}
// 4. Start an E2E test suite on the AUT.
// This example uses jmeter to run tests against WebGoat8
stage('Run Jmeter'){
steps{
script{
sh "/var/jenkins/apache-jmeter-5.4.1/bin/jmeter.sh -n -t WebGoat8.jmx"
}
}
}
// 5. Stop the CxIAST scan and open tickets.
// CxFlow can be used in server mode, or in CLI mode
stage('Stop Scan - Server mode'){
steps{
script{
// If bug tracker is Jira
sh "curl --header \"Content-Type:application/json\" \
--header \"X-Atlassian-Token:xxxx\" \
--request POST \
--data '{\"success\":\"true\",\"message\":\"ok\"}' \
http://CXFLOW-URL/iast/stop-scan-and-create-jira-issue/${appTag}"
}
// ------------- OR ------------
// If bug tracker is GitHub
sh "curl --header \"Content-Type:application/json\" \
--header \"X-Atlassian-Token:xxxx\" \
--request POST \
--data '{\"success\":\"true\",\"message\":\"ok\", \"assignee\":\"talilabok\", \"repoName\":\"cxflow-github\", \"namespace\":\"CxIAST\"}' \
http://CXFLOW-URL/iast/stop-scan-and-create-github-issue/${appTag}"
}
}
// ---------OR-----------
stage("Stop Scan - Cli Mode"){
steps{
script{
// If bug tracker is Jira
dir("cx-flow"){
echo "tag is ${APP_TAG}"
sh """
java -jar cx-flow*.jar --spring.config.location=/var/jenkins/CxFlow/application.yml --iast --scan-tag=\"${appTag}\" --bug-tracker=\"jira\"
"""
}
// ------------- OR ------------
// If bug tracker is GitHub
dir("cx-flow"){
sh """
java -jar cx-flow*.jar --spring.config.location=/var/jenkins/CxFlow/application.yml --iast --scan-tag=\"${appTag}\" --bug-tracker=\"githubissue\" --github.token=\"${githubToken}\" --repo-name=\"cxflow-github\" --namespace=\"CxIAST\" --assignee=\"CxIastCi\"
"""
}
}
}
}
}
}
server:
port: CXFLOW-PORT # used when CxFlow is in server mode
logging:
pattern:
console: "%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{15}){cyan} [%clr(%X{cx}){blue}] %clr(:){faint} %replace(%m){'([\\|])','\\$1'}%n%wEx"
file: "%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{15}){cyan} [%clr(%X{cx}){blue}] %clr(:){faint} %replace(%m){'([\\|])','\\$1'}%n%wEx"
file:
name: cx-flow.log
cx-flow:
# Agreed upon shared API token
token: xxxx
bug-tracker: JIRA
bug-tracker-impl:
- CxXml
- Json
- JIRA
- GitLab
- GitHub
- Csv
- Azure
- Rally
- ServiceNow
- Sarif
-SonarQube
branches:
- develop
- master
- security
filter-severity:
- High
filter-category:
filter-cwe:
filter-status:
mitre-url: https://cwe.mitre.org/data/definitions/%s.html
sonarqube:
file-path: C:\Checkmarx\SonarQube\cxSonarQube.json
checkmarx:
username: xxxxx
password: xxxxx
client-secret: xxxxx
base-url: http://cx.local
multi-tenant: true
configuration: Default Configuration
scan-preset: Checkmarx Default
team: \CxServer\SP\Checkmarx
url: ${checkmarx.base-url}/cxrestapi
#WSDL Config
portal-url: ${checkmarx.base-url}/cxwebinterface/Portal/CxWebService.asmx
jira:
url: https://cx-iast-ci.atlassian.net
username: xxxxx
token: xxxxx
project: CIC
issue-type: Bug
priorities:
X: Y # Map Checkmarx severity : to JIRA Priority
open-transition: Reopen Issue
close-transition: Close Issue
iast:
url: http://CXIAST-HOSTNAME
manager-port: CXIAST-PORT
username: xxxxx
password: xxxxx
update-token-seconds: 150 # By default token live only 5 minutes
filter-severity:
- HIGH
- MEDIUM
- LOW
# - INFO
thresholds-severity:
HIGH: -1
MEDIUM: -1
LOW: -1
INFO: -1
cxgo:
client-secret: xxx
base-url: https://cxgo-url
portal-url: https://cxgo-url
# CxOD Business unit that will contain the project/application/scan
team: \Demo\CxFlow
url: ${cxgo.base-url}
multi-tenant: true
configuration: Default Configuration
scan-preset: 1,2,3,4,5,9
cx-integrations:
url: https://cx.local
read-multi-tenant-configuration: false
rally:
token: xxxx
rally-project-id: xxxx
rally-workspace-id: xxxx
url: https://rallydev.com
api-url: https://xxxxx.rallydev.com/slm/webservice/v2.0
servicenow:
token: 123
servicenow-project-id: xxxx
servicenow-workspace-id: xxxx
url: https://servicenow.com
apiUrl: https://xxxxx.service-now.com/api/now/table
username: xxxx
password: xxxx
github:
webhook-token: xxxx
token: xxxxx
url: https://github.com
api-url: https://api.github.com/repos/
false-positive-label: false-positive
block-merge: true
gitlab:
webhook-token: xxxx
token: xxxx
url: https://gitlab.com
api-url: https://gitlab.com/api/v4/
false-positive-label: false-positive
block-merge: true
azure:
webhook-token: xxxx
token: xxxx
url: https://dev.azure.com
issue-type: issue
api-version: 5.0
false-positive-label: false-positive
json:
file-name-format: "[TEAM]-[PROJECT]-[TIME].json"
data-folder: "/tmp/cxflow"
cx-xml:
file-name-format: "[TEAM]-[PROJECT]-[TIME].xml"
data-folder: "/tmp/cxflow"
csv:
file-name-format: "[TEAM]-[PROJECT]-[TIME].csv"
data-folder: "/tmp/cxflow"
include-header: true
fields:
- header: Application
name: application
default-value: unknown
- header: severity
name: severity
- header: Vulnerability ID
name: summary
prefix: "[APP]:"
- header: file
name: filename
- header: Vulnerability ID
name: summary
- header: Vulnerability Name
name: category
- header: Category ID
name: cwe
- header: Description
name: description
- header: Severity
name: severity
- header: recommendation
name: recommendation
- header: Similarity ID
name: similarity-id
The following must be set up:
-
SonarQube Server (refer to SonarQube Setup Guide on here)
-
SonarQube Scanner (refer to SonarQube Scanner Installation on here)
-
Generate the Sonar Qube Issue Report by configuring bug tracker as SonarQube.
Configure cx-flow to generate SonarQube Report for SAST or SCA:
cx-flow:
bug-tracker: SonarQube
bug-tracker-impl:
-SonarQube
sonarqube:
file-path: C:\Checkmarx\SonarQube\cxSonarQube.json
Upload the CxFlow Sonar Qube Report generated for SAST or SCA scan:
- Set Sonar Scanner in Windows PATH Varaible.
- Edit {SONAR_SCANNER_HOME}\conf\sonar-scanner.properties for below properties:
sonar.externalIssuesReportPaths={PATH_TO_CX_FLOW_SONAR_REPORT}
- Run the below command from the folder where sonar-project.properties is located:
sonar-scanner -X
Issue Severity Mapping for SAST and SCA:
SAST / SCA Seveirty | SonarQube Severity |
---|---|
High |
CRITICAL |
Medium |
MAJOR |
Low |
MINOR |
Information |
INFO |
CxFlow supports creating branched project in CxSAST from a project created from default branch of a repository when event type is PUSH on the feature branch, or from a project created from the target branch of a PR when event type is PULL on the feature branch, without incrementing the count of utilized licensed project.
- Set cx-branch option under checkmarx section in the application yml file to true.
checkmarx:
...
cx-branch:true
-
When a PUSH event is created from any feature branch of a repository, and a licensed project for the repository's default branch is not already present in CxSAST, then a licensed project for the default branch is first created with no scans and then a branched project is created from it for the feature branch.
- No project exists in CxSAST for the repository
JavaVulnerabilityLabE
- The repository has two branches defined
master
which is default, andfeature-branch
which is a feature branch. -
PUSH event is created from the
feature-branch
branch of the repository. - Licensed project for default branch
master
with namejavaVulnerabilityLabE-master
is created. - Branched project for
feature-branch
with nameJavaVulnerabilityLabE-feature-branch
is created.
- No project exists in CxSAST for the repository
-
When a PULL event is created from any feature branch of a repository to a target branch and a project for the target branch does not exists in CxSAST then s licensed project for target branch is first created and then a branched project for the default branch is created.
-
PULL event is created from
feature-branch
to a target branchdemo-master
- Licensed project for target branch
demo-master
with nameJavaVulnerabilityLabE-demo-master
is created. - Branched project for
feature-branch
with nameJavaVulnerabilityLabE-feature-branch
is created.
-
PULL event is created from