Practice 10 ‐ Static analysis

This tutorial will demonstrate setting up a local SonarQube server for static analysis, configuring a project for static analysis, and developing our own project-specific analysis rule.

Setting up a local SonarQube instance

Installing Docker Compose

We have already installed docker-compose to the virtual machines in the lab. If you use your own copy of the virtual machine, you can install docker-compose as follows:

sudo dnf install podman podman-docker
sudo systemctl enable --now podman.socket
wget https://github.com/docker/compose/releases/download/v2.30.3/docker-compose-linux-x86_64
chmod a+x docker-compose-linux-x86_64 
sudo mv docker-compose-linux-x86_64 /usr/local/bin/docker-compose
sudo touch /etc/containers/nodocker

Installing SonarQube Community Edition

In this section, we will use docker-compose to create our own SonarQube Community Edition server.

For more information, see the Sonar documentation at https://docs.sonarsource.com/sonarqube/latest/ setup-and-upgrade/install-the-server/installing-sonarqube-from-docker/

We deliberately use the embedded H2 database, which is not suitable for production use, but lets us quickly set up a test server.

mkdir sonar
cd sonar
wget https://raw.githubusercontent.com/SonarSource/docker-sonarqube/refs/heads/master/example-compose-files/sq-with-h2/docker-compose.yml
sudo docker-compose up -d

Since we ran docker-compose in the sonar directory, the newly started Docker container will be named sonar-sonarqube-1.

One the server has booted up, configure it as follows:

Open http://localhost:9000 in your browser.
Login with user: admin and password: admin.
Change the password when prompted (use, e.g., LaborImage1234! to satisfy all password rules).

Setting up a sample project

We have created a sample project at https://github.com/ftsrg-edu/ase-lab-practice-10-example

Clone it into the directory ~/Shingler, then run ./gradlew build in the directory to build the project.

Changes from the previous lab

The sample code is identical to the practice-2b-end, but we have configured it with the SonarScanner Gradle plugin to upload the static analysis result to the SonarQube server.

We updated buildSrc/build.gradle.kts to add the Gradle plugin repository to build script, and make SonarScanner available for our convention plugins:

https://github.com/ftsrg-edu/ase-lab-practice-10-example/blob/7b276da713ed741b793cfd91c8d762fa92bf8306/buildSrc/build.gradle.kts#L5-L14

repositories {
    mavenCentral()
    maven {
        url = uri("https://plugins.gradle.org/m2/")
    }
}

dependencies {
    implementation("org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:5.1.0.4882")
}

We apply the org.sonarqube plugin to all Java subprojects in our hu.bme.mit.ase.shingler.gradle.java convenion plugin.

https://github.com/ftsrg-edu/ase-lab-practice-10-example/blob/7b276da713ed741b793cfd91c8d762fa92bf8306/buildSrc/src/main/kotlin/hu/bme/mit/ase/shingler/gradle/java.gradle.kts#L7

plugins {
    java
    jacoco
    `java-library`
    id("org.sonarqube")
}

We configure JaCoCo to emit XML test reports, which will be read by SonarScanner to track code coverage.

https://github.com/ftsrg-edu/ase-lab-practice-10-example/blob/7b276da713ed741b793cfd91c8d762fa92bf8306/buildSrc/src/main/kotlin/hu/bme/mit/ase/shingler/gradle/java.gradle.kts#L25-L30

tasks {
    jacocoTestReport {
        inputs.files(test.get().outputs)
        reports {
            xml.required = true
        }
    }
}

We also apply the org.sonarqube plugin the the build.gradle.kts file in the root of the repository to let Sonar know about all files in the repository.

Important

if you skip this step, Sonar will complain about missing files, because it will only recognize one of the subprojects in the repository instead of the whole repository.

https://github.com/ftsrg-edu/ase-lab-practice-10-example/blob/7b276da713ed741b793cfd91c8d762fa92bf8306/build.gradle.kts

plugins {
    id("org.sonarqube")
}

Lastly, we added some settings to gradle.properties:

https://github.com/ftsrg-edu/ase-lab-practice-10-example/blob/7b276da713ed741b793cfd91c8d762fa92bf8306/gradle.properties

# Increase the memory available for SonarScanner, as it may crash with the default amount.
org.gradle.jvmargs=-Xmx1g
systemProp.sonar.host.url=http://localhost:9000
systemProp.sonar.projectKey=Shingler
systemProp.sonar.token=

Configuring SonarScanner for the project

In the SonarQube web interface, create a local project with display name: Shingler, project key: Shingler, main branch name: main. Use the global setting for new code.
In the Sonar web interface select Other CI > Generate a project token with Expires in: No expiration.
Save the generated token to the systemProp.sonar.token key in gradle.properties.
Run ./gradlew build sonar to build the project and run SonarScanner.
If you reload the web interface, the newly found issues will appear shortly. This may take a while, as SonarQube will also have to perform server-side processing.

Warning

Saving the Sonar token to the source code is a bad practice for code that is available publicly, as it allows write access to the Sonar server to anyone who can read the code. In real-world projects, it is recommended to use environmental variables or a secred store like GitHub secrets to store the token.

Defining a new Sonar rule

For more information, see the Sonar documentation at https://docs.sonarsource.com/sonarqube/latest/extension-guide/adding-coding-rules/ and https://docs.sonarsource.com/sonarqube/latest/extension-guide/developing-a-plugin/plugin-basics/

The example project is based on the tutorial at https://github.com/SonarSource/sonar-java/blob/master/docs/CUSTOM_RULES_101.md

Case study

In this scenario, we want to flag uses of hu.bme.mit.ase.shingler.lib.VectorMultiplier#computeScalarProduct where we multiple a vector with itself and replace them with call to computeSquaredNorm. To this end, we will develop a custom Sonar plugin with a custom Java static analysis rule.

https://github.com/ftsrg-edu/ase-lab-practice-10-example/blob/7b276da713ed741b793cfd91c8d762fa92bf8306/lib/src/main/java/hu/bme/mit/ase/shingler/lib/VectorMultiplier.java#L9-L11

public interface VectorMultiplier {

    double computeScalarProduct(OccurrenceVector u, OccurrenceVector v);

    default double computeSquaredNorm(OccurrenceVector v) {
        return computeScalarProduct(v, v);
    }
}

Creating the Sonar plugin

The started project for creating a Sonar plugin is located in the branch https://github.com/ftsrg-edu/ase-labs/tree/practice-10

When importing into IntelliJ as a Maven project, select a Java 17 JDK (instead of a Java 21 JDK!) and Java language level of Java 8. This matches the environment in which Sonar plugin will run on the server.

Notable parts of the starter project:

Sonar plugins are build with Maven instead of Gradle, which is configured in pom.xml
- We set the name and description in the plugin in https://github.com/ftsrg-edu/ase-labs/blob/e56e43f57997e694f27d7aff5db39ccd7bb2578d/pom.xml#L12-L13
- The sonar-packaging-maven-plugin is configured in https://github.com/ftsrg-edu/ase-labs/blob/e56e43f57997e694f27d7aff5db39ccd7bb2578d/pom.xml#L99-L113 Most importantly, we set the pluginClass to hu.bme.mit.ase.sonar.ExamplePlugin, which will register elements of our plugin in Sonar.
ExamplePlugin registers the elements of our plugin in Sonar.
RulesList contains the list of rules to run in the main and test java source sets. Currently, both lists are empty.
ExampleRulesDefinition lads rule metadata. In particular,
- The constants REPOSITORY_KEY and REPOSITORY_NAME set the internal ID and user-visible name of our collection of rules.
- ScalarProductWithSelfRule.html is the description of our rule that will be shown in the SonarQube web interface.
- ScalarProductWithSelfRule.json is the rule metadata.
ExampleCheckRegistrar registers the implementations of the rules.
ScalarProductWithSelfRule will be the implementation of our rule. Currently, it is empty. The annotation @Rule(key = "ScalarProductWithSelfRule") sets the name of our rule.

Deploying the empty plugin to SonarQube

Run the sonar-ase-plugin > Lifecycle > package Maven task from the Maven window of IntelliJ. The plugin will be output to target/sonar-ase-plugin-1.0-SNAPSHOT.jar in the project directory.

Copy the plugin into the plugins directory of the Sonar Qube server.

sudo docker cp target/sonar-ase-plugin-1.0-SNAPSHOT.jar sonar-sonarqube-1:/opt/sonarqube/extensions/plugins/

Restart the server (issue these commands in the ~/sonar directory).
```
sudo docker-compose down
sudo docker-compose up -d
```
Once the server has booted up again, log in and click "I understand the risk" when installing a new plugin.
Go to Administration > Marketplace > Plugins > Installed. You will see the example plugin here.

Creating tests for a new rule

Open https://github.com/ftsrg-edu/ase-labs/blob/e56e43f57997e694f27d7aff5db39ccd7bb2578d/src/test/files/Example.java to add some test cases for the rule. Use a // Noncompliant comment to mark lines where an error should be emitted, but do not forget to also add negative test cases where no error should be emitted.

Modify https://github.com/ftsrg-edu/ase-labs/blob/e56e43f57997e694f27d7aff5db39ccd7bb2578d/src/test/java/hu/bme/mit/ase/sonar/ScalarProductWithSelfRuleTest.java to run the tests:

import org.sonar.java.checks.verifier.CheckVerifier;

class ScalarProductWithSelfRuleTest {
    @Test
    void test() {
        CheckVerifier.newVerifier()
                .onFile("src/test/files/Example.java")
                .withCheck(new ScalarProductWithSelfRule())
                .verifyIssues();
    }
}

What happens when you run this test? Sonar will fail to resolve classes like VectorMultiplier, since it has no access to the Shingler source files or binaries.

Fixing the test classpath

Copy ~/Shingler/lib/build/libs/lib.jar and ~/Shingler/logic/build/libs/logic.jar to src/test/files.

Adjust ScalarProductWithSelfRuleTest to add these jars to the Sonar analysis classpath:

import java.io.File;
import java.util.Arrays;

class ScalarProductWithSelfRuleTest {
    @Test
    void test() {
        CheckVerifier.newVerifier()
                .withClassPath(Arrays.asList(new File("src/test/files/lib.jar"),
                        new File("src/test/files/logic.jar")))
                .onFile("src/test/files/Example.java")
                .withCheck(new ScalarProductWithSelfRule())
                .verifyIssues();
    }
}

Now the test should not complain about resolving external classes. However, it should still fail, since we have not implemented our rule yet.

Implementing the new rule

First, modify ScalarProductWithSelfRule to visit all METHOD_INVOCATION nodes in the Abstract Syntax Tree (AST):

@Rule(key = "ScalarProductWithSelfRule")
public class ScalarProductWithSelfRule extends IssuableSubscriptionVisitor {
    @Override
    public List<Tree.Kind> nodesToVisit() {
        return Collections.singletonList(Tree.Kind.METHOD_INVOCATION);
    }

    @Override
    public void visitNode(Tree tree) {
        MethodInvocationTree methodInvocation = (MethodInvocationTree) tree;
        // TODO Implement the rest of the rule.
        reportIssue(methodInvocation.methodSelect(), "Replace this call with computeSquaredNorm.");
    }
}

Add a breakpoint to the visitNode method and use a debugger to discover the SonarJava analysis API and implement your rule. Evaluating expressions in the debugger is especially helpful to explore the structure of the AST and the available operations.

Besides the AST itself, you can use semantic information from SonarJava provided by the Symbol and MethodSymbol classes. This information is resolved from the analysis classpath (the lib and logic libraries in our example).

You can run the test to verify whether your rule works correctly.

Important

When writing custom Java rules, you can only use classes from package org.sonar.plugins.java.api. While other packages may be available in the IntelliJ content assist and during the tests, the classloader in the SonarQube server will block access to them due to the sandboxing of plugins.

Registering the new rule

Add the ScalarProductWithSelfRule to the RulesList. Should it run in the main source set, in the test source set, or in both?

Trying out the plugin

Run the sonar-ase-plugin > Lifecycle > package Maven task from the Maven window of IntelliJ. The plugin will be output to target/sonar-ase-plugin-1.0-SNAPSHOT.jar in the project directory.