feat: Enable starting scans with pull request argument (#300)

* Enabled starting scans with pull request argument Signed-off-by: Vasyl Kerimov <[email protected]> * Removed debug logs Signed-off-by: Vasyl Kerimov <[email protected]> * Formatting fixed Signed-off-by: Vasyl Kerimov <[email protected]> * Refactoring: moved GitHub connection establishment into separate service Signed-off-by: Vasyl Kerimov <[email protected]> * Formatting style fix Signed-off-by: Vasyl Kerimov <[email protected]> * Made a static method static Signed-off-by: Vasyl Kerimov <[email protected]> * doc: update readme, add signe scan description Signed-off-by: Mykola Rudyk <[email protected]> * fix: initialization procedure Signed-off-by: Mykola Rudyk <[email protected]> * Change format of github.pull.request Signed-off-by: Vasyl Kerimov <[email protected]> * README.md update Signed-off-by: Vasyl Kerimov <[email protected]> * fix: output results Signed-off-by: Mykola Rudyk <[email protected]> * Fix code style Signed-off-by: Vasyl Kerimov <[email protected]> * Fix code style more Signed-off-by: Vasyl Kerimov <[email protected]> * All necessary licenses added to the dump Signed-off-by: Vasyl Kerimov <[email protected]> * H2 properties added Signed-off-by: Vasyl Kerimov <[email protected]> * Removed repeat properties in singlescan profile properties file Signed-off-by: Vasyl Kerimov <[email protected]> * Updated README.md Signed-off-by: Vasyl Kerimov <[email protected]> * Reverted unnecessary change Signed-off-by: Vasyl Kerimov <[email protected]> * Formatting mistakes fixed Signed-off-by: Vasyl Kerimov <[email protected]> * Added license to a new file Signed-off-by: Vasyl Kerimov <[email protected]> * fix: small remarks Signed-off-by: Mykola Rudyk <[email protected]> * test: update unit tests for newly added features Signed-off-by: Mykola Rudyk <[email protected]> * Fixed README.md Signed-off-by: Vasyl Kerimov <[email protected]> * Fixed code style Signed-off-by: Vasyl Kerimov <[email protected]> * user column name reinstated Signed-off-by: Vasyl Kerimov <[email protected]> * Response to Oleg Kopysov's request Signed-off-by: Vasyl Kerimov <[email protected]> * test: update tests Signed-off-by: Mykola Rudyk <[email protected]> * Unnecessary imports deleted Signed-off-by: Vasyl Kerimov <[email protected]> * test: update tests Signed-off-by: Mykola Rudyk <[email protected]> * fix: fix style analyser warning Signed-off-by: Mykola Rudyk <[email protected]> * Fix to README.md Signed-off-by: Vasyl Kerimov <[email protected]> * fix: fix printing via internal loggin, add user input check Signed-off-by: Mykola Rudyk <[email protected]> * test: update unit tests to increase coverage Signed-off-by: Mykola Rudyk <[email protected]> * README.md instruction numbering fixed Signed-off-by: Vasyl Kerimov <[email protected]> * Removed credential from properties file Signed-off-by: Vasyl Kerimov <[email protected]> * Added htmlEscape, as per request Signed-off-by: Vasyl Kerimov <[email protected]> * Added ident Signed-off-by: Vasyl Kerimov <[email protected]> --------- Signed-off-by: Vasyl Kerimov <[email protected]> Signed-off-by: Mykola Rudyk <[email protected]> Co-authored-by: Mykola Rudyk <[email protected]> Co-authored-by: Oleg Kopysov <[email protected]>
Samsung · Dec 1, 2023 · 54f4d9b · 54f4d9b
1 parent 1776696
commit 54f4d9b
Show file tree

Hide file tree

Showing 15 changed files with 592 additions and 251 deletions.
diff --git a/README.md b/README.md
@@ -142,7 +142,7 @@ You can now create a new pull request or update an existing one with commits. LP
 
 ## How to Build and Run _LPVS_ from Source Code
 
-#### 1. Build Prerequisites
+### 1. Build Prerequisites
 
 Before building LPVS from source code, ensure that you have the following prerequisites installed:
 
@@ -160,7 +160,7 @@ Before building LPVS from source code, ensure that you have the following prereq
   sudo apt install mysql-server
   ```
 
-#### 2. Create Necessary MySQL Database and User (optional if not using a database)
+### 2. Create Necessary MySQL Database and User (optional if not using a database)
 
 2.1 Start the MySQL server:
    ```bash
@@ -194,7 +194,7 @@ Before building LPVS from source code, ensure that you have the following prereq
    spring.datasource.password=password
    ```
 
-#### 3. Setting up _LPVS_ `application.properties`
+### 3. Setting up _LPVS_ `application.properties`
 
 Fill in the following lines in the [`src/main/resources/application.properties`](src/main/resources/application.properties) file:
 
@@ -226,20 +226,25 @@ spring.datasource.password=
 
 Alternatively, you can provide the necessary values using the following environment variables: `LPVS_GITHUB_LOGIN`, `LPVS_GITHUB_TOKEN`, `LPVS_GITHUB_API_URL`, `LPVS_GITHUB_SECRET`, and `LPVS_LICENSE_CONFLICT`.
 
-#### 4. Build LPVS Application with Maven and Run it
-To build LPVS from source code and run it, follow these steps:
+### 4. Build LPVS Application with Maven and Run it
 
-4.1 Build the LPVS application using Maven:
+#### 4.a Service mode (default)
+
+To build LPVS from source code and run it in the default service mode, follow these steps:
+
+4.a.1 Build the LPVS application using Maven:
    ```bash
    mvn clean install
    ```
 
-4.2 Navigate to the target directory:
+4.a.2 Navigate to the target directory:
    ```bash
    cd target/
    ```
 
-4.3 Run the LPVS application using the following command:
+4.a.3. Run the LPVS application.
+
+   Service is run using the following command:
    ```bash
    java -jar lpvs-*.jar
    ```
@@ -252,6 +257,30 @@ To build LPVS from source code and run it, follow these steps:
 
 LPVS is now built and running. You can create a new pull request or update an existing one with commits, and LPVS will automatically start scanning and provide comments about the licenses found in the project.
 
+#### 4.b Single scan mode
+
+Alternatively, you can perform a one-time scan on a specific pull request using the single scan mode. Follow these steps:
+
+4.b.1. Begin by running the installation and navigating to the target directory, similar to the process in service mode (refer to steps 4.a.1 and 4.a.2):
+
+   ```bash
+   mvn clean install
+   cd target/
+   ```
+
+4.b.2. Execute the single scan with the following command:
+
+   ```bash
+   java -jar -Dgithub.token=<my-token> lpvs-*.jar --github.pull.request=<PR URL>
+   ```
+
+4.b.3. By default, the above command requires a pre-configured MySQL database. To avoid setting up the database, use the "singlescan" profile:
+   ```bash
+   java -jar -Dspring.profiles.active=singlescan -Dgithub.token=<my-token> lpvs-*.jar --github.pull-request=<PR URL>
+   ```
+
+These steps streamline the process, allowing you to run a scan on a single pull request without the need for a preconfigured database.
+
 ---
 
 ## Frontend Source Code (React)

diff --git a/pom.xml b/pom.xml
@@ -131,6 +131,11 @@
             <version>1.18.20</version>
             <scope>provided</scope>
         </dependency>
+        <dependency>
+            <groupId>com.h2database</groupId>
+            <artifactId>h2</artifactId>
+            <version>2.1.214</version>
+        </dependency>
 
     </dependencies>
 

diff --git a/src/main/java/com/lpvs/entity/LPVSQueue.java b/src/main/java/com/lpvs/entity/LPVSQueue.java
@@ -33,7 +33,7 @@ public class LPVSQueue implements Serializable {
     @Column(name = "scan_date")
     private Date date;
 
-    @Column(name = "user")
+    @Column(name = "user_id")
     private String userId;
 
     @Column(name = "review_system_type")

diff --git a/src/main/java/com/lpvs/service/LPVSDetectService.java b/src/main/java/com/lpvs/service/LPVSDetectService.java
@@ -9,12 +9,29 @@
 import com.lpvs.entity.LPVSFile;
 import com.lpvs.entity.LPVSQueue;
 import com.lpvs.service.scanner.scanoss.LPVSScanossDetectService;
+import com.lpvs.util.LPVSFileUtil;
+import com.nimbusds.jose.util.IOUtils;
+
 import lombok.extern.slf4j.Slf4j;
+import org.kohsuke.github.GHPullRequest;
+import org.kohsuke.github.GHRepository;
+import org.kohsuke.github.GitHub;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.beans.factory.annotation.Value;
+import org.springframework.boot.ExitCodeEvent;
+import org.springframework.boot.context.event.ApplicationReadyEvent;
+import org.springframework.context.ApplicationContext;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.event.EventListener;
 import org.springframework.stereotype.Service;
+import org.springframework.web.util.HtmlUtils;
+
 import javax.annotation.PostConstruct;
+
+import java.io.File;
+import java.io.IOException;
 import java.util.ArrayList;
+import java.util.Arrays;
 import java.util.List;
 
 @Service
@@ -25,11 +42,22 @@ public class LPVSDetectService {
 
     private LPVSScanossDetectService scanossDetectService;
 
+    private LPVSGitHubConnectionService gitHubConnectionService;
+
+    @Autowired private ApplicationEventPublisher eventPublisher;
+
+    @Value("${github.pull.request:}")
+    private String trigger;
+
+    @Autowired ApplicationContext ctx;
+
     @Autowired
     public LPVSDetectService(
             @Value("${scanner:scanoss}") String scannerType,
+            LPVSGitHubConnectionService gitHubConnectionService,
             LPVSScanossDetectService scanossDetectService) {
         this.scannerType = scannerType;
+        this.gitHubConnectionService = gitHubConnectionService;
         this.scanossDetectService = scanossDetectService;
     }
 
@@ -38,6 +66,77 @@ private void init() {
         log.info("License detection scanner: " + scannerType);
     }
 
+    @EventListener(ApplicationReadyEvent.class)
+    public void runOneScan() {
+        if (trigger != null && !HtmlUtils.htmlEscape(trigger).equals("")) {
+            try {
+                LPVSQueue webhookConfig =
+                        this.getInternalQueueByPullRequest(HtmlUtils.htmlEscape(trigger));
+                this.runScan(webhookConfig, LPVSDetectService.getPathByPullRequest(webhookConfig));
+                File scanResult = new File(LPVSFileUtil.getScanResultsJsonFilePath(webhookConfig));
+                if (scanResult.exists()) {
+                    String jsonTxt = IOUtils.readFileToString(scanResult);
+                    // ToDo: form html report and console output
+                    log.info(jsonTxt);
+                    log.info("\n\n\n Single scan finished successfully \n\n\n");
+                }
+            } catch (Exception ex) {
+                log.info("\n\n\n Single scan finished with errors \n\n\n");
+                log.error("Can't triger single scan " + ex);
+            }
+            // exiting application
+            eventPublisher.publishEvent(new ExitCodeEvent(new Object(), 0));
+        }
+    }
+
+    private static LPVSQueue getGitHubWebhookConfig(GHRepository repo, GHPullRequest pR) {
+        LPVSQueue webhookConfig = new LPVSQueue();
+        webhookConfig.setPullRequestUrl(
+                pR.getHtmlUrl() != null ? pR.getHtmlUrl().toString() : null);
+        if (pR.getHead() != null
+                && pR.getHead().getRepository() != null
+                && pR.getHead().getRepository().getHtmlUrl() != null) {
+            webhookConfig.setPullRequestFilesUrl(
+                    pR.getHead().getRepository().getHtmlUrl().toString());
+        } else {
+            webhookConfig.setPullRequestFilesUrl(webhookConfig.getPullRequestUrl());
+        }
+        webhookConfig.setPullRequestAPIUrl(pR.getUrl() != null ? pR.getUrl().toString() : null);
+        webhookConfig.setRepositoryUrl(
+                repo.getHtmlUrl() != null ? repo.getHtmlUrl().toString() : null);
+        webhookConfig.setUserId("Single scan run");
+        webhookConfig.setHeadCommitSHA(pR.getHead() != null ? pR.getHead().getSha() : null);
+        return webhookConfig;
+    }
+
+    public LPVSQueue getInternalQueueByPullRequest(String pullRequest) {
+        try {
+            if (pullRequest == null) return null;
+            String[] pullRequestSplit = pullRequest.split("/");
+            if (pullRequestSplit.length < 5) return null;
+            String pullRequestRepo =
+                    String.join(
+                            "/",
+                            Arrays.asList(pullRequestSplit)
+                                    .subList(
+                                            pullRequestSplit.length - 4,
+                                            pullRequestSplit.length - 2));
+            int pullRequestNum = Integer.parseInt(pullRequestSplit[pullRequestSplit.length - 1]);
+            GitHub gitHub = gitHubConnectionService.connectToGitHubApi();
+            GHRepository repo = gitHub.getRepository(pullRequestRepo);
+            GHPullRequest pR = repo.getPullRequest(pullRequestNum);
+            return LPVSDetectService.getGitHubWebhookConfig(repo, pR);
+        } catch (IOException e) {
+            log.error("Can't set up github client: " + e);
+        }
+        return null;
+    }
+
+    public static String getPathByPullRequest(LPVSQueue webhookConfig) {
+        if (webhookConfig == null) return null;
+        return LPVSFileUtil.getLocalDirectoryPath(webhookConfig);
+    }
+
     public List<LPVSFile> runScan(LPVSQueue webhookConfig, String path) throws Exception {
         if (scannerType.equals("scanoss")) {
             scanossDetectService.runScan(webhookConfig, path);

diff --git a/src/main/java/com/lpvs/service/LPVSGitHubConnectionService.java b/src/main/java/com/lpvs/service/LPVSGitHubConnectionService.java
@@ -0,0 +1,92 @@
+/**
+ * Copyright (c) 2023, Samsung Electronics Co., Ltd. All rights reserved.
+ *
+ * Use of this source code is governed by a MIT license that can be
+ * found in the LICENSE file.
+ */
+package com.lpvs.service;
+
+import com.lpvs.util.LPVSExitHandler;
+import lombok.extern.slf4j.Slf4j;
+import org.kohsuke.github.GitHub;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.stereotype.Service;
+
+import javax.annotation.PostConstruct;
+import java.io.IOException;
+import java.util.Optional;
+
+@Service
+@Slf4j
+public class LPVSGitHubConnectionService {
+
+    private String GITHUB_LOGIN;
+    private String GITHUB_AUTH_TOKEN;
+    private String GITHUB_API_URL;
+
+    private static final String GITHUB_LOGIN_PROP_NAME = "github.login";
+    private static final String GITHUB_AUTH_TOKEN_PROP_NAME = "github.token";
+    private static final String GITHUB_API_URL_PROP_NAME = "github.api.url";
+
+    private static final String GITHUB_LOGIN_ENV_VAR_NAME = "LPVS_GITHUB_LOGIN";
+    private static final String GITHUB_AUTH_TOKEN_ENV_VAR_NAME = "LPVS_GITHUB_TOKEN";
+    private static final String GITHUB_API_URL_ENV_VAR_NAME = "LPVS_GITHUB_API_URL";
+
+    private LPVSExitHandler exitHandler;
+
+    @Autowired
+    public LPVSGitHubConnectionService(
+            @Value("${" + GITHUB_LOGIN_PROP_NAME + "}") String GITHUB_LOGIN,
+            @Value("${" + GITHUB_AUTH_TOKEN_PROP_NAME + "}") String GITHUB_AUTH_TOKEN,
+            @Value("${" + GITHUB_API_URL_PROP_NAME + "}") String GITHUB_API_URL,
+            LPVSExitHandler exitHandler) {
+        this.GITHUB_LOGIN =
+                Optional.ofNullable(GITHUB_LOGIN)
+                        .filter(s -> !s.isEmpty())
+                        .orElse(
+                                Optional.ofNullable(System.getenv(GITHUB_LOGIN_ENV_VAR_NAME))
+                                        .orElse(""));
+        this.GITHUB_AUTH_TOKEN =
+                Optional.ofNullable(GITHUB_AUTH_TOKEN)
+                        .filter(s -> !s.isEmpty())
+                        .orElse(
+                                Optional.ofNullable(System.getenv(GITHUB_AUTH_TOKEN_ENV_VAR_NAME))
+                                        .orElse(""));
+        this.GITHUB_API_URL =
+                Optional.ofNullable(GITHUB_API_URL)
+                        .filter(s -> !s.isEmpty())
+                        .orElse(
+                                Optional.ofNullable(System.getenv(GITHUB_API_URL_ENV_VAR_NAME))
+                                        .orElse(""));
+        this.exitHandler = exitHandler;
+    }
+
+    @PostConstruct
+    private void checks() {
+        if (this.GITHUB_AUTH_TOKEN.isEmpty()) {
+            log.error(
+                    GITHUB_AUTH_TOKEN_ENV_VAR_NAME
+                            + "("
+                            + GITHUB_AUTH_TOKEN_PROP_NAME
+                            + ") is not set.");
+            exitHandler.exit(-1);
+        }
+    }
+
+    public GitHub connectToGitHubApi() throws IOException {
+        GitHub gH;
+        if (GITHUB_AUTH_TOKEN.isEmpty()) setGithubTokenFromEnv();
+        if (GITHUB_API_URL.isEmpty()) gH = GitHub.connect(GITHUB_LOGIN, GITHUB_AUTH_TOKEN);
+        else
+            gH =
+                    GitHub.connectToEnterpriseWithOAuth(
+                            GITHUB_API_URL, GITHUB_LOGIN, GITHUB_AUTH_TOKEN);
+        return gH;
+    }
+
+    public void setGithubTokenFromEnv() {
+        if (System.getenv("LPVS_GITHUB_TOKEN") != null)
+            GITHUB_AUTH_TOKEN = System.getenv("LPVS_GITHUB_TOKEN");
+    }
+}