SUBMARINE-1138. New SSO function based on OIDC

### What is this PR for? Use pac4j to support OIDC and default login action, and fix some user rest api question. Currently, it is a preview version, which is mainly modified for the background and adapted to the front-end processing. The current purpose is to summarize the core of the modification and test cicd. So please do not merge the current changes! ### What type of PR is it? Improvement ### Todos * [x] - User `pac4j-oidc` to support OIDC SSO based on cookie/session * [x] - Support rest api with header token * [x] - Front end modification. The 302 redirection of httpclient is not handled at present * [x] - Remove jdk1.8 support * [x] - Optimized cookie configuration * [x] - automatically create new user when logged in * [x] - Support clustering session by jdbc * [x] - Change mybatis log to SLF4J * [x] - Add some tests * [x] - Add some more documents about oidc support ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-1138 ### How should this be tested? Need to add some test later. ### Screenshots (if appropriate) ### Questions: * Do the license files need updating? No * Are there breaking changes for older versions? Yes * Does this need new documentation? Yes Author: cdmikechen <[email protected]> Signed-off-by: Kevin <[email protected]> Closes #1019 from cdmikechen/SUBMARINE-1138-0.8.0-pacj4j5.7 and squashes the following commits: 359109b [cdmikechen] SysUserService singleton 3501790 [cdmikechen] Add cookie document 0a45576 [cdmikechen] Fix document ad9d190 [cdmikechen] Add test d1a3304 [cdmikechen] revert authType b575241 [cdmikechen] remove SUBMARINE_AUTH_TYPE in image c552673 [cdmikechen] Fix test error 2e29658 [cdmikechen] Remove derby and upgrade jdk11 version c8644ce [cdmikechen] update jdk11 2803bda [cdmikechen] Adjustment code e9a1b8a [cdmikechen] Support jdk11 and pac4j 5.6.1 Add cookie samesite/httponly/securite eef1373 [cdmikechen] Test python-sdk 2ce98c1 [cdmikechen] Dealing with automatic user creation 1c98d2f [cdmikechen] Commit for python check fix 9ecb7cb [cdmikechen] Add api paths auth checks 220c49a [cdmikechen] Change auth type to flow type 3df16b4 [cdmikechen] Use servlet to replace static auth type check js 9409914 [cdmikechen] Handle front-end workbench oidc support 8b78695 [cdmikechen] deal with 401 16fe1a1 [cdmikechen] Add @context to fix error 90eb5c5 [cdmikechen] Add token to rest api header 0f8f263 [cdmikechen] Add oidc backend support(excluding the addition of oidc users)
apache · Dec 26, 2022 · 9347fe5 · 9347fe5
1 parent 4d234b0
commit 9347fe5
Show file tree

Hide file tree

Showing 50 changed files with 1,746 additions and 218 deletions.
diff --git a/.github/workflows/deploy_docker_images.yml b/.github/workflows/deploy_docker_images.yml
@@ -35,10 +35,10 @@ jobs:
         with:
           username: ${{ secrets.DOCKER_USERNAME }}
           password: ${{ secrets.DOCKER_PASSWORD }}
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:

diff --git a/.github/workflows/master.yml b/.github/workflows/master.yml
@@ -67,10 +67,10 @@ jobs:
       - uses: actions/checkout@v2
         with:
           fetch-depth: 50
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
@@ -146,10 +146,10 @@ jobs:
       - uses: actions/checkout@v2
         with:
           fetch-depth: 50
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
@@ -247,10 +247,10 @@ jobs:
         with:
           path: ./submarine-test/test-e2e/target/jacoco.exec
           key: ${{ runner.os }}-docker-${{ github.sha }}
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
@@ -293,10 +293,10 @@ jobs:
         with:
           path: ./submarine-test/test-k8s/target/jacoco.exec
           key: ${{ runner.os }}-docker-${{ github.sha }}
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
@@ -367,10 +367,10 @@ jobs:
         with:
           path: ./submarine-commons/commons-runtime/target/jacoco.exec
           key: ${{ runner.os }}-docker-${{ github.sha }}
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
@@ -403,10 +403,10 @@ jobs:
         with:
           path: ./submarine-client/target/jacoco.exec
           key: ${{ runner.os }}-docker-${{ github.sha }}
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
@@ -472,10 +472,10 @@ jobs:
           path: |
             ./submarine-serve/target/jacoco.exec
           key: ${{ runner.os }}-docker-${{ github.sha }}
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
@@ -588,10 +588,10 @@ jobs:
           path: |
             ./submarine-server/server-submitter/submitter-k8s/target/jacoco.exec
           key: ${{ runner.os }}-docker-${{ github.sha }}
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
@@ -620,18 +620,18 @@ jobs:
       - uses: actions/checkout@v2
       - uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - run: mvn org.apache.rat:apache-rat-plugin:check
   linter:
     name: Check Style
     runs-on: ubuntu-latest
     timeout-minutes: 30
     steps:
       - uses: actions/checkout@v2
-      - name: Set up JDK 1.8
+      - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: "1.8"
+          java-version: "11"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
@@ -754,16 +754,10 @@ jobs:
           path: ~/.sonar/cache
           key: ${{ runner.os }}-sonar
           restore-keys: ${{ runner.os }}-sonar
-      - name: Set up JDK 1.8
-        uses: actions/setup-java@v1
-        with:
-          java-version: "1.8"
       - name: Set up Maven 3.6.3
         uses: stCarolas/setup-maven@v4
         with:
           maven-version: 3.6.3
-      - name: Build the project with JDK 8
-        run: mvn install -DskipTests
       - name: Set up JDK 11
         uses: actions/setup-java@v1
         with:

diff --git a/conf/log4j.properties b/conf/log4j.properties
@@ -57,3 +57,6 @@ log4j.appender.console.target=System.err
 log4j.appender.console.layout=org.apache.log4j.PatternLayout
 log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} [%t]: %p %c{2}: %m%n
 log4j.appender.console.encoding=UTF-8
+
+# mybatis sql debug
+log4j.logger.org.apache.submarine.server.database=DEBUG
diff --git a/dev-support/docker-images/submarine/Dockerfile b/dev-support/docker-images/submarine/Dockerfile
@@ -13,7 +13,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM alpine:3.10
+FROM alpine:3.16.3
 MAINTAINER Apache Software Foundation <[email protected]>
 
 # If you are in China, enabling the following two lines of code can speed up the build of the image, but it may cause failure in travis.
@@ -23,10 +23,10 @@ MAINTAINER Apache Software Foundation <[email protected]>
 
 # INSTALL openjdk
 RUN apk update && \
-    apk add --no-cache openjdk8 bash tini && \
+    apk add --no-cache openjdk11 bash tini && \
     rm -rf /tmp/* /var/cache/apk/*
 
-ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk/jre
+ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk/jre
 
 # Install Submarine
 ADD ./tmp/submarine-dist-*.tar.gz /opt/

diff --git a/pom.xml b/pom.xml
@@ -48,7 +48,7 @@
 
   <properties>
     <!-- language versions -->
-    <java.version>1.8</java.version>
+    <java.version>11</java.version>
     <go.version>1.11.8</go.version>
 
     <!-- plugin versions -->
@@ -127,9 +127,8 @@
     <commons-text.version>1.4</commons-text.version>
     <zip4j.version>1.3.2</zip4j.version>
     <commons-collections.version>3.2.2</commons-collections.version>
-    <nimbus-jose-jwt.version>7.9</nimbus-jose-jwt.version>
+    <nimbus-jose-jwt.version>9.21</nimbus-jose-jwt.version>
     <mybatis-generator.version>1.3.7</mybatis-generator.version>
-    <derby.version>10.15.1.3</derby.version>
     <zeppelin.version>0.9.0-preview1</zeppelin.version>
     <jgit.version>5.13.0.202109080827-r</jgit.version>
     <atomix.version>3.1.5</atomix.version>
@@ -149,7 +148,8 @@
     <protobuf-java.version>3.14.0</protobuf-java.version>
     <joda-time.version>2.10.8</joda-time.version>
     <!--  pac4j  -->
-    <pac4j.version>4.5.6</pac4j.version>
+    <pac4j.version>5.6.1</pac4j.version>
+    <reflections.version>0.10.2</reflections.version>
   </properties>
 
   <modules>

diff --git a/...ons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfVars.java b/...ons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfVars.java
@@ -86,6 +86,11 @@ public enum ConfVars {
     ENVIRONMENT_CONDA_MIN_VERSION("environment.conda.min.version", "4.0.1"),
     ENVIRONMENT_CONDA_MAX_VERSION("environment.conda.max.version", "4.11.10"),
 
+    /* cookie setting */
+    SUBMARINE_COOKIE_HTTP_ONLY("submarine.cookie.http.only", false),
+    SUBMARINE_COOKIE_SECURE("submarine.cookie.secure", false),
+    SUBMARINE_COOKIE_SAMESITE("submarine.cookie.samesite", ""),
+
     /* auth */
     SUBMARINE_AUTH_TYPE("submarine.auth.type", "none"),
     SUBMARINE_AUTH_DEFAULT_SECRET("submarine.auth.default.secret", "SUBMARINE_SECRET_12345678901234567890"),

diff --git a/...e-server/server-api/src/main/java/org/apache/submarine/server/api/workbench/UserInfo.java b/...e-server/server-api/src/main/java/org/apache/submarine/server/api/workbench/UserInfo.java
@@ -24,7 +24,7 @@ public class UserInfo {
   private final String username;
   private final String password;
   private final String avatar;
-  private final int status;
+  private final String status;
   private final String telephone;
   private final String lastLoginIp;
   private final long lastLoginTime;
@@ -60,7 +60,7 @@ public static class Builder {
     private String username;
     private String password;
     private String avatar;
-    private int status = 0;
+    private String status;
     private String telephone;
     private String lastLoginIp;
     private long lastLoginTime;
@@ -91,7 +91,7 @@ public Builder avatar(String avatar) {
       return this;
     }
 
-    public Builder status(int status) {
+    public Builder status(String status) {
       this.status = status;
       return this;
     }
@@ -146,6 +146,66 @@ public UserInfo build() {
     }
   }
 
+  public String getId() {
+    return id;
+  }
+
+  public String getName() {
+    return name;
+  }
+
+  public String getUsername() {
+    return username;
+  }
+
+  public String getPassword() {
+    return password;
+  }
+
+  public String getAvatar() {
+    return avatar;
+  }
+
+  public String getStatus() {
+    return status;
+  }
+
+  public String getTelephone() {
+    return telephone;
+  }
+
+  public String getLastLoginIp() {
+    return lastLoginIp;
+  }
+
+  public long getLastLoginTime() {
+    return lastLoginTime;
+  }
+
+  public String getCreatorId() {
+    return creatorId;
+  }
+
+  public long getCreateTime() {
+    return createTime;
+  }
+
+  public String getMerchantCode() {
+    return merchantCode;
+  }
+
+  public int getDeleted() {
+    return deleted;
+  }
+
+  public String getRoleId() {
+    return roleId;
+  }
+
+  public Role getRole() {
+    return role;
+  }
+
   @Override
   public String toString() {
     return "User{" +

diff --git a/submarine-server/server-core/pom.xml b/submarine-server/server-core/pom.xml
@@ -258,12 +258,6 @@
       <!-- mysql-connector-java uses the GPL license. When we release the version in Submarine-dist, we exclude mysql-connector-java jar -->
     </dependency>
 
-    <dependency>
-      <groupId>org.apache.derby</groupId>
-      <artifactId>derby</artifactId>
-      <version>${derby.version}</version>
-    </dependency>
-
     <dependency>
       <groupId>org.json</groupId>
       <artifactId>json</artifactId>
@@ -465,6 +459,53 @@
       </exclusions>
     </dependency>
 
+    <dependency>
+      <groupId>org.pac4j</groupId>
+      <artifactId>pac4j-oidc</artifactId>
+      <version>${pac4j.version}</version>
+      <exclusions>
+        <exclusion>
+          <groupId>com.fasterxml.jackson.core</groupId>
+          <artifactId>jackson-databind</artifactId>
+        </exclusion>
+        <exclusion>
+          <groupId>org.ow2.asm</groupId>
+          <artifactId>asm</artifactId>
+        </exclusion>
+      </exclusions>
+    </dependency>
+
+    <dependency>
+      <groupId>org.pac4j</groupId>
+      <artifactId>pac4j-javaee</artifactId>
+      <version>${pac4j.version}</version>
+    </dependency>
+
+    <dependency>
+      <groupId>org.reflections</groupId>
+      <artifactId>reflections</artifactId>
+      <version>${reflections.version}</version>
+      <exclusions>
+        <exclusion>
+          <groupId>org.javassist</groupId>
+          <artifactId>javassist</artifactId>
+        </exclusion>
+      </exclusions>
+    </dependency>
+
+    <dependency>
+      <groupId>org.mockito</groupId>
+      <artifactId>mockito-core</artifactId>
+      <version>${mockito.version}</version>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>com.github.tomakehurst</groupId>
+      <artifactId>wiremock-jre8-standalone</artifactId>
+      <version>${wiremock.version}</version>
+      <scope>test</scope>
+    </dependency>
+
   </dependencies>
 
   <build>