From 9347fe5df6ac0e579df7b75421578eedb5342545 Mon Sep 17 00:00:00 2001 From: cdmikechen Date: Sun, 11 Dec 2022 13:56:03 +0800 Subject: [PATCH] SUBMARINE-1138. New SSO function based on OIDC ### What is this PR for? Use pac4j to support OIDC and default login action, and fix some user rest api question. Currently, it is a preview version, which is mainly modified for the background and adapted to the front-end processing. The current purpose is to summarize the core of the modification and test cicd. So please do not merge the current changes! ### What type of PR is it? Improvement ### Todos * [x] - User `pac4j-oidc` to support OIDC SSO based on cookie/session * [x] - Support rest api with header token * [x] - Front end modification. The 302 redirection of httpclient is not handled at present * [x] - Remove jdk1.8 support * [x] - Optimized cookie configuration * [x] - automatically create new user when logged in * [x] - Support clustering session by jdbc * [x] - Change mybatis log to SLF4J * [x] - Add some tests * [x] - Add some more documents about oidc support ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-1138 ### How should this be tested? Need to add some test later. ### Screenshots (if appropriate) ### Questions: * Do the license files need updating? No * Are there breaking changes for older versions? Yes * Does this need new documentation? Yes Author: cdmikechen Signed-off-by: Kevin Closes #1019 from cdmikechen/SUBMARINE-1138-0.8.0-pacj4j5.7 and squashes the following commits: 359109b8 [cdmikechen] SysUserService singleton 3501790e [cdmikechen] Add cookie document 0a455761 [cdmikechen] Fix document ad9d1902 [cdmikechen] Add test d1a3304e [cdmikechen] revert authType b5752418 [cdmikechen] remove SUBMARINE_AUTH_TYPE in image c5526736 [cdmikechen] Fix test error 2e296587 [cdmikechen] Remove derby and upgrade jdk11 version c8644cea [cdmikechen] update jdk11 2803bda4 [cdmikechen] Adjustment code e9a1b8ac [cdmikechen] Support jdk11 and pac4j 5.6.1 Add cookie samesite/httponly/securite eef13732 [cdmikechen] Test python-sdk 2ce98c1c [cdmikechen] Dealing with automatic user creation 1c98d2fc [cdmikechen] Commit for python check fix 9ecb7cbc [cdmikechen] Add api paths auth checks 220c49a0 [cdmikechen] Change auth type to flow type 3df16b4b [cdmikechen] Use servlet to replace static auth type check js 94099147 [cdmikechen] Handle front-end workbench oidc support 8b786954 [cdmikechen] deal with 401 16fe1a17 [cdmikechen] Add @Context to fix error 90eb5c5c [cdmikechen] Add token to rest api header 0f8f2636 [cdmikechen] Add oidc backend support(excluding the addition of oidc users) --- .github/workflows/deploy_docker_images.yml | 4 +- .github/workflows/master.yml | 44 ++-- conf/log4j.properties | 3 + .../docker-images/submarine/Dockerfile | 6 +- pom.xml | 8 +- .../commons/utils/SubmarineConfVars.java | 5 + .../server/api/workbench/UserInfo.java | 66 +++++- submarine-server/server-core/pom.xml | 53 ++++- .../submarine/server/SubmarineServer.java | 81 ++++++- .../server/rest/workbench/LoginRestApi.java | 20 +- .../server/rest/workbench/SysUserRestApi.java | 105 +++++++-- .../server/rest/workbench/SystemRestApi.java | 2 +- .../rest/workbench/annotation/NoneAuth.java | 33 +++ .../server/security/SecurityFactory.java | 10 +- .../server/security/SecurityProvider.java | 102 ++++++++- .../server/security/common/AuthFlowType.java | 43 ++++ .../server/security/common/CommonConfig.java | 5 + .../server/security/common/CommonFilter.java | 139 ++++++++++-- .../common/RegistryUserActionAdapter.java | 86 ++++++++ .../security/oidc/OidcCallbackResource.java | 49 +++++ .../server/security/oidc/OidcConfig.java | 54 +++++ .../server/security/oidc/OidcFilter.java | 79 +++++++ .../security/oidc/OidcSecurityProvider.java | 149 +++++++++++++ .../server/security/simple/SimpleFilter.java | 21 +- .../simple/SimpleSecurityProvider.java | 46 ++-- .../server/utils/response/DictAnnotation.java | 4 +- .../src/main/resources/log4j.properties | 3 + .../security/MockHttpServletRequest.java | 2 +- .../oidc/MockOidcHttpServletRequest.java | 37 ++++ .../security/oidc/SubmarineAuthOidcTest.java | 206 ++++++++++++++++++ .../{ => simple}/SubmarineAuthSimpleTest.java | 8 +- .../security/openid-configuration.json | 145 ++++++++++++ .../test/resources/security/user-info.json | 12 + .../server/database/utils/MyBatisUtil.java | 13 +- .../workbench/entity/SysUserEntity.java | 24 ++ .../workbench/service/SysUserService.java | 52 ++++- .../src/main/resources/mybatis-config.xml | 3 +- .../server-submitter/submitter-k8s/pom.xml | 4 - .../workbench-web/src/WEB-INF/web.xml | 6 - .../workbench-web/src/app/app.component.ts | 1 - .../workbench-web/src/app/app.module.ts | 10 +- .../src/app/core/auth/api-token-injector.ts | 60 +++++ .../src/app/core/auth/auth.guard.ts | 2 +- .../workbench/workbench-routing.module.ts | 2 +- .../src/app/services/auth.service.ts | 50 +++-- .../src/assets/security/provider.js | 24 ++ .../workbench-web/src/index.html | 1 + .../workbench-web/src/types/index.d.ts | 28 +++ .../workbench-web/tsconfig.json | 2 +- .../wip-designs/security-implementation.md | 52 ++--- 50 files changed, 1746 insertions(+), 218 deletions(-) create mode 100644 submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/annotation/NoneAuth.java create mode 100644 submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/AuthFlowType.java create mode 100644 submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/RegistryUserActionAdapter.java create mode 100644 submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcCallbackResource.java create mode 100644 submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcConfig.java create mode 100644 submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcFilter.java create mode 100644 submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcSecurityProvider.java create mode 100644 submarine-server/server-core/src/test/java/org/apache/submarine/server/security/oidc/MockOidcHttpServletRequest.java create mode 100644 submarine-server/server-core/src/test/java/org/apache/submarine/server/security/oidc/SubmarineAuthOidcTest.java rename submarine-server/server-core/src/test/java/org/apache/submarine/server/security/{ => simple}/SubmarineAuthSimpleTest.java (95%) create mode 100644 submarine-server/server-core/src/test/resources/security/openid-configuration.json create mode 100644 submarine-server/server-core/src/test/resources/security/user-info.json create mode 100644 submarine-workbench/workbench-web/src/app/core/auth/api-token-injector.ts create mode 100644 submarine-workbench/workbench-web/src/assets/security/provider.js create mode 100644 submarine-workbench/workbench-web/src/types/index.d.ts diff --git a/.github/workflows/deploy_docker_images.yml b/.github/workflows/deploy_docker_images.yml index 4955dc4966..3fa0f01006 100644 --- a/.github/workflows/deploy_docker_images.yml +++ b/.github/workflows/deploy_docker_images.yml @@ -35,10 +35,10 @@ jobs: with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: diff --git a/.github/workflows/master.yml b/.github/workflows/master.yml index 1b0745f34e..ab8c7ac85e 100644 --- a/.github/workflows/master.yml +++ b/.github/workflows/master.yml @@ -67,10 +67,10 @@ jobs: - uses: actions/checkout@v2 with: fetch-depth: 50 - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: @@ -146,10 +146,10 @@ jobs: - uses: actions/checkout@v2 with: fetch-depth: 50 - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: @@ -247,10 +247,10 @@ jobs: with: path: ./submarine-test/test-e2e/target/jacoco.exec key: ${{ runner.os }}-docker-${{ github.sha }} - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: @@ -293,10 +293,10 @@ jobs: with: path: ./submarine-test/test-k8s/target/jacoco.exec key: ${{ runner.os }}-docker-${{ github.sha }} - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: @@ -367,10 +367,10 @@ jobs: with: path: ./submarine-commons/commons-runtime/target/jacoco.exec key: ${{ runner.os }}-docker-${{ github.sha }} - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: @@ -403,10 +403,10 @@ jobs: with: path: ./submarine-client/target/jacoco.exec key: ${{ runner.os }}-docker-${{ github.sha }} - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: @@ -472,10 +472,10 @@ jobs: path: | ./submarine-serve/target/jacoco.exec key: ${{ runner.os }}-docker-${{ github.sha }} - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: @@ -588,10 +588,10 @@ jobs: path: | ./submarine-server/server-submitter/submitter-k8s/target/jacoco.exec key: ${{ runner.os }}-docker-${{ github.sha }} - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: @@ -620,7 +620,7 @@ jobs: - uses: actions/checkout@v2 - uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - run: mvn org.apache.rat:apache-rat-plugin:check linter: name: Check Style @@ -628,10 +628,10 @@ jobs: timeout-minutes: 30 steps: - uses: actions/checkout@v2 - - name: Set up JDK 1.8 + - name: Set up JDK 11 uses: actions/setup-java@v1 with: - java-version: "1.8" + java-version: "11" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: @@ -754,16 +754,10 @@ jobs: path: ~/.sonar/cache key: ${{ runner.os }}-sonar restore-keys: ${{ runner.os }}-sonar - - name: Set up JDK 1.8 - uses: actions/setup-java@v1 - with: - java-version: "1.8" - name: Set up Maven 3.6.3 uses: stCarolas/setup-maven@v4 with: maven-version: 3.6.3 - - name: Build the project with JDK 8 - run: mvn install -DskipTests - name: Set up JDK 11 uses: actions/setup-java@v1 with: diff --git a/conf/log4j.properties b/conf/log4j.properties index 2f44c7217d..125e24f8d3 100644 --- a/conf/log4j.properties +++ b/conf/log4j.properties @@ -57,3 +57,6 @@ log4j.appender.console.target=System.err log4j.appender.console.layout=org.apache.log4j.PatternLayout log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} [%t]: %p %c{2}: %m%n log4j.appender.console.encoding=UTF-8 + +# mybatis sql debug +log4j.logger.org.apache.submarine.server.database=DEBUG diff --git a/dev-support/docker-images/submarine/Dockerfile b/dev-support/docker-images/submarine/Dockerfile index ca76d318a9..507c7d8d0b 100644 --- a/dev-support/docker-images/submarine/Dockerfile +++ b/dev-support/docker-images/submarine/Dockerfile @@ -13,7 +13,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -FROM alpine:3.10 +FROM alpine:3.16.3 MAINTAINER Apache Software Foundation # If you are in China, enabling the following two lines of code can speed up the build of the image, but it may cause failure in travis. @@ -23,10 +23,10 @@ MAINTAINER Apache Software Foundation # INSTALL openjdk RUN apk update && \ - apk add --no-cache openjdk8 bash tini && \ + apk add --no-cache openjdk11 bash tini && \ rm -rf /tmp/* /var/cache/apk/* -ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk/jre +ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk/jre # Install Submarine ADD ./tmp/submarine-dist-*.tar.gz /opt/ diff --git a/pom.xml b/pom.xml index 66a266a69f..b1df264f5e 100644 --- a/pom.xml +++ b/pom.xml @@ -48,7 +48,7 @@ - 1.8 + 11 1.11.8 @@ -127,9 +127,8 @@ 1.4 1.3.2 3.2.2 - 7.9 + 9.21 1.3.7 - 10.15.1.3 0.9.0-preview1 5.13.0.202109080827-r 3.1.5 @@ -149,7 +148,8 @@ 3.14.0 2.10.8 - 4.5.6 + 5.6.1 + 0.10.2 diff --git a/submarine-commons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfVars.java b/submarine-commons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfVars.java index 6b0b9e45c6..6dbe18e43c 100644 --- a/submarine-commons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfVars.java +++ b/submarine-commons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfVars.java @@ -86,6 +86,11 @@ public enum ConfVars { ENVIRONMENT_CONDA_MIN_VERSION("environment.conda.min.version", "4.0.1"), ENVIRONMENT_CONDA_MAX_VERSION("environment.conda.max.version", "4.11.10"), + /* cookie setting */ + SUBMARINE_COOKIE_HTTP_ONLY("submarine.cookie.http.only", false), + SUBMARINE_COOKIE_SECURE("submarine.cookie.secure", false), + SUBMARINE_COOKIE_SAMESITE("submarine.cookie.samesite", ""), + /* auth */ SUBMARINE_AUTH_TYPE("submarine.auth.type", "none"), SUBMARINE_AUTH_DEFAULT_SECRET("submarine.auth.default.secret", "SUBMARINE_SECRET_12345678901234567890"), diff --git a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/workbench/UserInfo.java b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/workbench/UserInfo.java index c14eb86559..edbdaaa2ff 100644 --- a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/workbench/UserInfo.java +++ b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/workbench/UserInfo.java @@ -24,7 +24,7 @@ public class UserInfo { private final String username; private final String password; private final String avatar; - private final int status; + private final String status; private final String telephone; private final String lastLoginIp; private final long lastLoginTime; @@ -60,7 +60,7 @@ public static class Builder { private String username; private String password; private String avatar; - private int status = 0; + private String status; private String telephone; private String lastLoginIp; private long lastLoginTime; @@ -91,7 +91,7 @@ public Builder avatar(String avatar) { return this; } - public Builder status(int status) { + public Builder status(String status) { this.status = status; return this; } @@ -146,6 +146,66 @@ public UserInfo build() { } } + public String getId() { + return id; + } + + public String getName() { + return name; + } + + public String getUsername() { + return username; + } + + public String getPassword() { + return password; + } + + public String getAvatar() { + return avatar; + } + + public String getStatus() { + return status; + } + + public String getTelephone() { + return telephone; + } + + public String getLastLoginIp() { + return lastLoginIp; + } + + public long getLastLoginTime() { + return lastLoginTime; + } + + public String getCreatorId() { + return creatorId; + } + + public long getCreateTime() { + return createTime; + } + + public String getMerchantCode() { + return merchantCode; + } + + public int getDeleted() { + return deleted; + } + + public String getRoleId() { + return roleId; + } + + public Role getRole() { + return role; + } + @Override public String toString() { return "User{" + diff --git a/submarine-server/server-core/pom.xml b/submarine-server/server-core/pom.xml index 311ee5a661..f582c69b8f 100644 --- a/submarine-server/server-core/pom.xml +++ b/submarine-server/server-core/pom.xml @@ -258,12 +258,6 @@ - - org.apache.derby - derby - ${derby.version} - - org.json json @@ -465,6 +459,53 @@ + + org.pac4j + pac4j-oidc + ${pac4j.version} + + + com.fasterxml.jackson.core + jackson-databind + + + org.ow2.asm + asm + + + + + + org.pac4j + pac4j-javaee + ${pac4j.version} + + + + org.reflections + reflections + ${reflections.version} + + + org.javassist + javassist + + + + + + org.mockito + mockito-core + ${mockito.version} + test + + + com.github.tomakehurst + wiremock-jre8-standalone + ${wiremock.version} + test + + diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/SubmarineServer.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/SubmarineServer.java index 44498802c9..5a3f9b8b93 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/SubmarineServer.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/SubmarineServer.java @@ -18,12 +18,16 @@ */ package org.apache.submarine.server; +import org.apache.commons.lang3.StringUtils; import org.apache.log4j.PropertyConfigurator; +import org.apache.submarine.server.database.utils.MyBatisUtil; import org.apache.submarine.server.rest.provider.YamlEntityProvider; import org.apache.submarine.server.security.SecurityFactory; import org.apache.submarine.server.security.SecurityProvider; +import org.apache.submarine.server.security.common.AuthFlowType; import org.apache.submarine.server.workbench.websocket.NotebookServer; import org.apache.submarine.server.websocket.WebSocketServer; +import org.eclipse.jetty.http.HttpCookie; import org.eclipse.jetty.http.HttpVersion; import org.eclipse.jetty.server.Handler; import org.eclipse.jetty.server.HttpConfiguration; @@ -33,6 +37,8 @@ import org.eclipse.jetty.server.ServerConnector; import org.eclipse.jetty.server.SslConnectionFactory; import org.eclipse.jetty.server.handler.HandlerList; +import org.eclipse.jetty.server.session.DatabaseAdaptor; +import org.eclipse.jetty.server.session.JDBCSessionDataStoreFactory; import org.eclipse.jetty.server.session.SessionHandler; import org.eclipse.jetty.servlet.DefaultServlet; import org.eclipse.jetty.servlet.ServletContextHandler; @@ -119,6 +125,9 @@ protected void configure() { setupRestApiContextHandler(webApp, conf); + // Cookie config + setCookieConfig(webApp); + // Notebook server setupNotebookServer(webApp, conf, sharedServiceLocator); @@ -199,6 +208,51 @@ private static WebAppContext setupWebAppContext(HandlerList handlers, webApp.setTempDirectory(warTempDirectory); } + // add security filter + Optional securityProvider = SecurityFactory.getSecurityProvider(); + if (securityProvider.isPresent()) { + SecurityProvider provider = securityProvider.get(); + Class filterClass = provider.getFilterClass(); + // add filter + LOG.info("Add {} to support auth", filterClass); + webApp.addFilter(filterClass, "/*", EnumSet.of(DispatcherType.REQUEST)); + // add flow type result to front end + AuthFlowType type = provider.getAuthFlowType(); + // If using session, we can add JDBCSessionDataStoreFactory to support clustering session + // This solves two problems: + // 1. session loss after service restart + // 2. session sharing when multiple replicas + if (type == AuthFlowType.SESSION) { + // Configure a JDBCSessionDataStoreFactory. + JDBCSessionDataStoreFactory sessionDataStoreFactory = new JDBCSessionDataStoreFactory(); + sessionDataStoreFactory.setGracePeriodSec(3600); + sessionDataStoreFactory.setSavePeriodSec(0); + // add datasource (current mybatis) to factory + DatabaseAdaptor adaptor = new DatabaseAdaptor(); + adaptor.setDatasource(MyBatisUtil.getDatasource()); + sessionDataStoreFactory.setDatabaseAdaptor(adaptor); + // Add the SessionDataStoreFactory as a bean on the server. + jettyWebServer.addBean(sessionDataStoreFactory); + } + + ServletHolder authProviderServlet = new ServletHolder(new HttpServlet() { + private static final long serialVersionUID = 1L; + private final String staticProviderJs = String.format( + "(function () { window.GLOBAL_CONFIG = { \"type\": \"%s\" }; })();", type.getType() + ); + private static final String contentType = "application/javascript"; + private static final String encoding = "UTF-8"; + @Override + protected void doGet(HttpServletRequest req, HttpServletResponse resp) + throws ServletException, IOException { + resp.setContentType(contentType); + resp.setCharacterEncoding(encoding); + resp.getWriter().write(staticProviderJs); + } + }); + webApp.addServlet(authProviderServlet, "/assets/security/provider.js"); + } + webApp.addServlet(new ServletHolder(new DefaultServlet()), "/"); // When requesting the workbench page, the content of index.html needs to be returned, // otherwise a 404 error will be displayed @@ -207,19 +261,30 @@ private static WebAppContext setupWebAppContext(HandlerList handlers, webApp.addServlet(new ServletHolder(RefreshServlet.class), "/user/*"); webApp.addServlet(new ServletHolder(RefreshServlet.class), "/workbench/*"); - // add security filter - Optional securityProvider = SecurityFactory.getSecurityProvider(); - if (securityProvider.isPresent()) { - Class filterClass = securityProvider.get().getFilterClass(); - LOG.info("Add {} to support auth", filterClass); - webApp.addFilter(filterClass, "/*", EnumSet.of(DispatcherType.REQUEST)); - } - handlers.setHandlers(new Handler[]{webApp}); return webApp; } + /** + * Session cookie config + */ + public static void setCookieConfig(WebAppContext webapp) { + // http only + webapp.getSessionHandler().getSessionCookieConfig().setHttpOnly( + conf.getBoolean(SubmarineConfVars.ConfVars.SUBMARINE_COOKIE_HTTP_ONLY) + ); + // same site: NONE("None"), STRICT("Strict"), LAX("Lax"); + String sameSite = conf.getString(SubmarineConfVars.ConfVars.SUBMARINE_COOKIE_SAMESITE); + if (StringUtils.isNoneBlank(sameSite)) { + webapp.getSessionHandler().setSameSite(HttpCookie.SameSite.valueOf(sameSite.toUpperCase())); + } + // secure + webapp.getSessionHandler().getSessionCookieConfig().setSecure( + conf.getBoolean(SubmarineConfVars.ConfVars.SUBMARINE_COOKIE_SECURE) + ); + } + private static Server setupJettyServer(SubmarineConfiguration conf) { ThreadPool threadPool = new QueuedThreadPool(conf.getInt(SubmarineConfVars.ConfVars.SUBMARINE_SERVER_JETTY_THREAD_POOL_MAX), diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/LoginRestApi.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/LoginRestApi.java index f385a99766..4feb6dd2cb 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/LoginRestApi.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/LoginRestApi.java @@ -21,6 +21,7 @@ import com.google.gson.Gson; import com.google.gson.reflect.TypeToken; import org.apache.ibatis.session.SqlSession; +import org.apache.submarine.server.rest.workbench.annotation.NoneAuth; import org.apache.submarine.server.rest.workbench.annotation.SubmarineApi; import org.apache.submarine.server.database.workbench.entity.SysUserEntity; import org.apache.submarine.server.database.workbench.mappers.SysUserMapper; @@ -54,6 +55,7 @@ public LoginRestApi() { @POST @Path("/login") @SubmarineApi + @NoneAuth public Response login(String loginParams) { HashMap mapParams = gson.fromJson(loginParams, new TypeToken>() {}.getType()); @@ -80,7 +82,8 @@ public Response login(String loginParams) { claimsMap.put("exp", new Date().getTime() + CommonConfig.MAX_AGE); claimsMap.put("sub", "submarine"); claimsMap.put("jti", sysUser.getId()); - + // TODO(cdmikechen) By default the simple token is used, + // in other cases such as ldap it may need to be returned as an interface String token = SimpleLoginConfig.getJwtGenerator().generate(claimsMap); sysUser.setToken(token); } else { @@ -105,21 +108,6 @@ public Response login(String loginParams) { .build(); } - /** - * Get user by unique name - */ - public SysUserEntity getUserByName(String name) throws Exception { - SysUserEntity sysUser = null; - try (SqlSession sqlSession = MyBatisUtil.getSqlSession()) { - SysUserMapper sysUserMapper = sqlSession.getMapper(SysUserMapper.class); - sysUser = sysUserMapper.getUserByUniqueName(name); - } catch (Exception e) { - LOG.error(e.getMessage(), e); - throw new Exception(e); - } - return sysUser; - } - @POST @Path("/2step-code") @SubmarineApi diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/SysUserRestApi.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/SysUserRestApi.java index d8e41e75b3..b4d053f128 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/SysUserRestApi.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/SysUserRestApi.java @@ -19,8 +19,10 @@ package org.apache.submarine.server.rest.workbench; import com.github.pagehelper.PageInfo; -import com.google.gson.Gson; +import org.apache.submarine.server.rest.workbench.annotation.NoneAuth; +import org.apache.submarine.server.security.SecurityFactory; +import org.apache.submarine.server.security.SecurityProvider; import org.apache.submarine.server.utils.response.JsonResponse; import org.apache.submarine.server.utils.response.JsonResponse.ListResult; import org.apache.submarine.server.rest.workbench.annotation.SubmarineApi; @@ -30,11 +32,14 @@ import org.apache.submarine.server.api.workbench.Permission; import org.apache.submarine.server.api.workbench.Role; import org.apache.submarine.server.api.workbench.UserInfo; +import org.pac4j.core.profile.CommonProfile; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import javax.inject.Inject; import javax.inject.Singleton; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; import javax.ws.rs.DELETE; import javax.ws.rs.GET; import javax.ws.rs.POST; @@ -42,18 +47,22 @@ import javax.ws.rs.Path; import javax.ws.rs.Produces; import javax.ws.rs.QueryParam; +import javax.ws.rs.core.Context; import javax.ws.rs.core.Response; import java.util.ArrayList; import java.util.List; +import java.util.Optional; + +import static org.apache.submarine.server.database.workbench.service.SysUserService.DEFAULT_ADMIN_UID; @Path("/sys/user") @Produces("application/json") @Singleton public class SysUserRestApi { + private static final Logger LOG = LoggerFactory.getLogger(SysUserRestApi.class); - private SysUserService userService = new SysUserService(); - private static final Gson gson = new Gson(); + private static final SysUserService userService = SysUserService.INSTANCE; @Inject public SysUserRestApi() { @@ -69,7 +78,7 @@ public Response queryPageList(@QueryParam("userName") String userName, @QueryParam("field") String field, @QueryParam("pageNo") int pageNo, @QueryParam("pageSize") int pageSize) { - LOG.info("queryDictList userName:{}, email:{}, deptCode:{}, " + + LOG.debug("queryDictList userName:{}, email:{}, deptCode:{}, " + "column:{}, field:{}, pageNo:{}, pageSize:{}", userName, email, deptCode, column, field, pageNo, pageSize); @@ -107,6 +116,8 @@ public Response edit(SysUserEntity sysUser) { @POST @Path("/add") @SubmarineApi + /* This is temporarily marked as not requiring validation in the way that on the login page failed */ + @NoneAuth public Response add(SysUserEntity sysUser) { LOG.info("add({})", sysUser.toString()); @@ -159,7 +170,53 @@ public Response changePassword(SysUserEntity sysUser) { @GET @Path("/info") @SubmarineApi - public Response info() { + public Response info(@Context HttpServletRequest hsRequest, @Context HttpServletResponse hsResponse) { + UserInfo userInfo = null; + // get SecurityProvider to use perform method to get user info + Optional securityProvider = SecurityFactory.getSecurityProvider(); + if (securityProvider.isPresent()) { + Optional profileOpt = securityProvider.get().perform(hsRequest, hsResponse); + if (profileOpt.isPresent()) { + // Get user information + SysUserEntity sysUser = userService.getUserByName(profileOpt.get().getUsername()); + if (sysUser != null) { + // Create user info + UserInfo.Builder userInfoBuilder = new UserInfo.Builder(sysUser.getId(), sysUser.getUserName()); + userInfo = userInfoBuilder + .username(sysUser.getUserName()) + .password("******") + .avatar(sysUser.getAvatar()) + .status(sysUser.getStatus()) + .telephone(sysUser.getPhone()) + .lastLoginIp("******") + .lastLoginTime(System.currentTimeMillis()) + .creatorId(sysUser.getUserName()) + .createTime(sysUser.getCreateTime().getTime()) + .merchantCode("") + .deleted(0) + .roleId("default") + .role(createDefaultRole()).build(); + } + } + } + if (userInfo == null) { // user not found + return new JsonResponse.Builder<>(Response.Status.OK). + success(false) + .message("User can not be found!") + .build(); + } else { + return new JsonResponse.Builder(Response.Status.OK) + .success(true) + .result(userInfo) + .build(); + } + } + + /** + * Create default role + */ + private Role createDefaultRole() { + // TODO(cdmikechen): Will do after the role function is completed List actions = new ArrayList(); Action action1 = new Action("add", false, "add"); Action action2 = new Action("query", false, "query"); @@ -223,17 +280,35 @@ public Response info() { permissions.add(permission12); Role.Builder roleBuilder = new Role.Builder("admin", "admin"); - Role role = roleBuilder.describe("Permission").status(1).creatorId("system") - .createTime(1497160610259L).deleted(0).permissions(permissions).build(); - - UserInfo.Builder userInfoBuilder = new UserInfo.Builder("4291d7da9005377ec9aec4a71ea837f", "admin"); - UserInfo userInfo = userInfoBuilder.username("admin").password("") - .avatar("/avatar2.jpg").status(1).telephone("").lastLoginIp("27.154.74.117") - .lastLoginTime(1534837621348L).creatorId("admin") - .createTime(1497160610259L).merchantCode("TLif2btpzg079h15bk") - .deleted(0).roleId("admin").role(role).build(); + return roleBuilder.describe("Permission") + .status(1) + .creatorId("system") + .createTime(System.currentTimeMillis()) + .deleted(0) + .permissions(permissions) + .build(); + } - return new JsonResponse.Builder(Response.Status.OK).success(true).result(userInfo).build(); + /** + * Create default user + */ + private UserInfo createDefaultUser() { + LOG.warn("Can not get user info, use a default admin user"); + UserInfo.Builder userInfoBuilder = new UserInfo.Builder(DEFAULT_ADMIN_UID, "admin"); + return userInfoBuilder.username("admin") + .password("") + .avatar("/avatar2.jpg") + .status("1") + .telephone("") + .lastLoginIp("******") + .lastLoginTime(System.currentTimeMillis()) + .creatorId("admin") + .createTime(System.currentTimeMillis()) + .merchantCode("TLif2btpzg079h15bk") + .deleted(0) + .roleId("admin") + .role(createDefaultRole()) + .build(); } @POST diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/SystemRestApi.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/SystemRestApi.java index 733dd846ea..9e5a8d2c5b 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/SystemRestApi.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/SystemRestApi.java @@ -49,7 +49,7 @@ public class SystemRestApi { private static final Logger LOG = LoggerFactory.getLogger(SystemRestApi.class); - private SysUserService userService = new SysUserService(); + private static final SysUserService userService = SysUserService.INSTANCE; @Inject public SystemRestApi() { diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/annotation/NoneAuth.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/annotation/NoneAuth.java new file mode 100644 index 0000000000..ca6b1ceefa --- /dev/null +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/workbench/annotation/NoneAuth.java @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.submarine.server.rest.workbench.annotation; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Identifies methods that do not require auth checks + */ +@Retention(RetentionPolicy.RUNTIME) +@Target({ElementType.METHOD}) +public @interface NoneAuth { +} diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/SecurityFactory.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/SecurityFactory.java index d2232443f5..01fd6f2e49 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/SecurityFactory.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/SecurityFactory.java @@ -21,6 +21,7 @@ import org.apache.submarine.commons.utils.SubmarineConfVars; import org.apache.submarine.commons.utils.SubmarineConfiguration; +import org.apache.submarine.server.security.oidc.OidcSecurityProvider; import org.apache.submarine.server.security.simple.SimpleSecurityProvider; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -39,10 +40,15 @@ public static SimpleSecurityProvider getSimpleSecurityProvider() { return (SimpleSecurityProvider) providerMap.get("simple"); } + public static OidcSecurityProvider getPac4jSecurityProvider() { + return (OidcSecurityProvider) providerMap.get("oidc"); + } + static { - // int provider map + // init provider map providerMap = new HashMap<>(); providerMap.put("simple", new SimpleSecurityProvider()); + providerMap.put("oidc", new OidcSecurityProvider()); } public static void addProvider(String name, SecurityProvider provider) { @@ -51,7 +57,7 @@ public static void addProvider(String name, SecurityProvider provider) { public static Optional getSecurityProvider() { String authType = SubmarineConfiguration.getInstance() - .getString(SubmarineConfVars.ConfVars.SUBMARINE_AUTH_TYPE); + .getString(SubmarineConfVars.ConfVars.SUBMARINE_AUTH_TYPE); if (providerMap.containsKey(authType)) { return Optional.ofNullable(providerMap.get(authType)); } else { diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/SecurityProvider.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/SecurityProvider.java index c775705b94..ee78d1b367 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/SecurityProvider.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/SecurityProvider.java @@ -19,8 +19,22 @@ package org.apache.submarine.server.security; +import org.apache.submarine.server.security.common.AuthFlowType; import org.pac4j.core.config.Config; +import org.pac4j.core.context.WebContext; +import org.pac4j.core.context.session.SessionStore; +import org.pac4j.core.engine.CallbackLogic; +import org.pac4j.core.engine.DefaultCallbackLogic; +import org.pac4j.core.engine.DefaultLogoutLogic; +import org.pac4j.core.engine.DefaultSecurityLogic; +import org.pac4j.core.engine.LogoutLogic; +import org.pac4j.core.engine.SecurityLogic; +import org.pac4j.core.http.adapter.HttpActionAdapter; import org.pac4j.core.profile.CommonProfile; +import org.pac4j.core.util.FindBest; +import org.pac4j.jee.context.JEEContextFactory; +import org.pac4j.jee.context.session.JEESessionStoreFactory; +import org.pac4j.jee.http.adapter.JEEHttpActionAdapter; import javax.servlet.Filter; import javax.servlet.http.HttpServletRequest; @@ -30,32 +44,104 @@ /** * Provide security methods for different authentication types */ -public interface SecurityProvider { +public abstract class SecurityProvider { - String DEFAULT_AUTHORIZER = "isAuthenticated"; + protected final String DEFAULT_AUTHORIZER = "isAuthenticated"; + + protected Config pac4jConfig; + + /** + * Get authentication flow type + */ + public AuthFlowType getAuthFlowType() { + return AuthFlowType.TOKEN; + } /** * Get filter class */ - Class getFilterClass(); + public abstract Class getFilterClass(); /** * Get pac4j config */ - Config getConfig(); + public Config getConfig() { + if (this.pac4jConfig == null) this.pac4jConfig = createConfig(); + return pac4jConfig; + } + + /** + * Create pac4j config + */ + protected abstract Config createConfig(); /** * Get pac4j client */ - String getClient(HttpServletRequest httpServletRequest); + public abstract String getClient(HttpServletRequest httpServletRequest); + + /** + * Create {@link WebContext} + */ + protected WebContext createWebContext(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { + return FindBest.webContextFactory(null, getConfig(), JEEContextFactory.INSTANCE) + .newContext(hsRequest, hsResponse); + } + + /** + * Create {@link SessionStore} + */ + protected SessionStore createSessionStore(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { + return FindBest.sessionStoreFactory(null, getConfig(), JEESessionStoreFactory.INSTANCE) + .newSessionStore(hsRequest, hsResponse); + } + + /** + * Create {@link HttpActionAdapter} + */ + protected HttpActionAdapter createHttpActionAdapter() { + return FindBest.httpActionAdapter(null, getConfig(), JEEHttpActionAdapter.INSTANCE); + } + + /** + * Create {@link SecurityLogic} + */ + protected SecurityLogic createSecurityLogic() { + return FindBest.securityLogic(null, getConfig(), DefaultSecurityLogic.INSTANCE); + } + + /** + * Create {@link CallbackLogic} + */ + protected CallbackLogic createCallbackLogic() { + return FindBest.callbackLogic(null, getConfig(), DefaultCallbackLogic.INSTANCE); + } + + /** + * Create {@link LogoutLogic} + */ + protected LogoutLogic createLogoutLogic() { + return FindBest.logoutLogic(null, this.pac4jConfig, DefaultLogoutLogic.INSTANCE); + } /** * Process authentication information and return user profile */ - Optional perform(HttpServletRequest hsRequest, HttpServletResponse hsResponse); + public abstract Optional perform(HttpServletRequest hsRequest, HttpServletResponse hsResponse); + + /** + * Handling login perform + */ + public void login(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { } /** - * Get user profile + * Handling callback perform */ - Optional getProfile(HttpServletRequest hsRequest, HttpServletResponse hsResponse); + public void callback(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { } + + /** + * Handling logout perform + */ + public void logout(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { } + } diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/AuthFlowType.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/AuthFlowType.java new file mode 100644 index 0000000000..af7600d188 --- /dev/null +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/AuthFlowType.java @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.submarine.server.security.common; + +/** + * Type of the authentication flow + */ +public enum AuthFlowType { + + /* Use header token to pass authentication information by default */ + TOKEN("token"), + + /* Using session to pass authentication information is generally suitable for sso */ + SESSION("session"); + + private final String type; + + AuthFlowType(String type) { + this.type = type; + } + + public String getType() { + return type; + } + +} diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/CommonConfig.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/CommonConfig.java index 83f1bb84be..118018aa54 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/CommonConfig.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/CommonConfig.java @@ -31,6 +31,11 @@ public class CommonConfig { public static final int MAX_AGE; + public static final String AGENT_HEADER = "User-Agent"; + // python sdk agent header (submarine-sdk/pysubmarine/submarine/client/api_client.py#93) + // We only deal with front and server, py-sdk is not dealt with now + public static final String PYTHON_USER_AGENT_REGREX = "^OpenAPI-Generator/[\\w\\-\\.]+/python$"; + static { SubmarineConfiguration conf = SubmarineConfiguration.getInstance(); MAX_AGE = conf.getInt(ConfVars.SUBMARINE_AUTH_MAX_AGE_ENV); diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/CommonFilter.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/CommonFilter.java index a0a0fdc697..aa0f0d8965 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/CommonFilter.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/CommonFilter.java @@ -19,28 +19,133 @@ package org.apache.submarine.server.security.common; -import org.pac4j.core.context.JEEContext; -import org.pac4j.core.context.session.JEESessionStore; -import org.pac4j.core.context.session.SessionStore; -import org.pac4j.core.engine.DefaultCallbackLogic; -import org.pac4j.core.engine.DefaultLogoutLogic; -import org.pac4j.core.engine.DefaultSecurityLogic; -import org.pac4j.core.http.adapter.HttpActionAdapter; -import org.pac4j.core.http.adapter.JEEHttpActionAdapter; -import org.pac4j.core.profile.CommonProfile; -import org.pac4j.core.profile.UserProfile; +import org.apache.commons.lang3.StringUtils; +import org.apache.submarine.server.rest.workbench.annotation.NoneAuth; +import org.reflections.Reflections; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.servlet.FilterConfig; +import javax.servlet.ServletException; +import javax.servlet.http.HttpServletRequest; +import javax.ws.rs.DELETE; +import javax.ws.rs.GET; +import javax.ws.rs.HEAD; +import javax.ws.rs.OPTIONS; +import javax.ws.rs.POST; +import javax.ws.rs.Path; +import javax.ws.rs.PATCH; +import javax.ws.rs.PUT; +import java.lang.annotation.Annotation; +import java.lang.reflect.Method; +import java.util.Arrays; +import java.util.HashSet; +import java.util.Set; +import java.util.regex.Pattern; +import java.util.stream.Stream; + +import static org.apache.submarine.server.security.common.CommonConfig.PYTHON_USER_AGENT_REGREX; +import static org.reflections.scanners.Scanners.SubTypes; +import static org.reflections.scanners.Scanners.TypesAnnotated; + public class CommonFilter { - public static final HttpActionAdapter DEFAULT_HTTP_ACTION_ADAPTER = JEEHttpActionAdapter.INSTANCE; + private static final Logger LOG = LoggerFactory.getLogger(CommonFilter.class); + + /* Supported http method */ + protected final Set> SUPPORT_HTTP_METHODS = + new HashSet>() {{ + add(GET.class); + add(PUT.class); + add(POST.class); + add(DELETE.class); + add(PATCH.class); + add(OPTIONS.class); + add(HEAD.class); + }}; + + /* api with the full path */ + protected final Set REST_API_PATHS = new HashSet<>(16); + /* api with the regrex path */ + protected final Set REST_REGREX_API_PATHS = new HashSet<>(16); + + /** + * Filter init + */ + public void init(FilterConfig filterConfig) throws ServletException { + // Scan rest api class by annotations @Path + Reflections reflections = new Reflections("org.apache.submarine.server.rest"); + Set> rests = reflections.get(SubTypes.of(TypesAnnotated.with(Path.class)).asClass()); + for (Class rest : rests) { + // get path + Path pathAnno = rest.getAnnotation(Path.class); + String path = pathAnno.value(); + if (path.startsWith("/")) path = path.substring(1); + if (path.endsWith("/")) path = path.substring(0, path.length() - 1); + // loop method annotations + Method[] methods = rest.getDeclaredMethods(); + for (Method method : methods) { + addSupportedApiPath(path, method); + } + } + LOG.info("Get security filter rest api path = {} and regrex api path = {}", + REST_API_PATHS, REST_REGREX_API_PATHS); + } - public static final DefaultCallbackLogic CALLBACK_LOGIC = - new DefaultCallbackLogic<>(); + /** + * Add supported api path + */ + private void addSupportedApiPath(String path, Method method) { + Stream annotations = Arrays.stream(method.getAnnotations()); + // Only methods marked as REST http method + if (annotations.anyMatch(annotation -> SUPPORT_HTTP_METHODS.contains(annotation.annotationType()))) { + // Methods with the @NoneAuth require no authentication + if (method.getAnnotation(NoneAuth.class) != null) return; + Path pathAnno = method.getAnnotation(Path.class); + String endpoint = pathAnno == null ? "" : pathAnno.value(); - public static final DefaultSecurityLogic SECURITY_LOGIC = - new DefaultSecurityLogic<>(); + // If endpoint is empty, the api is used as the path + if ("".equals(endpoint) || "/".equals(endpoint)) { + REST_API_PATHS.add(String.format("/api/%s", path)); + } else { + if (endpoint.startsWith("/")) endpoint = endpoint.substring(1); + if (endpoint.endsWith("/")) endpoint = endpoint.substring(0, endpoint.length() - 1); + String api = String.format("/api/%s/%s", path, endpoint); + if (api.matches("(.*)\\{\\w+\\}(.*)")) { + REST_REGREX_API_PATHS.add(api.replaceAll("\\{\\w+\\}", "((?!\\/).)*")); + } else { + REST_API_PATHS.add(api); + } + } + } + } - public static final DefaultLogoutLogic LOGOUT_LOGIC = new DefaultLogoutLogic<>(); + /** + * Check if uri is in the list of known apis + */ + private boolean isSupportedRest(String uri) { + // Return true if found in the full path + if (REST_API_PATHS.contains(uri)) return true; + // Otherwise, do a match on the regrex path + for (String api : REST_REGREX_API_PATHS) { + if (Pattern.matches(api, uri)) { + return true; + } + } + return false; + } - public static final SessionStore SESSION_STORE = new JEESessionStore(); + /** + * Check whether the endpoint requires authorization verification + */ + protected boolean isProtectedApi(HttpServletRequest httpServletRequest) { + // If it is called by python, temporarily passed + String agentHeader = httpServletRequest.getHeader(CommonConfig.AGENT_HEADER); + if (StringUtils.isNoneBlank(agentHeader) && agentHeader.matches(PYTHON_USER_AGENT_REGREX)) { + return false; + } + // Now we just verify the api + return isSupportedRest(httpServletRequest.getRequestURI()); + } } diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/RegistryUserActionAdapter.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/RegistryUserActionAdapter.java new file mode 100644 index 0000000000..8ff0b1b96d --- /dev/null +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/common/RegistryUserActionAdapter.java @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.submarine.server.security.common; + +import org.apache.commons.lang3.ObjectUtils; +import org.apache.submarine.server.database.workbench.entity.SysUserEntity; +import org.apache.submarine.server.database.workbench.service.SysUserService; +import org.pac4j.core.context.WebContext; +import org.pac4j.core.exception.http.HttpAction; +import org.pac4j.core.profile.ProfileManager; +import org.pac4j.core.profile.UserProfile; +import org.pac4j.jee.context.session.JEESessionStore; +import org.pac4j.jee.http.adapter.JEEHttpActionAdapter; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.Date; +import java.util.Optional; + +import static org.apache.submarine.server.database.workbench.service.SysUserService.DEFAULT_ADMIN_UID; +import static org.apache.submarine.server.database.workbench.service.SysUserService.DEFAULT_CREATE_USER_PASSWORD; + +/** + * Triggers automatic creation of non-existent users + * when authenticating third party logins to the adapter at the same time + */ +public class RegistryUserActionAdapter extends JEEHttpActionAdapter { + + private static final SysUserService userService = SysUserService.INSTANCE; + + private final Logger LOG = LoggerFactory.getLogger(RegistryUserActionAdapter.class); + + @Override + public Object adapt(HttpAction action, WebContext context) { + super.adapt(action, context); + // get profile + //final SessionStore store = FindBest.sessionStore(null, Config.INSTANCE, JEESessionStore.INSTANCE); + ProfileManager manager = new ProfileManager(context, JEESessionStore.INSTANCE); + Optional profile = manager.getProfile(); + // every time call back, check if this user is exists + profile.ifPresent(this::createUndefinedUser); + return null; + } + + /** + * Create a user that does not exist + */ + public void createUndefinedUser(UserProfile profile) { + LOG.trace("Check user if exists ..."); + try { + // If the user does not exist then create + userService.getOrCreateUser(profile.getUsername(), () -> { + SysUserEntity entity = new SysUserEntity(); + entity.setUserName(profile.getUsername()); + entity.setRealName(profile.getUsername()); + entity.setPassword(DEFAULT_CREATE_USER_PASSWORD); + entity.setEmail(ObjectUtils.identityToString(profile.getAttribute("email"))); + entity.setPhone(ObjectUtils.identityToString(profile.getAttribute("phone"))); + entity.setAvatar(ObjectUtils.identityToString(profile.getAttribute("picture"))); + entity.setDeleted(0); + entity.setCreateBy(DEFAULT_ADMIN_UID); + entity.setCreateTime(new Date()); + return entity; + }); + } catch (Exception e) { + LOG.error("Get error when creating user, skip ...", e); + } + } +} diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcCallbackResource.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcCallbackResource.java new file mode 100644 index 0000000000..1e6162f6c1 --- /dev/null +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcCallbackResource.java @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.submarine.server.security.oidc; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.ws.rs.GET; +import javax.ws.rs.Path; +import javax.ws.rs.core.Response; + +/** + * Fixed Callback endpoint used after successful login with Identity Provider e.g. OAuth server. + * See https://www.pac4j.org/blog/understanding-the-callback-endpoint.html + */ +@Path(OidcCallbackResource.SELF_URL) +public class OidcCallbackResource { + + public static final String SELF_URL = "/auth/oidc/callback"; + + private static final Logger LOG = LoggerFactory.getLogger(OidcCallbackResource.class); + + @GET + public Response callback() { + LOG.error( + "This endpoint is to be handled by the pac4j filter to redirect users," + + " request should never reach here.", + new RuntimeException() + ); + return Response.serverError().build(); + } +} diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcConfig.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcConfig.java new file mode 100644 index 0000000000..1a8fb9d244 --- /dev/null +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcConfig.java @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.submarine.server.security.oidc; + +import org.apache.submarine.commons.utils.SubmarineConfiguration; +import org.apache.submarine.server.security.common.CommonConfig; + +public class OidcConfig extends CommonConfig { + + public static final String CLIENT_ID_ENV = "SUBMARINE_AUTH_OIDC_CLIENT_ID"; + public static final String CLIENT_ID_PROP = "submarine.auth.oidc.client.id"; + + public static final String CLIENT_SECRET_ENV = "SUBMARINE_AUTH_OIDC_CLIENT_SECRET"; + public static final String CLIENT_SECRET_PROP = "submarine.auth.oidc.client.secret"; + + public static final String DISCOVER_URI_ENV = "SUBMARINE_AUTH_OIDC_DISCOVER_URI"; + public static final String DISCOVER_URI_PROP = "submarine.auth.oidc.discover.uri"; + + public static final String LOGOUT_REDIRECT_URI_ENV = "SUBMARINE_AUTH_OIDC_LOGOUT_URI"; + public static final String LOGOUT_REDIRECT_URI_PROP = "submarine.auth.oidc.logout.uri"; + + public static final String CLIENT_ID; + + public static final String CLIENT_SECRET; + + public static final String DISCOVER_URI; + + public static final String LOGOUT_REDIRECT_URI; + + static { + SubmarineConfiguration conf = SubmarineConfiguration.getInstance(); + CLIENT_ID = conf.getString(CLIENT_ID_ENV, CLIENT_ID_PROP, ""); + CLIENT_SECRET = conf.getString(CLIENT_SECRET_ENV, CLIENT_SECRET_PROP, ""); + DISCOVER_URI = conf.getString(DISCOVER_URI_ENV, DISCOVER_URI_PROP, ""); + LOGOUT_REDIRECT_URI = conf.getString(LOGOUT_REDIRECT_URI_ENV, LOGOUT_REDIRECT_URI_PROP, ""); + } +} diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcFilter.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcFilter.java new file mode 100644 index 0000000000..58f392debc --- /dev/null +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcFilter.java @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.submarine.server.security.oidc; + +import org.apache.submarine.server.security.SecurityFactory; +import org.apache.submarine.server.security.common.CommonFilter; +import org.pac4j.oidc.profile.OidcProfile; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.servlet.Filter; +import javax.servlet.FilterChain; +import javax.servlet.ServletException; +import javax.servlet.ServletRequest; +import javax.servlet.ServletResponse; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; +import java.io.IOException; +import java.util.Optional; + +public class OidcFilter extends CommonFilter implements Filter { + + private static final Logger LOG = LoggerFactory.getLogger(OidcFilter.class); + + private final OidcSecurityProvider provider; + + public OidcFilter() { + this.provider = SecurityFactory.getPac4jSecurityProvider(); + } + + @Override + public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) + throws IOException, ServletException { + HttpServletRequest httpServletRequest = (HttpServletRequest) request; + HttpServletResponse httpServletResponse = (HttpServletResponse) response; + switch (httpServletRequest.getRequestURI()) { + case OidcCallbackResource.SELF_URL: + provider.callback(httpServletRequest, httpServletResponse); + break; + case OidcConfig.LOGOUT_ENDPOINT: + provider.logout(httpServletRequest, httpServletResponse); + break; + default: + Optional profile = provider.perform(httpServletRequest, httpServletResponse); + if (profile.isPresent()) { + chain.doFilter(request, response); + } else { + // There are some static resources that are also within the service, + // so we need to filter out the api + if (isProtectedApi(httpServletRequest)) { + httpServletResponse.sendError(HttpServletResponse.SC_UNAUTHORIZED, + "The token/session is not valid."); + } + } + } + } + + @Override + public void destroy() { + + } +} diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcSecurityProvider.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcSecurityProvider.java new file mode 100644 index 0000000000..faa65eab85 --- /dev/null +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/oidc/OidcSecurityProvider.java @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.submarine.server.security.oidc; + +import org.apache.commons.lang3.StringUtils; +import org.apache.submarine.commons.utils.SubmarineConfVars; +import org.apache.submarine.commons.utils.SubmarineConfiguration; +import org.apache.submarine.server.security.SecurityProvider; +import org.apache.submarine.server.security.common.AuthFlowType; +import org.apache.submarine.server.security.common.RegistryUserActionAdapter; +import org.pac4j.core.config.Config; +import org.pac4j.core.context.WebContext; +import org.pac4j.core.context.session.SessionStore; +import org.pac4j.core.http.callback.NoParameterCallbackUrlResolver; +import org.pac4j.core.http.url.DefaultUrlResolver; +import org.pac4j.core.matching.matcher.csrf.CsrfTokenGeneratorMatcher; +import org.pac4j.core.matching.matcher.csrf.DefaultCsrfTokenGenerator; +import org.pac4j.core.profile.UserProfile; +import org.pac4j.http.client.direct.HeaderClient; +import org.pac4j.oidc.client.OidcClient; +import org.pac4j.oidc.config.OidcConfiguration; +import org.pac4j.oidc.credentials.authenticator.UserInfoOidcAuthenticator; +import org.pac4j.oidc.profile.OidcProfile; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; +import java.util.Collection; +import java.util.Map; +import java.util.Optional; + +import static org.pac4j.core.matching.matcher.DefaultMatchers.CSRF_TOKEN; + +public class OidcSecurityProvider extends SecurityProvider { + + private static final Logger LOG = LoggerFactory.getLogger(OidcSecurityProvider.class); + + private final RegistryUserActionAdapter userActionAdapter = new RegistryUserActionAdapter(); + + @Override + public AuthFlowType getAuthFlowType() { + return AuthFlowType.SESSION; + } + + + @Override + public Class getFilterClass() { + return OidcFilter.class; + } + + @Override + public Config createConfig() { + // oidc config + OidcConfiguration oidcConf = new OidcConfiguration(); + oidcConf.setClientId(OidcConfig.CLIENT_ID); + oidcConf.setSecret(OidcConfig.CLIENT_SECRET); + oidcConf.setDiscoveryURI(OidcConfig.DISCOVER_URI); + oidcConf.setExpireSessionWithToken(true); + oidcConf.setUseNonce(true); + oidcConf.setReadTimeout(5000); + oidcConf.setMaxAge(OidcConfig.MAX_AGE); + // oidc client + OidcClient oidcClient = new OidcClient(oidcConf); + oidcClient.setUrlResolver(new DefaultUrlResolver(true)); + oidcClient.setCallbackUrlResolver(new NoParameterCallbackUrlResolver()); + // header client + HeaderClient headerClient = new HeaderClient(OidcConfig.AUTH_HEADER, + OidcConfig.BEARER_HEADER_PREFIX, new UserInfoOidcAuthenticator(oidcConf)); + Config config = new Config(OidcCallbackResource.SELF_URL, oidcClient, headerClient); + // add csrfToken matcher + SubmarineConfiguration conf = SubmarineConfiguration.getInstance(); + CsrfTokenGeneratorMatcher csrftgm = new CsrfTokenGeneratorMatcher(new DefaultCsrfTokenGenerator()); + csrftgm.setSecure(conf.getBoolean(SubmarineConfVars.ConfVars.SUBMARINE_COOKIE_SECURE)); + csrftgm.setHttpOnly(conf.getBoolean(SubmarineConfVars.ConfVars.SUBMARINE_COOKIE_HTTP_ONLY)); + config.setMatchers(Map.of(CSRF_TOKEN, csrftgm)); + return config; + } + + @Override + public String getClient(HttpServletRequest httpServletRequest) { + return "OidcClient,HeaderClient"; + } + + @Override + public Optional perform(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { + // perform get profile + UserProfile profile = (UserProfile) createSecurityLogic().perform( + createWebContext(hsRequest, hsResponse), + createSessionStore(hsRequest, hsResponse), + getConfig(), + (WebContext ctx, SessionStore store, Collection profiles, Object... parameters) -> { + if (profiles.isEmpty()) { + LOG.warn("No profiles found after OIDC auth."); + return null; + } else { + return profiles.iterator().next(); + } + }, + createHttpActionAdapter(), + getClient(hsRequest), DEFAULT_AUTHORIZER, "" + ); + return Optional.ofNullable((OidcProfile) profile); + } + + @Override + public void callback(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { + // perform callback + createCallbackLogic().perform( + createWebContext(hsRequest, hsResponse), + createSessionStore(hsRequest, hsResponse), + getConfig(), userActionAdapter, "/", false, "oidcClient" + ); + } + + @Override + public void logout(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { + String redirectUrl = OidcConfig.LOGOUT_REDIRECT_URI; + if (StringUtils.isBlank(redirectUrl)) { + redirectUrl = hsRequest.getParameter("redirect_url"); + } + // perform logout + createLogoutLogic().perform( + createWebContext(hsRequest, hsResponse), + createSessionStore(hsRequest, hsResponse), + getConfig(), + createHttpActionAdapter(), + redirectUrl, "/", true, true, true + ); + } + +} diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/simple/SimpleFilter.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/simple/SimpleFilter.java index 4bb919890c..1a3d425082 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/simple/SimpleFilter.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/simple/SimpleFilter.java @@ -27,7 +27,6 @@ import javax.servlet.Filter; import javax.servlet.FilterChain; -import javax.servlet.FilterConfig; import javax.servlet.ServletException; import javax.servlet.ServletRequest; import javax.servlet.ServletResponse; @@ -50,22 +49,22 @@ public SimpleFilter() { this.provider = SecurityFactory.getSimpleSecurityProvider(); } - @Override - public void init(FilterConfig filterConfig) throws ServletException { - } - @Override public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException { HttpServletRequest httpServletRequest = (HttpServletRequest) servletRequest; HttpServletResponse httpServletResponse = (HttpServletResponse) servletResponse; - // check header token - Optional profile = provider.perform(httpServletRequest, httpServletResponse); - // If the token can be correctly parsed then continue processing, otherwise return 401 - if (profile.isPresent()) { - filterChain.doFilter(servletRequest, servletResponse); + if (isProtectedApi(httpServletRequest)) { + // check header token + Optional profile = provider.perform(httpServletRequest, httpServletResponse); + // If the token can be correctly parsed then continue processing, otherwise return 401 + if (profile.isPresent()) { + filterChain.doFilter(servletRequest, servletResponse); + } else { + httpServletResponse.sendError(HttpServletResponse.SC_UNAUTHORIZED, "The token is not valid."); + } } else { - httpServletResponse.sendError(HttpServletResponse.SC_UNAUTHORIZED, "The token is not valid."); + filterChain.doFilter(servletRequest, servletResponse); } } diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/simple/SimpleSecurityProvider.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/simple/SimpleSecurityProvider.java index 3a3516654f..0a10846d44 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/simple/SimpleSecurityProvider.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/security/simple/SimpleSecurityProvider.java @@ -21,11 +21,9 @@ import org.apache.submarine.server.security.SecurityProvider; import org.apache.submarine.server.security.common.CommonConfig; -import org.apache.submarine.server.security.common.CommonFilter; import org.pac4j.core.config.Config; -import org.pac4j.core.context.JEEContext; -import org.pac4j.core.matching.matcher.PathMatcher; -import org.pac4j.core.profile.ProfileManager; +import org.pac4j.core.context.WebContext; +import org.pac4j.core.context.session.SessionStore; import org.pac4j.core.profile.UserProfile; import org.pac4j.http.client.direct.HeaderClient; import org.pac4j.jwt.profile.JwtProfile; @@ -37,36 +35,24 @@ import java.util.Collection; import java.util.Optional; -public class SimpleSecurityProvider implements SecurityProvider { +public class SimpleSecurityProvider extends SecurityProvider { private static final Logger LOG = LoggerFactory.getLogger(SimpleSecurityProvider.class); - private Config pac4jConfig; - @Override public Class getFilterClass() { return SimpleFilter.class; } @Override - public Config getConfig() { + public Config createConfig() { if (pac4jConfig != null) { return pac4jConfig; } - // header client HeaderClient headerClient = new HeaderClient(CommonConfig.AUTH_HEADER, CommonConfig.BEARER_HEADER_PREFIX, SimpleLoginConfig.getJwtAuthenticator()); - - Config pac4jConfig = new Config(headerClient); - // skip web static resources - pac4jConfig.addMatcher("static", new PathMatcher().excludeRegex( - "^/.*(\\.map|\\.js|\\.css|\\.ico|\\.svg|\\.png|\\.html|\\.htm)$")); - // skip login rest api - pac4jConfig.addMatcher("api", new PathMatcher().excludeRegex("^/api/auth/login$")); - this.pac4jConfig = pac4jConfig; - - return pac4jConfig; + return new Config(headerClient); } @Override @@ -76,11 +62,11 @@ public String getClient(HttpServletRequest httpServletRequest) { @Override public Optional perform(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { - JEEContext context = new JEEContext(hsRequest, hsResponse, CommonFilter.SESSION_STORE); - UserProfile profile = CommonFilter.SECURITY_LOGIC.perform( - context, - pac4jConfig, - (JEEContext ctx, Collection profiles, Object... parameters) -> { + UserProfile profile = (UserProfile) createSecurityLogic().perform( + createWebContext(hsRequest, hsResponse), + createSessionStore(hsRequest, hsResponse), + getConfig(), + (WebContext ctx, SessionStore store, Collection profiles, Object... parameters) -> { if (profiles.isEmpty()) { LOG.warn("No profiles found with default auth."); return null; @@ -88,15 +74,9 @@ public Optional perform(HttpServletRequest hsRequest, HttpServletRes return profiles.iterator().next(); } }, - CommonFilter.DEFAULT_HTTP_ACTION_ADAPTER, - getClient(hsRequest), DEFAULT_AUTHORIZER, "static,api", null); + createHttpActionAdapter(), + getClient(hsRequest), DEFAULT_AUTHORIZER, null + ); return Optional.ofNullable((JwtProfile) profile); } - - @Override - public Optional getProfile(HttpServletRequest hsRequest, HttpServletResponse hsResponse) { - JEEContext context = new JEEContext(hsRequest, hsResponse, CommonFilter.SESSION_STORE); - ProfileManager manager = new ProfileManager<>(context); - return manager.get(true); - } } diff --git a/submarine-server/server-core/src/main/java/org/apache/submarine/server/utils/response/DictAnnotation.java b/submarine-server/server-core/src/main/java/org/apache/submarine/server/utils/response/DictAnnotation.java index 110cc0df05..3d73eb83a5 100644 --- a/submarine-server/server-core/src/main/java/org/apache/submarine/server/utils/response/DictAnnotation.java +++ b/submarine-server/server-core/src/main/java/org/apache/submarine/server/utils/response/DictAnnotation.java @@ -173,7 +173,9 @@ public static boolean parseDictAnnotation(Object result) throws Exception { return true; } else { - LOG.warn("Unsupported parse {} Dict Annotation!", result.getClass()); + // When it contains lists, mostly arraylists and the like, + // we should do as a trace/debug level as it does not affect the data returned + LOG.trace("Unsupported parse {} Dict Annotation!", result.getClass()); } return false; diff --git a/submarine-server/server-core/src/main/resources/log4j.properties b/submarine-server/server-core/src/main/resources/log4j.properties index 55e02b6d20..5e71979ac4 100644 --- a/submarine-server/server-core/src/main/resources/log4j.properties +++ b/submarine-server/server-core/src/main/resources/log4j.properties @@ -15,3 +15,6 @@ log4j.appender.stdout = org.apache.log4j.ConsoleAppender log4j.appender.stdout.Target = System.out log4j.appender.stdout.layout = org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern = [%-5p] %d{yyyy-MM-dd HH:mm:ss,SSS} method:%l%n%m%n + +# mybatis sql debug +log4j.logger.org.apache.submarine.server.database=DEBUG diff --git a/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/MockHttpServletRequest.java b/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/MockHttpServletRequest.java index 49d8e04054..ad8a198c52 100644 --- a/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/MockHttpServletRequest.java +++ b/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/MockHttpServletRequest.java @@ -134,7 +134,7 @@ public String getRequestedSessionId() { @Override public String getRequestURI() { - return null; + return "/api/sys/user/info"; } private StringBuffer requestUrl; diff --git a/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/oidc/MockOidcHttpServletRequest.java b/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/oidc/MockOidcHttpServletRequest.java new file mode 100644 index 0000000000..887ac54907 --- /dev/null +++ b/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/oidc/MockOidcHttpServletRequest.java @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.submarine.server.security.oidc; + +import org.apache.submarine.server.security.MockHttpServletRequest; +import org.mockito.Mockito; + +import javax.servlet.http.HttpSession; + +import static org.mockito.Mockito.when; + +public class MockOidcHttpServletRequest extends MockHttpServletRequest { + + @Override + public HttpSession getSession(boolean create) { + HttpSession session = Mockito.mock(HttpSession.class); + when(session.getId()).thenReturn("id"); + return session; + } +} diff --git a/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/oidc/SubmarineAuthOidcTest.java b/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/oidc/SubmarineAuthOidcTest.java new file mode 100644 index 0000000000..7a83a36bbb --- /dev/null +++ b/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/oidc/SubmarineAuthOidcTest.java @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.submarine.server.security.oidc; + +import com.github.tomakehurst.wiremock.client.WireMock; +import com.github.tomakehurst.wiremock.junit.WireMockRule; +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.google.gson.reflect.TypeToken; +import org.apache.submarine.commons.utils.SubmarineConfVars; +import org.apache.submarine.commons.utils.SubmarineConfiguration; + +import org.apache.submarine.server.api.environment.EnvironmentId; +import org.apache.submarine.server.api.workbench.UserInfo; +import org.apache.submarine.server.rest.workbench.SysUserRestApi; +import org.apache.submarine.server.security.SecurityFactory; +import org.apache.submarine.server.security.SecurityProvider; +import org.apache.submarine.server.security.common.RegistryUserActionAdapter; +import org.apache.submarine.server.utils.gson.EnvironmentIdDeserializer; +import org.apache.submarine.server.utils.gson.EnvironmentIdSerializer; +import org.apache.submarine.server.utils.response.JsonResponse; +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.mockito.Mockito; +import org.pac4j.core.config.Config; +import org.pac4j.core.profile.UserProfile; +import org.pac4j.core.util.Pac4jConstants; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.servlet.FilterChain; +import javax.servlet.ServletException; +import javax.servlet.http.HttpServletResponse; +import javax.ws.rs.core.Response; +import java.io.File; +import java.io.IOException; +import java.io.PrintWriter; +import java.io.StringWriter; +import java.lang.reflect.Type; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; + +import static com.github.tomakehurst.wiremock.client.WireMock.aResponse; +import static com.github.tomakehurst.wiremock.client.WireMock.urlEqualTo; +import static org.apache.submarine.server.security.oidc.OidcConfig.CLIENT_ID_PROP; +import static org.apache.submarine.server.security.oidc.OidcConfig.CLIENT_SECRET_PROP; +import static org.apache.submarine.server.security.oidc.OidcConfig.DISCOVER_URI_PROP; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +public class SubmarineAuthOidcTest { + + private static final Logger LOG = LoggerFactory.getLogger(SubmarineAuthOidcTest.class); + + private static final SubmarineConfiguration conf = SubmarineConfiguration.getInstance(); + + private SysUserRestApi sysUserRestApi; + + private RegistryUserActionAdapter userActionAdapter; + + private static final GsonBuilder gsonBuilder = new GsonBuilder() + .registerTypeAdapter(EnvironmentId.class, new EnvironmentIdSerializer()) + .registerTypeAdapter(EnvironmentId.class, new EnvironmentIdDeserializer()); + private static final Gson gson = gsonBuilder.setDateFormat("yyyy-MM-dd HH:mm:ss").create(); + + @Rule + public final WireMockRule wireMockRule = new WireMockRule(8080); + + @Before + public void before() { + conf.updateConfiguration("submarine.auth.type", "oidc"); + conf.updateConfiguration(CLIENT_ID_PROP, "test"); + conf.updateConfiguration(CLIENT_SECRET_PROP, "secret"); + conf.updateConfiguration(DISCOVER_URI_PROP, + "http://localhost:8080/auth/realms/test-login/.well-known/openid-configuration"); + conf.setJdbcUrl("jdbc:mysql://127.0.0.1:3306/submarine_test?" + + "useUnicode=true&" + + "characterEncoding=UTF-8&" + + "autoReconnect=true&" + + "failOverReadOnly=false&" + + "zeroDateTimeBehavior=convertToNull&" + + "useSSL=false"); + conf.setJdbcUserName("submarine_test"); + conf.setJdbcPassword("password_test"); + + sysUserRestApi = new SysUserRestApi(); + userActionAdapter = new RegistryUserActionAdapter(); + + // Add oidc mock endpoint + // Based on the token, we currently use the following two endpoints: + // 1. openid-configuration + String openidConfig = getResourceFileContent("security/openid-configuration.json"); + wireMockRule.stubFor( + WireMock.get(urlEqualTo("/auth/realms/test-login/.well-known/openid-configuration")) + .willReturn(aResponse().withHeader("Content-Type", "application/json") + .withBody(openidConfig) + ) + ); + // 2. userinfo + String userInfo = getResourceFileContent("security/user-info.json"); + wireMockRule.stubFor( + WireMock.get(urlEqualTo("/auth/realms/test-login/protocol/openid-connect/userinfo")) + .willReturn(aResponse().withHeader("Content-Type", "application/json") + .withBody(userInfo) + ) + ); + } + + public static String getResourceFileContent(String resource) { + File file = new File(Objects.requireNonNull( + SubmarineAuthOidcTest.class.getClassLoader().getResource(resource)).getPath() + ); + try { + return new String(Files.readAllBytes(Paths.get(file.toString()))); + } catch (IOException e) { + LOG.error("Can not find file: " + resource, e); + return null; + } + } + + @Test + public void testOidcType() throws ServletException, IOException { + // test auth type config + String authType = conf.getString(SubmarineConfVars.ConfVars.SUBMARINE_AUTH_TYPE); + assertEquals(authType, "oidc"); + + // test provider + Optional providerOptional = SecurityFactory.getSecurityProvider(); + SecurityProvider provider = providerOptional.get(); + assertNotNull(provider); + assertEquals(provider.getFilterClass(), OidcFilter.class); + Config config = provider.getConfig(); + assertTrue(config.getClients().findClient("headerClient").isPresent()); + assertTrue(config.getClients().findClient("oidcClient").isPresent()); + + // create filter involved objects + // 1. test filter + OidcFilter filterTest = new OidcFilter(); + filterTest.init(null); + // 2. filter chain + FilterChain mockFilterChain = Mockito.mock(FilterChain.class); + // 3. http request + MockOidcHttpServletRequest mockRequest = new MockOidcHttpServletRequest(); + mockRequest.setRequestURL(new StringBuffer("/api/sys/user/info")); + // 4. http response + HttpServletResponse mockResponse = Mockito.mock(HttpServletResponse.class); + StringWriter out = new StringWriter(); + PrintWriter printOut = new PrintWriter(out); + when(mockResponse.getWriter()).thenReturn(printOut); + + // test no header + filterTest.doFilter(mockRequest, mockResponse, mockFilterChain); + verify(mockResponse).sendError(HttpServletResponse.SC_UNAUTHORIZED, + "The token/session is not valid."); + + // test header, here we use a fake Token to simulate login + mockRequest.setHeader("Authorization", "Bearer XXX"); + filterTest.doFilter(mockRequest, mockResponse, mockFilterChain); + verify(mockFilterChain).doFilter(mockRequest, mockResponse); + assertNotNull(mockRequest.getAttribute(Pac4jConstants.USER_PROFILES)); + + // Since we are not callback user, we can simulate creating oidc user + Map profiles = (Map) + mockRequest.getAttribute(Pac4jConstants.USER_PROFILES); + UserProfile profile = profiles.get("HeaderClient"); + userActionAdapter.createUndefinedUser(profile); + + // test get user info + Response response = sysUserRestApi.info(mockRequest, mockResponse); + assertEquals(response.getStatus(), Response.Status.OK.getStatusCode()); + String entity = (String) response.getEntity(); + Type type = new TypeToken>() { }.getType(); + JsonResponse jsonResponse = gson.fromJson(entity, type); + assertEquals(jsonResponse.getResult().getName(), "oidc_test"); + } + + @After + public void after() { + conf.updateConfiguration("submarine.auth.type", "none"); + } +} diff --git a/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/SubmarineAuthSimpleTest.java b/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/simple/SubmarineAuthSimpleTest.java similarity index 95% rename from submarine-server/server-core/src/test/java/org/apache/submarine/server/security/SubmarineAuthSimpleTest.java rename to submarine-server/server-core/src/test/java/org/apache/submarine/server/security/simple/SubmarineAuthSimpleTest.java index 284362f4d0..05b8d10304 100644 --- a/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/SubmarineAuthSimpleTest.java +++ b/submarine-server/server-core/src/test/java/org/apache/submarine/server/security/simple/SubmarineAuthSimpleTest.java @@ -17,7 +17,7 @@ * under the License. */ -package org.apache.submarine.server.security; +package org.apache.submarine.server.security.simple; import com.google.gson.Gson; import com.google.gson.GsonBuilder; @@ -28,7 +28,9 @@ import org.apache.submarine.server.database.workbench.entity.SysUserEntity; import org.apache.submarine.server.rest.workbench.LoginRestApi; import org.apache.submarine.server.rest.workbench.SysUserRestApi; -import org.apache.submarine.server.security.simple.SimpleFilter; +import org.apache.submarine.server.security.MockHttpServletRequest; +import org.apache.submarine.server.security.SecurityFactory; +import org.apache.submarine.server.security.SecurityProvider; import org.apache.submarine.server.utils.gson.EnvironmentIdDeserializer; import org.apache.submarine.server.utils.gson.EnvironmentIdSerializer; import org.apache.submarine.server.utils.response.JsonResponse; @@ -126,7 +128,7 @@ public void testSimpleType() throws ServletException, IOException { FilterChain mockFilterChain = Mockito.mock(FilterChain.class); // 3. http request MockHttpServletRequest mockRequest = new MockHttpServletRequest(); - mockRequest.setRequestURL(new StringBuffer("/test/url")); + mockRequest.setRequestURL(new StringBuffer("/api/sys/user/info")); // 4. http response HttpServletResponse mockResponse = Mockito.mock(HttpServletResponse.class); StringWriter out = new StringWriter(); diff --git a/submarine-server/server-core/src/test/resources/security/openid-configuration.json b/submarine-server/server-core/src/test/resources/security/openid-configuration.json new file mode 100644 index 0000000000..021eca04a3 --- /dev/null +++ b/submarine-server/server-core/src/test/resources/security/openid-configuration.json @@ -0,0 +1,145 @@ +{ + "issuer": "http://localhost:8080/auth/realms/test-login", + "authorization_endpoint": "http://localhost:8080/auth/realms/test-login/protocol/openid-connect/auth", + "token_endpoint": "http://localhost:8080/auth/realms/test-login/protocol/openid-connect/token", + "token_introspection_endpoint": "http://localhost:8080/auth/realms/test-login/protocol/openid-connect/token/introspect", + "userinfo_endpoint": "http://localhost:8080/auth/realms/test-login/protocol/openid-connect/userinfo", + "end_session_endpoint": "http://localhost:8080/auth/realms/test-login/protocol/openid-connect/logout", + "jwks_uri": "http://localhost:8080/auth/realms/test-login/protocol/openid-connect/certs", + "check_session_iframe": "http://localhost:8080/auth/realms/test-login/protocol/openid-connect/login-status-iframe.html", + "grant_types_supported": [ + "authorization_code", + "implicit", + "refresh_token", + "password", + "client_credentials" + ], + "response_types_supported": [ + "code", + "none", + "id_token", + "token", + "id_token token", + "code id_token", + "code token", + "code id_token token" + ], + "subject_types_supported": [ + "public", + "pairwise" + ], + "id_token_signing_alg_values_supported": [ + "PS384", + "ES384", + "RS384", + "HS256", + "HS512", + "ES256", + "RS256", + "HS384", + "ES512", + "PS256", + "PS512", + "RS512" + ], + "id_token_encryption_alg_values_supported": [ + "RSA-OAEP", + "RSA1_5" + ], + "id_token_encryption_enc_values_supported": [ + "A128GCM", + "A128CBC-HS256" + ], + "userinfo_signing_alg_values_supported": [ + "PS384", + "ES384", + "RS384", + "HS256", + "HS512", + "ES256", + "RS256", + "HS384", + "ES512", + "PS256", + "PS512", + "RS512", + "none" + ], + "request_object_signing_alg_values_supported": [ + "PS384", + "ES384", + "RS384", + "HS256", + "HS512", + "ES256", + "RS256", + "HS384", + "ES512", + "PS256", + "PS512", + "RS512", + "none" + ], + "response_modes_supported": [ + "query", + "fragment", + "form_post" + ], + "registration_endpoint": "http://localhost:8080/auth/realms/test-login/clients-registrations/openid-connect", + "token_endpoint_auth_methods_supported": [ + "private_key_jwt", + "client_secret_basic", + "client_secret_post", + "tls_client_auth", + "client_secret_jwt" + ], + "token_endpoint_auth_signing_alg_values_supported": [ + "PS384", + "ES384", + "RS384", + "HS256", + "HS512", + "ES256", + "RS256", + "HS384", + "ES512", + "PS256", + "PS512", + "RS512" + ], + "claims_supported": [ + "aud", + "sub", + "iss", + "auth_time", + "name", + "given_name", + "family_name", + "preferred_username", + "email", + "acr" + ], + "claim_types_supported": [ + "normal" + ], + "claims_parameter_supported": false, + "scopes_supported": [ + "openid", + "offline_access", + "microprofile-jwt", + "profile", + "email", + "address", + "phone", + "roles", + "web-origins" + ], + "request_parameter_supported": true, + "request_uri_parameter_supported": true, + "code_challenge_methods_supported": [ + "plain", + "S256" + ], + "tls_client_certificate_bound_access_tokens": true, + "introspection_endpoint": "http://localhost:8080/auth/realms/test-login/protocol/openid-connect/token/introspect" +} diff --git a/submarine-server/server-core/src/test/resources/security/user-info.json b/submarine-server/server-core/src/test/resources/security/user-info.json new file mode 100644 index 0000000000..3bbb8d324d --- /dev/null +++ b/submarine-server/server-core/src/test/resources/security/user-info.json @@ -0,0 +1,12 @@ +{ + "uid": "oidc_test", + "sub": "677ecaf0-2d94-4fa5-8cc1-f3caecac09f9", + "email_verified": false, + "displayName": "oidc_test", + "name": "oidc_test", + "preferred_username": "oidc_test", + "given_name": "oidc_test", + "family_name": "0", + "userId": "ffa47159-a80c-450c-999a-dd9420d393da", + "policy": "public_read_write_policy" +} diff --git a/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/utils/MyBatisUtil.java b/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/utils/MyBatisUtil.java index 69ab6c1e7e..35907810c2 100755 --- a/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/utils/MyBatisUtil.java +++ b/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/utils/MyBatisUtil.java @@ -26,6 +26,7 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import javax.sql.DataSource; import java.io.IOException; import java.io.Reader; import java.util.Properties; @@ -46,8 +47,9 @@ public class MyBatisUtil { String jdbcUrl = conf.getJdbcUrl(); String jdbcUserName = conf.getJdbcUserName(); String jdbcPassword = conf.getJdbcPassword(); - LOG.info("MyBatisUtil -> jdbcClassName: {}, jdbcUrl: {}, jdbcUserName: {}, jdbcPassword: {}", - jdbcClassName, jdbcUrl, jdbcUserName, jdbcPassword); + // We need to protect the password in logging + LOG.info("MyBatisUtil -> jdbcClassName: {}, jdbcUrl: {}, jdbcUserName: {}, jdbcPassword: ****", + jdbcClassName, jdbcUrl, jdbcUserName); Properties props = new Properties(); props.setProperty("jdbc.driverClassName", jdbcClassName); @@ -71,6 +73,13 @@ public static SqlSession getSqlSession() { return sqlSessionFactory.openSession(); } + /** + * Get datasource {@link org.apache.ibatis.datasource.pooled.PooledDataSource} + */ + public static DataSource getDatasource() { + return sqlSessionFactory.getConfiguration().getEnvironment().getDataSource(); + } + private static void checkCalledByTestMethod() { StackTraceElement[] stackTraceElements = Thread.currentThread().getStackTrace(); for (StackTraceElement element : stackTraceElements) { diff --git a/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/workbench/entity/SysUserEntity.java b/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/workbench/entity/SysUserEntity.java index 203ea7130c..cb14a6fbd6 100644 --- a/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/workbench/entity/SysUserEntity.java +++ b/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/workbench/entity/SysUserEntity.java @@ -161,4 +161,28 @@ public void setBirthday(Date birthday) { this.birthday = birthday; } + @Override + public String toString() { + return "SysUserEntity{" + + "userName='" + userName + '\'' + + ", realName='" + realName + '\'' + + ", password='" + password + '\'' + + ", avatar='" + avatar + '\'' + + ", sex='" + sex + '\'' + + ", status='" + status + '\'' + + ", phone='" + phone + '\'' + + ", email='" + email + '\'' + + ", deptCode='" + deptCode + '\'' + + ", deptName='" + deptName + '\'' + + ", roleCode='" + roleCode + '\'' + + ", birthday=" + birthday + + ", deleted=" + deleted + + ", token='" + token + '\'' + + ", id='" + id + '\'' + + ", createBy='" + createBy + '\'' + + ", createTime=" + createTime + + ", updateBy='" + updateBy + '\'' + + ", updateTime=" + updateTime + + '}'; + } } diff --git a/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/workbench/service/SysUserService.java b/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/workbench/service/SysUserService.java index 91d2c12ef4..47b2db00d1 100755 --- a/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/workbench/service/SysUserService.java +++ b/submarine-server/server-database/src/main/java/org/apache/submarine/server/database/workbench/service/SysUserService.java @@ -29,13 +29,22 @@ import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.function.Supplier; public class SysUserService { + + public static final SysUserService INSTANCE = new SysUserService(); + private static final Logger LOG = LoggerFactory.getLogger(SysUserService.class); - private static String GET_USER_BY_NAME_STATEMENT + private static final String GET_USER_BY_NAME_STATEMENT = "org.apache.submarine.server.database.workbench.mappers.SysUserMapper.getUserByName"; + // default user is admin + public static final String DEFAULT_ADMIN_UID = "e9ca23d68d884d4ebb19d07889727dae"; + // default password is `password` by angular markAsDirty method + public static final String DEFAULT_CREATE_USER_PASSWORD = "5f4dcc3b5aa765d61d8327deb882cf99"; + public SysUserEntity getUserByName(String name, String password) throws Exception { SysUserEntity sysUser = null; try (SqlSession sqlSession = MyBatisUtil.getSqlSession()) { @@ -54,6 +63,47 @@ public SysUserEntity getUserByName(String name, String password) throws Exceptio return sysUser; } + /** + * Get user by unique name + */ + public SysUserEntity getUserByName(String name) { + try (SqlSession sqlSession = MyBatisUtil.getSqlSession()) { + SysUserMapper sysUserMapper = sqlSession.getMapper(SysUserMapper.class); + return sysUserMapper.getUserByUniqueName(name); + } catch (Exception e) { + LOG.error(e.getMessage(), e); + throw e; + } + } + + /** + * Get or create undefined user: + * 1. If present, determine if reactivation is required + * 1. If not present, create user + */ + public SysUserEntity getOrCreateUser(String username, Supplier entitySupplier) { + LOG.trace("Check user if exists ..."); + try (SqlSession sqlSession = MyBatisUtil.getSqlSession()) { + SysUserMapper sysUserMapper = sqlSession.getMapper(SysUserMapper.class); + SysUserEntity sysUser = sysUserMapper.getUserByUniqueName(username); + if (sysUser == null) { + // if user is undefined, create this user + sysUser = entitySupplier.get(); + LOG.info("Can not find this user, need to create! User entity: {}", sysUser); + sysUserMapper.add(sysUser); + sqlSession.commit(); + } else if (sysUser.getDeleted() == 1) { + LOG.info("Reset this user {} to active", username); + sysUserMapper.activeUser(sysUser.getId()); + sqlSession.commit(); + } + return sysUser; + } catch (Exception e) { + LOG.error("Get error when creating user, skip ...", e); + return null; + } + } + public SysUserEntity login(HashMap mapParams) throws Exception { SysUserEntity sysUser = null; try (SqlSession sqlSession = MyBatisUtil.getSqlSession()) { diff --git a/submarine-server/server-database/src/main/resources/mybatis-config.xml b/submarine-server/server-database/src/main/resources/mybatis-config.xml index ecdc40ba67..897000393d 100755 --- a/submarine-server/server-database/src/main/resources/mybatis-config.xml +++ b/submarine-server/server-database/src/main/resources/mybatis-config.xml @@ -20,11 +20,12 @@ + - + diff --git a/submarine-server/server-submitter/submitter-k8s/pom.xml b/submarine-server/server-submitter/submitter-k8s/pom.xml index 609126e393..d04b3096c8 100644 --- a/submarine-server/server-submitter/submitter-k8s/pom.xml +++ b/submarine-server/server-submitter/submitter-k8s/pom.xml @@ -87,10 +87,6 @@ mysql mysql-connector-java - - org.apache.derby - derby - diff --git a/submarine-workbench/workbench-web/src/WEB-INF/web.xml b/submarine-workbench/workbench-web/src/WEB-INF/web.xml index 63f9a23c14..d1b256b631 100644 --- a/submarine-workbench/workbench-web/src/WEB-INF/web.xml +++ b/submarine-workbench/workbench-web/src/WEB-INF/web.xml @@ -56,10 +56,4 @@ 2 - - - true - true - - diff --git a/submarine-workbench/workbench-web/src/app/app.component.ts b/submarine-workbench/workbench-web/src/app/app.component.ts index 833cdaee92..0f29e6f98e 100644 --- a/submarine-workbench/workbench-web/src/app/app.component.ts +++ b/submarine-workbench/workbench-web/src/app/app.component.ts @@ -33,7 +33,6 @@ export class AppComponent implements OnInit { ngOnInit(): void { this.router.events.pipe(filter((event) => event instanceof NavigationEnd)).subscribe(() => { const paths = this.router.url.split('/'); - this.title.setTitle(`Submarine - ${paths[paths.length - 1]}`); }); } diff --git a/submarine-workbench/workbench-web/src/app/app.module.ts b/submarine-workbench/workbench-web/src/app/app.module.ts index 77c9937399..209d3727af 100644 --- a/submarine-workbench/workbench-web/src/app/app.module.ts +++ b/submarine-workbench/workbench-web/src/app/app.module.ts @@ -21,10 +21,11 @@ import { NgModule } from '@angular/core'; import { BrowserModule } from '@angular/platform-browser'; import { registerLocaleData } from '@angular/common'; -import { HttpClientModule } from '@angular/common/http'; +import { HttpClientModule, HTTP_INTERCEPTORS } from '@angular/common/http'; import en from '@angular/common/locales/en'; import { FormsModule } from '@angular/forms'; import { BrowserAnimationsModule } from '@angular/platform-browser/animations'; +import { ApiTokenInjector } from "@submarine/core/auth/api-token-injector"; import { LocalStorageService } from '@submarine/services'; import { en_US, NgZorroAntdModule, NZ_I18N } from 'ng-zorro-antd'; import { AppRoutingModule } from './app-routing.module'; @@ -44,7 +45,12 @@ registerLocaleData(en); HttpClientModule, BrowserAnimationsModule ], - providers: [{ provide: NZ_I18N, useValue: en_US }, LocalStorageService], + providers: [ + { provide: NZ_I18N, useValue: en_US }, + // add injector to set header a token when calling rest api + { provide: HTTP_INTERCEPTORS, useClass: ApiTokenInjector, multi: true }, + LocalStorageService + ], bootstrap: [AppComponent] }) export class AppModule {} diff --git a/submarine-workbench/workbench-web/src/app/core/auth/api-token-injector.ts b/submarine-workbench/workbench-web/src/app/core/auth/api-token-injector.ts new file mode 100644 index 0000000000..c214065b5b --- /dev/null +++ b/submarine-workbench/workbench-web/src/app/core/auth/api-token-injector.ts @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import {HttpErrorResponse, HttpEvent, HttpHandler, HttpInterceptor, HttpRequest} from '@angular/common/http'; +import {Injectable} from '@angular/core'; +import {Router} from '@angular/router'; +import {of, throwError, Observable} from 'rxjs'; +import {catchError} from "rxjs/operators"; +import {AuthService} from "../../services"; + +@Injectable() +export class ApiTokenInjector implements HttpInterceptor { + + constructor(private authService: AuthService, private router: Router) { } + + private handleAuthError(err: HttpErrorResponse): Observable { + // handle auth error or rethrow + if (err.status === 401 || err.status === 403) { + if ("session" !== this.authService.getFlowType()) { + // remove token cache + this.authService.removeToken(); + // navigate to login + this.router.navigate(['/user/login']); + } + return of(err.message); + } + return throwError(err); + } + + intercept(request: HttpRequest, next: HttpHandler): Observable> { + // If there is a token in localstorage and not with session, set the token into the header + const checkToken = "session" !== this.authService.getFlowType() && !!this.authService.getToken(); + let handler; + if (checkToken) { + handler = next.handle(request.clone({ + setHeaders: {Authorization: `Bearer ${this.authService.getToken()}`} + })); + } else { + handler = next.handle(request); + } + // handle unauthorized exception (like 401) + return handler.pipe(catchError(x => this.handleAuthError(x))); + } +} diff --git a/submarine-workbench/workbench-web/src/app/core/auth/auth.guard.ts b/submarine-workbench/workbench-web/src/app/core/auth/auth.guard.ts index 63a9a8b009..783a261205 100644 --- a/submarine-workbench/workbench-web/src/app/core/auth/auth.guard.ts +++ b/submarine-workbench/workbench-web/src/app/core/auth/auth.guard.ts @@ -34,7 +34,7 @@ export class AuthGuard implements CanActivate { } checkLogin(url: string): boolean { - if (this.authService.isLoggedIn) { + if (this.authService.isLoggedIn || this.authService.getFlowType() === "session") { return true; } diff --git a/submarine-workbench/workbench-web/src/app/pages/workbench/workbench-routing.module.ts b/submarine-workbench/workbench-web/src/app/pages/workbench/workbench-routing.module.ts index e872c1c6b4..4e24b29630 100644 --- a/submarine-workbench/workbench-web/src/app/pages/workbench/workbench-routing.module.ts +++ b/submarine-workbench/workbench-web/src/app/pages/workbench/workbench-routing.module.ts @@ -97,7 +97,7 @@ const routes: Routes = [ useValue: (route: ActivatedRouteSnapshot, state: RouterStateSnapshot) => { const disablePaths = ['home', 'data', 'workspace', 'interpreter']; let currentPage = state.url.split('/')[2]; - console.log('currentPage', currentPage); + // console.log('currentPage', currentPage); if (disablePaths.includes(currentPage)) return false; else return true; }, diff --git a/submarine-workbench/workbench-web/src/app/services/auth.service.ts b/submarine-workbench/workbench-web/src/app/services/auth.service.ts index e7ad4080c4..76bf583a08 100644 --- a/submarine-workbench/workbench-web/src/app/services/auth.service.ts +++ b/submarine-workbench/workbench-web/src/app/services/auth.service.ts @@ -32,17 +32,37 @@ import { LocalStorageService } from './local-storage.service'; export class AuthService { isLoggedIn = false; authTokenKey = 'auth_token'; - // store the URL so we can redirect after logging in redirectUrl: string; + // auth flow type: token, session + flowType: string = "token"; constructor( private localStorageService: LocalStorageService, private baseApi: BaseApiService, private httpClient: HttpClient ) { - const authToken = this.localStorageService.get(this.authTokenKey); - this.isLoggedIn = !!authToken; + this.flowType = window.GLOBAL_CONFIG.type + // console.log(`auth type = ${this.authType}`) + if (this.flowType === "session") { + this.isLoggedIn = true; + } else { + const authToken = this.localStorageService.get(this.authTokenKey); + this.isLoggedIn = !!authToken; + } + } + + getFlowType(): string { + return this.flowType; + } + + getToken() { + return this.localStorageService.get(this.authTokenKey); + } + + removeToken() { + this.isLoggedIn = false; + this.localStorageService.remove(this.authTokenKey); } login(userForm: { userName: string; password: string }): Observable { @@ -66,15 +86,19 @@ export class AuthService { } logout() { - return this.httpClient.post>(this.baseApi.getRestApi('/auth/logout'), {}).pipe( - map((res) => { - if (res.result) { - this.isLoggedIn = false; - this.localStorageService.remove(this.authTokenKey); - } - - return res.result; - }) - ); + if (this.flowType === "session") { + this.removeToken(); + const url = window.location.origin + window.location.pathname + window.location.href = '/auth/logout?redirect_url=' + url; + } else { + return this.httpClient.post>(this.baseApi.getRestApi('/auth/logout'), {}).pipe( + map((res) => { + if (res.result) { + this.removeToken(); + } + return res.result; + }) + ); + } } } diff --git a/submarine-workbench/workbench-web/src/assets/security/provider.js b/submarine-workbench/workbench-web/src/assets/security/provider.js new file mode 100644 index 0000000000..eadb66e58e --- /dev/null +++ b/submarine-workbench/workbench-web/src/assets/security/provider.js @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +(function () { + window.GLOBAL_CONFIG = { + "type": "simple" + }; +})(); diff --git a/submarine-workbench/workbench-web/src/index.html b/submarine-workbench/workbench-web/src/index.html index a88d57fcd3..f1595b7818 100644 --- a/submarine-workbench/workbench-web/src/index.html +++ b/submarine-workbench/workbench-web/src/index.html @@ -25,6 +25,7 @@ + diff --git a/submarine-workbench/workbench-web/src/types/index.d.ts b/submarine-workbench/workbench-web/src/types/index.d.ts new file mode 100644 index 0000000000..aaf9ca74c5 --- /dev/null +++ b/submarine-workbench/workbench-web/src/types/index.d.ts @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +export {}; + +declare global { + interface Window { + GLOBAL_CONFIG: { + type: string + }; + } +} diff --git a/submarine-workbench/workbench-web/tsconfig.json b/submarine-workbench/workbench-web/tsconfig.json index cc513fce36..670584aeac 100644 --- a/submarine-workbench/workbench-web/tsconfig.json +++ b/submarine-workbench/workbench-web/tsconfig.json @@ -18,7 +18,7 @@ "importHelpers": true, "target": "es5", "typeRoots": [ - "node_modules/@types" + "node_modules/@types", "./src/types" ], "lib": [ "es2018", diff --git a/website/docs/designDocs/wip-designs/security-implementation.md b/website/docs/designDocs/wip-designs/security-implementation.md index b799f2e816..c4e129f09f 100644 --- a/website/docs/designDocs/wip-designs/security-implementation.md +++ b/website/docs/designDocs/wip-designs/security-implementation.md @@ -107,28 +107,28 @@ The field names and values are defined in the OpenID Connect Discovery Specifica ### Configuration -| Attribute | Description | Type | Default | Comment | -| ---- | ---- | ---- | ---- | ---- | -| submarine.auth.type | Supported authentication types, currently available are: none, simple, oauth2/oidc, ldap, kerberos, saml, cas | string | none | Only one authentication method can be supported at any one time | -| submarine.auth.token.maxAge | Expiry time of the token (minite) | int | 1 day | | -| submarine.auth.refreshToken.maxAge | Expiry time of the refresh token (minite) | int | 1 hour | | -| submarine.auth.oauth2.client.id | OAuth2 client id | string | | | -| submarine.auth.oauth2.client.secret | OAuth2 client secret| string | | | -| submarine.auth.oauth2.client.flows | OAuth2 flows, can be: authorizationCode, implicit, password or clientCredentials | string | | | -| submarine.auth.oauth2.scopes | The available scopes for the OAuth2 security scheme. A map between the scope name and a short description for it. | string | | | -| submarine.auth.oauth2.token.uri | OAuth2 access token uri | string | | | -| submarine.auth.oauth2.refresh.uri | OAuth2 refresh token uri | string | | | -| submarine.auth.oauth2.authorization.uri | OAuth2 authorization uri | string | | | -| submarine.auth.oauth2.logout.uri | OAuth2 logout uri | string | | | -| submarine.auth.oidc.client.id | OIDC client id | string | | | -| submarine.auth.oidc.client.secret | OIDC client Secret| string | | | -| submarine.auth.oidc.client.scopes | The available scopes for the OIDC security scheme. A map between the scope name and a short description for it.| string | | | -| submarine.auth.oidc.useNonce | Whether to use nonce during login process | string | | | -| submarine.auth.oidc.discover.uri | OIDC discovery uri | string | | | -| submarine.auth.oidc.logout.uri | OIDC logout uri | string | | | -| submarine.auth.ladp.provider.uri | LDAP provider uri | string | | | -| submarine.auth.ladp.baseDn | LDAP base DN | string | | base DN is the base LDAP distinguished name for your LDAP server. For example, ou=dev,dc=xyz,dc=com | -| submarine.auth.ladp.domain | LDAP AD domain | string | | AD domain is the domain name of the AD server. For example, corp.domain.com | +| Attribute | Description | Type | Default | Comment | +|-----------------------------------------|-------------------------------------------------------------------------------------------------------------------|---------|---------|-----------------------------------------------------------------------------------------------------| +| submarine.auth.type | Supported authentication types, currently available are: none, simple, oauth2/oidc, ldap, kerberos, saml, cas | string | none | Only one authentication method can be supported at any one time | +| submarine.auth.token.maxAge | Expiry time of the token (minute) | int | 1 day | | +| submarine.auth.refreshToken.maxAge | Expiry time of the refresh token (minute) | int | 1 hour | | +| submarine.cookie.http.only | HttpOnly Cookie | boolean | false | | +| submarine.cookie.secure | Secure Cookie | boolean | false | | +| submarine.cookie.samesite | SameSite Cookie, can be Lax, Strict, None(or empty) | string | | https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite | +| submarine.auth.oauth2.client.id | OAuth2 client id | string | | | +| submarine.auth.oauth2.client.secret | OAuth2 client secret | string | | | +| submarine.auth.oauth2.client.flows | OAuth2 flows, can be: authorizationCode, implicit, password or clientCredentials | string | | | +| submarine.auth.oauth2.scopes | The available scopes for the OAuth2 security scheme. A map between the scope name and a short description for it. | string | | | +| submarine.auth.oauth2.token.uri | OAuth2 access token uri | string | | | +| submarine.auth.oauth2.refresh.uri | OAuth2 refresh token uri | string | | | +| submarine.auth.oauth2.authorization.uri | OAuth2 authorization uri | string | | | +| submarine.auth.oauth2.logout.uri | OAuth2 logout uri | string | | | +| submarine.auth.oidc.client.id | OIDC client id | string | | | +| submarine.auth.oidc.client.secret | OIDC client Secret | string | | | +| submarine.auth.oidc.discover.uri | OIDC discovery uri | string | | | +| submarine.auth.ladp.provider.uri | LDAP provider uri | string | | | +| submarine.auth.ladp.baseDn | LDAP base DN | string | | base DN is the base LDAP distinguished name for your LDAP server. For example, ou=dev,dc=xyz,dc=com | +| submarine.auth.ladp.domain | LDAP AD domain | string | | AD domain is the domain name of the AD server. For example, corp.domain.com | ### Design and implementation @@ -147,10 +147,10 @@ Describe the design of relevant user tables, user registration/modification/dele and the processing logic associated with authenticated login (including the mapping of attributes for automatically registered users when integrating with other authentication platforms, etc.). -We use `sys_user` table to store user information for submarines. -When `submarine.auth.type` is `simple`, the user's login operation will match `user_name` and `password` (encrypted) in `sys_user`. Only when the user name and password match will the login succeed. -When `submarine.auth.type` is `ldap`, the user's login will operation request the LDAP and verify that the username and password are correct. A new record will be added to the `sys_user` table if the logged-in user does not exist. -When logging in using other third-party authentication (OAuth2/OpenID Connect (OIDC), SAML, CAS etc.), the login page will automatically jump to the third-party service and revert back to the submarine after a successful login. A new record will be added to the `sys_user` table if the logged-in user does not exist. +We use `sys_user` table to store user information for submarines. +When `submarine.auth.type` is `simple`, the user's login operation will match `user_name` and `password` (encrypted) in `sys_user`. Only when the user name and password match will the login succeed. +When `submarine.auth.type` is `ldap`, the user's login will operation request the LDAP and verify that the username and password are correct. A new record will be added to the `sys_user` table if the logged-in user does not exist. +When logging in using other third-party authentication (OAuth2/OpenID Connect (OIDC), SAML, CAS etc.), the login page will automatically jump to the third-party service and revert back to the submarine after a successful login. A new record will be added to the `sys_user` table if the logged-in user does not exist. #### Department [TODO]