添加了管理员全局搜索Value值的功能 (#5182)

* 添加了管理员全局搜索Value值的功能 * Update ItemControllerTest.java * Update GlobalSearchValueController.js * Improved some project codes * Optimized some issues * Added some relevant documents * Update docs/zh/portal/apollo-user-guide.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update docs/zh/portal/apollo-user-guide.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update README.md * Added usage documentation * Update docs/zh/README.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Fixed some issues * Fixed the front-end * Update GlobalSearchValueController.js * Optimized some of the logic * Added some unit tests --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
apolloconfig · Sep 20, 2024 · 7e7b090 · 7e7b090
1 parent 31e6486
commit 7e7b090
Show file tree

Hide file tree

Showing 43 changed files with 1,581 additions and 56 deletions.
diff --git a/CHANGES.md b/CHANGES.md
@@ -7,7 +7,7 @@ Apollo 2.4.0
 ------------------
 * [Update the server config link in system info page](https://github.com/apolloconfig/apollo/pull/5204)
 * [Feature support portal restTemplate Client connection pool config](https://github.com/apolloconfig/apollo/pull/5200)
-
+* [Feature added the ability for administrators to globally search for Value](https://github.com/apolloconfig/apollo/pull/5182)
 
 ------------------
-All issues and pull requests are [here](https://github.com/apolloconfig/apollo/milestone/15?closed=1)
+All issues and pull requests are [here](https://github.com/apolloconfig/apollo/milestone/15?closed=1)
diff --git a/README.md b/README.md
@@ -37,36 +37,31 @@ Demo Environment:
   * The same codebase could have different configurations when deployed in different clusters
   * With the namespace concept, it is easy to support multiple applications to share the same configurations, while also allowing them to customize the configurations
   * Multiple languages is provided in user interface(currently Chinese and English)
-
 * **Configuration changes takes effect in real time (hot release)**
   * After the user modified the configuration and released it in Apollo, the sdk will receive the latest configurations in real time (1 second) and notify the application
-
 * **Release version management**
   * Every configuration releases are versioned, which is friendly to support configuration rollback
-
 * **Grayscale release**
   * Support grayscale configuration release, for example, after clicking release, it will only take effect for some application instances. After a period of observation, we could push the configurations to all application instances if there is no problem
-
+* **Global Search Configuration Items**
+  * A fuzzy search of the key and value of a configuration item finds in which application, environment, cluster, namespace the configuration item with the corresponding value is used
+  * It is easy for administrators and SRE roles to quickly and easily find and change the configuration values of resources by highlighting, paging and jumping through configurations
 * **Authorization management, release approval and operation audit**
   * Great authorization mechanism is designed for applications and configurations management, and the management of configurations is divided into two operations: editing and publishing, therefore greatly reducing human errors
   * All operations have audit logs for easy tracking of problems
-
 * **Client side configuration information monitoring**
   * It's very easy to see which instances are using the configurations and what versions they are using
-
 * **Rich SDKs available**
   * Provides native sdks of Java and .Net to facilitate application integration
   * Support Spring Placeholder, Annotation and Spring Boot ConfigurationProperties for easy application use (requires Spring 3.1.1+)
   * Http APIs are provided, so non-Java and .Net applications can integrate conveniently
   * Rich third party sdks are also available, e.g. Golang, Python, NodeJS, PHP, C, etc
-
 * **Open platform API**
   * Apollo itself provides a unified configuration management interface, which supports features such as multi-environment, multi-data center configuration management, permissions, and process governance
   * However, for the sake of versatility, Apollo will not put too many restrictions on the modification of the configuration, as long as it conforms to the basic format, it can be saved.
   * In our research, we found that for some users, their configurations may have more complicated formats, such as xml, json, and the format needs to be verified
   * There are also some users such as DAL, which not only have a specific format, but also need to verify the entered value before saving, such as checking whether the database, username and password match
   * For this type of application, Apollo allows the application to modify and release configurations through open APIs, which has great authorization and permission control mechanism built in
-
 * **Simple deployment**
   * As an infrastructure service, the configuration center has very high availability requirements, which forces Apollo to rely on external dependencies as little as possible
   * Currently, the only external dependency is MySQL, so the deployment is very simple. Apollo can run as long as Java and MySQL are installed

diff --git a/...vice/src/main/java/com/ctrip/framework/apollo/adminservice/controller/ItemController.java b/...vice/src/main/java/com/ctrip/framework/apollo/adminservice/controller/ItemController.java
@@ -27,6 +27,7 @@
 import com.ctrip.framework.apollo.biz.service.ReleaseService;
 import com.ctrip.framework.apollo.biz.utils.ConfigChangeContentBuilder;
 import com.ctrip.framework.apollo.common.dto.ItemDTO;
+import com.ctrip.framework.apollo.common.dto.ItemInfoDTO;
 import com.ctrip.framework.apollo.common.dto.PageDTO;
 import com.ctrip.framework.apollo.common.exception.BadRequestException;
 import com.ctrip.framework.apollo.common.exception.NotFoundException;
@@ -201,6 +202,14 @@ public List<ItemDTO> findDeletedItems(@PathVariable("appId") String appId,
     return Collections.emptyList();
   }
 
+  @GetMapping("/items-search/key-and-value")
+  public PageDTO<ItemInfoDTO> getItemInfoBySearch(@RequestParam(value = "key", required = false) String key,
+                                                  @RequestParam(value = "value", required = false) String value,
+                                                  Pageable limit) {
+    Page<ItemInfoDTO> pageItemInfoDTO = itemService.getItemInfoBySearch(key, value, limit);
+    return new PageDTO<>(pageItemInfoDTO.getContent(), limit, pageItemInfoDTO.getTotalElements());
+  }
+
   @GetMapping("/items/{itemId}")
   public ItemDTO get(@PathVariable("itemId") long itemId) {
     Item item = itemService.findOne(itemId);

diff --git a/.../src/test/java/com/ctrip/framework/apollo/adminservice/controller/ItemControllerTest.java b/.../src/test/java/com/ctrip/framework/apollo/adminservice/controller/ItemControllerTest.java
@@ -21,18 +21,19 @@
 import com.ctrip.framework.apollo.biz.entity.Commit;
 import com.ctrip.framework.apollo.biz.repository.CommitRepository;
 import com.ctrip.framework.apollo.biz.repository.ItemRepository;
-import com.ctrip.framework.apollo.common.dto.AppDTO;
-import com.ctrip.framework.apollo.common.dto.ClusterDTO;
-import com.ctrip.framework.apollo.common.dto.ItemDTO;
-import com.ctrip.framework.apollo.common.dto.NamespaceDTO;
+import com.ctrip.framework.apollo.biz.service.ItemService;
+import com.ctrip.framework.apollo.common.dto.*;
+
 import java.util.List;
 import java.util.Objects;
 import org.junit.Assert;
 import org.junit.Test;
 import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.core.ParameterizedTypeReference;
+import org.springframework.data.domain.Page;
+import org.springframework.data.domain.PageRequest;
 import org.springframework.data.domain.Pageable;
-import org.springframework.http.HttpStatus;
-import org.springframework.http.ResponseEntity;
+import org.springframework.http.*;
 import org.springframework.test.context.jdbc.Sql;
 import org.springframework.test.context.jdbc.Sql.ExecutionPhase;
 
@@ -48,6 +49,9 @@ public class ItemControllerTest extends AbstractControllerTest {
   @Autowired
   private ItemRepository itemRepository;
 
+  @Autowired
+  private ItemService itemService;
+
   @Test
   @Sql(scripts = "/controller/test-itemset.sql", executionPhase = ExecutionPhase.BEFORE_TEST_METHOD)
   @Sql(scripts = "/controller/cleanup.sql", executionPhase = ExecutionPhase.AFTER_TEST_METHOD)
@@ -58,7 +62,7 @@ public void testCreate() {
     ClusterDTO cluster = restTemplate.getForObject(clusterBaseUrl(), ClusterDTO.class, app.getAppId(), "default");
     assert cluster != null;
     NamespaceDTO namespace = restTemplate.getForObject(namespaceBaseUrl(),
-        NamespaceDTO.class, app.getAppId(), cluster.getName(), "application");
+            NamespaceDTO.class, app.getAppId(), cluster.getName(), "application");
 
     String itemKey = "test-key";
     String itemValue = "test-value";
@@ -68,12 +72,12 @@ public void testCreate() {
     item.setDataChangeLastModifiedBy("apollo");
 
     ResponseEntity<ItemDTO> response = restTemplate.postForEntity(itemBaseUrl(),
-        item, ItemDTO.class, app.getAppId(), cluster.getName(), namespace.getNamespaceName());
+            item, ItemDTO.class, app.getAppId(), cluster.getName(), namespace.getNamespaceName());
     Assert.assertEquals(HttpStatus.OK, response.getStatusCode());
     Assert.assertEquals(itemKey, Objects.requireNonNull(response.getBody()).getKey());
 
     List<Commit> commitList = commitRepository.findByAppIdAndClusterNameAndNamespaceNameOrderByIdDesc(app.getAppId(), cluster.getName(), namespace.getNamespaceName(),
-        Pageable.ofSize(10));
+            Pageable.ofSize(10));
     Assert.assertEquals(1, commitList.size());
 
     Commit commit = commitList.get(0);
@@ -93,15 +97,15 @@ public void testUpdate() {
     ClusterDTO cluster = restTemplate.getForObject(clusterBaseUrl(), ClusterDTO.class, app.getAppId(), "default");
     assert cluster != null;
     NamespaceDTO namespace = restTemplate.getForObject(namespaceBaseUrl(),
-        NamespaceDTO.class, app.getAppId(), cluster.getName(), "application");
+            NamespaceDTO.class, app.getAppId(), cluster.getName(), "application");
 
     String itemKey = "test-key";
     String itemValue = "test-value-updated";
 
     long itemId = itemRepository.findByKey(itemKey, Pageable.ofSize(1))
-        .getContent()
-        .get(0)
-        .getId();
+            .getContent()
+            .get(0)
+            .getId();
     ItemDTO item = new ItemDTO(itemKey, itemValue, "", 1);
     item.setDataChangeLastModifiedBy("apollo");
 
@@ -115,7 +119,7 @@ public void testUpdate() {
     });
 
     List<Commit> commitList = commitRepository.findByAppIdAndClusterNameAndNamespaceNameOrderByIdDesc(app.getAppId(), cluster.getName(), namespace.getNamespaceName(),
-        Pageable.ofSize(10));
+            Pageable.ofSize(10));
     assertThat(commitList).hasSize(2);
   }
 
@@ -131,23 +135,44 @@ public void testDelete() {
     ClusterDTO cluster = restTemplate.getForObject(clusterBaseUrl(), ClusterDTO.class, app.getAppId(), "default");
     assert cluster != null;
     NamespaceDTO namespace = restTemplate.getForObject(namespaceBaseUrl(),
-        NamespaceDTO.class, app.getAppId(), cluster.getName(), "application");
+            NamespaceDTO.class, app.getAppId(), cluster.getName(), "application");
 
     String itemKey = "test-key";
 
     long itemId = itemRepository.findByKey(itemKey, Pageable.ofSize(1))
-        .getContent()
-        .get(0)
-        .getId();
+            .getContent()
+            .get(0)
+            .getId();
 
     String deleteUrl = url(  "/items/{itemId}?operator=apollo");
     restTemplate.delete(deleteUrl, itemId);
     assertThat(itemRepository.findById(itemId).isPresent())
-        .isFalse();
+            .isFalse();
 
     assert namespace != null;
     List<Commit> commitList = commitRepository.findByAppIdAndClusterNameAndNamespaceNameOrderByIdDesc(app.getAppId(), cluster.getName(), namespace.getNamespaceName(),
-        Pageable.ofSize(10));
+            Pageable.ofSize(10));
     assertThat(commitList).hasSize(2);
   }
+
+  @Test
+  @Sql(scripts = "/controller/test-itemset.sql", executionPhase = ExecutionPhase.BEFORE_TEST_METHOD)
+  @Sql(scripts = "/controller/cleanup.sql", executionPhase = ExecutionPhase.AFTER_TEST_METHOD)
+  public void testSearch() {
+    this.testCreate();
+
+    String itemKey = "test-key";
+    String itemValue = "test-value";
+    Page<ItemInfoDTO> itemInfoDTOS = itemService.getItemInfoBySearch(itemKey, itemValue, PageRequest.of(0, 200));
+    HttpHeaders headers = new HttpHeaders();
+    HttpEntity<Void> entity = new HttpEntity<>(headers);
+    ResponseEntity<PageDTO<ItemInfoDTO>> response = restTemplate.exchange(
+            url("/items-search/key-and-value?key={key}&value={value}&page={page}&size={size}"),
+            HttpMethod.GET,
+            entity,
+            new ParameterizedTypeReference<PageDTO<ItemInfoDTO>>() {},
+            itemKey, itemValue, 0, 200
+    );
+    assertThat(itemInfoDTOS.getContent().toString()).isEqualTo(response.getBody().getContent().toString());
+  }
 }
diff --git a/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/repository/ItemRepository.java b/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/repository/ItemRepository.java
@@ -18,11 +18,13 @@
 
 import com.ctrip.framework.apollo.biz.entity.Item;
 
+import com.ctrip.framework.apollo.common.dto.ItemInfoDTO;
 import org.springframework.data.domain.Page;
 import org.springframework.data.domain.Pageable;
 import org.springframework.data.jpa.repository.Modifying;
 import org.springframework.data.jpa.repository.Query;
 import org.springframework.data.repository.PagingAndSortingRepository;
+import org.springframework.data.repository.query.Param;
 
 import java.util.Date;
 import java.util.List;
@@ -43,6 +45,21 @@ public interface ItemRepository extends PagingAndSortingRepository<Item, Long> {
 
   Item findFirst1ByNamespaceIdOrderByLineNumDesc(Long namespaceId);
 
+  @Query("SELECT new com.ctrip.framework.apollo.common.dto.ItemInfoDTO(n.appId, n.clusterName, n.namespaceName, i.key, i.value) " +
+          "FROM Item i RIGHT JOIN Namespace n ON i.namespaceId = n.id " +
+          "WHERE i.key LIKE %:key% AND i.value LIKE %:value% AND i.isDeleted = 0")
+  Page<ItemInfoDTO> findItemsByKeyAndValueLike(@Param("key") String key, @Param("value") String value, Pageable pageable);
+
+  @Query("SELECT new com.ctrip.framework.apollo.common.dto.ItemInfoDTO(n.appId, n.clusterName, n.namespaceName, i.key, i.value) " +
+          "FROM Item i RIGHT JOIN Namespace n ON i.namespaceId = n.id " +
+          "WHERE i.key LIKE %:key% AND i.isDeleted = 0")
+  Page<ItemInfoDTO> findItemsByKeyLike(@Param("key") String key, Pageable pageable);
+
+  @Query("SELECT new com.ctrip.framework.apollo.common.dto.ItemInfoDTO(n.appId, n.clusterName, n.namespaceName, i.key, i.value) " +
+          "FROM Item i RIGHT JOIN Namespace n ON i.namespaceId = n.id " +
+          "WHERE i.value LIKE %:value% AND i.isDeleted = 0")
+  Page<ItemInfoDTO> findItemsByValueLike(@Param("value") String value, Pageable pageable);
+
   @Modifying
   @Query("update Item set IsDeleted = true, DeletedAt = ROUND(UNIX_TIMESTAMP(NOW(4))*1000), DataChange_LastModifiedBy = ?2 where NamespaceId = ?1 and IsDeleted = false")
   int deleteByNamespaceId(long namespaceId, String operator);

diff --git a/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/service/ItemService.java b/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/service/ItemService.java
@@ -22,6 +22,7 @@
 import com.ctrip.framework.apollo.biz.entity.Item;
 import com.ctrip.framework.apollo.biz.entity.Namespace;
 import com.ctrip.framework.apollo.biz.repository.ItemRepository;
+import com.ctrip.framework.apollo.common.dto.ItemInfoDTO;
 import com.ctrip.framework.apollo.common.exception.BadRequestException;
 import com.ctrip.framework.apollo.common.exception.NotFoundException;
 import com.ctrip.framework.apollo.common.utils.BeanUtils;
@@ -33,10 +34,7 @@
 import org.springframework.stereotype.Service;
 import org.springframework.transaction.annotation.Transactional;
 
-import java.util.Collections;
-import java.util.Date;
-import java.util.List;
-import java.util.Map;
+import java.util.*;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 
@@ -146,6 +144,18 @@ public Page<Item> findItemsByNamespace(String appId, String clusterName, String
     return itemRepository.findByNamespaceId(namespace.getId(), pageable);
   }
 
+  public Page<ItemInfoDTO> getItemInfoBySearch(String key, String value, Pageable limit) {
+    Page<ItemInfoDTO> itemInfoDTOs;
+    if (key.isEmpty() && !value.isEmpty()) {
+      itemInfoDTOs = itemRepository.findItemsByValueLike(value, limit);
+    } else if (value.isEmpty() && !key.isEmpty()) {
+      itemInfoDTOs = itemRepository.findItemsByKeyLike(key, limit);
+    } else {
+      itemInfoDTOs = itemRepository.findItemsByKeyAndValueLike(key, value, limit);
+    }
+    return itemInfoDTOs;
+  }
+
   @Transactional
   public Item save(Item entity) {
     checkItemKeyLength(entity.getKey());

diff --git a/apollo-biz/src/test/java/com/ctrip/framework/apollo/biz/service/ItemServiceTest.java b/apollo-biz/src/test/java/com/ctrip/framework/apollo/biz/service/ItemServiceTest.java
@@ -18,10 +18,13 @@
 
 import com.ctrip.framework.apollo.biz.AbstractIntegrationTest;
 import com.ctrip.framework.apollo.biz.entity.Item;
+import com.ctrip.framework.apollo.common.dto.ItemInfoDTO;
 import com.ctrip.framework.apollo.common.exception.BadRequestException;
 import org.junit.Assert;
 import org.junit.Test;
 import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.data.domain.Page;
+import org.springframework.data.domain.PageRequest;
 import org.springframework.test.context.jdbc.Sql;
 
 public class ItemServiceTest extends AbstractIntegrationTest {
@@ -71,4 +74,26 @@ public void testUpdateItem() {
         Assert.assertEquals("v1-new", dbItem.getValue());
     }
 
+    @Test
+    @Sql(scripts = {"/sql/namespace-test.sql","/sql/item-test.sql"}, executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD)
+    @Sql(scripts = "/sql/clean.sql", executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
+    public void testSearchItem() {
+        ItemInfoDTO itemInfoDTO = new ItemInfoDTO();
+        itemInfoDTO.setAppId("testApp");
+        itemInfoDTO.setClusterName("default");
+        itemInfoDTO.setNamespaceName("application");
+        itemInfoDTO.setKey("k1");
+        itemInfoDTO.setValue("v1");
+
+        String itemKey = "k1";
+        String itemValue = "v1";
+        Page<ItemInfoDTO> ExpectedItemInfoDTOSByKeyAndValue = itemService.getItemInfoBySearch(itemKey, itemValue, PageRequest.of(0,200));
+        Page<ItemInfoDTO> ExpectedItemInfoDTOSByKey = itemService.getItemInfoBySearch(itemKey,"", PageRequest.of(0,200));
+        Page<ItemInfoDTO> ExpectedItemInfoDTOSByValue = itemService.getItemInfoBySearch("", itemValue, PageRequest.of(0,200));
+        Assert.assertEquals(itemInfoDTO.toString(), ExpectedItemInfoDTOSByKeyAndValue.getContent().get(0).toString());
+        Assert.assertEquals(itemInfoDTO.toString(), ExpectedItemInfoDTOSByKey.getContent().get(0).toString());
+        Assert.assertEquals(itemInfoDTO.toString(), ExpectedItemInfoDTOSByValue.getContent().get(0).toString());
+
+    }
+
 }