diff --git a/CHANGES.md b/CHANGES.md index e67b0b78c40..ed756c3f80e 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -7,7 +7,7 @@ Apollo 2.4.0 ------------------ * [Update the server config link in system info page](https://github.com/apolloconfig/apollo/pull/5204) * [Feature support portal restTemplate Client connection pool config](https://github.com/apolloconfig/apollo/pull/5200) - +* [Feature added the ability for administrators to globally search for Value](https://github.com/apolloconfig/apollo/pull/5182) ------------------ -All issues and pull requests are [here](https://github.com/apolloconfig/apollo/milestone/15?closed=1) +All issues and pull requests are [here](https://github.com/apolloconfig/apollo/milestone/15?closed=1) \ No newline at end of file diff --git a/README.md b/README.md index 2dbe99867c3..3b1a1f7d7ce 100644 --- a/README.md +++ b/README.md @@ -37,36 +37,31 @@ Demo Environment: * The same codebase could have different configurations when deployed in different clusters * With the namespace concept, it is easy to support multiple applications to share the same configurations, while also allowing them to customize the configurations * Multiple languages is provided in user interface(currently Chinese and English) - * **Configuration changes takes effect in real time (hot release)** * After the user modified the configuration and released it in Apollo, the sdk will receive the latest configurations in real time (1 second) and notify the application - * **Release version management** * Every configuration releases are versioned, which is friendly to support configuration rollback - * **Grayscale release** * Support grayscale configuration release, for example, after clicking release, it will only take effect for some application instances. After a period of observation, we could push the configurations to all application instances if there is no problem - +* **Global Search Configuration Items** + * A fuzzy search of the key and value of a configuration item finds in which application, environment, cluster, namespace the configuration item with the corresponding value is used + * It is easy for administrators and SRE roles to quickly and easily find and change the configuration values of resources by highlighting, paging and jumping through configurations * **Authorization management, release approval and operation audit** * Great authorization mechanism is designed for applications and configurations management, and the management of configurations is divided into two operations: editing and publishing, therefore greatly reducing human errors * All operations have audit logs for easy tracking of problems - * **Client side configuration information monitoring** * It's very easy to see which instances are using the configurations and what versions they are using - * **Rich SDKs available** * Provides native sdks of Java and .Net to facilitate application integration * Support Spring Placeholder, Annotation and Spring Boot ConfigurationProperties for easy application use (requires Spring 3.1.1+) * Http APIs are provided, so non-Java and .Net applications can integrate conveniently * Rich third party sdks are also available, e.g. Golang, Python, NodeJS, PHP, C, etc - * **Open platform API** * Apollo itself provides a unified configuration management interface, which supports features such as multi-environment, multi-data center configuration management, permissions, and process governance * However, for the sake of versatility, Apollo will not put too many restrictions on the modification of the configuration, as long as it conforms to the basic format, it can be saved. * In our research, we found that for some users, their configurations may have more complicated formats, such as xml, json, and the format needs to be verified * There are also some users such as DAL, which not only have a specific format, but also need to verify the entered value before saving, such as checking whether the database, username and password match * For this type of application, Apollo allows the application to modify and release configurations through open APIs, which has great authorization and permission control mechanism built in - * **Simple deployment** * As an infrastructure service, the configuration center has very high availability requirements, which forces Apollo to rely on external dependencies as little as possible * Currently, the only external dependency is MySQL, so the deployment is very simple. Apollo can run as long as Java and MySQL are installed diff --git a/apollo-adminservice/src/main/java/com/ctrip/framework/apollo/adminservice/controller/ItemController.java b/apollo-adminservice/src/main/java/com/ctrip/framework/apollo/adminservice/controller/ItemController.java index 2e7e25d5912..f628748d2fa 100644 --- a/apollo-adminservice/src/main/java/com/ctrip/framework/apollo/adminservice/controller/ItemController.java +++ b/apollo-adminservice/src/main/java/com/ctrip/framework/apollo/adminservice/controller/ItemController.java @@ -27,6 +27,7 @@ import com.ctrip.framework.apollo.biz.service.ReleaseService; import com.ctrip.framework.apollo.biz.utils.ConfigChangeContentBuilder; import com.ctrip.framework.apollo.common.dto.ItemDTO; +import com.ctrip.framework.apollo.common.dto.ItemInfoDTO; import com.ctrip.framework.apollo.common.dto.PageDTO; import com.ctrip.framework.apollo.common.exception.BadRequestException; import com.ctrip.framework.apollo.common.exception.NotFoundException; @@ -201,6 +202,14 @@ public List findDeletedItems(@PathVariable("appId") String appId, return Collections.emptyList(); } + @GetMapping("/items-search/key-and-value") + public PageDTO getItemInfoBySearch(@RequestParam(value = "key", required = false) String key, + @RequestParam(value = "value", required = false) String value, + Pageable limit) { + Page pageItemInfoDTO = itemService.getItemInfoBySearch(key, value, limit); + return new PageDTO<>(pageItemInfoDTO.getContent(), limit, pageItemInfoDTO.getTotalElements()); + } + @GetMapping("/items/{itemId}") public ItemDTO get(@PathVariable("itemId") long itemId) { Item item = itemService.findOne(itemId); diff --git a/apollo-adminservice/src/test/java/com/ctrip/framework/apollo/adminservice/controller/ItemControllerTest.java b/apollo-adminservice/src/test/java/com/ctrip/framework/apollo/adminservice/controller/ItemControllerTest.java index f406d01e0f9..84e791f6c1d 100644 --- a/apollo-adminservice/src/test/java/com/ctrip/framework/apollo/adminservice/controller/ItemControllerTest.java +++ b/apollo-adminservice/src/test/java/com/ctrip/framework/apollo/adminservice/controller/ItemControllerTest.java @@ -21,18 +21,19 @@ import com.ctrip.framework.apollo.biz.entity.Commit; import com.ctrip.framework.apollo.biz.repository.CommitRepository; import com.ctrip.framework.apollo.biz.repository.ItemRepository; -import com.ctrip.framework.apollo.common.dto.AppDTO; -import com.ctrip.framework.apollo.common.dto.ClusterDTO; -import com.ctrip.framework.apollo.common.dto.ItemDTO; -import com.ctrip.framework.apollo.common.dto.NamespaceDTO; +import com.ctrip.framework.apollo.biz.service.ItemService; +import com.ctrip.framework.apollo.common.dto.*; + import java.util.List; import java.util.Objects; import org.junit.Assert; import org.junit.Test; import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.core.ParameterizedTypeReference; +import org.springframework.data.domain.Page; +import org.springframework.data.domain.PageRequest; import org.springframework.data.domain.Pageable; -import org.springframework.http.HttpStatus; -import org.springframework.http.ResponseEntity; +import org.springframework.http.*; import org.springframework.test.context.jdbc.Sql; import org.springframework.test.context.jdbc.Sql.ExecutionPhase; @@ -48,6 +49,9 @@ public class ItemControllerTest extends AbstractControllerTest { @Autowired private ItemRepository itemRepository; + @Autowired + private ItemService itemService; + @Test @Sql(scripts = "/controller/test-itemset.sql", executionPhase = ExecutionPhase.BEFORE_TEST_METHOD) @Sql(scripts = "/controller/cleanup.sql", executionPhase = ExecutionPhase.AFTER_TEST_METHOD) @@ -58,7 +62,7 @@ public void testCreate() { ClusterDTO cluster = restTemplate.getForObject(clusterBaseUrl(), ClusterDTO.class, app.getAppId(), "default"); assert cluster != null; NamespaceDTO namespace = restTemplate.getForObject(namespaceBaseUrl(), - NamespaceDTO.class, app.getAppId(), cluster.getName(), "application"); + NamespaceDTO.class, app.getAppId(), cluster.getName(), "application"); String itemKey = "test-key"; String itemValue = "test-value"; @@ -68,12 +72,12 @@ public void testCreate() { item.setDataChangeLastModifiedBy("apollo"); ResponseEntity response = restTemplate.postForEntity(itemBaseUrl(), - item, ItemDTO.class, app.getAppId(), cluster.getName(), namespace.getNamespaceName()); + item, ItemDTO.class, app.getAppId(), cluster.getName(), namespace.getNamespaceName()); Assert.assertEquals(HttpStatus.OK, response.getStatusCode()); Assert.assertEquals(itemKey, Objects.requireNonNull(response.getBody()).getKey()); List commitList = commitRepository.findByAppIdAndClusterNameAndNamespaceNameOrderByIdDesc(app.getAppId(), cluster.getName(), namespace.getNamespaceName(), - Pageable.ofSize(10)); + Pageable.ofSize(10)); Assert.assertEquals(1, commitList.size()); Commit commit = commitList.get(0); @@ -93,15 +97,15 @@ public void testUpdate() { ClusterDTO cluster = restTemplate.getForObject(clusterBaseUrl(), ClusterDTO.class, app.getAppId(), "default"); assert cluster != null; NamespaceDTO namespace = restTemplate.getForObject(namespaceBaseUrl(), - NamespaceDTO.class, app.getAppId(), cluster.getName(), "application"); + NamespaceDTO.class, app.getAppId(), cluster.getName(), "application"); String itemKey = "test-key"; String itemValue = "test-value-updated"; long itemId = itemRepository.findByKey(itemKey, Pageable.ofSize(1)) - .getContent() - .get(0) - .getId(); + .getContent() + .get(0) + .getId(); ItemDTO item = new ItemDTO(itemKey, itemValue, "", 1); item.setDataChangeLastModifiedBy("apollo"); @@ -115,7 +119,7 @@ public void testUpdate() { }); List commitList = commitRepository.findByAppIdAndClusterNameAndNamespaceNameOrderByIdDesc(app.getAppId(), cluster.getName(), namespace.getNamespaceName(), - Pageable.ofSize(10)); + Pageable.ofSize(10)); assertThat(commitList).hasSize(2); } @@ -131,23 +135,44 @@ public void testDelete() { ClusterDTO cluster = restTemplate.getForObject(clusterBaseUrl(), ClusterDTO.class, app.getAppId(), "default"); assert cluster != null; NamespaceDTO namespace = restTemplate.getForObject(namespaceBaseUrl(), - NamespaceDTO.class, app.getAppId(), cluster.getName(), "application"); + NamespaceDTO.class, app.getAppId(), cluster.getName(), "application"); String itemKey = "test-key"; long itemId = itemRepository.findByKey(itemKey, Pageable.ofSize(1)) - .getContent() - .get(0) - .getId(); + .getContent() + .get(0) + .getId(); String deleteUrl = url( "/items/{itemId}?operator=apollo"); restTemplate.delete(deleteUrl, itemId); assertThat(itemRepository.findById(itemId).isPresent()) - .isFalse(); + .isFalse(); assert namespace != null; List commitList = commitRepository.findByAppIdAndClusterNameAndNamespaceNameOrderByIdDesc(app.getAppId(), cluster.getName(), namespace.getNamespaceName(), - Pageable.ofSize(10)); + Pageable.ofSize(10)); assertThat(commitList).hasSize(2); } + + @Test + @Sql(scripts = "/controller/test-itemset.sql", executionPhase = ExecutionPhase.BEFORE_TEST_METHOD) + @Sql(scripts = "/controller/cleanup.sql", executionPhase = ExecutionPhase.AFTER_TEST_METHOD) + public void testSearch() { + this.testCreate(); + + String itemKey = "test-key"; + String itemValue = "test-value"; + Page itemInfoDTOS = itemService.getItemInfoBySearch(itemKey, itemValue, PageRequest.of(0, 200)); + HttpHeaders headers = new HttpHeaders(); + HttpEntity entity = new HttpEntity<>(headers); + ResponseEntity> response = restTemplate.exchange( + url("/items-search/key-and-value?key={key}&value={value}&page={page}&size={size}"), + HttpMethod.GET, + entity, + new ParameterizedTypeReference>() {}, + itemKey, itemValue, 0, 200 + ); + assertThat(itemInfoDTOS.getContent().toString()).isEqualTo(response.getBody().getContent().toString()); + } } diff --git a/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/repository/ItemRepository.java b/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/repository/ItemRepository.java index d482ce63e97..c866b902814 100644 --- a/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/repository/ItemRepository.java +++ b/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/repository/ItemRepository.java @@ -18,11 +18,13 @@ import com.ctrip.framework.apollo.biz.entity.Item; +import com.ctrip.framework.apollo.common.dto.ItemInfoDTO; import org.springframework.data.domain.Page; import org.springframework.data.domain.Pageable; import org.springframework.data.jpa.repository.Modifying; import org.springframework.data.jpa.repository.Query; import org.springframework.data.repository.PagingAndSortingRepository; +import org.springframework.data.repository.query.Param; import java.util.Date; import java.util.List; @@ -43,6 +45,21 @@ public interface ItemRepository extends PagingAndSortingRepository { Item findFirst1ByNamespaceIdOrderByLineNumDesc(Long namespaceId); + @Query("SELECT new com.ctrip.framework.apollo.common.dto.ItemInfoDTO(n.appId, n.clusterName, n.namespaceName, i.key, i.value) " + + "FROM Item i RIGHT JOIN Namespace n ON i.namespaceId = n.id " + + "WHERE i.key LIKE %:key% AND i.value LIKE %:value% AND i.isDeleted = 0") + Page findItemsByKeyAndValueLike(@Param("key") String key, @Param("value") String value, Pageable pageable); + + @Query("SELECT new com.ctrip.framework.apollo.common.dto.ItemInfoDTO(n.appId, n.clusterName, n.namespaceName, i.key, i.value) " + + "FROM Item i RIGHT JOIN Namespace n ON i.namespaceId = n.id " + + "WHERE i.key LIKE %:key% AND i.isDeleted = 0") + Page findItemsByKeyLike(@Param("key") String key, Pageable pageable); + + @Query("SELECT new com.ctrip.framework.apollo.common.dto.ItemInfoDTO(n.appId, n.clusterName, n.namespaceName, i.key, i.value) " + + "FROM Item i RIGHT JOIN Namespace n ON i.namespaceId = n.id " + + "WHERE i.value LIKE %:value% AND i.isDeleted = 0") + Page findItemsByValueLike(@Param("value") String value, Pageable pageable); + @Modifying @Query("update Item set IsDeleted = true, DeletedAt = ROUND(UNIX_TIMESTAMP(NOW(4))*1000), DataChange_LastModifiedBy = ?2 where NamespaceId = ?1 and IsDeleted = false") int deleteByNamespaceId(long namespaceId, String operator); diff --git a/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/service/ItemService.java b/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/service/ItemService.java index 7dffa366c5e..3c0d23c1b62 100644 --- a/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/service/ItemService.java +++ b/apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/service/ItemService.java @@ -22,6 +22,7 @@ import com.ctrip.framework.apollo.biz.entity.Item; import com.ctrip.framework.apollo.biz.entity.Namespace; import com.ctrip.framework.apollo.biz.repository.ItemRepository; +import com.ctrip.framework.apollo.common.dto.ItemInfoDTO; import com.ctrip.framework.apollo.common.exception.BadRequestException; import com.ctrip.framework.apollo.common.exception.NotFoundException; import com.ctrip.framework.apollo.common.utils.BeanUtils; @@ -33,10 +34,7 @@ import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; -import java.util.Collections; -import java.util.Date; -import java.util.List; -import java.util.Map; +import java.util.*; import java.util.regex.Matcher; import java.util.regex.Pattern; @@ -146,6 +144,18 @@ public Page findItemsByNamespace(String appId, String clusterName, String return itemRepository.findByNamespaceId(namespace.getId(), pageable); } + public Page getItemInfoBySearch(String key, String value, Pageable limit) { + Page itemInfoDTOs; + if (key.isEmpty() && !value.isEmpty()) { + itemInfoDTOs = itemRepository.findItemsByValueLike(value, limit); + } else if (value.isEmpty() && !key.isEmpty()) { + itemInfoDTOs = itemRepository.findItemsByKeyLike(key, limit); + } else { + itemInfoDTOs = itemRepository.findItemsByKeyAndValueLike(key, value, limit); + } + return itemInfoDTOs; + } + @Transactional public Item save(Item entity) { checkItemKeyLength(entity.getKey()); diff --git a/apollo-biz/src/test/java/com/ctrip/framework/apollo/biz/service/ItemServiceTest.java b/apollo-biz/src/test/java/com/ctrip/framework/apollo/biz/service/ItemServiceTest.java index 3b0ed5f6a4b..6d8003f1d58 100644 --- a/apollo-biz/src/test/java/com/ctrip/framework/apollo/biz/service/ItemServiceTest.java +++ b/apollo-biz/src/test/java/com/ctrip/framework/apollo/biz/service/ItemServiceTest.java @@ -18,10 +18,13 @@ import com.ctrip.framework.apollo.biz.AbstractIntegrationTest; import com.ctrip.framework.apollo.biz.entity.Item; +import com.ctrip.framework.apollo.common.dto.ItemInfoDTO; import com.ctrip.framework.apollo.common.exception.BadRequestException; import org.junit.Assert; import org.junit.Test; import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.data.domain.Page; +import org.springframework.data.domain.PageRequest; import org.springframework.test.context.jdbc.Sql; public class ItemServiceTest extends AbstractIntegrationTest { @@ -71,4 +74,26 @@ public void testUpdateItem() { Assert.assertEquals("v1-new", dbItem.getValue()); } + @Test + @Sql(scripts = {"/sql/namespace-test.sql","/sql/item-test.sql"}, executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD) + @Sql(scripts = "/sql/clean.sql", executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD) + public void testSearchItem() { + ItemInfoDTO itemInfoDTO = new ItemInfoDTO(); + itemInfoDTO.setAppId("testApp"); + itemInfoDTO.setClusterName("default"); + itemInfoDTO.setNamespaceName("application"); + itemInfoDTO.setKey("k1"); + itemInfoDTO.setValue("v1"); + + String itemKey = "k1"; + String itemValue = "v1"; + Page ExpectedItemInfoDTOSByKeyAndValue = itemService.getItemInfoBySearch(itemKey, itemValue, PageRequest.of(0,200)); + Page ExpectedItemInfoDTOSByKey = itemService.getItemInfoBySearch(itemKey,"", PageRequest.of(0,200)); + Page ExpectedItemInfoDTOSByValue = itemService.getItemInfoBySearch("", itemValue, PageRequest.of(0,200)); + Assert.assertEquals(itemInfoDTO.toString(), ExpectedItemInfoDTOSByKeyAndValue.getContent().get(0).toString()); + Assert.assertEquals(itemInfoDTO.toString(), ExpectedItemInfoDTOSByKey.getContent().get(0).toString()); + Assert.assertEquals(itemInfoDTO.toString(), ExpectedItemInfoDTOSByValue.getContent().get(0).toString()); + + } + } diff --git a/apollo-common/src/main/java/com/ctrip/framework/apollo/common/dto/ItemInfoDTO.java b/apollo-common/src/main/java/com/ctrip/framework/apollo/common/dto/ItemInfoDTO.java new file mode 100644 index 00000000000..a28794ed656 --- /dev/null +++ b/apollo-common/src/main/java/com/ctrip/framework/apollo/common/dto/ItemInfoDTO.java @@ -0,0 +1,88 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package com.ctrip.framework.apollo.common.dto; + + +public class ItemInfoDTO extends BaseDTO{ + private String appId; + private String clusterName; + private String namespaceName; + private String key; + private String value; + + public ItemInfoDTO() { + } + + public ItemInfoDTO(String appId, String clusterName, String namespaceName, String key, String value) { + this.appId = appId; + this.clusterName = clusterName; + this.namespaceName = namespaceName; + this.key = key; + this.value = value; + } + + public String getAppId() { + return appId; + } + + public void setAppId(String appId) { + this.appId = appId; + } + + public String getClusterName() { + return clusterName; + } + + public void setClusterName(String clusterName) { + this.clusterName = clusterName; + } + + public String getNamespaceName() { + return namespaceName; + } + + public void setNamespaceName(String namespaceName) { + this.namespaceName = namespaceName; + } + + public String getKey() { + return key; + } + + public void setKey(String key) { + this.key = key; + } + + public String getValue() { + return value; + } + + public void setValue(String value) { + this.value = value; + } + + @Override + public String toString() { + return "ItemInfoDTO{" + + "appId='" + appId + '\'' + + ", clusterName='" + clusterName + '\'' + + ", namespaceName='" + namespaceName + '\'' + + ", key='" + key + '\'' + + ", value='" + value + '\'' + + '}'; + } +} diff --git a/apollo-common/src/main/java/com/ctrip/framework/apollo/common/http/SearchResponseEntity.java b/apollo-common/src/main/java/com/ctrip/framework/apollo/common/http/SearchResponseEntity.java new file mode 100644 index 00000000000..0eb6cf446da --- /dev/null +++ b/apollo-common/src/main/java/com/ctrip/framework/apollo/common/http/SearchResponseEntity.java @@ -0,0 +1,67 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package com.ctrip.framework.apollo.common.http; + +import org.springframework.http.HttpStatus; + +public class SearchResponseEntity { + + private T body; + private boolean hasMoreData; + private Object message; + private int code; + + public static SearchResponseEntity ok(T body){ + SearchResponseEntity SearchResponseEntity = new SearchResponseEntity<>(); + SearchResponseEntity.message = HttpStatus.OK.getReasonPhrase(); + SearchResponseEntity.code = HttpStatus.OK.value(); + SearchResponseEntity.body = body; + SearchResponseEntity.hasMoreData = false; + return SearchResponseEntity; + } + + public static SearchResponseEntity okWithMessage(T body, Object message){ + SearchResponseEntity SearchResponseEntity = new SearchResponseEntity<>(); + SearchResponseEntity.message = message; + SearchResponseEntity.code = HttpStatus.OK.value(); + SearchResponseEntity.body = body; + SearchResponseEntity.hasMoreData = true; + return SearchResponseEntity; + } + + public static SearchResponseEntity error(HttpStatus httpCode, Object message){ + SearchResponseEntity SearchResponseEntity = new SearchResponseEntity<>(); + SearchResponseEntity.message = message; + SearchResponseEntity.code = httpCode.value(); + return SearchResponseEntity; + } + + public int getCode() { + return code; + } + + public Object getMessage() { + return message; + } + + public T getBody() { + return body; + } + + public boolean isHasMoreData() {return hasMoreData;} + +} \ No newline at end of file diff --git a/apollo-common/src/test/java/com/ctrip/framework/apollo/common/dto/ItemInfoDTOTest.java b/apollo-common/src/test/java/com/ctrip/framework/apollo/common/dto/ItemInfoDTOTest.java new file mode 100644 index 00000000000..5a138b691cf --- /dev/null +++ b/apollo-common/src/test/java/com/ctrip/framework/apollo/common/dto/ItemInfoDTOTest.java @@ -0,0 +1,61 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except + * in compliance with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ +package com.ctrip.framework.apollo.common.dto; + + +import org.junit.Before; +import org.junit.Test; + +import static org.junit.Assert.assertEquals; + +public class ItemInfoDTOTest { + + private ItemInfoDTO itemInfoDTO; + + @Before + public void setUp() { + itemInfoDTO = new ItemInfoDTO("testAppId", "testClusterName", "testNamespaceName", "testKey", "testValue"); + } + + @Test + public void testGetAppId_ShouldReturnCorrectAppId() { + assertEquals("testAppId", itemInfoDTO.getAppId()); + } + + @Test + public void testGetClusterName_ShouldReturnCorrectClusterName() { + assertEquals("testClusterName", itemInfoDTO.getClusterName()); + } + + @Test + public void testGetNamespaceName_ShouldReturnCorrectNamespaceName() { + assertEquals("testNamespaceName", itemInfoDTO.getNamespaceName()); + } + + @Test + public void testGetKey_ShouldReturnCorrectKey() { + assertEquals("testKey", itemInfoDTO.getKey()); + } + + @Test + public void testGetValue_ShouldReturnCorrectValue() { + assertEquals("testValue", itemInfoDTO.getValue()); + } + + @Test + public void testToString_ShouldReturnExpectedString() { + assertEquals("ItemInfoDTO{appId='testAppId', clusterName='testClusterName', namespaceName='testNamespaceName', key='testKey', value='testValue'}", itemInfoDTO.toString()); + } +} diff --git a/apollo-common/src/test/java/com/ctrip/framework/apollo/common/http/SearchResponseEntityTest.java b/apollo-common/src/test/java/com/ctrip/framework/apollo/common/http/SearchResponseEntityTest.java new file mode 100644 index 00000000000..14e250004eb --- /dev/null +++ b/apollo-common/src/test/java/com/ctrip/framework/apollo/common/http/SearchResponseEntityTest.java @@ -0,0 +1,62 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except + * in compliance with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ +package com.ctrip.framework.apollo.common.http; + + +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.junit.MockitoJUnitRunner; +import org.springframework.http.HttpStatus; + +import static org.junit.Assert.*; + +@RunWith(MockitoJUnitRunner.class) +public class SearchResponseEntityTest { + + @Test + public void testOk_WithValidBody_ShouldReturnOkResponse() { + String body = "test body"; + SearchResponseEntity response = SearchResponseEntity.ok(body); + + assertEquals(HttpStatus.OK.value(), response.getCode()); + assertEquals(HttpStatus.OK.getReasonPhrase(), response.getMessage()); + assertEquals(body, response.getBody()); + assertFalse(response.isHasMoreData()); + } + + @Test + public void testOkWithMessage_WithValidBodyAndMessage_ShouldReturnOkResponseWithMessage() { + String body = "test body"; + String message = "test message"; + SearchResponseEntity response = SearchResponseEntity.okWithMessage(body, message); + + assertEquals(HttpStatus.OK.value(), response.getCode()); + assertEquals(message, response.getMessage()); + assertEquals(body, response.getBody()); + assertTrue(response.isHasMoreData()); + } + + @Test + public void testError_WithValidCodeAndMessage_ShouldReturnErrorResponse() { + HttpStatus httpCode = HttpStatus.BAD_REQUEST; + String message = "error message"; + SearchResponseEntity response = SearchResponseEntity.error(httpCode, message); + + assertEquals(httpCode.value(), response.getCode()); + assertEquals(message, response.getMessage()); + assertEquals(null, response.getBody()); + assertFalse(response.isHasMoreData()); + } +} diff --git a/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/api/AdminServiceAPI.java b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/api/AdminServiceAPI.java index 3d3f23bff4a..b8035aae7a2 100644 --- a/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/api/AdminServiceAPI.java +++ b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/api/AdminServiceAPI.java @@ -184,6 +184,9 @@ public static class ItemAPI extends API { private final ParameterizedTypeReference> openItemPageDTO = new ParameterizedTypeReference>() {}; + private final ParameterizedTypeReference> pageItemInfoDTO = + new ParameterizedTypeReference>() {}; + public List findItems(String appId, Env env, String clusterName, String namespaceName) { ItemDTO[] itemDTOs = restTemplate.get(env, "apps/{appId}/clusters/{clusterName}/namespaces/{namespaceName}/items", @@ -198,6 +201,15 @@ public List findDeletedItems(String appId, Env env, String clusterName, return Arrays.asList(itemDTOs); } + public PageDTO getPerEnvItemInfoBySearch(Env env, String key, String value, int page, int size){ + ResponseEntity> + entity = + restTemplate.get(env, + "items-search/key-and-value?key={key}&value={value}&page={page}&size={size}", + pageItemInfoDTO, key, value, page, size); + return entity.getBody(); + } + public ItemDTO loadItem(Env env, String appId, String clusterName, String namespaceName, String key) { return restTemplate.get(env, "apps/{appId}/clusters/{clusterName}/namespaces/{namespaceName}/items/{key}", ItemDTO.class, appId, clusterName, namespaceName, key); diff --git a/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/component/config/PortalConfig.java b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/component/config/PortalConfig.java index 93b9e93daf2..a5a4a9d81ea 100644 --- a/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/component/config/PortalConfig.java +++ b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/component/config/PortalConfig.java @@ -86,6 +86,8 @@ public List portalSupportedEnvs() { return envs; } + public int getPerEnvSearchMaxResults() {return getIntProperty("apollo.portal.search.perEnvMaxResults", 200);} + /** * @return the relationship between environment and its meta server. empty if meet exception */ diff --git a/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/controller/GlobalSearchController.java b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/controller/GlobalSearchController.java new file mode 100644 index 00000000000..bb44b22932f --- /dev/null +++ b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/controller/GlobalSearchController.java @@ -0,0 +1,54 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package com.ctrip.framework.apollo.portal.controller; + + +import com.ctrip.framework.apollo.common.exception.BadRequestException; +import com.ctrip.framework.apollo.common.http.SearchResponseEntity; +import com.ctrip.framework.apollo.portal.component.config.PortalConfig; +import com.ctrip.framework.apollo.portal.entity.vo.ItemInfo; +import com.ctrip.framework.apollo.portal.service.GlobalSearchService; +import org.springframework.security.access.prepost.PreAuthorize; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.RestController; + +import java.util.List; + +@RestController +public class GlobalSearchController { + private final GlobalSearchService globalSearchService; + private final PortalConfig portalConfig; + + public GlobalSearchController(final GlobalSearchService globalSearchService, final PortalConfig portalConfig) { + this.globalSearchService = globalSearchService; + this.portalConfig = portalConfig; + } + + @PreAuthorize(value = "@permissionValidator.isSuperAdmin()") + @GetMapping("/global-search/item-info/by-key-or-value") + public SearchResponseEntity> getItemInfoBySearch(@RequestParam(value = "key", required = false, defaultValue = "") String key, + @RequestParam(value = "value", required = false , defaultValue = "") String value) { + + if(key.isEmpty() && value.isEmpty()) { + throw new BadRequestException("Please enter at least one search criterion in either key or value."); + } + + return globalSearchService.getAllEnvItemInfoBySearch(key, value, 0, portalConfig.getPerEnvSearchMaxResults()); + } + +} diff --git a/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/entity/vo/ItemInfo.java b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/entity/vo/ItemInfo.java new file mode 100644 index 00000000000..f4d8dbf3974 --- /dev/null +++ b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/entity/vo/ItemInfo.java @@ -0,0 +1,100 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package com.ctrip.framework.apollo.portal.entity.vo; + +public class ItemInfo { + + private String appId; + private String envName; + private String clusterName; + private String namespaceName; + private String key; + private String value; + + public ItemInfo() { + } + + public ItemInfo(String appId, String envName, String clusterName, + String namespaceName, String key, String value) { + this.appId = appId; + this.envName = envName; + this.clusterName = clusterName; + this.namespaceName = namespaceName; + this.key = key; + this.value = value; + } + + public String getAppId() { + return appId; + } + + public void setAppId(String appId) { + this.appId = appId; + } + + public String getEnvName() { + return envName; + } + + public void setEnvName(String envName) { + this.envName = envName; + } + + public String getClusterName() { + return clusterName; + } + + public void setClusterName(String clusterName) { + this.clusterName = clusterName; + } + + public String getNamespaceName() { + return namespaceName; + } + + public void setNamespaceName(String namespaceName) { + this.namespaceName = namespaceName; + } + + public String getKey() { + return key; + } + + public void setKey(String key) { + this.key = key; + } + + public String getValue() { + return value; + } + + public void setValue(String value) { + this.value = value; + } + + @Override + public String toString() { + return "ItemInfo{" + + "appId='" + appId + '\'' + + ", envName='" + envName + '\'' + + ", clusterName='" + clusterName + '\'' + + ", namespaceName='" + namespaceName + '\'' + + ", key='" + key + '\'' + + ", value='" + value + '\'' + + '}'; + } +} diff --git a/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/GlobalSearchService.java b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/GlobalSearchService.java new file mode 100644 index 00000000000..5c7cbdcf5ba --- /dev/null +++ b/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/GlobalSearchService.java @@ -0,0 +1,77 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package com.ctrip.framework.apollo.portal.service; + +import com.ctrip.framework.apollo.common.dto.ItemInfoDTO; +import com.ctrip.framework.apollo.common.dto.PageDTO; +import com.ctrip.framework.apollo.common.http.SearchResponseEntity; +import com.ctrip.framework.apollo.portal.api.AdminServiceAPI; +import com.ctrip.framework.apollo.portal.component.PortalSettings; +import com.ctrip.framework.apollo.portal.entity.vo.ItemInfo; +import com.ctrip.framework.apollo.portal.environment.Env; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.stereotype.Service; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.atomic.AtomicBoolean; + +@Service +public class GlobalSearchService { + + private static final Logger LOGGER = LoggerFactory.getLogger(GlobalSearchService.class); + private final AdminServiceAPI.ItemAPI itemAPI; + private final PortalSettings portalSettings; + + public GlobalSearchService(AdminServiceAPI.ItemAPI itemAPI, PortalSettings portalSettings) { + this.itemAPI = itemAPI; + this.portalSettings = portalSettings; + } + + public SearchResponseEntity> getAllEnvItemInfoBySearch(String key, String value, int page, int size) { + List activeEnvs = portalSettings.getActiveEnvs(); + List envBeyondLimit = new ArrayList<>(); + AtomicBoolean hasMoreData = new AtomicBoolean(false); + List allEnvItemInfos = new ArrayList<>(); + activeEnvs.forEach(env -> { + PageDTO perEnvItemInfoDTOs = itemAPI.getPerEnvItemInfoBySearch(env, key, value, page, size); + if (!perEnvItemInfoDTOs.hasContent()) { + return; + } + perEnvItemInfoDTOs.getContent().forEach(itemInfoDTO -> { + try { + ItemInfo itemInfo = new ItemInfo(itemInfoDTO.getAppId(),env.getName(),itemInfoDTO.getClusterName(),itemInfoDTO.getNamespaceName(),itemInfoDTO.getKey(),itemInfoDTO.getValue()); + allEnvItemInfos.add(itemInfo); + } catch (Exception e) { + LOGGER.error("Error converting ItemInfoDTO to ItemInfo for item: {}", itemInfoDTO, e); + } + }); + if(perEnvItemInfoDTOs.getTotal() > size){ + envBeyondLimit.add(env.getName()); + hasMoreData.set(true); + } + }); + if(hasMoreData.get()){ + return SearchResponseEntity.okWithMessage(allEnvItemInfos,String.format( + "In %s , more than %d items found (Exceeded the maximum search quantity for a single environment). Please enter more precise criteria to narrow down the search scope.", + String.join(" , ", envBeyondLimit), size)); + } + return SearchResponseEntity.ok(allEnvItemInfos); + } + +} diff --git a/apollo-portal/src/main/resources/static/global_search_value.html b/apollo-portal/src/main/resources/static/global_search_value.html new file mode 100644 index 00000000000..1d59578914b --- /dev/null +++ b/apollo-portal/src/main/resources/static/global_search_value.html @@ -0,0 +1,196 @@ + + + + + + + + + + + + + + {{'Global.Title' | translate }} + + + + +
+
+
+ +
+
+
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/apollo-portal/src/main/resources/static/i18n/en.json b/apollo-portal/src/main/resources/static/i18n/en.json index dbe1d698520..81c3bded0a6 100644 --- a/apollo-portal/src/main/resources/static/i18n/en.json +++ b/apollo-portal/src/main/resources/static/i18n/en.json @@ -893,5 +893,28 @@ "ApolloAuditLog.ParentSpan": "parent operation", "ApolloAuditLog.FollowsFromSpan": "last operation", "ApolloAuditLog.FieldChangeHistory": "Field Change History", - "ApolloAuditLog.InfluenceEntity": "Audit entity influenced" + "ApolloAuditLog.InfluenceEntity": "Audit entity influenced", + "Global.Title": "Global Search for Value", + "Global.App": "App ID", + "Global.Env": "Env Name", + "Global.Cluster": "Cluster Name", + "Global.NameSpace": "NameSpace Name", + "Global.Key": "Key", + "Global.Value": "Value", + "Global.ValueSearch.Tips" : "(Fuzzy search, key can be the name or content of the configuration item, value is the value of the configuration item.)", + "Global.Operate" : "Operate", + "Global.Expand" : "Expand", + "Global.Abbreviate" : "Abbreviate", + "Global.JumpToEditPage" : "Jump to edit page", + "Item.GlobalSearchByKey": "Search by Key", + "Item.GlobalSearchByValue": "Search by Value", + "Item.GlobalSearch": "Search", + "Item.GlobalSearchSystemError": "System error, please try again or contact the system administrator", + "Item.GlobalSearch.Tips": "Search hint", + "ApolloGlobalSearch.NoData" : "No data yet, please search or add", + "Paging.TotalItems.part1" : "Total of", + "Paging.TotalItems.part2" : "records", + "Paging.DisplayNumber" : "per/Page", + "Paging.PageNumberOne" : "First", + "Paging.PageNumberLast" : "Last" } diff --git a/apollo-portal/src/main/resources/static/i18n/zh-CN.json b/apollo-portal/src/main/resources/static/i18n/zh-CN.json index e166237c306..2ef309b46dc 100644 --- a/apollo-portal/src/main/resources/static/i18n/zh-CN.json +++ b/apollo-portal/src/main/resources/static/i18n/zh-CN.json @@ -893,5 +893,28 @@ "ApolloAuditLog.ParentSpan": "父操作", "ApolloAuditLog.FollowsFromSpan": "前操作", "ApolloAuditLog.FieldChangeHistory": "属性变动历史", - "ApolloAuditLog.InfluenceEntity": "影响的审计实体" + "ApolloAuditLog.InfluenceEntity": "影响的审计实体", + "Global.Title": "Value的全局搜索", + "Global.App": "应用ID", + "Global.Env": "环境", + "Global.Cluster": "集群名", + "Global.NameSpace": "命名空间", + "Global.Key": "Key", + "Global.Value": "Value", + "Global.ValueSearch.Tips" : "(模糊搜索,key可为配置项名称或content,value为配置项值)", + "Global.Operate" : "操作", + "Global.Expand" : "展开", + "Global.Abbreviate" : "缩略", + "Global.JumpToEditPage" : "跳转到编辑页面", + "Item.GlobalSearchByKey": "按照Key值检索", + "Item.GlobalSearchByValue": "按照Value值检索", + "Item.GlobalSearch": "查询", + "Item.GlobalSearchSystemError": "系统出错,请重试或联系系统负责人", + "Item.GlobalSearch.Tips": "搜索提示", + "ApolloGlobalSearch.NoData" : "暂无数据,请进行检索或者添加", + "Paging.TotalItems.part1" : "共", + "Paging.TotalItems.part2" : "条记录", + "Paging.DisplayNumber" : "条/页", + "Paging.PageNumberOne" : "首页", + "Paging.PageNumberLast" : "尾页" } diff --git a/apollo-portal/src/main/resources/static/img/nodata.png b/apollo-portal/src/main/resources/static/img/nodata.png new file mode 100644 index 00000000000..1cb236546ea Binary files /dev/null and b/apollo-portal/src/main/resources/static/img/nodata.png differ diff --git a/apollo-portal/src/main/resources/static/scripts/app.js b/apollo-portal/src/main/resources/static/scripts/app.js index bac2dd5fde8..88ecb81e522 100644 --- a/apollo-portal/src/main/resources/static/scripts/app.js +++ b/apollo-portal/src/main/resources/static/scripts/app.js @@ -64,6 +64,8 @@ var diff_item_module = angular.module('diff_item', ['app.service', 'apollo.direc var namespace_module = angular.module('namespace', ['app.service', 'apollo.directive', 'app.util', 'toastr', 'angular-loading-bar', 'valdr']); //server config var server_config_manage_module = angular.module('server_config_manage', ['app.service', 'apollo.directive', 'app.util', 'toastr', 'angular-loading-bar']); +// Value的全局检索 +var global_search_value_module = angular.module('global_search_value', ['app.service', 'apollo.directive', 'app.util', 'toastr', 'angular-loading-bar', 'ngSanitize']); //setting var setting_module = angular.module('setting', ['app.service', 'apollo.directive', 'app.util', 'toastr', 'angular-loading-bar', 'valdr']); //role diff --git a/apollo-portal/src/main/resources/static/scripts/controller/GlobalSearchValueController.js b/apollo-portal/src/main/resources/static/scripts/controller/GlobalSearchValueController.js new file mode 100644 index 00000000000..ee720aacc58 --- /dev/null +++ b/apollo-portal/src/main/resources/static/scripts/controller/GlobalSearchValueController.js @@ -0,0 +1,273 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +global_search_value_module.controller('GlobalSearchValueController', + ['$scope', '$window', '$translate', 'toastr', 'AppUtil', 'GlobalSearchValueService', 'PermissionService', GlobalSearchValueController]); + +function GlobalSearchValueController($scope, $window, $translate, toastr, AppUtil, GlobalSearchValueService, PermissionService) { + + $scope.allItemInfo = []; + $scope.pageItemInfo = []; + $scope.itemInfoSearchKey = ''; + $scope.itemInfoSearchValue = ''; + $scope.needToBeHighlightedKey = ''; + $scope.needToBeHighlightedValue = ''; + $scope.isShowHighlightKeyword = []; + $scope.isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword = []; + $scope.isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword = []; + $scope.isAllItemInfoDisplayValueInARow = []; + $scope.isPageItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword = []; + $scope.isPageItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword = []; + $scope.isPageItemInfoDisplayValueInARow = []; + $scope.currentPage = 1; + $scope.pageSize = '10'; + $scope.totalItems = 0; + $scope.totalPages = 0; + $scope.pagesArray = []; + $scope.tempKey = ''; + $scope.tempValue = ''; + + $scope.getItemInfoByKeyAndValue = getItemInfoByKeyAndValue; + $scope.highlightKeyword = highlightKeyword; + $scope.jumpToTheEditingPage = jumpToTheEditingPage; + $scope.isShowAllValue = isShowAllValue; + $scope.convertPageSizeToInt = convertPageSizeToInt; + $scope.changePage = changePage; + $scope.getPagesArray = getPagesArray; + $scope.determineDisplayKeyOrValueWithoutShowHighlightKeyword = determineDisplayKeyOrValueWithoutShowHighlightKeyword; + $scope.determineDisplayValueInARow = determineDisplayValueInARow; + + init(); + function init() { + initPermission(); + } + + function initPermission() { + PermissionService.has_root_permission() + .then(function (result) { + $scope.isRootUser = result.hasPermission; + }); + } + + function getItemInfoByKeyAndValue(itemInfoSearchKey, itemInfoSearchValue) { + $scope.currentPage = 1; + $scope.itemInfoSearchKey = itemInfoSearchKey || ''; + $scope.itemInfoSearchValue = itemInfoSearchValue || ''; + $scope.allItemInfo = []; + $scope.pageItemInfo = []; + $scope.isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword = []; + $scope.isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword = []; + $scope.isAllItemInfoDisplayValueInARow = []; + $scope.isPageItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword = []; + $scope.isPageItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword = []; + $scope.isPageItemInfoDisplayValueInARow = []; + $scope.tempKey = itemInfoSearchKey || ''; + $scope.tempValue = itemInfoSearchValue || ''; + $scope.isShowHighlightKeyword = []; + GlobalSearchValueService.findItemInfoByKeyAndValue($scope.itemInfoSearchKey, $scope.itemInfoSearchValue) + .then(handleSuccess).catch(handleError); + function handleSuccess(result) { + let allItemInfo = []; + let isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword = []; + let isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword = []; + let isAllItemInfoDisplayValueInARow = []; + if(($scope.itemInfoSearchKey === '') && !($scope.itemInfoSearchValue === '')){ + $scope.needToBeHighlightedValue = $scope.itemInfoSearchValue; + $scope.needToBeHighlightedKey = ''; + result.body.forEach((itemInfo, index) => { + allItemInfo.push(itemInfo); + isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword[index] = '0'; + isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword[index] = determineDisplayKeyOrValueWithoutShowHighlightKeyword(itemInfo.value, itemInfoSearchValue); + isAllItemInfoDisplayValueInARow[index] = determineDisplayValueInARow(itemInfo.value, itemInfoSearchValue); + }); + }else if(!($scope.itemInfoSearchKey === '') && ($scope.itemInfoSearchValue === '')){ + $scope.needToBeHighlightedKey = $scope.itemInfoSearchKey; + $scope.needToBeHighlightedValue = ''; + result.body.forEach((itemInfo, index) => { + allItemInfo.push(itemInfo); + isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword[index] = determineDisplayKeyOrValueWithoutShowHighlightKeyword(itemInfo.key, itemInfoSearchKey); + isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword[index] = '0'; + }); + }else{ + $scope.needToBeHighlightedKey = $scope.itemInfoSearchKey; + $scope.needToBeHighlightedValue = $scope.itemInfoSearchValue; + result.body.forEach((itemInfo, index) => { + allItemInfo.push(itemInfo); + isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword[index] = determineDisplayKeyOrValueWithoutShowHighlightKeyword(itemInfo.value, itemInfoSearchValue); + isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword[index] = determineDisplayKeyOrValueWithoutShowHighlightKeyword(itemInfo.key, itemInfoSearchKey); + isAllItemInfoDisplayValueInARow[index] = determineDisplayValueInARow(itemInfo.value, itemInfoSearchValue); + }); + } + $scope.totalItems = allItemInfo.length; + $scope.allItemInfo = allItemInfo; + $scope.totalPages = Math.ceil($scope.totalItems / parseInt($scope.pageSize, 10)); + const startIndex = ($scope.currentPage - 1) * parseInt($scope.pageSize, 10); + const endIndex = Math.min(startIndex + parseInt($scope.pageSize, 10), allItemInfo.length); + $scope.pageItemInfo = allItemInfo.slice(startIndex, endIndex); + $scope.isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword = isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword; + $scope.isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword = isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword; + $scope.isAllItemInfoDisplayValueInARow = isAllItemInfoDisplayValueInARow; + $scope.isPageItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword = isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword.slice(startIndex, endIndex); + $scope.isPageItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword = isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword.slice(startIndex, endIndex); + $scope.isPageItemInfoDisplayValueInARow = isAllItemInfoDisplayValueInARow.slice(startIndex, endIndex); + getPagesArray(); + if(result.hasMoreData){ + toastr.warning(result.message, $translate.instant('Item.GlobalSearch.Tips')); + } + } + + function handleError(error) { + $scope.itemInfo = []; + toastr.error(AppUtil.errorMsg(error), $translate.instant('Item.GlobalSearchSystemError')); + } + } + + function convertPageSizeToInt() { + getItemInfoByKeyAndValue($scope.tempKey, $scope.tempValue); + } + + function changePage(page) { + if (page >= 1 && page <= $scope.totalPages) { + $scope.currentPage = page; + $scope.isShowHighlightKeyword = []; + $scope.isPageItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword = []; + $scope.isPageItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword = []; + $scope.isPageItemInfoDisplayValueInARow = []; + $scope.itemInfoSearchKey = $scope.tempKey; + $scope.itemInfoSearchValue = $scope.tempValue; + const startIndex = ($scope.currentPage - 1)* parseInt($scope.pageSize, 10); + const endIndex = Math.min(startIndex + parseInt($scope.pageSize, 10), $scope.totalItems); + $scope.pageItemInfo = $scope.allItemInfo.slice(startIndex, endIndex); + $scope.isPageItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword = $scope.isAllItemInfoDirectlyDisplayValueWithoutShowHighlightKeyword.slice(startIndex, endIndex); + $scope.isPageItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword = $scope.isAllItemInfoDirectlyDisplayKeyWithoutShowHighlightKeyword.slice(startIndex, endIndex); + $scope.isPageItemInfoDisplayValueInARow = $scope.isAllItemInfoDisplayValueInARow.slice(startIndex, endIndex); + getPagesArray(); + } + } + + function getPagesArray() { + const pageRange = 2; + let pagesArray = []; + let currentPage = $scope.currentPage; + let totalPages = $scope.totalPages; + if (totalPages <= (pageRange * 2) + 4) { + for (let i = 1; i <= totalPages; i++) { + pagesArray.push(i); + } + } else { + if (currentPage <= (pageRange + 2)) { + for (let i = 1; i <= pageRange * 2 + 2; i++) { + pagesArray.push(i); + } + pagesArray.push('...'); + pagesArray.push(totalPages); + } else if (currentPage >= (totalPages - (pageRange + 1))) { + for (let i = totalPages - pageRange * 2 - 1 ; i <= totalPages; i++) { + pagesArray.push(i); + } + pagesArray.unshift('...'); + pagesArray.unshift(1); + } else { + for (let i = (currentPage - pageRange); i <= currentPage + pageRange; i++) { + pagesArray.push(i); + } + pagesArray.unshift('...'); + pagesArray.unshift(1); + pagesArray.push('...'); + pagesArray.push(totalPages); + } + } + $scope.pagesArray = pagesArray; + } + + function determineDisplayValueInARow(value, highlight) { + var valueColumn = document.getElementById('valueColumn'); + var testElement = document.createElement('span'); + setupTestElement(testElement, valueColumn); + testElement.innerText = value; + document.body.appendChild(testElement); + const position = determinePosition(value, highlight); + let displayValue = '0'; + if (testElement.scrollWidth > testElement.offsetWidth) { + displayValue = position; + } else { + if (testElement.scrollWidth === testElement.offsetWidth) { + return '0'; + } + switch (position) { + case '1': + testElement.innerText = value + '...' + '| ' + $translate.instant('Global.Expand'); + break; + case '2': + testElement.innerText = '...' + value + '| ' + $translate.instant('Global.Expand'); + break; + case '3': + testElement.innerText = '...' + value + '...' + '| ' + $translate.instant('Global.Expand'); + break; + default: + return '0'; + } + if (testElement.scrollWidth === testElement.offsetWidth) { + displayValue = '0'; + } else { + displayValue = position; + } + } + document.body.removeChild(testElement); + return displayValue; + } + + function setupTestElement(element, valueColumn) { + element.style.visibility = 'hidden'; + element.style.position = 'absolute'; + element.style.whiteSpace = 'nowrap'; + element.style.display = 'inline-block'; + element.style.fontFamily = '"Open Sans", sans-serif'; + const devicePixelRatio = window.devicePixelRatio; + const zoomLevel = Math.round((window.outerWidth / window.innerWidth) * 100) / 100; + element.style.fontSize = 13 * devicePixelRatio * zoomLevel + 'px'; + element.style.padding = 8 * devicePixelRatio * zoomLevel + 'px'; + element.style.width = valueColumn.offsetWidth * devicePixelRatio * zoomLevel + 'px'; + } + + function determinePosition(value, highlight) { + const position = value.indexOf(highlight); + if (position === -1) return '-1'; + if (position === 0) return '1'; + if (position + highlight.length === value.length) return '2'; + return "3"; + } + + function determineDisplayKeyOrValueWithoutShowHighlightKeyword(keyorvalue, highlight) { + return keyorvalue === highlight ? '0' : '-1'; + } + + function jumpToTheEditingPage(appid,env,cluster){ + let url = AppUtil.prefixPath() + "/config.html#/appid=" + appid + "&" +"env=" + env + "&" + "cluster=" + cluster; + window.open(url, '_blank'); + } + + function highlightKeyword(fulltext,keyword) { + if (!keyword || keyword.length === 0) return fulltext; + let regex = new RegExp("(" + keyword + ")", "g"); + return fulltext.replace(regex, '$1'); + } + + function isShowAllValue(index){ + $scope.isShowHighlightKeyword[index] = !$scope.isShowHighlightKeyword[index]; + } + +} diff --git a/apollo-portal/src/main/resources/static/scripts/services/GlobalSearchValueService.js b/apollo-portal/src/main/resources/static/scripts/services/GlobalSearchValueService.js new file mode 100644 index 00000000000..52a345449b7 --- /dev/null +++ b/apollo-portal/src/main/resources/static/scripts/services/GlobalSearchValueService.js @@ -0,0 +1,40 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +appService.service('GlobalSearchValueService', ['$resource', '$q', 'AppUtil', function ($resource, $q, AppUtil) { + let global_search_resource = $resource('', {}, { + get_item_Info_by_key_and_Value: { + isArray: false, + method: 'GET', + url: AppUtil.prefixPath() + '/global-search/item-info/by-key-or-value', + params: { + key: 'key', + value: 'value' + } + } + }); + return { + findItemInfoByKeyAndValue:function (key,value){ + let d = $q.defer(); + global_search_resource.get_item_Info_by_key_and_Value({key: key,value: value},function (result) { + d.resolve(result); + }, function (error) { + d.reject(error); + }); + return d.promise; + } + } +}]); diff --git a/apollo-portal/src/main/resources/static/views/common/nav.html b/apollo-portal/src/main/resources/static/views/common/nav.html index c2f73b213dd..a78f2b68dd0 100644 --- a/apollo-portal/src/main/resources/static/views/common/nav.html +++ b/apollo-portal/src/main/resources/static/views/common/nav.html @@ -66,6 +66,7 @@
  • {{'Common.Nav.SystemInfo' | translate }}
  • {{'Common.Nav.ConfigExport' | translate }}
  • {{'ApolloAuditLog.Title' | translate }}
    • +
  • {{'Global.Title' | translate }}
    • diff --git a/apollo-portal/src/test/java/com/ctrip/framework/apollo/portal/controller/GlobalSearchControllerTest.java b/apollo-portal/src/test/java/com/ctrip/framework/apollo/portal/controller/GlobalSearchControllerTest.java new file mode 100644 index 00000000000..03231dcc59e --- /dev/null +++ b/apollo-portal/src/test/java/com/ctrip/framework/apollo/portal/controller/GlobalSearchControllerTest.java @@ -0,0 +1,135 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package com.ctrip.framework.apollo.portal.controller; + +/** + * @author hujiyuan 2024-08-10 + */ + +import com.ctrip.framework.apollo.common.http.SearchResponseEntity; +import com.ctrip.framework.apollo.portal.component.config.PortalConfig; +import com.ctrip.framework.apollo.portal.entity.vo.ItemInfo; +import com.ctrip.framework.apollo.portal.service.GlobalSearchService; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.MockitoJUnitRunner; +import org.springframework.http.MediaType; +import org.springframework.test.web.servlet.MockMvc; +import org.springframework.test.web.servlet.request.MockMvcRequestBuilders; +import org.springframework.test.web.servlet.setup.MockMvcBuilders; + +import java.util.*; + +import static org.mockito.Mockito.*; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; + +@RunWith(MockitoJUnitRunner.class) +public class GlobalSearchControllerTest { + + private MockMvc mockMvc; + + @Mock + private PortalConfig portalConfig; + + @Mock + private GlobalSearchService globalSearchService; + + @InjectMocks + private GlobalSearchController globalSearchController; + + private final int perEnvSearchMaxResults = 200; + + @Before + public void setUp() { + when(portalConfig.getPerEnvSearchMaxResults()).thenReturn(perEnvSearchMaxResults); + mockMvc = MockMvcBuilders.standaloneSetup(globalSearchController).build(); + } + + @Test + public void testGet_ItemInfo_BySearch_WithKeyAndValueAndActiveEnvs_ReturnEmptyItemInfos() throws Exception { + when(globalSearchService.getAllEnvItemInfoBySearch(anyString(), anyString(),eq(0),eq(perEnvSearchMaxResults))).thenReturn(SearchResponseEntity.ok(new ArrayList<>())); + mockMvc.perform(MockMvcRequestBuilders.get("/global-search/item-info/by-key-or-value") + .contentType(MediaType.APPLICATION_JSON) + .param("key", "query-key") + .param("value", "query-value")) + .andExpect(status().isOk()) + .andExpect(content().json("{\"body\":[],\"hasMoreData\":false,\"message\":\"OK\",\"code\":200}")); + verify(portalConfig,times(1)).getPerEnvSearchMaxResults(); + verify(globalSearchService,times(1)).getAllEnvItemInfoBySearch(anyString(), anyString(),eq(0),eq(perEnvSearchMaxResults)); + } + + @Test + public void testGet_ItemInfo_BySearch_WithKeyAndValueAndActiveEnvs_ReturnExpectedItemInfos_ButOverPerEnvLimit() throws Exception { + List allEnvMockItemInfos = new ArrayList<>(); + allEnvMockItemInfos.add(new ItemInfo("appid1","env1","cluster1","namespace1","query-key","query-value")); + allEnvMockItemInfos.add(new ItemInfo("appid2","env2","cluster2","namespace2","query-key","query-value")); + when(globalSearchService.getAllEnvItemInfoBySearch(eq("query-key"), eq("query-value"),eq(0),eq(perEnvSearchMaxResults))).thenReturn(SearchResponseEntity.okWithMessage(allEnvMockItemInfos,"In DEV , PRO , more than "+perEnvSearchMaxResults+" items found (Exceeded the maximum search quantity for a single environment). Please enter more precise criteria to narrow down the search scope.")); + mockMvc.perform(MockMvcRequestBuilders.get("/global-search/item-info/by-key-or-value") + .contentType(MediaType.APPLICATION_JSON) + .param("key", "query-key") + .param("value", "query-value")) + .andExpect(status().isOk()) + .andExpect(content().json("{\"body\":[" + + " { \"appId\": \"appid1\",\n" + + " \"envName\": \"env1\",\n" + + " \"clusterName\": \"cluster1\",\n" + + " \"namespaceName\": \"namespace1\",\n" + + " \"key\": \"query-key\",\n" + + " \"value\": \"query-value\"}," + + " { \"appId\": \"appid2\",\n" + + " \"envName\": \"env2\",\n" + + " \"clusterName\": \"cluster2\",\n" + + " \"namespaceName\": \"namespace2\",\n" + + " \"key\": \"query-key\",\n" + + " \"value\": \"query-value\"}],\"hasMoreData\":true,\"message\":\"In DEV , PRO , more than 200 items found (Exceeded the maximum search quantity for a single environment). Please enter more precise criteria to narrow down the search scope.\",\"code\":200}")); + verify(portalConfig,times(1)).getPerEnvSearchMaxResults(); + verify(globalSearchService, times(1)).getAllEnvItemInfoBySearch(eq("query-key"), eq("query-value"),eq(0),eq(perEnvSearchMaxResults)); + } + + @Test + public void testGet_ItemInfo_BySearch_WithKeyAndValueAndActiveEnvs_ReturnExpectedItemInfos() throws Exception { + List allEnvMockItemInfos = new ArrayList<>(); + allEnvMockItemInfos.add(new ItemInfo("appid1","env1","cluster1","namespace1","query-key","query-value")); + allEnvMockItemInfos.add(new ItemInfo("appid2","env2","cluster2","namespace2","query-key","query-value")); + when(globalSearchService.getAllEnvItemInfoBySearch(eq("query-key"), eq("query-value"),eq(0),eq(perEnvSearchMaxResults))).thenReturn(SearchResponseEntity.ok(allEnvMockItemInfos)); + + mockMvc.perform(MockMvcRequestBuilders.get("/global-search/item-info/by-key-or-value") + .contentType(MediaType.APPLICATION_JSON) + .param("key", "query-key") + .param("value", "query-value")) + .andExpect(status().isOk()) + .andExpect(content().json("{\"body\":[" + + " { \"appId\": \"appid1\",\n" + + " \"envName\": \"env1\",\n" + + " \"clusterName\": \"cluster1\",\n" + + " \"namespaceName\": \"namespace1\",\n" + + " \"key\": \"query-key\",\n" + + " \"value\": \"query-value\"}," + + " { \"appId\": \"appid2\",\n" + + " \"envName\": \"env2\",\n" + + " \"clusterName\": \"cluster2\",\n" + + " \"namespaceName\": \"namespace2\",\n" + + " \"key\": \"query-key\",\n" + + " \"value\": \"query-value\"}],\"hasMoreData\":false,\"message\":\"OK\",\"code\":200}")); + verify(globalSearchService, times(1)).getAllEnvItemInfoBySearch(eq("query-key"), eq("query-value"),eq(0),eq(perEnvSearchMaxResults)); + } + +} diff --git a/apollo-portal/src/test/java/com/ctrip/framework/apollo/portal/service/GlobalSearchServiceTest.java b/apollo-portal/src/test/java/com/ctrip/framework/apollo/portal/service/GlobalSearchServiceTest.java new file mode 100644 index 00000000000..661a692447d --- /dev/null +++ b/apollo-portal/src/test/java/com/ctrip/framework/apollo/portal/service/GlobalSearchServiceTest.java @@ -0,0 +1,127 @@ +/* + * Copyright 2024 Apollo Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package com.ctrip.framework.apollo.portal.service; + +/** + * @author hujiyuan 2024-08-10 + */ + +import com.ctrip.framework.apollo.common.dto.ItemInfoDTO; +import com.ctrip.framework.apollo.common.dto.PageDTO; +import com.ctrip.framework.apollo.common.http.SearchResponseEntity; +import com.ctrip.framework.apollo.portal.api.AdminServiceAPI; +import com.ctrip.framework.apollo.portal.component.PortalSettings; +import com.ctrip.framework.apollo.portal.entity.vo.ItemInfo; +import com.ctrip.framework.apollo.portal.environment.Env; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.Mockito; +import org.mockito.junit.MockitoJUnitRunner; +import org.springframework.data.domain.PageRequest; + +import java.util.ArrayList; +import java.util.List; + +import static org.junit.Assert.assertEquals; +import static org.mockito.ArgumentMatchers.*; +import static org.mockito.Mockito.*; +import static org.mockito.Mockito.when; + +@RunWith(MockitoJUnitRunner.class) +public class GlobalSearchServiceTest { + + @Mock + private AdminServiceAPI.ItemAPI itemAPI; + + @Mock + private PortalSettings portalSettings; + + @InjectMocks + private GlobalSearchService globalSearchService; + + private final List activeEnvs = new ArrayList<>(); + + @Before + public void setUp() { + when(portalSettings.getActiveEnvs()).thenReturn(activeEnvs); + } + + @Test + public void testGet_PerEnv_ItemInfo_BySearch_withKeyAndValue_ReturnExpectedItemInfos() { + activeEnvs.add(Env.DEV); + activeEnvs.add(Env.PRO); + + ItemInfoDTO itemInfoDTO = new ItemInfoDTO("TestApp","TestCluster","TestNamespace","TestKey","TestValue"); + List mockItemInfoDTOs = new ArrayList<>(); + mockItemInfoDTOs.add(itemInfoDTO); + Mockito.when(itemAPI.getPerEnvItemInfoBySearch(any(Env.class), eq("TestKey"), eq("TestValue"), eq(0), eq(1))).thenReturn(new PageDTO<>(mockItemInfoDTOs, PageRequest.of(0, 1), 1L)); + SearchResponseEntity> mockItemInfos = globalSearchService.getAllEnvItemInfoBySearch("TestKey", "TestValue", 0, 1); + assertEquals(2, mockItemInfos.getBody().size()); + + List devMockItemInfos = new ArrayList<>(); + List proMockItemInfos = new ArrayList<>(); + List allEnvMockItemInfos = new ArrayList<>(); + devMockItemInfos.add(new ItemInfo("TestApp", Env.DEV.getName(), "TestCluster", "TestNamespace", "TestKey", "TestValue")); + proMockItemInfos.add(new ItemInfo("TestApp", Env.PRO.getName(), "TestCluster", "TestNamespace", "TestKey", "TestValue")); + allEnvMockItemInfos.addAll(devMockItemInfos); + allEnvMockItemInfos.addAll(proMockItemInfos); + + verify(itemAPI,times(2)).getPerEnvItemInfoBySearch(any(Env.class), eq("TestKey"), eq("TestValue"), eq(0), eq(1)); + verify(portalSettings,times(1)).getActiveEnvs(); + assertEquals(allEnvMockItemInfos.toString(), mockItemInfos.getBody().toString()); + } + + @Test + public void testGet_PerEnv_ItemInfo_withKeyAndValue_BySearch_ReturnEmptyItemInfos() { + activeEnvs.add(Env.DEV); + activeEnvs.add(Env.PRO); + Mockito.when(itemAPI.getPerEnvItemInfoBySearch(any(Env.class), anyString(), anyString(), eq(0), eq(1))) + .thenReturn(new PageDTO<>(new ArrayList<>(), PageRequest.of(0, 1), 0L)); + SearchResponseEntity> result = globalSearchService.getAllEnvItemInfoBySearch("NonExistentKey", "NonExistentValue", 0, 1); + assertEquals(0, result.getBody().size()); + } + + @Test + public void testGet_PerEnv_ItemInfo_BySearch_withKeyAndValue_ReturnExpectedItemInfos_ButOverPerEnvLimit() { + activeEnvs.add(Env.DEV); + activeEnvs.add(Env.PRO); + + ItemInfoDTO itemInfoDTO = new ItemInfoDTO("TestApp","TestCluster","TestNamespace","TestKey","TestValue"); + List mockItemInfoDTOs = new ArrayList<>(); + mockItemInfoDTOs.add(itemInfoDTO); + Mockito.when(itemAPI.getPerEnvItemInfoBySearch(any(Env.class), eq("TestKey"), eq("TestValue"), eq(0), eq(1))).thenReturn(new PageDTO<>(mockItemInfoDTOs, PageRequest.of(0, 1), 2L)); + SearchResponseEntity> mockItemInfos = globalSearchService.getAllEnvItemInfoBySearch("TestKey", "TestValue", 0, 1); + assertEquals(2, mockItemInfos.getBody().size()); + + List devMockItemInfos = new ArrayList<>(); + List proMockItemInfos = new ArrayList<>(); + List allEnvMockItemInfos = new ArrayList<>(); + devMockItemInfos.add(new ItemInfo("TestApp", Env.DEV.getName(), "TestCluster", "TestNamespace", "TestKey", "TestValue")); + proMockItemInfos.add(new ItemInfo("TestApp", Env.PRO.getName(), "TestCluster", "TestNamespace", "TestKey", "TestValue")); + allEnvMockItemInfos.addAll(devMockItemInfos); + allEnvMockItemInfos.addAll(proMockItemInfos); + String message = "In DEV , PRO , more than 1 items found (Exceeded the maximum search quantity for a single environment). Please enter more precise criteria to narrow down the search scope."; + verify(itemAPI,times(2)).getPerEnvItemInfoBySearch(any(Env.class), eq("TestKey"), eq("TestValue"), eq(0), eq(1)); + verify(portalSettings,times(1)).getActiveEnvs(); + assertEquals(allEnvMockItemInfos.toString(), mockItemInfos.getBody().toString()); + assertEquals(message, mockItemInfos.getMessage()); + } + +} diff --git a/changes/changes-2.4.0.md b/changes/changes-2.4.0.md new file mode 100644 index 00000000000..615d7c1d7a4 --- /dev/null +++ b/changes/changes-2.4.0.md @@ -0,0 +1,13 @@ +Changes by Version +================== +Release Notes. + +Apollo 2.4.0 + +------------------ +* [Update the server config link in system info page](https://github.com/apolloconfig/apollo/pull/5204) +* [Feature support portal restTemplate Client connection pool config](https://github.com/apolloconfig/apollo/pull/5200) +* [Feature added the ability for administrators to globally search for Value](https://github.com/apolloconfig/apollo/pull/5182) + +------------------ +All issues and pull requests are [here](https://github.com/apolloconfig/apollo/milestone/15?closed=1) diff --git a/docs/en/README.md b/docs/en/README.md index 46fb251756f..d76c4540aa2 100644 --- a/docs/en/README.md +++ b/docs/en/README.md @@ -37,6 +37,10 @@ Demo Environment: * **Grayscale release** * Support grayscale configuration release, for example, after clicking release, it will only take effect for some application instances. After a period of observation, we could push the configurations to all application instances if there is no problem +- **Global Search Configuration Items** + - A fuzzy search of the key and value of a configuration item finds in which application, environment, cluster, namespace the configuration item with the corresponding value is used + - It is easy for administrators and SRE roles to quickly and easily find and change the configuration values of resources by highlighting, paging and jumping through configurations + * **Authorization management, release approval and operation audit** * Great authorization mechanism is designed for applications and configurations management, and the management of configurations is divided into two operations: editing and publishing, therefore greatly reducing human errors * All operations have audit logs for easy tracking of problems diff --git a/docs/en/deployment/distributed-deployment-guide.md b/docs/en/deployment/distributed-deployment-guide.md index 3251d4c5d5e..4dbe8dd7586 100644 --- a/docs/en/deployment/distributed-deployment-guide.md +++ b/docs/en/deployment/distributed-deployment-guide.md @@ -769,9 +769,9 @@ apollo.service.registry.cluster=same name with apollo Cluster ``` 2. (optional) If you want to customize Config Service and Admin Service's uri for Client, -for example when deploying on the intranet, -if you don't want to expose the intranet ip, -you can add a property in `config/application-github.properties` of the Config Service and Admin Service installation package + for example when deploying on the intranet, + if you don't want to expose the intranet ip, + you can add a property in `config/application-github.properties` of the Config Service and Admin Service installation package ```properties apollo.service.registry.uri=http://your-ip-or-domain:${server.port}/ ``` @@ -1447,6 +1447,14 @@ The default is true, which makes it easy to quickly search for configurations by If set to false, this feature is disabled +### 3.1.14 apollo.portal.search.perEnvMaxResults - set the Administrator Tool-Global Search for Value function's maximum number of search results for a single individual environment + +> For versions 2.4.0 and above + +Default is 200, which means that each environment will return up to 200 results in a single search operation. + +Modifying this parameter may affect the performance of the search function, so before modifying it, you should conduct sufficient testing and adjust the value of `apollo.portal.search.perEnvMaxResults` appropriately according to the actual business requirements and system resources to balance the performance and the number of search results. + ## 3.2 Adjusting ApolloConfigDB configuration Configuration items are uniformly stored in the ApolloConfigDB.ServerConfig table. It should be noted that each environment's ApolloConfigDB.ServerConfig needs to be configured separately, and the modification takes effect in real time for one minute afterwards. diff --git a/docs/en/design/apollo-design.md b/docs/en/design/apollo-design.md index f078c1881d2..0061f696f20 100644 --- a/docs/en/design/apollo-design.md +++ b/docs/en/design/apollo-design.md @@ -130,7 +130,7 @@ Why do we use Eureka as a service registry instead of the traditional zk and etc ### 1.3.2 Admin Service * Provide configuration management interface -* Provides interfaces for configuration modification, publishing, etc. +* Provides interfaces for configuration modification, publishing, retrieval, etc. * Interface service object is Portal ### 1.3.3 Meta Server diff --git a/docs/en/design/apollo-introduction.md b/docs/en/design/apollo-introduction.md index a001f5a3a81..f846e6be867 100644 --- a/docs/en/design/apollo-introduction.md +++ b/docs/en/design/apollo-introduction.md @@ -78,7 +78,13 @@ It is precisely based on the particularity of configuration that Apollo has been * **Client configuration information monitoring** * You can easily see which instances the configuration is being used on the interface +**Global Search Configuration Items** + +- A fuzzy search of the key and value of a configuration item finds in which application, environment, cluster, namespace the configuration item with the corresponding value is used +- It is easy for administrators and SRE roles to quickly and easily find and change the configuration values of resources by highlighting, paging and jumping through configurations + **Java and .Net native clients available** + * Provides native clients of Java and .Net for easy application integration * Support Spring Placeholder, Annotation and Spring Boot's ConfigurationProperties for easy application use (requires Spring 3.1.1+) * Also provides Http interface, non-Java and .Net applications can also be easily used diff --git a/docs/en/images/Configuration query-Non properties.png b/docs/en/images/Configuration query-Non properties.png new file mode 100644 index 00000000000..1a355074cbc Binary files /dev/null and b/docs/en/images/Configuration query-Non properties.png differ diff --git a/docs/en/images/Configuration query-properties.png b/docs/en/images/Configuration query-properties.png new file mode 100644 index 00000000000..2f8fd771109 Binary files /dev/null and b/docs/en/images/Configuration query-properties.png differ diff --git a/docs/en/images/System-parameterization-of-global-search-configuration-items.png b/docs/en/images/System-parameterization-of-global-search-configuration-items.png new file mode 100644 index 00000000000..4f47b58cd7b Binary files /dev/null and b/docs/en/images/System-parameterization-of-global-search-configuration-items.png differ diff --git a/docs/en/portal/apollo-user-guide.md b/docs/en/portal/apollo-user-guide.md index cb84a8e6faa..56674a741fa 100644 --- a/docs/en/portal/apollo-user-guide.md +++ b/docs/en/portal/apollo-user-guide.md @@ -133,6 +133,20 @@ The rollback mechanism here is similar to the release system, where the rollback The rollback in Apollo is a similar mechanism. Clicking rollback rolls back the configuration published to the client to the previous published version, which means that the configuration read by the client will be restored to the previous version, but the configuration in the edited state on the page will not be rolled back, so that the developer can re-publish after fixing the configuration. +## 1.7 Configuration queries (administrator privileges) + +After a configuration has been added or modified, the administrator user can make a query for the configuration item it belongs to as well as jump to modifications by going to the `Administrator Tools - Global Search for Value` page. + +The query here is a fuzzy search, where at least one of the key and value of the configuration item is searched to find out in which application, environment, cluster, namespace the configuration is used. + +- Properties format configuration can be retrieved directly from the key and value + +![Configuration query-properties](../images/Configuration query-properties.png) + +- xml, json, yml, yaml, txt and other formats configuration, because the storage of content-value storage, so you can key = content, value = configuration item content, retrieval + +![Configuration query-Non properties](../images/Configuration query-Non properties.png) + # II. Public component access guide ## 2.1 Difference between public components and common applications @@ -482,6 +496,19 @@ Apollo has added an access key mechanism since version 1.6.0, so that only authe 3. Client-side [configure access key](en/client/java-sdk-user-guide?id=_1244-configuring-access-keys) . +## 6.3 System parameterization of global search configuration items + +Starting from version 2.4.0, apollo-portal adds the ability to globally search for configuration items by fuzzy retrieval of the key and value of a configuration item to find out which application, environment, cluster, or namespace the configuration item with the corresponding value is used in. In order to prevent memory overflow (OOM) problems when performing global view searches of configuration items, we introduce a system parameter `apollo.portal.search.perEnvMaxResults`, which is used to limit the number of maximum search results per environment configuration item in a single search. By default, this value is set to `200`, but administrators can adjust it to suit their actual needs. + +**Setting method：** + +1. Log in to the Apollo Configuration Center interface with a super administrator account. +2. Just go to the `Administrator Tools - System Parameters` page and add or modify the `apollo.portal.search.perEnvMaxResults` configuration item. + +Please note that modifications to system parameters may affect the performance of the search function, so you should perform adequate testing and ensure that you understand exactly what the parameters do before making changes. + +![System-parameterization-of-global-search-configuration-items](../images/System-parameterization-of-global-search-configuration-items.png) + # VII. Best practices ## 7.1 Security Related diff --git a/docs/zh/README.md b/docs/zh/README.md index e54c216f0a3..09c04f85647 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -40,6 +40,10 @@ Java客户端不依赖任何框架，能够运行于所有Java运行时环境， * **灰度发布** * 支持配置的灰度发布，比如点了发布后，只对部分应用实例生效，等观察一段时间没问题后再推给所有应用实例。 +- **配置项的全局视角搜索** + - 通过对配置项的key与value进行的模糊检索，找到拥有对应值的配置项在哪个应用、环境、集群、命名空间中被使用。 + - 通过高亮显示、分页与跳转配置等操作，便于让管理员以及SRE角色快速、便捷地找到与更改资源的配置值。 + * **权限管理、发布审核、操作审计** * 应用和配置的管理都有完善的权限管理机制，对配置的管理还分为了编辑和发布两个环节，从而减少人为的错误。 * 所有的操作都有审计日志，可以方便的追踪问题。 diff --git a/docs/zh/deployment/distributed-deployment-guide.md b/docs/zh/deployment/distributed-deployment-guide.md index 4d21e1c80aa..f83d62ac692 100644 --- a/docs/zh/deployment/distributed-deployment-guide.md +++ b/docs/zh/deployment/distributed-deployment-guide.md @@ -1392,6 +1392,14 @@ portal上“帮助”链接的地址，默认是Apollo github的wiki首页，可 如果设置为 false，则关闭此功能 +### 3.1.14 apollo.portal.search.perEnvMaxResults - 设置管理员工具-value的全局搜索功能单次单独环境最大搜索结果的数量 + +> 适用于2.4.0及以上版本 + +默认为200，意味着每个环境在单次搜索操作中最多返回200条结果 + +修改该参数可能会影响搜索功能的性能，因此在修改之前应该进行充分的测试，根据实际业务需求和系统资源情况，适当调整`apollo.portal.search.perEnvMaxResults`的值，以平衡性能和搜索结果的数量 + ## 3.2 调整ApolloConfigDB配置 配置项统一存储在ApolloConfigDB.ServerConfig表中，需要注意每个环境的ApolloConfigDB.ServerConfig都需要单独配置，修改完一分钟实时生效。 diff --git a/docs/zh/design/apollo-design.md b/docs/zh/design/apollo-design.md index e2724800c5f..7cd1b49b2c0 100644 --- a/docs/zh/design/apollo-design.md +++ b/docs/zh/design/apollo-design.md @@ -136,7 +136,7 @@ sequenceDiagram ### 1.3.2 Admin Service * 提供配置管理接口 -* 提供配置修改、发布等接口 +* 提供配置修改、发布、检索等接口 * 接口服务对象为Portal ### 1.3.3 Meta Server diff --git a/docs/zh/design/apollo-introduction.md b/docs/zh/design/apollo-introduction.md index e05da35ec54..166af68c3a4 100644 --- a/docs/zh/design/apollo-introduction.md +++ b/docs/zh/design/apollo-introduction.md @@ -69,26 +69,30 @@ Apollo支持4个维度管理Key-Value格式的配置： * **灰度发布** * 支持配置的灰度发布，比如点了发布后，只对部分应用实例生效，等观察一段时间没问题后再推给所有应用实例 +* **配置项的全局视角搜索** + * 通过对配置项的key与value进行的模糊检索，找到拥有对应值的配置项在哪个应用、环境、集群、命名空间中被使用 + * 通过高亮显示、分页与跳转配置等操作，便于让管理员以及SRE角色快速、便捷地找到与更改资源的配置值 + * **权限管理、发布审核、操作审计** - * 应用和配置的管理都有完善的权限管理机制，对配置的管理还分为了编辑和发布两个环节，从而减少人为的错误。 - * 所有的操作都有审计日志，可以方便地追踪问题 + * 应用和配置的管理都有完善的权限管理机制，对配置的管理还分为了编辑和发布两个环节，从而减少人为的错误。 + * 所有的操作都有审计日志，可以方便地追踪问题 * **客户端配置信息监控** - * 可以在界面上方便地看到配置在被哪些实例使用 + * 可以在界面上方便地看到配置在被哪些实例使用 * **提供Java和.Net原生客户端** - * 提供了Java和.Net的原生客户端，方便应用集成 - * 支持Spring Placeholder, Annotation和Spring Boot的ConfigurationProperties，方便应用使用（需要Spring 3.1.1+） - * 同时提供了Http接口，非Java和.Net应用也可以方便地使用 + * 提供了Java和.Net的原生客户端，方便应用集成 + * 支持Spring Placeholder, Annotation和Spring Boot的ConfigurationProperties，方便应用使用（需要Spring 3.1.1+） + * 同时提供了Http接口，非Java和.Net应用也可以方便地使用 * **提供开放平台API** - * Apollo自身提供了比较完善的统一配置管理界面，支持多环境、多数据中心配置管理、权限、流程治理等特性。不过Apollo出于通用性考虑，不会对配置的修改做过多限制，只要符合基本的格式就能保存，不会针对不同的配置值进行针对性的校验，如数据库用户名、密码，Redis服务地址等 - * 对于这类应用配置，Apollo支持应用方通过开放平台API在Apollo进行配置的修改和发布，并且具备完善的授权和权限控制 + * Apollo自身提供了比较完善的统一配置管理界面，支持多环境、多数据中心配置管理、权限、流程治理等特性。不过Apollo出于通用性考虑，不会对配置的修改做过多限制，只要符合基本的格式就能保存，不会针对不同的配置值进行针对性的校验，如数据库用户名、密码，Redis服务地址等 + * 对于这类应用配置，Apollo支持应用方通过开放平台API在Apollo进行配置的修改和发布，并且具备完善的授权和权限控制 * **部署简单** - * 配置中心作为基础服务，可用性要求非常高，这就要求Apollo对外部依赖尽可能地少 - * 目前唯一的外部依赖是MySQL，所以部署非常简单，只要安装好Java和MySQL就可以让Apollo跑起来 - * Apollo还提供了打包脚本，一键就可以生成所有需要的安装包，并且支持自定义运行时参数 + * 配置中心作为基础服务，可用性要求非常高，这就要求Apollo对外部依赖尽可能地少 + * 目前唯一的外部依赖是MySQL，所以部署非常简单，只要安装好Java和MySQL就可以让Apollo跑起来 + * Apollo还提供了打包脚本，一键就可以生成所有需要的安装包，并且支持自定义运行时参数 # 3、Apollo at a glance diff --git a/docs/zh/images/Configuration query-Non properties.png b/docs/zh/images/Configuration query-Non properties.png new file mode 100644 index 00000000000..1a355074cbc Binary files /dev/null and b/docs/zh/images/Configuration query-Non properties.png differ diff --git a/docs/zh/images/Configuration query-properties.png b/docs/zh/images/Configuration query-properties.png new file mode 100644 index 00000000000..2f8fd771109 Binary files /dev/null and b/docs/zh/images/Configuration query-properties.png differ diff --git a/docs/zh/images/System-parameterization-of-global-search-configuration-items.png b/docs/zh/images/System-parameterization-of-global-search-configuration-items.png new file mode 100644 index 00000000000..4f47b58cd7b Binary files /dev/null and b/docs/zh/images/System-parameterization-of-global-search-configuration-items.png differ diff --git a/docs/zh/portal/apollo-user-guide.md b/docs/zh/portal/apollo-user-guide.md index 28ec5cc0fbc..ef42e470222 100644 --- a/docs/zh/portal/apollo-user-guide.md +++ b/docs/zh/portal/apollo-user-guide.md @@ -123,6 +123,20 @@ Apollo目前提供Java客户端，具体信息请点击[Java客户端使用文 Apollo中的回滚也是类似的机制，点击回滚后是将发布到客户端的配置回滚到上一个已发布版本，也就是说客户端读取到的配置会恢复到上一个版本，但页面上编辑状态的配置是不会回滚的，从而开发可以在修复配置后重新发布。 +## 1.7 配置查询（管理员权限） + +在配置添加或修改后，管理员用户可以通过进入 `管理员工具 - Value的全局搜索` 页面，来对配置项进行所属查询以及跳转修改。 + +这里的查询为模糊检索，通过对配置项的key与value至少一项进行检索，找到该配置在哪个应用、环境、集群、命名空间中被使用。 + +- properties格式配置可以直接通过对key与value进行检索 + +![Configuration query-properties](../images/Configuration query-properties.png) + +- xml、json、yml、yaml、txt等格式配置，由于存储时以content-value进行存储，故可以通过key=content、value=配置项内容，进行检索 + +![Configuration query-Non properties](../images/Configuration query-Non properties.png) + # 二、公共组件接入指南 ## 2.1 公共组件和普通应用的区别 @@ -226,9 +240,9 @@ Apollo目前提供Java客户端，具体信息请点击[Java客户端使用文 3. 关联成功后，页面会自动跳转到Namespace权限管理页面 1. 分配修改权限 -![namespace-permission-edit](https://cdn.jsdelivr.net/gh/apolloconfig/apollo@master/doc/images/namespace-permission-edit.png) + ![namespace-permission-edit](https://cdn.jsdelivr.net/gh/apolloconfig/apollo@master/doc/images/namespace-permission-edit.png) 2. 分配发布权限 -![namespace-publish-permission](https://cdn.jsdelivr.net/gh/apolloconfig/apollo@master/doc/images/namespace-publish-permission.png) + ![namespace-publish-permission](https://cdn.jsdelivr.net/gh/apolloconfig/apollo@master/doc/images/namespace-publish-permission.png) 4. 点击“返回”回到项目页面 @@ -448,13 +462,26 @@ Apollo目前提供Java客户端，具体信息请点击[Java客户端使用文 Apollo从1.6.0版本开始增加访问密钥机制，从而只有经过身份验证的客户端才能访问敏感配置。如果应用开启了访问密钥，客户端需要配置密钥，否则无法获取配置。 1. 项目管理员打开管理密钥页面 -![管理密钥入口](https://user-images.githubusercontent.com/837658/94990081-f4d3cd80-05ab-11eb-9470-fed5ec6de92e.png) + ![管理密钥入口](https://user-images.githubusercontent.com/837658/94990081-f4d3cd80-05ab-11eb-9470-fed5ec6de92e.png) 2. 为项目的每个环境生成访问密钥，注意默认是禁用的，建议在客户端都配置完成后再开启 -![密钥配置页面](https://user-images.githubusercontent.com/837658/94990150-788dba00-05ac-11eb-9a12-727fdb872e42.png) + ![密钥配置页面](https://user-images.githubusercontent.com/837658/94990150-788dba00-05ac-11eb-9a12-727fdb872e42.png) 3. 客户端侧[配置访问密钥](zh/client/java-sdk-user-guide#_1244-配置访问密钥) +## 6.3 全局搜索配置项的系统参数设置 + +从2.4.0版本开始，apollo-portal增加了全局搜索配置项的功能，通过对配置项的key与value进行的模糊检索，找到拥有对应值的配置项在哪个应用、环境、集群、命名空间中被使用。为了防止在进行配置项的全局视角搜索时出现内存溢出（OOM）的问题，我们引入了一个系统参数`apollo.portal.search.perEnvMaxResults`。这个参数用于限制每个环境配置项单次最大搜索结果的数量。默认情况下，这个值被设置为`200`，但管理员可以根据实际需求进行调整。 + +**设置方法：** + +1. 用超级管理员账号登录到Apollo配置中心的界面 +2. 进入`管理员工具 - 系统参数`页面新增或修改`apollo.portal.search.perEnvMaxResults`配置项即可 + +请注意，修改系统参数可能会影响搜索功能的性能，因此在修改之前应该进行充分的测试，并确保理解参数的具体作用。 + +![System-parameterization-of-global-search-configuration-items](../images/System-parameterization-of-global-search-configuration-items.png) + # 七、最佳实践 ## 7.1 安全相关