Document vector search

modules/howtos/examples/Search.java

@@ -16,6 +16,7 @@
 
 import static com.couchbase.client.java.search.SearchOptions.searchOptions;
 import java.util.HashMap;
+import java.util.List;
 import java.util.Map;
 import java.util.concurrent.atomic.AtomicInteger;
 import com.couchbase.client.core.error.CouchbaseException;
@@ -28,11 +29,14 @@
 import com.couchbase.client.java.kv.MutationState;
 import com.couchbase.client.java.search.HighlightStyle;
 import com.couchbase.client.java.search.SearchQuery;
+import com.couchbase.client.java.search.SearchRequest;
 import com.couchbase.client.java.search.facet.SearchFacet;
 import com.couchbase.client.java.search.result.ReactiveSearchResult;
 import com.couchbase.client.java.search.result.SearchResult;
 import com.couchbase.client.java.search.result.SearchRow;
 import com.couchbase.client.java.search.sort.SearchSort;
+import com.couchbase.client.java.search.vector.VectorQuery;
+import com.couchbase.client.java.search.vector.VectorSearch;
 import org.reactivestreams.Subscription;
 import reactor.core.publisher.BaseSubscriber;
 import reactor.core.publisher.Mono;
@@ -170,6 +174,48 @@ protected void hookOnNext(SearchRow row) {
       });
       // end::backpressure[]
     }
+
+    // This will come from an external source, such as an embeddings API.
+    float[] vectorQuery = null;
+    float[] anotherVectorQuery = null;
+
+    {
+      // tag::vector1[]
+      SearchRequest request = SearchRequest
+              .create(VectorSearch.create(VectorQuery.create("vector_field", vectorQuery)));
+
+      SearchResult result = scope.search("travel-sample-index", request);
+      // end::vector1[]
+    }
+
+    {
+      // tag::vector2[]
+      SearchRequest request = SearchRequest.create(SearchQuery.matchAll())
+              .vectorSearch(VectorSearch.create(VectorQuery.create("vector_field", vectorQuery)));
+
+      SearchResult result = scope.search("travel-sample-index", request);
+      // end::vector2[]
+    }
+
+    {
+      // tag::vector3[]
+      SearchRequest request = SearchRequest
+              .create(VectorSearch.create(List.of(
+                              VectorQuery.create("vector_field", vectorQuery).numCandidates(2).boost(0.3),
+                              VectorQuery.create("vector_field", anotherVectorQuery).numCandidates(5).boost(0.7)));
+
+      SearchResult result = scope.search("travel-sample-index", request);
+      // end::vector3[]
+    }
+
+    {
+      // tag::vector4[]
+      SearchRequest request = SearchRequest.create(SearchQuery.matchAll());
+
+      SearchResult result = scope.search("travel-sample-index", request);
+      // end::vector4[]
+    }
+
   }
 
   static void process(SearchRow value) {

modules/howtos/pages/full-text-searching-with-sdk.adoc

@@ -17,20 +17,27 @@ It uses natural language processing for indexing and querying documents, provide
 
 Some of the supported query-types include simple queries like Match and Term queries, range queries like Date Range and Numeric Range and compound queries for conjunctions, disjunctions and/or boolean queries.
 
+The Full Text Search service also supports vector search from Couchbase Server 7.6 onwards.
+
 == Getting Started
 
 After familiarizing yourself with how to create and query a Search index in the UI you can query it from the SDK. 
-Intentionally the API itself is very similar to the query and analytics ones, the main difference being that you cannot cast the resulting rows into a domain object directly but rather get a `SearchRow` returned. 
-The reason for this is that a search row ("hit") has more metadata associated with it than you potentially want to look at.
+
+There are two APIs for querying search: `cluster.searchQuery()`, and `cluster.search()`.
+Both are also available at the Scope level.
+
+The former API supports FTS queries (`SearchQuery`), while the latter additionally supports the `VectorSearch` added in 7.6.
+Most of this documentation will focus on the former API, as the latter is in @Stability.Volatile status.
+
+We will perform an FTS query here - see the <<vector search>> section for examples of that.
 
 [source,java]
 ----
 include::example$Search.java[tag=simple,indent=0]
 ----
 
 Let's break it down. 
-A Search query is always performed at the `Cluster` level, using the `searchQuery` method. 
-It takes the name of the index and the type of query  as required arguments and then allows to provide additional options if needed 
+The `searchQuery` API takes the name of the index and the type of query  as required arguments and then allows to provide additional options if needed
 (in the example above, no options are specified).
 
 Once a result returns you can iterate over the returned rows, and/or access the `SearchMetaData` associated with the query. 
@@ -41,13 +48,6 @@ If something goes wrong during the execution of the search query, a subclass of
 Exception in thread "main" com.couchbase.client.core.error.IndexNotFoundException: Index not found {"completed":true,"coreId":1,"httpStatus":400,"idempotent":true,"lastDispatchedFrom":"127.0.0.1:53818","lastDispatchedTo":"127.0.0.1:8094","requestId":3,"requestType":"SearchRequest","service":{"indexName":"unknown-index","type":"search"},"status":"INVALID_ARGS","timeoutMs":75000,"timings":{"dispatchMicros":18289,"totalMicros":1359398}}
 ----
 
-[NOTE]
-.Open Buckets and Cluster-Level Queries
-====
-If you are using a cluster older than Couchbase Server 6.5, it is required that there is at least one bucket open before performing a cluster-level query.
-If you fail to do so, the SDK will return a `FeatureNotAvailableException` with a descriptive error message asking you to open one.
-====
-
 == Search Queries
 
 The second mandatory argument in the example above used `SearchQuery.queryString("query")` to specify the query to run against the search index. 
@@ -109,6 +109,72 @@ This makes sure that even with partial data usually Search results are useable,
 so if you absolutely need to check if all partitions are present in the result double check the error 
 (and not only catch an exception on the query itself).
 
+== Scoped vs Global Indexes
+The FTS APIs exist at both the `Cluster` and `Scope` levels.
+
+This is because FTS supports, as of Couchbase Server 7.6, a new form of "scoped index" in addition to the traditional "global index".
+
+It's important to use the `Cluster.searchQuery()` / `Cluster.search()` for global indexes, and `Scope.searchQuery()` / `Scope.search()` for scoped indexes.
+
+== Vector Search
+As of Couchbase Server 7.6, the FTS service supports vector search in additional to traditional full text search queries.
+// todo link to the server docs when available
+
+=== Examples
+==== Single vector query
+In this first example we are performing a single vector query:
+[source,java]
+----
+include::example$Search.java[tag=vector1,indent=0]
+----
+
+Let's break this down.
+We create a `SearchRequest`, which can contain a traditional FTS query `SearchQuery` and/or the new `VectorSearch`.
+Here we are just using the latter.
+
+The `VectorSearch` allows us to perform one or more `VectorQuery` s.
+
+The `VectorQuery` itself takes the name of the document field that contains embedded vectors ("vector_field" here), plus actual vector query in the form of a `float[]`.
+
+(Note that Couchbase itself is not involved in generating the vectors, and these will come from an external source such as an embeddings API.)
+
+Finally we execute the `SearchRequest` against the FTS index "travel-sample-index", which has previously been setup to vector index the "vector_field" field.
+
+This happens to be a scoped index so we are using `scope.search()`.
+If it was a global index we would use `cluster.search()` instead - see <<Scoped vs Global Indexes>>.
+
+It returns the same `SearchResult` detailed earlier.
+
+==== Multiple vector queries
+You can run multiple vector queries together:
+
+[source,java]
+----
+include::example$Search.java[tag=vector3,indent=0]
+----
+
+How the results are combined (ANDed or ORed) can be controlled with `vectorSearchOptions().vectorQueryCombination()`.
+
+==== Combining FTS and vector queries
+You can combine a traditional FTS query with vector queries:
+
+[source,java]
+----
+include::example$Search.java[tag=vector2,indent=0]
+----
+
+How the results are combined (ANDed or ORed) can be controlled with `vectorSearchOptions().vectorQueryCombination()`.
+
+==== FTS queries
+And note that traditional FTS queries, without vector search, are also supported with the new `cluster.search()` / `scope.search()` APIs:
+
+[source,java]
+----
+include::example$Search.java[tag=vector4,indent=0]
+----
+
+The `SearchQuery` is created in the same way as detailed earlier.
+
 == Search Options
 
 The Search Service provides an array of options to customize your query. The following table lists them all:
@@ -217,10 +283,10 @@ Please see the xref:transcoders-nonjson.adoc[documentation on transcoding and se
 
 == Reactive And Async APIs
 
-In addition to the blocking API on `Cluster`, the SDK provides reactive and async APIs on `ReactiveCluster` or `AsyncCluster` respectively. 
+In addition to the blocking API on `Cluster`, the SDK provides reactive and async APIs on `ReactiveCluster` or `AsyncCluster` respectively (and similar for `Scope`, `ReactiveScope` and `AsyncScope`).
 If you are in doubt of which API to use, we recommend looking at the reactive first: 
 it builds on top of reactor, a powerful library that allows you to compose reactive computations and deal with error handling and other related concerns (like retry) in an elegant manner. 
-The async API on the other hand exposes a `CompletableFuture` and is more meant for lower level integration into other libraries or if you need the last drop of performance.
+The async API on the other hand exposes a `CompletableFuture` and is more intended for lower level integration into other libraries or if you need the last drop of performance.
 
 There is another reason for using the reactive API here: streaming large results with backpressure from the application side. 
 Both the blocking and async APIs have no means of signalling backpressure in a good way, so if you need it the reactive API is your best option.

pom.xml

@@ -22,7 +22,7 @@
         <dependency>
             <groupId>com.couchbase.client</groupId>
             <artifactId>java-client</artifactId>
-            <version>3.4.11</version>
+            <version>3.5.3-SNAPSHOT</version>
         </dependency>
         <dependency>
             <groupId>com.fasterxml.jackson.core</groupId>