similarity_top_k does not return expected number of nodes

[johnhu33]

Why does index.as_retriever(similarity_top_k=5) not returning expected number of nodes, when similarity_top_k=3 it returns 2 nodes, when similarity_top_k=5 it returns 3 nodes, when similarity_top_k=10 it returns 5 nodes.

[dosubot[bot]]

The index.as_retriever(similarity_top_k=5) method does not always return the expected number of nodes because the actual number of nodes returned depends on the results of the vector store query. The similarity_top_k parameter specifies the maximum number of top results to return based on similarity, but the actual number of nodes returned can be less if there are fewer nodes that meet the similarity criteria.

Here is a relevant code snippet that shows how the query is built and processed:

def _build_vector_store_query(
        self, query_bundle_with_embeddings: QueryBundle
    ) -> VectorStoreQuery:
        return VectorStoreQuery(
            query_embedding=query_bundle_with_embeddings.embedding,
            similarity_top_k=self._similarity_top_k,
            node_ids=self._node_ids,
            doc_ids=self._doc_ids,
            query_str=query_bundle_with_embeddings.query_str,
            mode=self._vector_store_query_mode,
            alpha=self._alpha,
            filters=self._filters,
            sparse_top_k=self._sparse_top_k,
        )

def _get_nodes_with_embeddings(
        self, query_bundle_with_embeddings: QueryBundle
    ) -> List[NodeWithScore]:
        query = self._build_vector_store_query(query_bundle_with_embeddings)
        query_result = self._vector_store.query(query, **self._kwargs)
        return self._build_node_list_from_query_result(query_result)

The _build_node_list_from_query_result method processes the query results and constructs the list of nodes to return. If the vector store query does not find enough nodes that meet the similarity criteria, fewer nodes will be returned.

def _build_node_list_from_query_result(
        self, query_result: VectorStoreQueryResult
    ) -> List[NodeWithScore]:
        if query_result.nodes is None:
            if query_result.ids is None:
                raise ValueError(
                    "Vector store query result should return at "
                    "least one of nodes or ids."
                )
            assert isinstance(self._index.index_struct, IndexDict)
            node_ids = [
                self._index.index_struct.nodes_dict[idx] for idx in query_result.ids
            ]
            nodes = self._docstore.get_nodes(node_ids)
            query_result.nodes = nodes
        else:
            for i in range(len(query_result.nodes)):
                source_node = query_result.nodes[i].source_node
                if (not self._vector_store.stores_text) or (
                    source_node is not None and source_node.node_type != ObjectType.TEXT
                ):
                    node_id = query_result.nodes[i].node_id
                    if self._docstore.document_exists(node_id):
                        query_result.nodes[i] = self._docstore.get_node(
                            node_id
                        )

        log_vector_store_query_result(query_result)

        node_with_scores: List[NodeWithScore] = []
        for ind, node in enumerate(query_result.nodes):
            score: Optional[float] = None
            if query_result.similarities is not None:
                score = query_result.similarities[ind]
            node_with_scores.append(NodeWithScore(node=node, score=score))

        return node_with_scores

Therefore, the discrepancy in the number of nodes returned (3 nodes for similarity_top_k=5, 2 nodes for similarity_top_k=3, and 5 nodes for similarity_top_k=10) is due to the actual results of the vector store query, which may not always have enough nodes that meet the similarity criteria to match the similarity_top_k value ^[1][2].

To continue talking to Dosu, mention @dosu.

[johnhu33]

@dosu, if there is not enough nodes that meet the similarity criteria, then why when we increase the similarity_top_k, more nodes would be returned?

[dosubot[bot]]

Increasing the similarity_top_k parameter results in more nodes being returned because it sets a higher limit on the number of nodes to be retrieved. The VectorIndexRetriever uses this parameter to limit the number of results. By increasing similarity_top_k, the query allows for more nodes to be included in the results, even if they do not meet the initial similarity criteria as closely as others. This means that the system will return more nodes up to the specified limit, regardless of their similarity scores ^[1][2].

To continue talking to Dosu, mention @dosu.

[johnhu33]

@dosu how to check nodes that initially returned from the retriever, redis do not return similarity_top_k

[dosubot[bot]]

To check the nodes that are initially returned from the retriever when using Redis, you can inspect the result_nodes variable after calling the retrieve method on the retriever. Here is the relevant part of the code:

result_nodes = retriever.retrieve(turtle_test["question"])

You can then examine the result_nodes to see the nodes that were returned. For example, you can print the result_nodes to the console:

print(result_nodes)

This will give you a list of nodes that were initially returned by the retriever.

Additionally, if you need to adjust the number of top results returned, you can set the similarity_top_k parameter in the KGTableRetriever class. Here is how you can set it:

retriever.similarity_top_k = 10  # Set the number of top k results to return

This ensures that the retriever returns the specified number of top results based on similarity. The similarity_top_k parameter is used in the retrieve method to determine the number of top results to retrieve. For example:

retriever = KGTableRetriever(index, similarity_top_k=10, graph_store=graph_store)
nodes = retriever.retrieve(QueryBundle("foo"))
print(nodes)

This will configure the retriever to return the top 10 results based on similarity ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.