Index is not loading from disk

[SilvioM97]

Hi,
I'm not able to load the index from disk in python.
I saved an index in this way:

index = HnswIndex(d, distance)
for sample in documents:
    index.add_data(sample)
index.build(m=m, max_m0=m0, ef_construction=ef_construction, n_threads=n_threads, neighbor_selecting="heuristic")
index.save(index_path)

The index is indeed saved, I see its memory occupancy on the disk.

Then I load it in this way:

index_n2 = HnswIndex(d)
loaded = index_n2.load(path_n2)

if loaded:
    print("Loaded n2 index")
else:
    print("Failed to load n2 index")

But the load fails, it prints "Failed to load n2 index" and I don't see it in main memory.
Why does this happen? What am I doing wrong?

[lancifollia]

Hello @SilvioM97 I apologize for the delay in responding.

I think there is a missing point in the python binding code. The load() function doesn't have return value.
https://github.com/kakao/n2/blob/dev/bindings/python/n2.pyx#L74C22-L74C31

After loading your index file, could you try running the search_by_id() function? (ignoring the loading function return value check)

index_n2 = HnswIndex(d)
loaded = index_n2.load(path_n2)

#if loaded:
#   print("Loaded n2 index")
#else:
#    print("Failed to load n2 index")

index_n2.search_by_id(0, 10)

In n2, virtual memory is pre-allocated on model load, and disk -> memory actually happens at inference time. This may have caused you to miss something in your memory check, so you may want to check that as well.