Error while compiling FIt-SNE v1.1.0 or 1.0.0 solved Fedora 31 5.6.8-200 while using with Seurat 3.1

[rafalwoycicki]

Hi,
I do not know if you had a problem similar to mine, but if yes so there is a solution which I have found. I have not found a similar subject here in issues so I am posting it, if it was and I have not seen it so I apologize.

First of all, I have tried to use FIt-SNE v1.2.1 with Seurat without success, but as learned from Seurat forum (as the wrapper is not updated to v1.2.1), I tried to install v.1.1.0 than.
My system is:
[ravwoy@work-ravwoy FIt-SNE-1.0.0]$ uname -a
Linux work-ravwoy 5.6.8-200.fc31.x86_64 #1 SMP Wed Apr 29 19:10:01 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
with 96 GB RAM and 12 cores (2 x Intel Xeon each with 6 cores).

I am not a developer, nor informatician, I am biologist who learned some programming and data analysis, co Bioinformatician with moreon the Bio site, so forgiove me not being professional.

While compiling v.1.1.0 (or v.1.0.0) I got this error:
[ravwoy@work-ravwoy FIt-SNE-1.0.0]$ g++ -std=c++11 -O3 src/sptree.cpp src/tsne.cpp src/nbodyfft.cpp -o bin/fast_tsne -pthread -lfftw3 -lm

In file included from src/tsne.cpp:42:
src/annoylib.h: In instantiation of ‘void AnnoyIndex<S, T, Distance, Random>::get_nns_by_item(S, size_t, size_t, std::vector<_Tp>, std::vector) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random; size_t = long unsigned int]’:
src/tsne.cpp:1213:76: required from here
src/annoylib.h:674:25: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
674 | _get_all_nns(m->v, n, search_k, result, distances);
| ~~~^
src/annoylib.h: In instantiation of ‘S AnnoyIndex<S, T, Distance, Random>::_make_tree(const std::vector<_Tp>&) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random]’:
src/annoylib.h:559:30: required from ‘void AnnoyIndex<S, T, Distance, Random>::build(int) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random]’
src/tsne.cpp:1208:25: required from here
src/annoylib.h:751:43: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
751 | bool side = D::side(m, n->v, _f, _random);
| ~~~^
src/annoylib.h: In instantiation of ‘void AnnoyIndex<S, T, Distance, Random>::_get_all_nns(const T*, size_t, size_t, std::vector<_Tp>, std::vector) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random; size_t = long unsigned int]’:
src/annoylib.h:674:9: required from ‘void AnnoyIndex<S, T, Distance, Random>::get_nns_by_item(S, size_t, size_t, std::vector<_Tp>, std::vector) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random; size_t = long unsigned int]’
src/tsne.cpp:1213:76: required from here
src/annoylib.h:809:36: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
809 | const S dst = nd->children;
| ~~~~^~~~~~~~
src/annoylib.h:813:76: error: cannot bind packed field ‘nd->Minkowski::Node<int, double>::children[1]’ to ‘int&’
813 | q.push(make_pair(D::pq_distance(d, margin, 1), nd->children[1]));
| ~~~~~~~~~~~~^
src/annoylib.h:814:76: error: cannot bind packed field ‘nd->Minkowski::Node<int, double>::children[0]’ to ‘int&’
814 | q.push(make_pair(D::pq_distance(d, margin, 0), nd->children[0]));
| ~~~~~~~~~~~~^
src/annoylib.h:828:66: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
828 | nns_dist.push_back(make_pair(D::distance(v, _get(j)->v, _f), j));
src/annoylib.h: In instantiation of ‘static void Euclidean::create_split(const std::vector<Minkowski::Node<S, T>>&, int, Random&, Minkowski::Node<S, T>) [with S = int; T = double; Random = Kiss32Random]’:
src/annoylib.h:745:24: required from ‘S AnnoyIndex<S, T, Distance, Random>::_make_tree(const std::vector<_Tp>&) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random]’
src/annoylib.h:559:30: required from ‘void AnnoyIndex<S, T, Distance, Random>::build(int) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random]’
src/tsne.cpp:1208:25: required from here
src/annoylib.h:389:22: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
389 | normalize(n->v, f);
| ~~~^
src/annoylib.h: In instantiation of ‘void two_means(const std::vector&, int, Random&, bool, T*, T*) [with T = double; Random = Kiss32Random; Distance = Euclidean; Node = Minkowski::Node<int, double>]’:
src/annoylib.h:385:56: required from ‘static void Euclidean::create_split(const std::vector<Minkowski::Node<S, T>>&, int, Random&, Minkowski::Node<S, T>) [with S = int; T = double; Random = Kiss32Random]’
src/annoylib.h:745:24: required from ‘S AnnoyIndex<S, T, Distance, Random>::_make_tree(const std::vector<_Tp>&) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random]’
src/annoylib.h:559:30: required from ‘void AnnoyIndex<S, T, Distance, Random>::build(int) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random]’
src/tsne.cpp:1208:25: required from here
src/annoylib.h:135:58: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
135 | T di = ic * Distance::distance(&iv[0], nodes[k]->v, f),
| ~~~~~~~~~~^
src/annoylib.h:136:64: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
136 | dj = jc * Distance::distance(&jv[0], nodes[k]->v, f);
| ~~~~~~~~~~^
src/annoylib.h:137:46: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
137 | T norm = cosine ? get_norm(nodes[k]->v, f) : 1.0;
| ~~~~~~~~~~^
src/annoylib.h: In instantiation of ‘T AnnoyIndex<S, T, Distance, Random>::get_distance(S, S) [with S = int; T = double; Distance = Euclidean; Random = Kiss32Random]’:
src/annoylib.h:666:7: required from here
src/annoylib.h:667:31: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
667 | const T *x = _get(i)->v;
src/annoylib.h:668:31: warning: taking address of packed member of ‘Minkowski::Node<int, double>’ may result in an unaligned pointer value [-Waddress-of-packed-member]
668 | const T *y = _get(j)->v;

So there were:

warnings: -Waddress-of-packed-member
errors in src/annoylib.h:813:76 and src/annoylib.h:814:76
Solution found to work:
I have chenged the code in annoylib.h in the lines 813 and 814 to include same additional code like in v1.2.1 version annoylib.h lines having same code, so:
instead of : q.push(make_pair(D::pq_distance(d, margin, 1), nd->children[1]));
q.push(make_pair(D::pq_distance(d, margin, 0), nd->children[0]));

I changed it to:
q.push(make_pair(D::pq_distance(d, margin, 1), static_cast(nd->children[1])));
q.push(make_pair(D::pq_distance(d, margin, 0), static_cast(nd->children[0])));

so added just "static_cast(.......) to the code.

After changing these lines and compiling with code same like for v1.2.1 version, so:

g++ -std=c++11 -O3 src/sptree.cpp src/tsne.cpp src/nbodyfft.cpp -o bin/fast_tsne -pthread -lfftw3 -lm -Wno-address-of-packed-member

The v.1.1.0 compiled and the results from Mouse Atlas were the same like in tutorial.

Best,
Rafał

[dkobak]

Hi. The problems with Annoy should have all been fixed in the current version. Why couldn't you install 1.2.1? I did not really understand that part.

[rafalwoycicki]

Hi, sorry, now I have updated the title.
I am using FIt-SNE with Seurat, and they did not update their wrapper so I had to use v1.1.0 or v1.0.0 version of FIt-SNE.

[dkobak]

I see. Somebody should make a PR to push the latest 1.2.1 version to Seurat. @linqiaozhi Can you do it? Or maybe @rafalwoycicki you want to do it?

[linqiaozhi]

@rafalwoycicki Thanks for posting your fix!

But yes, @dkobak is right, we need to fix the wrapper for Seurat so that it supports 1.2.1. See here satijalab/seurat#2792. I plan to get to it this week.

[rafalwoycicki]

@linqiaozhi Thanks, and in the meantime maybe this way around could help someone.

[saketkc]

Hi @linqiaozhi, I have updated the seurat version based on your wrapper:
satijalab/seurat#3281. Happy to hear your comments, thanks!

[rafalwoycicki]

Thanks a lot! Wysłane z konta pocztowego należącego do Rafała Kazimierza Wóycickiego

…

Wiadomość napisana przez Saket Choudhary ***@***.***> w dniu 22.07.2020, o godz. 20:33:  Hi @linqiaozhi, I have updated the seurat version based on your wrapper: satijalab/seurat#3281. Happy to hear your comments, thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[linqiaozhi]

Thanks so much for making this happen @saketkc! Looks great. Please feel free to close this once they merge your changes.

[saketkc]

This can now be closed as satijalab/seurat#3281 has been merged in the develop version and should be in the next release (Current release is at 3.2).

For anyone else bumping into this, if you would like to try the develop branch of Seurat, the instructions are here: https://satijalab.org/seurat/devinstall.html