-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
191 lines (131 loc) · 6.2 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
oooooooooooooooooooooooooo
O __ O
O \ / /\ / | O
O \ / / \ /___ | O
O | /----\ \ | O
O | / \ \__ |___ O
O O
oooooooooooooooooooooooooo
What is it?
===========
This is a library for performing efficient basic operations,
in particular kmeans and exhaustive nearest neighbor search function.
It offers three interfaces: C, python and matlab
Only a few function are exported in matlab, as most of the function
provided in the library are native matlab functions.
Documentation
=============
Online: http://yael.gforge.inria.fr/
Legal & Contact
===============
This library has been written and is maintained by Herve Jegou and Matthijs Douze.
Contact: [email protected] and [email protected]
Copyright (C) INRIA 2009-2011
Last change to this documentation: September 2011.
This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software.
See http://www.cecill.info/licences.en.html
INRIA may provide a licence for industrial applications.
Do not hesitate to contact us.
Prerequisites
=============
1- Blas and Lapack.
Any implementation should work as it is
wrapped with the Fortran calling conventions.
You might need to adjust the location of these libraries in
the makefile.inc file (generated by configure.sh)
The LD_LIBRARY_PATH environment variable should be set accordingly
2- Python and its development libraries (python-dev).
If only python is installed, not python-dev, you will get an error
saying that Python.h can not be found
3- swig (swig)
These two last pre-requisites are required by the python interface
and are not strictly mandatory. If you are not interested by this
python interface and that you don't have python-dev and swig installed,
you should remove the target _yael.so target in the Makefile of the
yael core directory.
The library is provided with a pre-compiled documentation (in doc
subdirectory). The reference documentation is generated by doxygen
while the reference manual is generated using sphinx.
Installation
============
1- ./configure.sh in the yael root directory.
For most configurations, nothing has to be done. However you might need
to adjust the variables defined in makefile.inc to fit your local config.
Moreover, we recommand to use --msse4 flags if your processor supports it,
in particular for better performance with Hamming distance computation:
./configure.sh --msse4
2- make
3- Try to compile the programs in progs and test
At this point, only the core C and python library are compiled.
If you need the matlab interface, you have to compile the mex files:
4- cd matlab ; matlab -r Make -nodisplay -nojvm
The mex executable should be in the PATH.
It is also possible to compile an version that works with Octave (see
the Makefile and octave_mex.sh)
Remarks
=======
The performance of the kmeans and of the most computationally intensive
functions heavily depend on the version of Lapack and Blas.
Timings might be scaled in the range from 1 to 3, depending on your choice.
In order to obtain the best performance, you should consider testing
the different version installed on your machine
(e.g., in /usr/lib and /usr/lib/atlas).
Troubleshooting
===============
LLVM compiler on the Mac
------------------------
In recent version of MacOS X (Maverick update), there is a compilation
issue trouble with the LLVM compiler when multi-threading is activated.
These occur both in all interface. While this problem is not related
directly to Yael, the problem may be solved by using gcc-4.9,
for instance as follows:
1) download the compiler,
$ brew install homebrew/versions/gcc49
For this to work, you need brew package installer. It can be obtained with
following command line:
$ ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go/install)"
Ensure that gcc-4.9 is in the path
2) Modify makefile.inc: replace
CC=gcc
by
CC=gcc-4.9
Alternately, you might consider deactivating multi-threading if you do not it.
This can be done in makefile.inc by replacing USEOPENMP=yes by USEOPENMP=no
This remark does not apply to Matlab, which is only compatible with LLVM,
at least for the common user.
32- and 64- bit integers in Fortran code
----------------------------------------
BLAS and Lapack are called with the fortran interface, which is most
standard. Some implementations are compiled with 32-bit integers
(eg. Atlas on most Linux distributions) and some with 64-bit integers
(eg. Matlab's -lmwblas and -lmwlapack libraries). The compilation
flags -DFINTEGER=long or -DFINTEGER=int switches between the two
calling conventions.
For example, if you have a license for Intel's MKL, you can use:
LAPACKLDFLAGS=-L MY_MKL_INSTALL_PATH -Wl,-rpath,MY_MKL_INSTALL_PATH \
-lmkl_intel_ilp64 -lmkl_sequential -lmkl_lapack -lmkl_core -lpthread -lm
LAPACKCFLAGS=-DFINTEGER=long
This selects the non-threaded version with 64-bit
integers. Multithreading will usually be performed by Yael.
For the multithreaded version (tested with MKL from composer_xe_2013):
LAPACKLDFLAGS=\
-L /home/clear/lear/intel/mkl/lib/intel64 -Wl,-rpath,/home/clear/lear/intel/mkl/lib/intel64 \
-L /home/clear/lear/intel/lib/intel64 -Wl,-rpath,/home/clear/lear/intel/lib/intel64 \
-lmkl_intel_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lpthread
NB that the MKL flags are a bit like black magic. A useful starting point
https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor
The computation of subsets of eigenvectors or singular values depends
on the ARPACK package, enabled with ./configure.sh --enable-arpack (from
http://www.caam.rice.edu/software/ARPACK/). Arpack should be compiled
with the same integer size as the blas/lapack implementation. To
compile ARPACK with gfortran and 64-bit integers, set
# %-----------------------------------------%
# | Various compilation programs and flags. |
# | You need to make sure these are correct |
# | for your system. |
# %-----------------------------------------%
#
FC = gfortran
FFLAGS = -g -O3 -fdefault-integer-8 -fPIC
in ARPACK's ARmake.inc