Using Scalapack/COSMA from Breeze

[lion241]

Hi!!

I have been looking into your repo and i wanted to understand if you are using BLAS/Lapack (perhaps calling C or Fortran) to perform matrix multiplication and in the case that the computer allows, do you also multithread in case the matrix size is big enough?

Finally is it possible with some installation in the cloud or locally to use the BLAS/Lapack library to ensure that operation can happen through that (assuming you do it through these packages)?

Sorry if this question is simple but I cant seem to be able to find a clear answer on this by looking around in the repo code.

[dlwh]

That's right. We basically use your native system BLAS/LApack if it's installed. This is now done via @luhenry's netlib https://github.com/luhenry/netlib . We don't provide any particular way of controlling the system BLAS's behavior. OpenBLAS provides some control via environment variables you can use. We usually call out to BLAS for any level 2 or level 3 operation. Many BLAS 1 operations (at least for small vectors) were historically faster to do in the jvm due to overhead. I need to revisit that now that we moved to luhenry's library.

…

On Tue, Jun 29, 2021 at 5:09 AM lion241 ***@***.***> wrote: Hi!! I have been looking into your repo and i wanted to understand if you are using BLAS/Lapack (perhaps calling C or Fortran) to perform matrix multiplication and in the case that the computer allows, do you also multithread in case the matrix size is big enough? Finally is it possible with some installation in the cloud or locally to use the BLAS/Lapack library to ensure that operation can happen through that (assuming you do it through these packages)? Sorry if this question is simple but I cant seem to be able to find a clear answer on this by looking around in the repo code. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#814>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAACLINHRQPXH2PSH5737WDTVGZWJANCNFSM47P7G6OA> .

[luhenry]

With https://github.com/luhenry/netlib, as long as you've OpenBLAS installed and on the dynamic loader path (see LD_LIBRARY_PATH and man dlopen), you should automatically use OpenBLAS. It can also work with Intel MKL when you specify -Ddev.ludovic.netlib.blas.nativeLibPath=/path/to/libmkl_rt.so

[lion241]

Assuming that i want to modify the BLAS/LApack library a little bit. Is there a way to ensure that breeze will use the modified version.

I am interested in using this to further speed up matrix multiplication:

https://github.com/eth-cscs/COSMA#cosma-in-production

I am sorry, I am not very familiar with such tasks, but my understanding is that I need to install scalapack or blas/lapack and then modify it based on the github above and then make sure breeze is pointing to the modified version.

I am not sure how I can make sure that breeze is pointing to the new version.

[dlwh]

It looks like COSMA exposes a Scalapack api, which I don't think @luhenry's library (nor Breeze) directly invokes. most of Breeze (at least the linear algebra parts) is just a fancy wrapper around BLAS/LAPACK/ARPACK (via https://github.com/luhenry/netlib). I've not done anything to make it work with scalapack.

I'd be happy to help provide some pointers, or you might be able to follow what @luhenry did for, e.g., LAPACK.

[luhenry]

https://github.com/luhenry/netlib supports calling into a BLAS/LAPACK/ARPACK library, whatever the implementation (OpenBLAS, Intel MKL, or others for example). If Scalapack provides such API, then you can call into it, but if it doesn't then it doesn't support calling into it.

This is the JNI wrapper file for LAPACK: https://github.com/luhenry/netlib/blob/master/lapack/src/main/native/jni.c