diff --git a/docs/index.rst b/docs/index.rst index ac2c21b7..bd624e46 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,83 +2,17 @@ ReSolve ******* -ReSolve is a library of GPU-resident linear solver. It contains -iterative and direct linear solvers designed to run on NVIDIA and AMD -GPUs, as well as on CPU devices. +ReSolve is a library of GPU-resident linear solver. It contains iterative and direct linear solvers designed to run on NVIDIA and AMD GPUs, as well as on CPU devices. -Getting started ---------------- -ReSolve is maintained and developed on the -`ReSolve Github Project `_. - -Dependencies: - KLU, AMD and COLAMD libraries from SuiteSparse - CUDA >= -11.4 - CMake >= 3.22 - -To build it: - -.. code:: shell - - $ git clone https://code.ornl.gov/peles/resolve.git - $ mkdir build && cd build - $ cmake ../resolve - $ make - -To install the library ----------------------- - -In the directory where you built the library run - -.. code:: shell - - $ make install - -To change the install location please use the CMAkePresets.json file as -mentioned in `test and deploy <#test-and-deploy>`__ - -To run it, download `test linear -systems `__ -and then edit script ```runResolve`` `__ to match locations -of your linear systems and binary installation. The script will emulate -nonlinear solver calling the linear solver repeatedly. - -To use the ReSolve library in your own project ----------------------------------------------- - -Make sure Resolve library is installed (see above) - -Below is an example CMakeList.txt file to use ReSolve library in your -project - -.. code:: cmake - - cmake_minimum_required(VERSION 3.20) - project(my_app LANGUAGES CXX) - - find_package(ReSolve CONFIG - PATHS ${ReSolve_DIR} ${ReSolve_DIR}/share/resolve/cmake - ENV LD_LIBRARY_PATH ENV DYLD_LIBRARY_PATH - REQUIRED) - - # Build your executable - add_executable(my_app my_app.cpp) - target_link_libraries(my_app PRIVATE ReSolve::ReSolve) +To learn how to get start with resolve please checkout our `User Guide `_ ============= Documentation ============= -User guides and source code documentation are always linked on this site. +Source code documentation are also linked on this site. `ReSolve Github Project `_. -`Source documentation `_ - -.. list-table:: - :align: center - - * - Functions - - `Source `_ - * - Namespaces - - `Source Documentation `_ - +`Source documentation `_ Contributing ------------ @@ -86,126 +20,32 @@ Contributing For all contributions to ReSolve please follow the `developer guidelines `__ -Test and Deploy ---------------- - -ReSolve as a library is tested every merge request via Gitlab pipelines -that execute various library tests including a test of ReSolve being -consumed as package within an external project as mentioned in `Using -ReSolve in your own -Project <#to-use-the-resolve-library-in-your-own-project>`__ - -To test your own install of ReSolve simply cd into your ReSolve build -directory and run - -.. code:: shell - - $ make test - -or - -.. code:: shell - - $ ctest - -Below is an example of what a functional ReSolve build will ouput on -Passing tests - -.. code:: text - - Test project /people/dane678/resolve/build - Start 1: resolve_consume - 1/9 Test #1: resolve_consume .................. Passed 16.51 sec - Start 2: klu_klu_test - 2/9 Test #2: klu_klu_test ..................... Passed 1.04 sec - Start 3: klu_rf_test - 3/9 Test #3: klu_rf_test ...................... Passed 1.04 sec - Start 4: klu_rf_fgmres_test - 4/9 Test #4: klu_rf_fgmres_test ............... Passed 3.14 sec - Start 5: klu_glu_test - 5/9 Test #5: klu_glu_test ..................... Passed 1.06 sec - Start 6: matrix_test - 6/9 Test #6: matrix_test ...................... Passed 0.03 sec - Start 7: matrix_handler_test - 7/9 Test #7: matrix_handler_test .............. Passed 0.97 sec - Start 8: vector_handler_test - 8/9 Test #8: vector_handler_test .............. Passed 0.98 sec - Start 9: logger_test - 9/9 Test #9: logger_test ...................... Passed 0.03 sec - -Important Notes ---------------- - -You can find default Cmake Configurations in the CMakePresets.json file, -which allows for easy switching between different CMake Configs. To -create your own CMake Configuration we encourage you to utlize a -CmakeUserPresets.json file. To learn more about cmake-presets please -checkout the cmake -`docs `__ - -For example if you wanted to build and install ReSolve on a High -Performance Computing Cluster such as PNNL’s Deception or ORNL’s Ascent -we encourage you to utilize our cluster preset. Using this preset will -set CMAKE_INSTALL_PREFIX to an install folder. To use this preset simply -call the preset flag in the cmake build step. - -.. code:: shell - - cmake -B build --preset cluster - -Developing Documentation Using Dev Container -------- - - Prerequisites - -1. install Docker Desktop and launch the app -2. install the "Remote Development" extension in VSCode -3. open your local clone of resolve in VSCode - -Build Container - -The build info for this container is in `.devcontainer/`. There is a Dockerfile and json file associated with the configuration. - -1. if connected, disconnect from the PNNL VPN -2. launch the container build - * `cmd shift p` to open the command pallette in vscode - * click `> Dev Container: rebuild and reopen container` - * this will start building the container, taking about 40 minutes - * click on the pop up with `(show log)` to view the progress - -3. Open new terminal within VSCODe and run the renderDocs.sh (note this takes a minute) -4. Open the link that was served to you after step 3 - -Note - pushing/pulling from git is not supported in a devcontainer, and should be done independently. - -For any questions or to report a bug please submit a `GitHub -issue `__. - Authors and acknowledgment -------------------------- -Primary authors of this project are Kasia Świrydowicz -kasia.swirydowicz@pnnl.gov and Slaven Peles peless@ornl.gov. +Primary authors of this project are: +* Kasia Świrydowicz kasia.swirydowicz@pnnl.gov (PNNL) +* Slaven Peles peless@ornl.gov (ORNL) ReSolve project would not be possible without significant contributions -from (in alphabetic ortder): - Maksudul Alam - Ryan Danehy - Nicholson -Koukpaizan - Jaelyn Litzinger - Phil Roth - Cameron Rutherford +from (in alphabetic order): +* Maksudul Alam (ORNL) +* Ryan Danehy (PNNL) +* Nicholson Koukpaizan (ORNL) +* Jaelyn Litzinger (PNNL) +* Phil Roth (ORNL) +* Cameron Rutherford (PNNL) -Development of this coede was supported by the Exascale Computing -Project (ECP), Project Number: 17-SC-20-SC, a collaborative effort of -two DOE organizations—the Office of Science and the National Nuclear -Security Administration—responsible for the planning and preparation of -a capable exascale ecosystem—including software, applications, hardware, -advanced system engineering, and early testbed platforms—to support the -nation’s exascale computing imperative. +Development of this code was supported by the Exascale Computing Project (ECP), Project Number: 17-SC-20-SC, a collaborative effort of two DOE organizations—the Office of Science and the National Nuclear Security Administration—responsible for the planning and preparation of a capable exascale ecosystem—including software, applications, hardware, advanced system engineering, and early testbed platforms—to support the nation’s exascale computing imperative. License ------- Copyright © 2023, UT-Battelle, LLC, and Battelle Memorial Institute. -ReSolve is a free software distributed under a BSD-style license. See -the `LICENSE `__ and `NOTICE `__ files for details. All +ReSolve is a free software distributed under a BSD-style license. + +See the `LICENSE `__ and `NOTICE `__ files for details. All new contributions to ReSolve must be made under the same licensing terms. @@ -220,3 +60,10 @@ terms. sphinx/licenses sphinx/notice sphinx/developer_guide/index + +.. toctree:: + :maxdepth: 2 + :hidden: + :caption: User Guide + + sphinx/guide \ No newline at end of file diff --git a/docs/sphinx/guide.rst b/docs/sphinx/guide.rst new file mode 100644 index 00000000..a3fe6da8 --- /dev/null +++ b/docs/sphinx/guide.rst @@ -0,0 +1,149 @@ +User Guide +==================== + +Getting started +--------------- +ReSolve is maintained and developed on the +`ReSolve Github Project `_. + +Dependencies: - KLU, AMD and COLAMD libraries from SuiteSparse - CUDA >= 11.4 - CMake >= 3.22 + +To build it: + +.. code:: shell + + $ git clone https://code.ornl.gov/peles/resolve.git + $ mkdir build && cd build + $ cmake ../resolve + $ make + +To install the library +---------------------- + +In the directory where you built the library run + +.. code:: shell + + $ make install + +To change the install location please use the CMAkePresets.json file as mentioned in `test and deploy <#test-and-deploy>`__ + +To run it, download `test linear systems `__ and then edit script ```runResolve`` `__ to match locations of your linear systems and binary installation. The script will emulate nonlinear solver calling the linear solver repeatedly. + +To use the ReSolve library in your own project +---------------------------------------------- + +Make sure Resolve library is installed (see above) + +Below is an example CMakeList.txt file to use ReSolve library in your project + +.. code:: cmake + + cmake_minimum_required(VERSION 3.20) + project(my_app LANGUAGES CXX) + + find_package(ReSolve CONFIG + PATHS ${ReSolve_DIR} ${ReSolve_DIR}/share/resolve/cmake + ENV LD_LIBRARY_PATH ENV DYLD_LIBRARY_PATH + REQUIRED) + + # Build your executable + add_executable(my_app my_app.cpp) + target_link_libraries(my_app PRIVATE ReSolve::ReSolve) + + +Test and Deploy +--------------- + +ReSolve as a library is tested every merge request via Gitlab pipelines +that execute various library tests including a test of ReSolve being +consumed as package within an external project as mentioned in `Using +ReSolve in your own +Project <#to-use-the-resolve-library-in-your-own-project>`__ + +To test your own install of ReSolve simply cd into your ReSolve build +directory and run + +.. code:: shell + + $ make test + +or + +.. code:: shell + + $ ctest + +Below is an example of what a functional ReSolve build will ouput on +Passing tests + +.. code:: text + + Test project /people/dane678/resolve/build + Start 1: resolve_consume + 1/9 Test #1: resolve_consume .................. Passed 16.51 sec + Start 2: klu_klu_test + 2/9 Test #2: klu_klu_test ..................... Passed 1.04 sec + Start 3: klu_rf_test + 3/9 Test #3: klu_rf_test ...................... Passed 1.04 sec + Start 4: klu_rf_fgmres_test + 4/9 Test #4: klu_rf_fgmres_test ............... Passed 3.14 sec + Start 5: klu_glu_test + 5/9 Test #5: klu_glu_test ..................... Passed 1.06 sec + Start 6: matrix_test + 6/9 Test #6: matrix_test ...................... Passed 0.03 sec + Start 7: matrix_handler_test + 7/9 Test #7: matrix_handler_test .............. Passed 0.97 sec + Start 8: vector_handler_test + 8/9 Test #8: vector_handler_test .............. Passed 0.98 sec + Start 9: logger_test + 9/9 Test #9: logger_test ...................... Passed 0.03 sec + +Important Notes +--------------- + +You can find default Cmake Configurations in the CMakePresets.json file, +which allows for easy switching between different CMake Configs. To +create your own CMake Configuration we encourage you to utlize a +CmakeUserPresets.json file. To learn more about cmake-presets please +checkout the cmake +`docs `__ + +For example if you wanted to build and install ReSolve on a High +Performance Computing Cluster such as PNNL’s Deception or ORNL’s Ascent +we encourage you to utilize our cluster preset. Using this preset will +set CMAKE_INSTALL_PREFIX to an install folder. To use this preset simply +call the preset flag in the cmake build step. + +.. code:: shell + + cmake -B build --preset cluster + +Developing Documentation Using Dev Container +------- + +Prerequisites + +#. install Docker Desktop and launch the app +#. install the "Remote Development" extension in VSCode +#. open your local clone of resolve in VSCode + +Build Container + +The build info for this container is in `.devcontainer/`. There is a Dockerfile and json file associated with the configuration. + +#. if connected, disconnect from the PNNL VPN +#. launch the container build + + * `cmd shift p` to open the command pallette in vscode + * click `> Dev Container: rebuild and reopen container` + * this will start building the container, taking about 40 minutes + * click on the pop up with `(show log)` to view the progress + +#. Open new terminal within VSCODe and run the renderDocs.sh (note this takes a minute) +#. Open the link that was served to you after step 3 + +Note - pushing/pulling from git is not supported in a devcontainer, and should be done independently. + +For any questions or to report a bug please submit a `GitHub +issue `__. diff --git a/resolve/LinSolverIterativeRandFGMRES.hpp b/resolve/LinSolverIterativeRandFGMRES.hpp index 07176980..315e01db 100644 --- a/resolve/LinSolverIterativeRandFGMRES.hpp +++ b/resolve/LinSolverIterativeRandFGMRES.hpp @@ -58,7 +58,7 @@ namespace ReSolve vector_type* d_V_{nullptr}; vector_type* d_Z_{nullptr}; // for performing Gram-Schmidt - vector_type* d_S_{nullptr}; + vector_type* d_S_{nullptr};///< this is where sketched vectors are stored real_type* h_H_{nullptr}; real_type* h_c_{nullptr}; diff --git a/resolve/RandSketchingCountSketch.cpp b/resolve/RandSketchingCountSketch.cpp index 7ce61550..86daf251 100644 --- a/resolve/RandSketchingCountSketch.cpp +++ b/resolve/RandSketchingCountSketch.cpp @@ -12,6 +12,13 @@ namespace ReSolve { + /** + * @brief Default constructor + * + * @post All class variables are set to nullptr. + * + * @todo There is little utility for the default constructor. Maybe remove?. + */ RandSketchingCountSketch::RandSketchingCountSketch() { h_labels_ = nullptr; @@ -22,6 +29,10 @@ namespace ReSolve } // destructor + /** + * @brief Default de-constructor + * + */ RandSketchingCountSketch::~RandSketchingCountSketch() { delete h_labels_; @@ -31,6 +42,16 @@ namespace ReSolve } // Actual sketching process + /** + * @brief Sketching method - it sketches a given vector (shrinks its size) + * + * @param[in] input - input vector, size _n_ + * @param[out] output - output vector, size _k_ + * + * @pre both vectors are allocated. Setup function from this class has been called. + * + * @return 0 of successful, -1 otherwise (TODO). + */ int RandSketchingCountSketch::Theta(vector_type* input, vector_type* output) { mem_.deviceSynchronize(); @@ -47,6 +68,18 @@ namespace ReSolve } // Setup the parameters, sampling matrices, permuations, etc + /** + * @brief Sketching method setup. + * + * This function allocated _labels_, and _flip_ arrays and, populates them. + * + * @param[in] n - size of base (non-sketched) vector + * @param[in] k - size of sketched vector. + * + * @post Everything is set up so you call call Theta. + * + * @return 0 of successful, -1 otherwise. + */ int RandSketchingCountSketch::setup(index_type n, index_type k) { k_rand_ = k; @@ -83,8 +116,18 @@ namespace ReSolve } //to be fixed, this can be done on the GPU + /** + * @brief Reset values in the arrays used for sketching. + * + * If the solver restarts, call this method between restarts. + * + * @post Everything is set up so you call call Theta. + * + * @return 0 of successful, -1 otherwise. + */ int RandSketchingCountSketch::reset() // if needed can be reset (like when Krylov method restarts) { + srand(static_cast(time(nullptr))); for (int i = 0; i < n_; ++i) { h_labels_[i] = rand() % k_rand_; diff --git a/resolve/RandSketchingCountSketch.hpp b/resolve/RandSketchingCountSketch.hpp index a796f12c..b4481394 100644 --- a/resolve/RandSketchingCountSketch.hpp +++ b/resolve/RandSketchingCountSketch.hpp @@ -30,10 +30,10 @@ namespace ReSolve { virtual int reset(); // if needed can be reset (like when Krylov method restarts) private: - index_type* h_labels_; - index_type* h_flip_; + index_type* h_labels_;///< label array size _n_, with values from _0_ to _k-1_ assigned by random + index_type* h_flip_; ///< flip array with valyes of 1 and -1 assigned by random - index_type* d_labels_; - index_type* d_flip_; + index_type* d_labels_; ///< h_labels GPU counterpart + index_type* d_flip_; ///< h_flip GPU counterpart }; } diff --git a/resolve/RandSketchingFWHT.cpp b/resolve/RandSketchingFWHT.cpp index 76e91674..dcb59e40 100644 --- a/resolve/RandSketchingFWHT.cpp +++ b/resolve/RandSketchingFWHT.cpp @@ -16,6 +16,13 @@ namespace ReSolve { using out = io::Logger; + /** + * @brief Default constructor + * + * @post All class variables are set to nullptr. + * + * @todo There is little utility for the default constructor. Maybe remove?. + */ RandSketchingFWHT::RandSketchingFWHT() { h_seq_ = nullptr; @@ -28,6 +35,11 @@ namespace ReSolve } // destructor + + /** + * @brief Default de-constructor + * + */ RandSketchingFWHT::~RandSketchingFWHT() { delete h_seq_; @@ -40,6 +52,18 @@ namespace ReSolve } // Actual sketching process + + /** + * @brief Sketching method - it sketches a given vector (shrinks its size) + * + * @param[in] input - input vector, size _n_ + * @param[out] output - output vector, size _k_ + * + * @pre both vectors are allocated. Setup function from this class has been called. + * @warning normal FWHT function requires scaling by 1/k. This function does not scale. + * + * @return 0 of successful, -1 otherwise (TODO). + */ int RandSketchingFWHT::Theta(vector_type* input, vector_type* output) { mem_.setZeroArrayOnDevice(d_aux_, N_); @@ -62,6 +86,21 @@ namespace ReSolve } // Setup the parameters, sampling matrices, permuations, etc + /** + * @brief Sketching method setup. + * + * This function allocated P(erm), D (diagonal scaling matrix) andpopulates + * them and allocates a bunch of other auxiliary arrays. + * + * + * @param[in] n - size of base (non-sketched) vector + * @param[in] k - size of sketched vector. + * + * @post Everything is set up so you call call Theta. + * + * @return 0 of successful, -1 otherwise. + */ + int RandSketchingFWHT::setup(index_type n, index_type k) { k_rand_ = k; @@ -122,6 +161,15 @@ namespace ReSolve } //to be fixed, this can be done on the GPU + /** + * @brief Reset values in the arrays used for sketching. + * + * If the solver restarts, call this method between restarts. + * + * @post Everything is set up so you call call Theta. + * + * @return 0 of successful, -1 otherwise. + */ int RandSketchingFWHT::reset() // if needed can be reset (like when Krylov method restarts) { srand(static_cast(time(nullptr))); diff --git a/resolve/RandSketchingFWHT.hpp b/resolve/RandSketchingFWHT.hpp index 4cab9e51..0ee3e026 100644 --- a/resolve/RandSketchingFWHT.hpp +++ b/resolve/RandSketchingFWHT.hpp @@ -29,16 +29,16 @@ namespace ReSolve { virtual int reset(); // if needed can be reset (like when Krylov method restarts) private: - index_type* h_seq_; - index_type* h_D_; - index_type* h_perm_; + index_type* h_seq_; ///< auxiliary variable used for Fisher-Yates algorithm + index_type* h_D_; ///< D is a diagonal matrix (FWHT computed y = PHDx), we store it as an array. D consists of _1_s and _-1_s + index_type* h_perm_; ///< permuation array, containing _k_ values in range of _0_ to _n-1_ - index_type* d_D_; - index_type* d_perm_; - real_type* d_aux_; + index_type* d_D_;///< device mirror of D + index_type* d_perm_; ///< device mirror of h_perm + real_type* d_aux_; ///< auxiliary variable needed to store partial results in FWHT application. - index_type N_; - index_type log2N_; - real_type one_over_k_; + index_type N_; ///< padded vector size + index_type log2N_; ///< log2 of N_, used multiple times so we store it + real_type one_over_k_;///< 1/k, used many times for scaling so we store the value to avoid recomputation }; } diff --git a/resolve/RandSketchingManager.cpp b/resolve/RandSketchingManager.cpp index 3cb42670..796379a4 100644 --- a/resolve/RandSketchingManager.cpp +++ b/resolve/RandSketchingManager.cpp @@ -4,6 +4,10 @@ namespace ReSolve { // constructor + /** + * @brief Simple constructor + * + */ RandSketchingManager::RandSketchingManager() { k_rand_ = 0; @@ -12,20 +16,39 @@ namespace ReSolve { } // destructor + /** + * @brief Destructor + * + */ RandSketchingManager::~RandSketchingManager() { } + /** + * @brief Returns the size of base (non-sketched) vector. + * + * @return Base vector size _n_. + */ index_type RandSketchingManager::getVectorSize() { return n_; } + /** + * @brief Returns the size of sketched vector. + * + * @return Sketched vector size _k_, generally _k_ =< _n_. + */ index_type RandSketchingManager::getSketchSize() { return k_rand_; } + /** + * @brief If padding is used, returns size of padded vector. + * + * @return Sketched vector size _N_, generally _N_ >= _n_. + */ index_type RandSketchingManager::getPaddedSize() { return N_; diff --git a/resolve/RandSketchingManager.hpp b/resolve/RandSketchingManager.hpp index baff7b0e..236c850f 100644 --- a/resolve/RandSketchingManager.hpp +++ b/resolve/RandSketchingManager.hpp @@ -20,6 +20,7 @@ namespace ReSolve { // Setup the parameters, sampling matrices, permuations, etc virtual int setup(index_type n, index_type k) = 0; + // Need to use with methods that restart virtual int reset() = 0; virtual index_type getVectorSize(); @@ -27,9 +28,9 @@ namespace ReSolve { virtual index_type getPaddedSize(); protected: - index_type n_;// size of base vector - index_type k_rand_; // size of sketched vector - index_type N_; // padded n -- generally N_ > n_ + index_type n_;///< size of base vector + index_type k_rand_; ///< size of sketched vector + index_type N_; ///< padded n -- generally N_ > n_ MemoryHandler mem_; ///< Device memory manager object }; diff --git a/resolve/matrix/MatrixHandler.hpp b/resolve/matrix/MatrixHandler.hpp index 8041807a..b1de27e7 100644 --- a/resolve/matrix/MatrixHandler.hpp +++ b/resolve/matrix/MatrixHandler.hpp @@ -32,7 +32,7 @@ namespace ReSolve { * This includes: * - Matrix format conversion: coo2csr, csr2csc * - Matrix vector product (SpMV) - * - Matrix 1-norm + * - Matrix Inf-norm * * The class uses pointer to implementation (PIMPL) idiom to create * multiple matrix operation implementations running on CUDA and HIP devices diff --git a/resolve/matrix/Sparse.cpp b/resolve/matrix/Sparse.cpp index bf54cbd4..4ab387c2 100644 --- a/resolve/matrix/Sparse.cpp +++ b/resolve/matrix/Sparse.cpp @@ -4,10 +4,20 @@ namespace ReSolve { namespace matrix { + /** + * @brief empty constructor that does absolutely nothing + */ Sparse::Sparse() { } + /** + * @brief basic constructor. It DOES NOT allocate any memory! + * + * @param[in] n - number of rows + * @param[in] m - number of columns + * @param[in] nnz - number of non-zeros + */ Sparse::Sparse(index_type n, index_type m, index_type nnz): @@ -37,6 +47,15 @@ namespace ReSolve { namespace matrix { owns_gpu_vals_ = false; } + /** + * @brief another basic constructor. It DOES NOT allocate any memory! + * + * @param[in] n - number of rows + * @param[in] m - number of columns + * @param[in] nnz - number of non-zeros + * @param[in] symmetric - true if symmetric, false if non-symmetric + * @param[in] expanded - true if expanded, false if not + */ Sparse::Sparse(index_type n, index_type m, index_type nnz, @@ -70,69 +89,134 @@ namespace ReSolve { namespace matrix { owns_gpu_data_ = false; owns_gpu_vals_ = false; } - + + /** + * @brief destructor + * */ Sparse::~Sparse() { this->destroyMatrixData(memory::HOST); this->destroyMatrixData(memory::DEVICE); } + /** + * @brief set the matrix update flags to false (for both HOST and DEVICE). + */ void Sparse::setNotUpdated() { h_data_updated_ = false; d_data_updated_ = false; } + /** + * @brief get number of matrix rows + * + * @return number of matrix rows. + */ index_type Sparse::getNumRows() { return this->n_; } + /** + * @brief get number of matrix columns + * + * @return number of matrix columns. + */ index_type Sparse::getNumColumns() { return this->m_; } + /** + * @brief get number of non-zeros in the matrix. + * + * @return number of non-zeros. + */ index_type Sparse::getNnz() { return this->nnz_; } + /** + * @brief get number of non-zeros in expanded matrix. + * + * @return number of non-zeros in expanded matrix. + */ index_type Sparse::getNnzExpanded() { return this->nnz_expanded_; } + /** + * @brief check if matrix is symmetric. + * + * @return true if symmetric, false otherwise. + */ bool Sparse::symmetric() { return is_symmetric_; } + /** + * @brief check if (symmetric) matrix is expanded. + * + * @return true if expanded, false otherwise. + */ bool Sparse::expanded() { return is_expanded_; } + /** + * @brief Set matrix symmetry property + * + * @param[in] symmetric - true to set matrix to symmetric and false to set to non-symmetric + */ void Sparse::setSymmetric(bool symmetric) { this->is_symmetric_ = symmetric; } + /** + * @brief Set matrix "expanded" property + * + * @param[in] expanded - true to set matrix to expanded and false to set to not expanded + */ void Sparse::setExpanded(bool expanded) { this->is_expanded_ = expanded; } + /** + * @brief Set number of non-zeros in expanded matrix. + * + * @param[in] nnz_expanded_new - new number of non-zeros in expanded matrix + */ void Sparse::setNnzExpanded(index_type nnz_expanded_new) { this->nnz_expanded_ = nnz_expanded_new; } + /** + * @brief Set number of non-zeros. + * + * @param[in] nnz_new - new number of non-zeros + */ void Sparse::setNnz(index_type nnz_new) { this->nnz_ = nnz_new; } + /** + * @brief Set the data to be updated on HOST or DEVICE. + * + * @param[in] memspace - memory space (HOST or DEVICE) of data that is set to "updated" + * + * @return 0 if successful, -1 if not. + * + * @note The method automatically sets the other mirror data to non-updated (but it does not copy). + */ int Sparse::setUpdated(memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -149,6 +233,20 @@ namespace ReSolve { namespace matrix { return 0; } + /** + * @brief Set the pointers for matrix row, column, value data. + * + * Useful if interfacing with other codes - this function only assigns pointers, + * but it does not allocate nor copy anything. Note that the data ownership + * flags would be left at default (false). + * + * @param[in] row_data - pointer to row data (array of integers) + * @param[in] col_data - pointer to column data (array of integers) + * @param[in] val_data - pointer to value data (array of real numbers) + * @param[in] memspace - memory space (HOST or DEVICE) of incoming data + * + * @return 0 if successful, -1 if not. + */ int Sparse::setMatrixData(index_type* row_data, index_type* col_data, real_type* val_data, memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -171,7 +269,16 @@ namespace ReSolve { namespace matrix { } return 0; } - + + /** + * @brief destroy matrix data (HOST or DEVICE) if the matrix owns it + * (will attempt to destroy all three arrays). + * + * @param[in] memspace - memory space (HOST or DEVICE) of incoming data + * + * @return 0 if successful, -1 if not. + * + */ int Sparse::destroyMatrixData(memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -199,6 +306,18 @@ namespace ReSolve { namespace matrix { } } + /** + * @brief updata matrix values using the _new_values_ provided either as HOST or as DEVICE array. + * + * This function will copy the data (not just assign a pointer) and allocate if needed. + * It also sets ownership and update flags. + * + * @param[in] new_vals - pointer to new values data (array of real numbers) + * @param[in] memspaceIn - memory space (HOST or DEVICE) of _new_vals_ + * @param[in] memspaceOut - memory space (HOST or DEVICE) of matrix values to be updated. + * + * @return 0 if successful, -1 if not. + */ int Sparse::updateValues(real_type* new_vals, memory::MemorySpace memspaceIn, memory::MemorySpace memspaceOut) { @@ -251,6 +370,16 @@ namespace ReSolve { namespace matrix { return 0; } + /** + * @brief updata matrix values using the _new_values_ provided either as HOST or as DEVICE array. + * + * This function only assigns a pointer, but does not copy. It sets update flags. + * + * @param[in] new_vals - pointer to new values data (array of real numbers) + * @param[in] memspace - memory space (HOST or DEVICE) of _new_vals_ + * + * @return 0 if successful, -1 if not. + */ int Sparse::setNewValues(real_type* new_vals, memory::MemorySpace memspace) { using namespace ReSolve::memory; diff --git a/resolve/matrix/Sparse.hpp b/resolve/matrix/Sparse.hpp index 96121acb..49a94dce 100644 --- a/resolve/matrix/Sparse.hpp +++ b/resolve/matrix/Sparse.hpp @@ -6,6 +6,13 @@ #include namespace ReSolve { namespace matrix { + /** + * @brief This class implements basic sparse matrix interface. (Almost) all sparse matrix formats store information about matrix rows and columns (as integers) and data (as real numbers). + * This class is virtualand implements only what is common for all basic formats. + * Note that regardless of how row/column information is stored, all values need to be stored, so all utilities needed for values are implemented in this class. + * + * @author Kasia Swirydowicz + */ class Sparse { public: @@ -59,25 +66,25 @@ namespace ReSolve { namespace matrix { protected: //size - index_type n_{0}; - index_type m_{0}; - index_type nnz_{0}; - index_type nnz_expanded_{0}; + index_type n_{0}; ///< number of rows + index_type m_{0}; ///< number of columns + index_type nnz_{0}; ///< number of non-zeros + index_type nnz_expanded_{0}; ///< number of non-zeros in an expanded matrix (for symmetric matrices) - bool is_symmetric_{false}; - bool is_expanded_{false}; + bool is_symmetric_{false}; ///< symmetry flag + bool is_expanded_{false}; ///< "expanded" flag //host data - index_type* h_row_data_{nullptr}; - index_type* h_col_data_{nullptr}; - real_type* h_val_data_{nullptr}; - bool h_data_updated_{false}; + index_type* h_row_data_{nullptr}; ///< row data (HOST) + index_type* h_col_data_{nullptr}; ///< column data (HOST) + real_type* h_val_data_{nullptr}; ///< value data (HOST) + bool h_data_updated_{false}; ///< HOST update flag //gpu data - index_type* d_row_data_{nullptr}; - index_type* d_col_data_{nullptr}; - real_type* d_val_data_{nullptr}; - bool d_data_updated_{false}; + index_type* d_row_data_{nullptr}; ///< row data (DEVICE) + index_type* d_col_data_{nullptr}; ///< column data (DEVICE) + real_type* d_val_data_{nullptr}; ///< value data (DEVICE) + bool d_data_updated_{false}; ///< DEVICE update flag void setNotUpdated(); diff --git a/resolve/vector/Vector.cpp b/resolve/vector/Vector.cpp index c022c0ee..ccef9e0c 100644 --- a/resolve/vector/Vector.cpp +++ b/resolve/vector/Vector.cpp @@ -4,6 +4,11 @@ namespace ReSolve { namespace vector { + /** + * @brief basic constructor. + * + * @param[in] n - Number of elements in the vector + */ Vector::Vector(index_type n): n_(n), k_(1), @@ -17,6 +22,12 @@ namespace ReSolve { namespace vector { { } + /** + * @brief multivector constructor. + * + * @param[in] n - Number of elements in the vector + * @param[in] k - Number of vectors in multivector + */ Vector::Vector(index_type n, index_type k) : n_(n), k_(k), @@ -30,6 +41,10 @@ namespace ReSolve { namespace vector { { } + /** + * @brief destructor. + * + */ Vector::~Vector() { if (owns_cpu_data_) delete [] h_data_; @@ -37,21 +52,45 @@ namespace ReSolve { namespace vector { } + /** + * @brief get the number of elements in a single vector. + * + * @return _n_, number of elements in the vector. + */ index_type Vector::getSize() const { return n_; } + /** + * @brief get the current number of elements in a single vector + * (use only for vectors with changing sizes, allocate for maximum expected size). + * + * @return _n_current_, number of elements currently in thr vector. + */ index_type Vector::getCurrentSize() { return n_current_; } + /** + * @brief get the total number of vectors in multivector. + * + * @return _k_, number of vectors in multivector, or 1 if the vector is not a multivector. + */ index_type Vector::getNumVectors() { return k_; } + /** + * @brief set the vector data variable (HOST or DEVICE) to the provided pointer. + * + * @param[in] data - Pointer to data + * @param[in] memspace - Memory space (HOST or DEVICE) + * + * @warning This function DOES NOT ALLOCATE any data, it only assigns the pointer. + */ void Vector::setData(real_type* data, memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -69,6 +108,13 @@ namespace ReSolve { namespace vector { } } + /** + * @brief set the flag to indicate that the data (HOST or DEVICE) has been updated. + * + * Important because of data mirroring approach. + * + * @param[in] memspace - Memory space (HOST or DEVICE) + */ void Vector::setDataUpdated(memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -84,12 +130,32 @@ namespace ReSolve { namespace vector { } } + /** + * @brief update vector values based on another vector data. + * + * @param[in] v - Vector, which data will be copied + * @param[in] memspaceIn - Memory space of the incoming data (HOST or DEVICE) + * @param[in] memspaceOut - Memory space the data will be copied to (HOST or DEVICE) + * + * @pre size of _v_ is equal or larger than the current vector size. + */ int Vector::update(Vector* v, memory::MemorySpace memspaceIn, memory::MemorySpace memspaceOut) { real_type* data = v->getData(memspaceIn); return update(data, memspaceIn, memspaceOut); } + /** + * @brief update vector data based on given input. + * + * This function allocates (if necessary) and copies the data. + * + * @param[in] data - Data that is to be copied + * @param[in] memspaceIn - Memory space of the incoming data (HOST or DEVICE) + * @param[in] memspaceOut - Memory space the data will be copied to (HOST or DEVICE) + * + * @return 0 if successful, -1 otherwise. + */ int Vector::update(real_type* data, memory::MemorySpace memspaceIn, memory::MemorySpace memspaceOut) { int control=-1; @@ -136,11 +202,34 @@ namespace ReSolve { namespace vector { return 0; } + /** + * @brief get a pointer to HOST or DEVICE vector data. + * + * @param[in] memspace - Memory space of the pointer (HOST or DEVICE) + * + * @return pointer to the vector data (HOST or DEVICE). In case of multivectors, vectors are stored column-wise. + * + * @note This function gives you access to the pointer, not to a copy. + * If you change the values using the pointer, the vector values will change too. + */ real_type* Vector::getData(memory::MemorySpace memspace) { return this->getData(0, memspace); } + /** + * @brief get a pointer to HOST or DEVICE data of a particular vector in a multivector. + * + * @param[in] i - Index of a vector in multivector + * @param[in] memspace - Memory space of the pointer (HOST or DEVICE) + * + * @return pointer to the _i_th vector data (HOST or DEVICE) within a multivector. + * + * @pre _i_ < _k_ i.e,, _i_ is smaller than the total number of vectors in multivector. + * + * @note This function gives you access to the pointer, not to a copy. + * If you change the values using the pointer, the vector values will change too. + */ real_type* Vector::getData(index_type i, memory::MemorySpace memspace) { if ((memspace == memory::HOST) && (cpu_updated_ == false) && (gpu_updated_ == true )) { @@ -165,6 +254,15 @@ namespace ReSolve { namespace vector { } + /** + * @brief copy internal vector data from HOST to DEVICE or from DEVICE to HOST + * + * @param[in] memspaceIn - Memory space of the data to copy FROM + * @param[in] memspaceOut - Memory space of the data to copy TO + * + * @return 0 if successful, -1 otherwise. + * + */ int Vector::copyData(memory::MemorySpace memspaceIn, memory::MemorySpace memspaceOut) { int control=-1; @@ -196,6 +294,12 @@ namespace ReSolve { namespace vector { return 0; } + /** + * @brief allocate vector data for HOST or DEVICE + * + * @param[in] memspace - Memory space of the data to be allocated + * + */ void Vector::allocate(memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -213,7 +317,12 @@ namespace ReSolve { namespace vector { } } - + /** + * @brief set vector data to zero. In case of multivectors, entire multivector is set to zero. + * + * @param[in] memspace - Memory space of the data to be set to 0 (HOST or DEVICE) + * + */ void Vector::setToZero(memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -237,6 +346,14 @@ namespace ReSolve { namespace vector { } } + /** + * @brief set the data of a single vector in a multivector to zero. + * + * @param[in] i - Index of a vector in a multivector + * @param[in] memspace - Memory space of the data to be set to 0 (HOST or DEVICE) + * + * @pre _i_ < _k_ i.e,, _i_ is smaller than the total number of vectors in multivector. + */ void Vector::setToZero(index_type j, memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -261,6 +378,15 @@ namespace ReSolve { namespace vector { } } + /** + * @brief set vector data to a given constant. + * + * In case of multivectors, entire multivector is set to the constant. + * + * @param[in] C - Constant (real number) + * @param[in] memspace - Memory space of the data to be set to 0 (HOST or DEVICE) + * + */ void Vector::setToConst(real_type C, memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -284,6 +410,15 @@ namespace ReSolve { namespace vector { } } + /** + * @brief set the data of a single vector in a multivector to a given constant. + * + * @param[in] j - Index of a vector in a multivector + * @param[in] C - Constant (real number) + * @param[in] memspace - Memory space of the data to be set to 0 (HOST or DEVICE) + * + * @pre _j_ < _k_ i.e,, _j_ is smaller than the total number of vectors in multivector. + */ void Vector::setToConst(index_type j, real_type C, memory::MemorySpace memspace) { using namespace ReSolve::memory; @@ -307,6 +442,19 @@ namespace ReSolve { namespace vector { } } + /** + * @brief get a pointer to HOST or DEVICE data of a specified vector in a multivector. + * + * @param[in] i - Index of a vector in a multivector + * @param[in] memspace - Memory space of the pointer (HOST or DEVICE) + * + * @return pointer to the _i_ th vector data (HOST or DEVICE). e + * + * @pre _i_ < _k_ i.e,, _i_ is smaller than the total number of vectors in multivector. + * + * @note This function gives you access to the pointer, not to a copy. + * If you change the values using the pointer, the vector values will change too. + */ real_type* Vector::getVectorData(index_type i, memory::MemorySpace memspace) { if (this->k_ < i) { @@ -316,6 +464,15 @@ namespace ReSolve { namespace vector { } } + /** + * @brief Set current lenght of the vector (use for vectors and multivectors that change size throughout computation). Note: vector needs to be allocated using maximum expected lenght. + * + * @param[in] new_n_current - New vector lenght + * + * @return 0 if successful, -1 otherwise. + * + * @pre _new_n_current_ <= _n_ i.e,, _new_n_current_ is smaller than the allocated vector lenght. + */ int Vector::setCurrentSize(int new_n_current) { if (new_n_current > n_) { @@ -326,6 +483,18 @@ namespace ReSolve { namespace vector { } } + /** + * @brief copy HOST or DEVICE data of a specified vector in a multivector to _dest_. + * + * @param[out] dest - Pointer to the memory to which data is copied + * @param[in] i - Index of a vector in a multivector + * @param[in] memspace - Memory space (HOST or DEVICE) to copy from + * + * @return 0 if successful, -1 otherwise. + * + * @pre _i_ < _k_ i.e,, _i_ is smaller than the total number of vectors in multivector. + * @pre _dest_ is allocated, and the size of _dest_ is at least _n_ (lenght of a single vector in the multivector). + */ int Vector::deepCopyVectorData(real_type* dest, index_type i, memory::MemorySpace memspaceOut) { using namespace ReSolve::memory; @@ -345,6 +514,18 @@ namespace ReSolve { namespace vector { } } + /** + * @brief copy HOST or DEVICE vector data to _dest_. + * + * In case of multivector, all data (size _k_ * _n_) is copied. + * + * @param[out] dest - Pointer to the memory to which data is copied + * @param[in] memspace - Memory space (HOST or DEVICE) to copy from + * + * @return 0 if successful, -1 otherwise. + * + * @pre _dest_ is allocated, and the size of _dest_ is at least _k_ * _n_ . + */ int Vector::deepCopyVectorData(real_type* dest, memory::MemorySpace memspaceOut) { using namespace ReSolve::memory; diff --git a/resolve/vector/Vector.hpp b/resolve/vector/Vector.hpp index f2403e4d..88641f00 100644 --- a/resolve/vector/Vector.hpp +++ b/resolve/vector/Vector.hpp @@ -4,6 +4,19 @@ #include namespace ReSolve { namespace vector { + /** + * @brief This class implements vectors (dense arrays) and multivectors and some basic utilities (get size, allocate, set data, get data, etc). + * + * + * What you need to know: + * - Multivectors are stored in one array, organized column-wise. A vector is a multivector of size 1. + * - There is a mirroring memory approach: the class has DEVICE and HOST data pointers. If needed, only one (or none) can be used or allocated. Unless triggered directly or by other function, the data is NOT automatically updated between HOST and DEVICE. + * - Constructor DOES NOT allocate memory. This has to be done separately. + * - You can get (and set) "raw" data easily, if needed. + * - There is memory ownership utility - vector can own memory (separate flags for HOST and DEVICE) or not, depending on how it is used. + * + * @author Kasia Swirydowicz + */ class Vector { public: @@ -36,14 +49,14 @@ namespace ReSolve { namespace vector { private: index_type n_; ///< size index_type k_; ///< k_ = 1 for vectors and k_>1 for multivectors (multivectors are accessed column-wise). - index_type n_current_; // if vectors dynamically change size, "current n_" keeps track of this. Needed for some solver implementations. - real_type* d_data_{nullptr}; - real_type* h_data_{nullptr}; - bool gpu_updated_; - bool cpu_updated_; + index_type n_current_; ///< if vectors dynamically changes size, "current n_" keeps track of this. Needed for some solver implementations. + real_type* d_data_{nullptr}; ///< DEVICE data array + real_type* h_data_{nullptr}; ///< HOST data array + bool gpu_updated_; ///< DEVICE data flag (updated or not) + bool cpu_updated_; ///< HOST data flag (updated or not) - bool owns_gpu_data_{false}; - bool owns_cpu_data_{false}; + bool owns_gpu_data_{false}; ///< data owneship flag for DEVICE data + bool owns_cpu_data_{false}; ///< data ownership flag for HOST data MemoryHandler mem_; ///< Device memory manager object };