From 0f655deb4300d09f7e0d752288eb7cd06a4b1e8f Mon Sep 17 00:00:00 2001 From: AntoinePrv Date: Mon, 13 Nov 2023 19:36:31 +0100 Subject: [PATCH] Simplify development doc --- docs/source/developer_zone/build_locally.rst | 294 ------------------ .../source/developer_zone/dev_environment.rst | 169 ++++++++++ docs/source/index.rst | 2 +- 3 files changed, 170 insertions(+), 295 deletions(-) delete mode 100644 docs/source/developer_zone/build_locally.rst create mode 100644 docs/source/developer_zone/dev_environment.rst diff --git a/docs/source/developer_zone/build_locally.rst b/docs/source/developer_zone/build_locally.rst deleted file mode 100644 index 1d70c916ff..0000000000 --- a/docs/source/developer_zone/build_locally.rst +++ /dev/null @@ -1,294 +0,0 @@ -============= -Build locally -============= - -Local development environment -============================= - -Clone the repo -************** - -Start by cloning the repo: - -.. code:: - - git clone https://github.com/mamba-org/mamba.git - -Then move to the newly created directory: - -.. code:: - - cd mamba - -Install dev requirements -************************ - -| Make sure you have Mamba's development requirements installed in your environment. -| To do so, you can use your existing ``mamba`` or ``micromamba`` installation: - -.. code:: - - mamba env update --name --file .//environment-dev.yml - -.. code:: - - micromamba install --name --file .//environment-dev.yml - -Pick the correct environment spec file depending on the project you want to build. - -If you don't have an existing env, refer to the :ref:`installation` page. - -The different parts of this repository are sub-projects, all relying (totally or partially) on ``cmake`` configuration. - -.. note:: - All ``cmake`` commands listed below use ``bash`` multi-line syntax. - On Windows, replace ``\`` trailing character with ``^``. - -.. note:: - Feel free to use your favorite generator: ``make``, ``ninja``, etc. - - -Build ``libmamba`` -================== - -``libmamba`` build is enabled using ``BUILD_LIBMAMBA`` ``cmake`` option. - -Shared library -************** - -``BUILD_SHARED`` option needs to be activated: - -.. code:: bash - - cmake -B build/ \ - -DBUILD_LIBMAMBA=ON \ - -DBUILD_SHARED=ON - cmake --build build/ - -.. note:: - ``libmamba`` ``cmake`` target represents the shared library - -Static library -************** - -``BUILD_STATIC`` option needs to be activated: - -.. code:: bash - - cmake -B build/ \ - -DBUILD_LIBMAMBA=ON \ - -DBUILD_STATIC=ON - cmake --build build/ - -.. note:: - ``libmamba-static`` ``cmake`` target represents the static library with static dependencies linkage - -.. note:: - The ``libmamba`` static library does not embed the dependencies but the ``cmake`` target will expose those dependencies to any executable linking on it - -.. note:: - The ``libmamba`` static lib still has few symbols required from system shared libraries (``glibc`` for instance) - -.. warning:: - This version of the library has a small difference versus the shared one, on the way the SSL backend of cURL is set - See `libmamba/src/core/fetch.cpp` for more information - -Tests -***** - -| C++ tests require ``libmamba`` to be built. -| To compile the ``gtest``-based C++ test suite, run: - -.. code:: - - cmake -B build/ \ - -DBUILD_LIBMAMBA=ON \ - -DBUILD_SHARED=ON \ - -DBUILD_LIBMAMBA_TESTS=ON - cmake --build build/ - -You should now be able to run: - -.. code:: - - ./build/libmamba/tests/test_libmamba - -Alternatively you can use: - -.. code:: - - cmake --build build/ --target test - -.. note:: - If you want to run specific or a subset of tests, you can use ``GTEST_FILTER`` environment variable or the ``--gtest_filter`` flag. - -Build ``libmambapy`` -==================== - -The Python bindings of ``libmamba``, ``libmambapy`` can be built by using the ``BUILD_LIBMAMBAPY`` ``cmake`` option. - -You can either rely on ``libmamba`` package already installed in your environment and run: - -.. code:: bash - - cmake -B build/ \ - -DBUILD_LIBMAMBAPY=ON - cmake --build build/ - -or rebuild ``libmamba`` at the same time: - -.. code:: bash - - cmake -B build/ \ - -DBUILD_LIBMAMBA=ON \ - -DBUILD_SHARED=ON \ - -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX \ - -DCMAKE_PREFIX_PATH=$CONDA_PREFIX \ - -DBUILD_LIBMAMBAPY=ON - cmake --build build/ - -.. note:: - When rebuilding ``libmamba``, you also need to install the library in a path it will be found. - Use for that the ``CMAKE_INSTALL_PREFIX`` ``cmake`` option to point your current development environment, and ``CMAKE_PREFIX_PATH`` ``cmake`` option to specify the installation prefixes to be searched: - - - ``-DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX`` [unix] - - ``-DCMAKE_PREFIX_PATH=$CONDA_PREFIX`` [unix] - - ``-DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX\\Library`` [win] - - ``-DCMAKE_PREFIX_PATH=$CONDA_PREFIX\\Library`` [win] - -You'll need to install the target to have the bindings Python sub-module correctly located, then you can use ``pip`` to install that Python package: - -.. code:: bash - - make install -C build/ - pip install -e libmambapy/ --no-deps - -.. note:: - The editable mode ``-e`` provided by ``pip`` allows to use the source directory as the Python package instead of copying sources inside the environment - You can then change the code without having to reinstall the package - -.. note:: - The ``--no-deps`` tells ``pip`` to skip the dependencies installation, since they are already installed (``libmamba`` installed using ``cmake``) - -.. note:: - ``libmambapy`` is dynamically linking against ``libmamba`` (shared) library - - -Build ``mamba`` -=============== - -You need to build and install ``libmambapy``, which is a dependency of ``mamba``, then install ``mamba``: - -.. code:: - - pip install -e ./mamba/ --no-deps - -.. note:: - Other dependencies are already installed from the `environment-dev.yml` file - - -Build ``micromamba`` -==================== - -Dynamically linked -****************** - -To build ``micromamba``, activate the ``BUILD_MICROMAMBA`` flag in ``cmake`` command: - -.. code:: bash - - cmake -B build/ \ - -DBUILD_MICROMAMBA=ON \ - -DBUILD_SHARED=ON - cmake --build build/ - -or rebuild ``libmamba`` at the same time: - -.. code:: bash - - cmake -B build/ \ - -DBUILD_LIBMAMBA=ON \ - -DBUILD_SHARED=ON \ - -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX \ - -DCMAKE_PREFIX_PATH=$CONDA_PREFIX \ - -DBUILD_MICROMAMBA=ON - cmake --build build/ - -.. note:: - If you need to install, use the ``CMAKE_INSTALL_PREFIX`` ``cmake`` option to point your current development environment: - - - ``-DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX`` [unix] - - ``-DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX\\Library`` [win] - - You may need to use the ``CMAKE_PREFIX_PATH`` ``cmake`` option as well, to specify the installation prefixes to be searched: - - - ``-DCMAKE_PREFIX_PATH=$CONDA_PREFIX`` [unix] - - ``-DCMAKE_PREFIX_PATH=$CONDA_PREFIX\\Library`` [win] - -.. note:: - ``micromamba`` will be dynamically linked against ``libmamba`` and all its dependencies (``libsolv``, ``libarchive``, etc.) - -.. note:: - ``MICROMAMBA_LINKAGE`` default value is ``DYNAMIC`` - -Statically linked -***************** - -You can also build ``micromamba`` as a fully statically linked executable. For that, you need to install extra requirements: - -.. code:: - - micromamba install --name --file ./libmamba/environment-static-dev.yml - -It will install the development version of dependencies, including static libraries. - -Now you can run ``cmake``: - -.. code:: bash - - cmake -B build/ \ - -DBUILD_MICROMAMBA=ON \ - -DBUILD_STATIC=ON - cmake --build build/ - -or with ``libmamba``: - -.. code:: bash - - cmake -B build/ \ - -DBUILD_LIBMAMBA=ON \ - -DBUILD_STATIC=ON \ - -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX \ - -DCMAKE_PREFIX_PATH=$CONDA_PREFIX \ - -DBUILD_MICROMAMBA=ON - cmake --build build/ - -.. note:: - If you decide to build both the static and shared libraries, ``micromamba`` will link with the shared one. To link - with the static library, pass the additional build option ``-DMICROMAMBA_LINKAGE=STATIC`` - -Tests -***** - -In order to run the Python-based test suite, you need to set the following environment variables: - -.. code:: - - export TEST_MAMBA_EXE=build/micromamba/micromamba - export MAMBA_ROOT_PREFIX=YOUR_MICROMAMBA_ROOT_PREFIX - -Then, you should be able to run the tests: - -.. code:: - - pytest micromamba/tests/ - -Since running all the tests would take a great amount of time, you could choose to run only a specific test. -To launch ``test_env`` for example, you can run: - -.. code:: - - pytest micromamba/tests/test_env.py - -.. note:: - You could also use ``pytest`` ``-k`` option to filter by test full name or substring. diff --git a/docs/source/developer_zone/dev_environment.rst b/docs/source/developer_zone/dev_environment.rst new file mode 100644 index 0000000000..7b79869dd0 --- /dev/null +++ b/docs/source/developer_zone/dev_environment.rst @@ -0,0 +1,169 @@ +======================= +Development Envrionment +======================= + +.. warning:: + + These instruction have some inaccuracies on Windows. + +Get the code and Mamba +====================== + +Clone the repo +************** + +Fork and clone the repository in your prefered manner. +Refer to Github documentation for how to do so. + +Install micromamba +****************** + +Setting the development environment requires conda, mamba, or micromamba. +Since some commands are automated with ``micromamba``, and to avoid any runtime linking issue, +it is best to work with micromamba. + +Refer to the :ref:`installation` page for how to install micromamba or mamba. + +Develop using Taskfile +====================== + +Many development opetation can be automated and chained using `Taskfile `_. +You can follow the installation instruction there, or install it via `conda-forge`. + +.. code:: bash + + micromamba create -n mamba -c conda-forge go-task + micromamba activate -n mamba + +Running commands +**************** + +Tasks and their description can be queried with: + +.. code:: bash + + task --list-all + +For instance to run ``libmamba`` tests, execute: + +.. code:: bash + + task test-libmamba + +Extra command line arguments can also be passed to the underlying command: + +.. code:: bash + + task test-libmamba -- --test-suite='util::*' + + +Development tools +***************** + +With any command, ``task`` will run ``create-dev-env`` with all development dependencies. +This environment can be activated to get development tools such as +`LSP `_ code completion and lint. + + +Running commands manually +========================= + +.. note:: + + Even if not using Taskfile, the file ``Taskfile.dist.yml`` can provide insights on + useful commands. + The CI files in ``.github/workflow`` provide an alternative way of developing Mamba. + +Install develoment dependencies +******************************* + +.. code:: bash + + micromamba create -n mamba -c conda-forge -f dev/environment.yml + micromamba activate -n mamba + +Compile ``libmmaba`` and other artifacts +**************************************** + +``libmamba`` is build using CMake. +Typically during development, development, we build everything dynamically using dependencies +from Conda-Forge. + +The first step is to configure the build options. +A recommended set is already provided as CMake Preset, but feel free to use any variations. + +.. note:: + All ``cmake`` commands listed below use ``bash`` multi-line syntax. + On Windows, replace ``\`` trailing character with ``^``. + +.. code:: bash + + cmake -B build/ -G Ninja --preset mamba-unix-shared-debug-dev + +Compilation can then be launched with: + +.. code:: bash + + cmake --build build/ --parallel + +``libmamba`` tests +****************** + +The test for libamba are written in C++. + +.. code:: bash + + ./build/libmamba/tests/test_libmamba + +``micromamba`` integration tests +******************************** + +Many ``micromamba`` integration tests are written through a Pytest Python wrapper. +The environment variable ``TEST_MAMBA_EXE`` controls which executable is being tested. + +.. code:: bash + + export TEST_MAMBA_EXE="${PWD}/build/micromamba/micromamba" + python -m pytest micromamba/tests + +``libmambapy`` tests +******************** + +To run the ``libmambapy`` tests, the Python package needs to be properly installed first. + +.. warning:: + + This needs to be done every time ``libamamba`` changes. + +.. code:: bash + + cmake --install build/ --prefix "${CONDA_PREFIX}" + +Then the Python bindings can be installed + +.. code:: bash + + python -m pip install --no-deps --no-build-isolation libmambapy/ + +Finally the tests can be run: + +.. code:: bash + + python -m pytest libmambapy/tests + +Code Formatting +=============== + +Code formatting is done using Pre Commit hooks. +Whichever way you decided to install development dependencies, we recommend installing +Pre Commit hooks with + +.. code:: bash + + pre-commit install + +Alternatively, the checks can be run manually + +.. code:: bash + + pre-commit run --all-files diff --git a/docs/source/index.rst b/docs/source/index.rst index 855751b251..7ec316eb90 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -62,5 +62,5 @@ You can try Mamba now by visiting the :ref:`installation page`! :hidden: developer_zone/contributing - developer_zone/build_locally + developer_zone/dev_environment developer_zone/internals