Merge pull request #122 from cvxgrp/semantic_versioning

Semantic versioning

.gitignore

@@ -95,6 +95,9 @@ ENV/
 # Spyder project settings
 .spyderproject
 
+# Codium
+.vscode/*
+
 # Rope project settings
 .ropeproject
 

README.rst

@@ -12,29 +12,50 @@ paper <https://web.stanford.edu/~boyd/papers/pdf/cvx_portfolio.pdf>`__.
 The documentation of the library is at
 `www.cvxportfolio.com <https://www.cvxportfolio.com>`__.
 
+
+.. Installation
+
 Installation
 ------------
 
-You can install our latest release with
+Cvxportolio is written in Python and can easily installed in any Python
+environment by simple:
 
-.. code:: bash
+.. code:: console
 
    pip install -U cvxportfolio
 
 You can see how this works on our `Installation and Hello
-World <https://youtu.be/1ThOKEu371M>`__ youtube video.
+World <https://youtu.be/1ThOKEu371M>`__ youtube video. 
+Anaconda installs 
+`are also supported <https://anaconda.org/conda-forge/cvxportfolio>`_.
+
+Cvxportfolio's main dependencies are `Cvxpy <https://www.cvxpy.org>`_ for
+interfacing with numerical solvers and `Pandas <https://pandas.pydata.org/>`_
+for interfacing with databases. We don't require any specific version of our
+dependencies and test against all recent ones (up to a few years ago).
+
 
-Testing locally
----------------
+.. Test
 
-After installing you can run our unit test suite in you local
-environment by
+Test
+----
 
-.. code:: bash
+After installing you can run our unit test suite in you local environment by
+
+.. code:: console
 
    python -m cvxportfolio.tests
 
-Simple Example
+We test against recent python versions (3.9, 3.10, 3.11) and recent versions
+of the main dependencies (from pandas 1.4, cvxpy 1.1, ..., up to the current
+versions) on all major operating systems. You can see the `automated testing code 
+<https://github.com/cvxgrp/cvxportfolio/blob/master/.github/workflows/test.yml>`_.
+
+
+.. Simple Example
+
+Simple example
 --------------
 
 In the following example market data is downloaded by a public source
@@ -73,8 +94,23 @@ includes holding and transaction costs, using the models described in
 the paper, and default parameters that are typical for the US stock
 market.
 
-Some Other Examples
--------------------
+Other examples
+--------------
+
+`Many examples 
+<https://www.cvxportfolio.com/en/latest/examples.html>`_ 
+are shown in the documentation website, along with
+their output and comments.
+
+`Even more example scripts
+<https://github.com/cvxgrp/cvxportfolio/blob/master/examples>`_ 
+are available in the code repository. 
+
+`The original examples from the paper 
+<https://github.com/cvxgrp/cvxportfolio/tree/0.0.X/examples>`_ 
+are visible in a dedicated branch,
+and are being translated to run with the stable versions (``1.0.*``) of the
+library. 
 
 We show in the example on `user-provided
 forecasters <https://github.com/cvxgrp/cvxportfolio/blob/master/examples/user_provided_forecasters.py>`__
@@ -93,6 +129,25 @@ how a simple sweep over hyper-parameters, taking advantage of our
 sophisticated parallel backtest machinery, quickly provides results on
 the best strategy to apply to any given selection of assets.
 
+
+.. Contributions
+
+Contributions
+-------------
+
+We welcome contributors and you don't need to sign a CLA. If you don't have
+a Github account you may also send a 
+`git patch via email <https://git-scm.com/docs/git-send-email>`_ to the 
+`project maintainer <https://github.com/enzbus>`_.
+
+Bug fixes, improvements in the documentations and examples,
+new constraints, new cost objects, ..., are good contributions and can be done
+even if you're not familiar with the low-level details on the library.
+For more advanced contributions we recommend reading the
+`TODOs and roadmap
+<https://github.com/cvxgrp/cvxportfolio/blob/master/TODOs_ROADMAP.rst>`_
+file.
+
 Development
 -----------
 
@@ -101,7 +156,7 @@ repository (or, `fork on
 Github <https://docs.github.com/en/get-started/quickstart/fork-a-repo>`__
 and then clone your fork)
 
-.. code:: bash
+.. code:: console
 
    git clone https://github.com/cvxgrp/cvxportfolio.git
    cd cvxportfolio
@@ -111,7 +166,7 @@ Then, you should have a look at our
 and possibly change the ``PYTHON`` variable to match your system’s
 python interpreter. Once you have done that,
 
-.. code:: bash
+.. code:: console
 
    make env
    make test
@@ -123,7 +178,7 @@ test suite.
 You activate the shell environment with one of scripts in ``env/bin``
 (or ``env\Scripts`` on Windows), for example if you use bash on POSIX
 
-.. code:: bash
+.. code:: console
 
    source env/bin/activate
 
@@ -137,21 +192,52 @@ Windows) like we do in the Makefile.
 Additionally, to match our CI/CD pipeline, you may set the following
 `git hooks <https://git-scm.com/docs/githooks>`__
 
-.. code:: bash
+.. code:: console
 
    echo "make lint" > .git/hooks/pre-commit
    chmod +x .git/hooks/pre-commit
    echo "make test" > .git/hooks/pre-push
    chmod +x .git/hooks/pre-push
 
-Examples from the paper
------------------------
 
-In branch `0.0.X <https://github.com/cvxgrp/cvxportfolio/tree/0.0.X>`__
-you can find the original material used to generate plots and results in
-the paper. As you may see from those ipython notebooks a lot of the
-logic that was implemented there, outside of Cvxportfolio proper, is
-being included and made automatic in newer versions of Cvxportfolio.
+Code style and quality
+----------------------
+
+Cvxportfolio follows the `PEP8 <https://peps.python.org/pep-0008/>`_
+specification for code style. This is enforced by the `Pylint
+<https://pylint.readthedocs.io/en/stable/>`_ automated linter, with options 
+in the `Pyproject 
+<https://github.com/cvxgrp/cvxportfolio/blob/master/pyproject.toml>`_
+configuration file.
+Pylint is also used to enforce code quality standards, along with some of its
+optional plugins.
+Docstrings are written in the `Sphinx style 
+<https://www.sphinx-doc.org/en/master/index.html>`_, are also checked by 
+Pylint, and are used to generate the documentation.
+
+.. Versions
+
+Versions and releases
+---------------------
+
+Cvxportfolio follows the `semantic versioning <https://semver.org>`_
+specification. No breaking change in its public API will be introduced
+until the next major version (``2.0.0``), which won't happen for some time. 
+New features in the public API are introduced with minor versions 
+(``1.1.0``, ``1.2.0``, ...), and only bug fixes at each revision.
+
+The history of our releases (source distributions and wheels) is visible on our 
+`PyPI page <https://pypi.org/project/cvxportfolio/#history>`_.
+
+Releases are also tagged in our git repository and include a short summary
+of changes in 
+`their commit messages <https://github.com/cvxgrp/cvxportfolio/tags>`_.
+
+We maintain a `document listing the planned changes and target releases
+<https://github.com/cvxgrp/cvxportfolio/blob/master/TODOs_ROADMAP.rst>`_. 
+
+
+.. Citing
 
 Citing
 ------------
@@ -200,7 +286,7 @@ The latter is also the first chapter of this PhD thesis:
 Licensing
 ---------
 
-Cvxportfolio is licensed under the `Apache 2.0 <http://www.apache.org/licenses/>`_ permissive
+Cvxportfolio is licensed under the `Apache 2.0 <https://www.apache.org/licenses/LICENSE-2.0>`_ permissive
 open source license.
 
 .. |CVXportfolio on PyPI| image:: https://img.shields.io/pypi/v/cvxportfolio.svg

TODOs_ROADMAP.rst

@@ -0,0 +1,110 @@
+TODOs and ROADMAP
+=================
+
+Cvxportfolio follows the `semantic versioning <https://semver.org>`_
+specification for changes in its public API. The public API is defined
+as:
+
+- public methods, *i.e.*, without a leading underscore ``_``
+- methods and objects clearly documented as such and/or used in the examples.
+
+*Internal* methods that are used by 
+Cvxportfolio objects to communicate with each other (or other tasks) and don't
+have a leading underscore, are considered public if they are exposed through 
+the HTML documentation and/or are used in the examples.
+
+In this document we list the planned
+changes, in particular the ones that are relevant for semantic versioning, and 
+their planned release.
+
+``cvxportfolio.cache``
+----------------------
+
+- [ ] Not part of public API; to be redesigned and probably dropped. Should use
+  ``_loader_*`` and ``_storer_*`` from ``cvxportfolio.data``. Target ``1.1.0``.
+
+``cvxportfolio.forecast``
+-------------------------
+
+- cache logic needs improvement, not easily exposable to third-parties now with ``dataclass.__hash__``
+
+  - drop decorator
+  - drop dataclass
+  - cache IO logic should be managed by forecaster not by simulator, could be done by ``initialize_estimator``; maybe enough to just
+    define it in the base class of forecasters
+- improve names of internal methods, clean them (lots of stuff can be re-used at universe change, ...)
+- generalize the mean estimator:
+
+  - use same code for ``past_returns``, ``past_returns**2``, ``past_volumes``, ...
+  - add rolling window option, should be in ``pd.Timedelta``
+  - add exponential moving avg, should be in half-life ``pd.Timedelta``
+- add same extras to the covariance estimator
+- goal: make this module crystal clear; third-party ML models should use it (at least for caching)
+
+``cvxportfolio.estimator``
+--------------------------
+
+- [ ] ``DataEstimator`` needs refactoring, too long and complex methods. Target 
+  ``1.0.4``. 
+- ``Estimator`` could define base logic for on-disk caching. By itself it
+  wouldn't do anything, actual functionality implemented by forecasters' base
+  class.
+
+  - [ ] ``initialize_estimator`` could get optional market data partial
+    signature for caching. Default None, no incompatible change.
+  - [ ] Could get a ``finalize_estimator`` method used for storing
+    data, like risk models on disk, doesn't need arguments; it can use the
+    partial signature got above. No incompatible change.
+
+``cvxportfolio.data``
+--------------------------
+
+``cvxportfolio.simulator``
+--------------------------
+
+``cvxportfolio.risks``
+----------------------
+
+``cvxportfolio.policies``
+-------------------------
+
+Optimization policies
+~~~~~~~~~~~~~~~~~~~~~
+
+- [ ] Improve behavior for infeasibility/unboundedness/solver error. Target 
+  ``1.1.0``.
+
+``cvxportfolio.constraints``
+----------------------------
+
+- [ ] Add missing constraints from the paper. Target ``1.1.0``.
+- [ ] Make ``MarketNeutral`` accept arbitrary benchmark (policy object).
+
+``cvxportfolio.result``
+-----------------------
+
+- [ ] Make ``BackTestResult`` interface methods with ``MarketSimulator`` 
+  public. 
+
+
+Development & testing
+---------------------
+
+- [ ] Add extra pylint checkers. 
+
+  - [ ] Code complexity. Target ``1.0.4``. 
+- [ ] Consider removing downloaded data from ``test_simulator.py``,
+  so only ``test_data.py`` requires internet. 
+
+Documentation
+-------------
+
+- Manual
+- Quickstart
+
+Examples
+--------
+
+- [ ] Restore examples from paper. Target ``1.0.4``, PR #118.
+- [ ] Expose more (all?) examples through HTML docs. Target ``1.0.4``, PR #118.
+- [ ] Consider making examples a package that can be pip installed.

docs/contributing.rst

@@ -0,0 +1,15 @@
+Contributing to Cvxportfolio
+============================
+
+Cvxportfolio is a collaborative project. Its development is hosted
+on Github at `this repository <https://github.com/cvxgrp/cvxportfolio>`_.
+
+You can open bug reports using the 
+`issue tracking <https://github.com/cvxgrp/cvxportfolio/issues>`_
+system there.
+
+The following is copied from the README file on the repository.
+
+.. include:: ../README.rst    
+    :start-after: .. Contributions
+    :end-before: .. Versions