Improved forecasters logic

TODOs_ROADMAP.rst

@@ -21,31 +21,30 @@ their planned release.
 ----------------------
 
 - [ ] Not part of public API; to be redesigned and probably dropped. Should use
-  ``_loader_*`` and ``_storer_*`` from ``cvxportfolio.data``. Target ``1.1.0``.
+  ``_loader_*`` and ``_storer_*`` from ``cvxportfolio.data``. 
 
 ``cvxportfolio.forecast``
 -------------------------
 
 - cache logic needs improvement, not easily exposable to third-parties now with ``dataclass.__hash__``
 
   - drop decorator
-  - drop dataclass
+  - drop dataclass, PR #133
   - cache IO logic should be managed by forecaster not by simulator, could be done by ``initialize_estimator``; maybe enough to just
     define it in the base class of forecasters
 - improve names of internal methods, clean them (lots of stuff can be re-used at universe change, ...)
 - generalize the mean estimator:
 
-  - use same code for ``past_returns``, ``past_returns**2``, ``past_volumes``, ...
-  - add rolling window option, should be in ``pd.Timedelta``
-  - add exponential moving avg, should be in half-life ``pd.Timedelta``
-- add same extras to the covariance estimator
+  - [X] use same code for ``past_returns``, ``past_returns**2``, ``past_volumes``, .... Done in #126, target ``1.2.0``
+  - [X] add rolling window option, should be in ``pd.Timedelta``. Done in #126, target ``1.2.0``
+  - [X] add exponential moving avg, should be in half-life ``pd.Timedelta``. Done in #126, target ``1.2.0``
+- [X] add same extras to the covariance estimator. Done in #126, target ``1.2.0``
 - goal: make this module crystal clear; third-party ML models should use it (at least for caching)
 
 ``cvxportfolio.estimator``
 --------------------------
 
-- [ ] ``DataEstimator`` needs refactoring, too long and complex methods. Target 
-  ``1.1.1``. 
+- [ ] ``DataEstimator`` needs refactoring, too long and complex methods.
 - ``Estimator`` could define base logic for on-disk caching. By itself it
   wouldn't do anything, actual functionality implemented by forecasters' base
   class.
@@ -65,8 +64,8 @@ their planned release.
   YF), total returns like other data sources, or neither for non-stocks assets.
   This would implement all data cleaning process as sequence of small steps
   in separate methods, with good logging. It would also implement data quality
-  check in the ``preload`` method to give feedback to the user. PR #125
-- [ ] Factor ``data.py`` in ``data/`` submodule. PR #125
+  check in the ``preload`` method to give feedback to the user. PR #127
+- [ ] Factor ``data.py`` in ``data/`` submodule. PR #127
 
 ``cvxportfolio.simulator``
 --------------------------
@@ -85,11 +84,12 @@ Partially public; only ``cvx.Gamma()`` (no arguments) and ``optimize_hyperparame
 (simple usage) are public, all the rest is not.
 
 - [ ] Clean up interface w/ ``MarketSimulator``, right now it calls private 
-  methods, maybe enough to make them public. Target ``1.1.1``.
+  methods, maybe enough to make them public.
 - [ ] Add risk/fine default ``GammaTrade``, ``GammaRisk`` (which are
   ``RangeHyperParameter``) modeled after original examples from paper.
-- [ ] Add ``Constant`` internal object throughout the library, also in ``DataEstimator``
+- [X] Add ``Constant`` internal object throughout the library, also in ``DataEstimator``
   in the case of scalar; it resolves to ``current_value`` if you pass a hyper-parameter.
+    Replaced with _resolve_hyperpar in #126.
 - [ ] Distinguish integer and positive hyper-parameters (also enforced by Constant).
 - [ ] Consider changing the increment/decrement model; hyperparameter object
   could instead return a ``neighbors`` set at each point. Probably cleaner.
@@ -98,7 +98,7 @@ Partially public; only ``cvx.Gamma()`` (no arguments) and ``optimize_hyperparame
 -------------------------
 
 - [ ] Add `AllIn` policy, which allocates all to a single name (like 
-  ``AllCash``). Target ``1.1.0``.
+  ``AllCash``).
 
 Optimization policies
 ~~~~~~~~~~~~~~~~~~~~~
@@ -115,7 +115,7 @@ Optimization policies
 ----------------------------
 
 - [ ] Add missing constraints from the paper.
-- [ ] Make ``MarketNeutral`` accept arbitrary benchmark (policy object).
+- [X] Make ``MarketNeutral`` accept arbitrary benchmark (policy object). Done in #126.
 
 ``cvxportfolio.result``
 -----------------------
@@ -144,7 +144,7 @@ Development & testing
 
 - [ ] Add extra pylint checkers. 
 
-  - [ ] Code complexity. Target ``1.1.1``. 
+  - [ ] Code complexity.
 - [ ] Consider removing downloaded data from ``test_simulator.py``,
   so only ``test_data.py`` requires internet. 
 

cvxportfolio/constraints/__init__.py

@@ -0,0 +1,45 @@
+# Copyright 2023 Enzo Busseti
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Here we define many realistic constraints that apply to :ref:`portfolio
+optimization trading policies <optimization-policies-page>`.
+
+Some of them, like :class:`LongOnly`, are
+very simple to use. Some others are more advanced,
+for example :class:`FactorNeutral`
+takes time-varying factor exposures as parameters.
+
+For a minimal example we present the classic Markowitz allocation.
+
+.. code-block:: python
+
+    import cvxportfolio as cvx
+
+    objective = cvx.ReturnsForecast() - gamma_risk * cvx.FullCovariance()
+
+    # the policy takes a list of constraint instances
+    constraints = [cvx.LongOnly(applies_to_cash=True)]
+
+    policy = cvx.SinglePeriodOptimization(objective, constraints)
+    print(cvx.MarketSimulator(universe).backtest(policy))
+
+With this, we require that the optimal post-trade weights
+found by the single-period optimization policy are non-negative.
+In our formulation the full portfolio weights vector (which includes
+the cash account) sums to one,
+see equation :math:`(4.9)` at page 43 of
+`the book <https://stanford.edu/~boyd/papers/pdf/cvx_portfolio.pdf>`_.
+"""
+
+from .base_constraints import *
+from .constraints import *

cvxportfolio/constraints/base_constraints.py

@@ -0,0 +1,135 @@
+# Copyright 2023 Enzo Busseti
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""This module defines base constraint classes."""
+
+
+from ..estimator import CvxpyExpressionEstimator, DataEstimator
+
+__all__ = ['Constraint', 'EqualityConstraint', 'InequalityConstraint']
+
+class Constraint(CvxpyExpressionEstimator):
+    """Base cvxpy constraint class."""
+
+    def compile_to_cvxpy(self, w_plus, z, w_plus_minus_w_bm):
+        """Compile constraint to cvxpy.
+
+        :param w_plus: Post-trade weights.
+        :type w_plus: cvxpy.Variable
+        :param z: Trade weights.
+        :type z: cvxpy.Variable
+        :param w_plus_minus_w_bm: Post-trade weights minus benchmark
+            weights.
+        :type w_plus_minus_w_bm: cvxpy.Variable
+        :returns: some cvxpy.constraints object, or list of those
+        :rtype: cvxpy.constraints, list
+        """
+        raise NotImplementedError # pragma: no cover
+
+
+class EqualityConstraint(Constraint):
+    """Base class for equality constraints.
+
+    This class is not exposed to the user, each equality
+    constraint inherits from this and overrides the
+    :func:`InequalityConstraint._compile_constr_to_cvxpy` and
+    :func:`InequalityConstraint._rhs` methods.
+
+    We factor this code in order to streamline the
+    design of :class:`SoftConstraint` costs.
+    """
+
+    def compile_to_cvxpy(self, w_plus, z, w_plus_minus_w_bm):
+        """Compile constraint to cvxpy.
+
+        :param w_plus: Post-trade weights.
+        :type w_plus: cvxpy.Variable
+        :param z: Trade weights.
+        :type z: cvxpy.Variable
+        :param w_plus_minus_w_bm: Post-trade weights minus benchmark
+            weights.
+        :type w_plus_minus_w_bm: cvxpy.Variable
+        :returns: Cvxpy constraints object.
+        :rtype: cvxpy.constraints
+        """
+        return self._compile_constr_to_cvxpy(w_plus, z, w_plus_minus_w_bm) ==\
+            self._rhs()
+
+    def _compile_constr_to_cvxpy(self, w_plus, z, w_plus_minus_w_bm):
+        """Cvxpy expression of the left-hand side of the constraint."""
+        raise NotImplementedError # pragma: no cover
+
+    def _rhs(self):
+        """Cvxpy expression of the right-hand side of the constraint."""
+        raise NotImplementedError # pragma: no cover
+
+
+class InequalityConstraint(Constraint):
+    """Base class for inequality constraints.
+
+    This class is not exposed to the user, each inequality
+    constraint inherits from this and overrides the
+    :func:`InequalityConstraint._compile_constr_to_cvxpy` and
+    :func:`InequalityConstraint._rhs` methods.
+
+    We factor this code in order to streamline the
+    design of :class:`SoftConstraint` costs.
+    """
+
+    def compile_to_cvxpy(self, w_plus, z, w_plus_minus_w_bm):
+        """Compile constraint to cvxpy.
+
+        :param w_plus: Post-trade weights.
+        :type w_plus: cvxpy.Variable
+        :param z: Trade weights.
+        :type z: cvxpy.Variable
+        :param w_plus_minus_w_bm: Post-trade weights minus benchmark
+            weights.
+        :type w_plus_minus_w_bm: cvxpy.Variable
+        :returns: Cvxpy constraints object.
+        :rtype: cvxpy.constraints
+        """
+        return self._compile_constr_to_cvxpy(w_plus, z, w_plus_minus_w_bm) <=\
+            self._rhs()
+
+    def _compile_constr_to_cvxpy(self, w_plus, z, w_plus_minus_w_bm):
+        """Cvxpy expression of the left-hand side of the constraint."""
+        raise NotImplementedError # pragma: no cover
+
+    def _rhs(self):
+        """Cvxpy expression of the right-hand side of the constraint."""
+        raise NotImplementedError # pragma: no cover
+
+
+class CostInequalityConstraint(InequalityConstraint):
+    """Linear inequality constraint applied to a cost term.
+
+    The user does not interact with this class directly,
+    it is returned by an expression such as ``cost <= value``
+    where ``cost`` is a :class:`Cost` instance and ``value``
+    is a scalar.
+    """
+
+    def __init__(self, cost, value):
+        self.cost = cost
+        self.value = DataEstimator(value, compile_parameter=True)
+
+    def _compile_constr_to_cvxpy(self, w_plus, z, w_plus_minus_w_bm):
+        """Compile constraint to cvxpy."""
+        return self.cost.compile_to_cvxpy(w_plus, z, w_plus_minus_w_bm)
+
+    def _rhs(self):
+        return self.value.parameter
+
+    def __repr__(self):
+        return self.cost.__repr__() + ' <= ' + self.value.__repr__()