cvxcovariance

The cvxcovariance package provides simple tools for creating an estimate $\hat\Sigma_t$ of the covariance $\Sigma_t$ of the $n$-dimensional return vectors $r_t$, $t=1,2,\ldots$, based on the observed returns $r_1, \ldots, r_{t-1}$. (Here $r_t$ is the return from $t-1$ to $t$.) The covariance predictor $\hat\Sigma_t$ is generated by blending $K$ different "expert" predictors $\hat\Sigma_t^{(1)},\ldots,\hat\Sigma_t^{(K)}$, by solving a convex optimization problem at each time step.

For a detailed description of the methodology, see our manuscript A Simple Method for Predicting Covariance Matrices of Financial Returns (in particular Section 3).

In the simplest case the user provides a $T\times n$ pandas DataFrame of returns $r_1,\ldots,r_T$ and $K$ half-life pairs, and gets back covariance predictors for each time step. (The $K$ experts are computed as iterated exponentially weighted moving average (IEWMA) predictors as described in Section 2.5 of the paper.) In the more general case, the user provides the $K$ expert predictors $\hat\Sigma_t^{(1)},\ldots,\hat\Sigma_t^{(K)}$, $t=1,\ldots,T$, and these are blended together by solving the convex optimization problems. In either case the result is returned as an iterator object over namedtuples: Result = namedtuple("Result", ["time", "mean", "covariance", "weights"]).

Note: at time $t$ the user is provided with $\hat\Sigma_{t+1}$, $\textit{i.e.}$, the covariance matrix for the next time step. So Result.covariance returns the covariance prediction for time+1.

Installation

To install the package, run the following command in the terminal:

pip install cvxcovariance

Usage

There are two alternative ways to use the package. The first is to use the from_ewmas function to create a combined multiple IEWMA (CM-IEWMA) predictor. The second is to provide your own covariance experts, via dictionaries, and pass them to the from_sigmas function. Both functions return an object of the _CovarianceCombination class, which can be used to solve the covariance combination problem.

CM-IEWMA

The from_ewmas function takes as input a pandas DataFrame of returns and the IEWMA half-life pairs (each pair consists of one half-life for volatility estimation and one for correlation estimation), and returns an iterator object that iterates over the CM-IEWMA covariance predictors defined via namedtuples. Through the namedtuple you can access the time, mean, covariance, and weights attributes. time is the timestamp. mean is the estimated mean of the return at the $\textit{next}$ timestamp, $\textit{i.e.}$ time+1, if the user wants to estimate the mean; per default the mean is set to zero, which is a reasonable assumption for many financial returns. covariance is the estimated covariance matrix for the $\textit{next}$ timestamp, $\textit{i.e.}$ time+1. weights are the $K$ weights attributed to the experts. Here is an example:

import pandas as pd
from cvx.covariance.combination import from_ewmas

# Load return data
returns = pd.read_csv("data/ff5_no_rf.csv", index_col=0, header=0, parse_dates=True).iloc[:1000]
n = returns.shape[1]

# Define half-life pairs for K=3 experts, (halflife_vola, halflife_cov)
halflife_pairs = [(10, 21), (21, 63), (63, 125)]

# Define the covariance combinator
combinator = from_ewmas(returns,
                        halflife_pairs,
                        min_periods_vola=n,  # min periods for volatility estimation
                        min_periods_cov=3 * n)  # min periods for correlation estimation

# Solve combination problem and loop through combination results to get predictors
covariance_predictors = {}
for predictor in combinator.solve(window=10):  # lookback window for optimization
    # From predictor we can access predictor.time, predictor.mean (=0 here),
    # predictor.covariance, and predictor.weights
    covariance_predictors[predictor.time] = predictor.covariance

Here covariance_predictors[t] is the covariance prediction for time $t+1$, $\textit{i.e.}$, it is uses knowledge of $r_1,\ldots,r_t$.

General covariance combination

The from_sigmas function takes as input a pandas DataFrame of returns and a dictionary of covariance predictors {key: {time: sigma}, where key is the key of an expert predictor and {time: sigma} is the expert predictions. For example, here we combine two EWMA covariance predictors from pandas:

import pandas as pd
from cvx.covariance.combination import from_sigmas

# Load return data
returns = pd.read_csv("data/ff5_no_rf.csv", index_col=0,
                      header=0, parse_dates=True).iloc[:1000]
n = returns.shape[1]

# Define 21 and 63 day EWMAs as dictionaries (K=2 experts)
ewma21 = returns.ewm(halflife=21, min_periods=5 * n).cov().dropna()
expert1 = {time: ewma21.loc[time] for time in ewma21.index.get_level_values(0).unique()}
ewma63 = returns.ewm(halflife=63, min_periods=5 * n).cov().dropna()
expert2 = {time: ewma63.loc[time] for time in ewma63.index.get_level_values(0).unique()}

# Create expert dictionary
experts = {1: expert1, 2: expert2}

# Define the covariance combinator
combinator = from_sigmas(sigmas=experts, returns=returns)

# Solve combination problem and loop through combination results to get predictors
covariance_predictors = {}
for predictor in combinator.solve(window=10):
    # From predictor we can access predictor.time, predictor.mean (=0 here),
    # predictor.covariance, and predictor.weights
    covariance_predictors[predictor.time] = predictor.covariance

Here covariance_predictors[t] is the covariance prediction for time $t+1$, $\textit{i.e.}$, it is uses knowledge of $r_1,\ldots,r_t$.

Poetry

We assume you share already the love for Poetry. Once you have installed poetry you can perform

make install

to replicate the virtual environment we have defined in pyproject.toml and locked in poetry.lock.

Jupyter

We install JupyterLab on fly within the aforementioned virtual environment. Executing

make jupyter

will install and start the jupyter lab.

Citing

If you want to reference our paper in your research, please consider citing us by using the following BibTeX:

@article{johansson2023covariance,
url = {http://dx.doi.org/10.1561/0800000047},
year = {2023},
volume = {12},
journal = {Foundations and Trends® in Econometrics},
title = {A Simple Method for Predicting Covariance Matrices of Financial Returns},
doi = {10.1561/0800000047},
issn = {1551-3076},
number = {4},
pages = {324-407},
author = {Kasper Johansson
and Mehmet G. Ogut
and Markus Pelger
and Thomas Schmelzer
and Stephen Boyd}
}