Merge pull request #2 from adalseno/latest

Latest merge into master

README.md

@@ -8,54 +8,31 @@ Authors:
 
 The repo structure follows this post: https://khuyentran1401.github.io/reproducible-data-science/structure_project/introduction.html
 
-## Tools used in this project
-* [Poetry](https://towardsdatascience.com/how-to-effortlessly-publish-your-python-package-to-pypi-using-poetry-44b305362f9f): Dependency management - [article](https://mathdatasimplified.com/2023/06/12/poetry-a-better-way-to-manage-python-dependencies/)
-* [hydra](https://hydra.cc/): Manage configuration files - [article](https://mathdatasimplified.com/2023/05/25/stop-hard-coding-in-a-data-science-project-use-configuration-files-instead/)
-* [pre-commit plugins](https://pre-commit.com/): Automate code reviewing formatting
-* [DVC](https://dvc.org/): Data version control - [article](https://mathdatasimplified.com/2023/02/20/introduction-to-dvc-data-version-control-tool-for-machine-learning-projects-2/)
-* [pdoc](https://github.com/pdoc3/pdoc): Automatically create an API documentation for your project
-
-## Set up the environment
-1. Install [Poetry](https://python-poetry.org/docs/#installation)
-2. Set up the environment:
-```bash
-make env 
-```
-
-## Install dependencies
-To install all dependencies for this project, run:
-```bash
-poetry install
-```
-
-To install a new package, run:
-```bash
-poetry add <package-name>
-```
-
-## Version your data
-To track changes to the "data" directory, type:
-```bash
-dvc add data
-```
-
-This command will create the "data.dvc" file, which contains a unique identifier and the location of the data directory in the file system.
-
-To keep track of the data associated with a particular version, commit the "data.dvc" file to Git:
-```bash
-git add data.dvc
-git commit -m "add data"
-```
-
-To push the data to remote storage, type:
-```bash
-dvc push 
-```
-
-## Auto-generate API documentation
-
-To auto-generate API document for your project, run:
-
-```bash
-make docs
-```
+## Main directories with their use and content
+
+    ├── data: data files
+    │   ├── final
+    │   ├── processed
+    │   └── raw: Raw files are not included and must be downloaded from source
+    │       
+    ├── docs: Documentation for the MMAR class and utility functions
+    │   
+    ├── notebooks: the notebooks used in the project
+    ├── src: source files. In some cases contains local copies of packages available on [Pypi](https://pypi.org/) for compatibility reasons.
+    │   ├── MMAR: the MMAR class
+    └── tests: test files (at present not used)
+
+
+The main purpose of this project is to provide a simple class to allow building a [Multifractal Model of Asset Returns](https://users.math.yale.edu/~bbm3/web_pdfs/Cowles1164.pdf) based on actual data.
+
+We used the class to compute the main parameters for the MMAR model and then used them to create a Monte Carlo simulation. We used the simulated data (based on SPY ETF) to improve option predictions using a basic trading strategy, which proved to be reasonably effective in out test-case:
+![Example strategy](other/strategy_example.png)
+
+
+* For S&P500 ETF (SPY) data, we used [Yahoo Finance.](https://finance.yahoo.com/quote/SPY?.tsrc=fin-srch) 
+* For option chains data, we used [OptinosDX](https://www.optionsdx.com/).
+* For interest rates, we used [FRED](https://fred.stlouisfed.org/series/TB3MS).
+* [Alpha Avantage](https://www.alphavantage.co/documentation/) provides more granular data (not only daily but also intraday), but those data haven't been used in the final version.
+* dot files, such as *.env*, are clearly not uploaded to GitHub. You must create them locally with your API keys.
+* There are many notebooks in the experiments branch that were not included in the final version, but might be interesting to explore. However, that branch is serveral commit behind, so don't expect the notebooks to work without some manual fixes.
+* A basic documentation for the MMAR class, built using [Sphinx](https://www.sphinx-doc.org/en/master/), is available on GitHub Pages.

docs/conf.py

@@ -17,7 +17,8 @@
 extensions = [
     'sphinx.ext.autodoc',
     'sphinx.ext.viewcode',
-    'sphinx.ext.napoleon'
+    'sphinx.ext.napoleon',
+    'sphinx.ext.mathjax'
 ]
 
 templates_path = ['_templates']

docs/html/modules.html

@@ -4,7 +4,7 @@
   <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
 
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-  <title>src &mdash; Options&#39; pricing with MMAR model 0.0.1 documentation</title>
+  <title>wqu_capstone &mdash; Options&#39; pricing with MMAR model 0.0.1 documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
       <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
 
@@ -44,7 +44,7 @@
         </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
               <!-- Local TOC -->
               <div class="local-toc"><ul>
-<li><a class="reference internal" href="#">src</a></li>
+<li><a class="reference internal" href="#">wqu_capstone</a></li>
 </ul>
 </div>
         </div>
@@ -61,7 +61,7 @@
           <div role="navigation" aria-label="Page navigation">
   <ul class="wy-breadcrumbs">
       <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
-      <li class="breadcrumb-item active">src</li>
+      <li class="breadcrumb-item active">wqu_capstone</li>
       <li class="wy-breadcrumbs-aside">
             <a href="_sources/modules.rst.txt" rel="nofollow"> View page source</a>
       </li>
@@ -71,20 +71,12 @@
           <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
            <div itemprop="articleBody">
 
-  <section id="src">
-<h1>src<a class="headerlink" href="#src" title="Link to this heading"></a></h1>
+  <section id="wqu-capstone">
+<h1>wqu_capstone<a class="headerlink" href="#wqu-capstone" title="Link to this heading"></a></h1>
 <div class="toctree-wrapper compound">
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="src.html">src package</a><ul>
 <li class="toctree-l2"><a class="reference internal" href="src.html#subpackages">Subpackages</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="src.HurstIndexSolver.html">src.HurstIndexSolver package</a><ul>
-<li class="toctree-l4"><a class="reference internal" href="src.HurstIndexSolver.html#submodules">Submodules</a></li>
-<li class="toctree-l4"><a class="reference internal" href="src.HurstIndexSolver.html#module-src.HurstIndexSolver.AddMethods">src.HurstIndexSolver.AddMethods module</a></li>
-<li class="toctree-l4"><a class="reference internal" href="src.HurstIndexSolver.html#module-src.HurstIndexSolver.HurstIndexSolver">src.HurstIndexSolver.HurstIndexSolver module</a></li>
-<li class="toctree-l4"><a class="reference internal" href="src.HurstIndexSolver.html#module-src.HurstIndexSolver.fgn">src.HurstIndexSolver.fgn module</a></li>
-<li class="toctree-l4"><a class="reference internal" href="src.HurstIndexSolver.html#module-src.HurstIndexSolver">Module contents</a></li>
-</ul>
-</li>
 <li class="toctree-l3"><a class="reference internal" href="src.MMAR.html">src.MMAR package</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="src.MMAR.html#submodules">Submodules</a></li>
 <li class="toctree-l4"><a class="reference internal" href="src.MMAR.html#module-src.MMAR.MMAR">src.MMAR.MMAR module</a></li>
@@ -94,30 +86,37 @@ <h1>src<a class="headerlink" href="#src" title="Link to this heading"></a></h
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="src.html#submodules">Submodules</a></li>
-<li class="toctree-l2"><a class="reference internal" href="src.html#module-src.process">src.process module</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="src.html#src.process.process_data"><code class="docutils literal notranslate"><span class="pre">process_data()</span></code></a></li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="src.html#module-src.train_model">src.train_model module</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="src.html#src.train_model.train_model"><code class="docutils literal notranslate"><span class="pre">train_model()</span></code></a></li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="src.html#src-utility-functions-module">src.utility_functions module</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="src.html#module-src.utility_functions">src.utility_functions module</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.backtest_strategy"><code class="docutils literal notranslate"><span class="pre">backtest_strategy()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.compare_qq_plots"><code class="docutils literal notranslate"><span class="pre">compare_qq_plots()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.compare_return_distribution"><code class="docutils literal notranslate"><span class="pre">compare_return_distribution()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.display_report"><code class="docutils literal notranslate"><span class="pre">display_report()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.get_df"><code class="docutils literal notranslate"><span class="pre">get_df()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.get_standard_returns"><code class="docutils literal notranslate"><span class="pre">get_standard_returns()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.objective_catboost"><code class="docutils literal notranslate"><span class="pre">objective_catboost()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.objective_lightgbm"><code class="docutils literal notranslate"><span class="pre">objective_lightgbm()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.objective_logistic_regression"><code class="docutils literal notranslate"><span class="pre">objective_logistic_regression()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.objective_random_forest"><code class="docutils literal notranslate"><span class="pre">objective_random_forest()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.objective_svc"><code class="docutils literal notranslate"><span class="pre">objective_svc()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.plot_feature_imp"><code class="docutils literal notranslate"><span class="pre">plot_feature_imp()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.plot_ks_comparison"><code class="docutils literal notranslate"><span class="pre">plot_ks_comparison()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.plot_multiple_return_comparison"><code class="docutils literal notranslate"><span class="pre">plot_multiple_return_comparison()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.plot_return_comparison"><code class="docutils literal notranslate"><span class="pre">plot_return_comparison()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.plot_simulated_paths"><code class="docutils literal notranslate"><span class="pre">plot_simulated_paths()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.plot_strategy"><code class="docutils literal notranslate"><span class="pre">plot_strategy()</span></code></a></li>
 <li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.select_threshold"><code class="docutils literal notranslate"><span class="pre">select_threshold()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="src.html#src.utility_functions.tune_params"><code class="docutils literal notranslate"><span class="pre">tune_params()</span></code></a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="src.html#module-src">Module contents</a></li>
 </ul>
 </li>
+<li class="toctree-l1"><a class="reference internal" href="tests.html">tests package</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="tests.html#submodules">Submodules</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tests.html#module-tests.test_utilities">tests.test_utilities module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tests.html#module-tests">Module contents</a></li>
+</ul>
+</li>
 </ul>
 </div>
 </section>

docs/html/py-modindex.html

@@ -73,7 +73,8 @@
    <h1>Python Module Index</h1>
 
    <div class="modindex-jumpbox">
-   <a href="#cap-s"><strong>s</strong></a>
+   <a href="#cap-s"><strong>s</strong></a> | 
+   <a href="#cap-t"><strong>t</strong></a>
    </div>
 
    <table class="indextable modindextable">
@@ -86,26 +87,6 @@ <h1>Python Module Index</h1>
        <td>
        <a href="src.html#module-src"><code class="xref">src</code></a></td><td>
        <em></em></td></tr>
-     <tr class="cg-1">
-       <td></td>
-       <td>&#160;&#160;&#160;
-       <a href="src.HurstIndexSolver.html#module-src.HurstIndexSolver"><code class="xref">src.HurstIndexSolver</code></a></td><td>
-       <em></em></td></tr>
-     <tr class="cg-1">
-       <td></td>
-       <td>&#160;&#160;&#160;
-       <a href="src.HurstIndexSolver.html#module-src.HurstIndexSolver.AddMethods"><code class="xref">src.HurstIndexSolver.AddMethods</code></a></td><td>
-       <em></em></td></tr>
-     <tr class="cg-1">
-       <td></td>
-       <td>&#160;&#160;&#160;
-       <a href="src.HurstIndexSolver.html#module-src.HurstIndexSolver.fgn"><code class="xref">src.HurstIndexSolver.fgn</code></a></td><td>
-       <em></em></td></tr>
-     <tr class="cg-1">
-       <td></td>
-       <td>&#160;&#160;&#160;
-       <a href="src.HurstIndexSolver.html#module-src.HurstIndexSolver.HurstIndexSolver"><code class="xref">src.HurstIndexSolver.HurstIndexSolver</code></a></td><td>
-       <em></em></td></tr>
      <tr class="cg-1">
        <td></td>
        <td>&#160;&#160;&#160;
@@ -119,17 +100,21 @@ <h1>Python Module Index</h1>
      <tr class="cg-1">
        <td></td>
        <td>&#160;&#160;&#160;
-       <a href="src.html#module-src.process"><code class="xref">src.process</code></a></td><td>
+       <a href="src.html#module-src.utility_functions"><code class="xref">src.utility_functions</code></a></td><td>
        <em></em></td></tr>
-     <tr class="cg-1">
-       <td></td>
-       <td>&#160;&#160;&#160;
-       <a href="src.html#module-src.train_model"><code class="xref">src.train_model</code></a></td><td>
+     <tr class="pcap"><td></td><td>&#160;</td><td></td></tr>
+     <tr class="cap" id="cap-t"><td></td><td>
+       <strong>t</strong></td><td></td></tr>
+     <tr>
+       <td><img src="_static/minus.png" class="toggler"
+              id="toggle-2" style="display: none" alt="-" /></td>
+       <td>
+       <a href="tests.html#module-tests"><code class="xref">tests</code></a></td><td>
        <em></em></td></tr>
-     <tr class="cg-1">
+     <tr class="cg-2">
        <td></td>
        <td>&#160;&#160;&#160;
-       <a href="src.html#module-src.utility_functions"><code class="xref">src.utility_functions</code></a></td><td>
+       <a href="tests.html#module-tests.test_utilities"><code class="xref">tests.test_utilities</code></a></td><td>
        <em></em></td></tr>
    </table>