Skip to content

Commit

Permalink
Updating documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
mmcentre committed Jun 30, 2024
1 parent f6a9261 commit 781fb38
Show file tree
Hide file tree
Showing 11 changed files with 73 additions and 66 deletions.
11 changes: 11 additions & 0 deletions doc/home.md
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,17 @@ documentation for Dorieh platform
Documentation utilities to simplify creation of consistent
documentation for Dorieh platform

* [cwl2md](members/cwl2md) Generates [Markdown](https://www.markdownguide.org/basic-syntax/)
documentation for a CWL tool or workflow
* [collector](members/collector) Generates automatic
[reStructuredText](https://docutils.sourceforge.io/rst.html)
templates for all Python modules
* [copy_section](members/copy_section) Copies a specified section from one markdown document
to another. This way we can collect summaries in one file<!-- end of section documentation utilities overview from dorieh.docutils -->

Documentation utilities to simplify creation of consistent
documentation for Dorieh platform

* [cwl2md](members/cwl2md) Generates [Markdown](https://www.markdownguide.org/basic-syntax/)
documentation for a CWL tool or workflow
* [collector](members/collector) Generates automatic
Expand Down
Binary file modified docs/.doctrees/TestingWorkflows.doctree
Binary file not shown.
Binary file modified docs/.doctrees/environment.pickle
Binary file not shown.
Binary file modified docs/.doctrees/home.doctree
Binary file not shown.
69 changes: 19 additions & 50 deletions docs/TestingWorkflows.html
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,6 @@
<li class="toctree-l2"><a class="reference internal" href="members/create_test.html">Utility to generate test queries</a></li>
<li class="toctree-l2"><a class="reference internal" href="members/dbt_runner.html">Utility to generate test queries</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">Testing bundled workflows</a><ul>
<li class="toctree-l3"><a class="reference internal" href="pipelines.html">Data Processing Pipelines</a></li>
<li class="toctree-l3"><a class="reference internal" href="pipeline/aqs.html">Full EPA AQS Processing Pipeline</a></li>
<li class="toctree-l3"><a class="reference internal" href="pipeline/airnow.html">Full EPA AirNow Processing Pipeline (including downloading shapefiles)</a></li>
<li class="toctree-l3"><a class="reference internal" href="pipeline/gridmet.html">Pipeline to aggregate data from Climatology Lab</a></li>
<li class="toctree-l3"><a class="reference internal" href="pipeline/pm25_yearly_download.html">Pipeline to aggregate data in NetCDF format over given geographies</a></li>
<li class="toctree-l3"><a class="reference internal" href="#introduction-to-testing-and-prerequisites">Introduction to testing and prerequisites</a></li>
<li class="toctree-l3"><a class="reference internal" href="#before-running-any-test">Before running any test</a></li>
<li class="toctree-l3"><a class="reference internal" href="#testing-aqs-workflow">Testing AQS workflow</a><ul>
Expand Down Expand Up @@ -162,49 +157,8 @@ <h1>Testing bundled workflows<a class="headerlink" href="#testing-bundled-workfl
</nav>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="DBT.html"><span class="doc std std-doc">Database Testing Framework</span></a></p>
</div>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="pipelines.html">Data Processing Pipelines</a><ul>
<li class="toctree-l2"><a class="reference internal" href="pipelines.html#introduction">Introduction</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipelines.html#running-workflows">Running Workflows</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipelines.html#testing-workflows">Testing workflows</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipelines.html#installing-python-dependencies">Installing Python dependencies</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipelines.html#published-and-tested-workflows">Published and tested workflows</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipelines.html#developing-your-own-workflows">Developing your own workflows</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipelines.html#example-of-a-workflow">Example of a workflow</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="pipeline/aqs.html">Full EPA AQS Processing Pipeline</a><ul>
<li class="toctree-l2"><a class="reference internal" href="pipeline/aqs.html#description">Description</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/aqs.html#inputs">Inputs</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/aqs.html#outputs">Outputs</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/aqs.html#steps">Steps</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="pipeline/airnow.html">Full EPA AirNow Processing Pipeline (including downloading shapefiles)</a><ul>
<li class="toctree-l2"><a class="reference internal" href="pipeline/airnow.html#description">Description</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/airnow.html#inputs">Inputs</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/airnow.html#outputs">Outputs</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/airnow.html#steps">Steps</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="pipeline/gridmet.html">Pipeline to aggregate data from Climatology Lab</a><ul>
<li class="toctree-l2"><a class="reference internal" href="pipeline/gridmet.html#description">Description</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/gridmet.html#inputs">Inputs</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/gridmet.html#outputs">Outputs</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/gridmet.html#steps">Steps</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="pipeline/pm25_yearly_download.html">Pipeline to aggregate data in NetCDF format over given geographies</a><ul>
<li class="toctree-l2"><a class="reference internal" href="pipeline/pm25_yearly_download.html#description">Description</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/pm25_yearly_download.html#inputs">Inputs</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/pm25_yearly_download.html#outputs">Outputs</a></li>
<li class="toctree-l2"><a class="reference internal" href="pipeline/pm25_yearly_download.html#steps">Steps</a></li>
</ul>
</li>
</ul>
<p><a class="reference internal" href="DBT.html"><span class="doc std std-doc">Database Testing Framework</span></a>
<a class="reference internal" href="pipelines.html"><span class="doc std std-doc">Data Processing Pipelines</span></a></p>
</div>
<section id="introduction-to-testing-and-prerequisites">
<h2><a class="toc-backref" href="#id7" role="doc-backlink">Introduction to testing and prerequisites</a><a class="headerlink" href="#introduction-to-testing-and-prerequisites" title="Permalink to this heading"></a></h2>
Expand Down Expand Up @@ -256,6 +210,10 @@ <h2><a class="toc-backref" href="#id8" role="doc-backlink">Before running any te
</section>
<section id="testing-aqs-workflow">
<h2><a class="toc-backref" href="#id9" role="doc-backlink">Testing AQS workflow</a><a class="headerlink" href="#testing-aqs-workflow" title="Permalink to this heading"></a></h2>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="pipeline/aqs.html"><span class="doc std std-doc">Full EPA AQS Processing Pipeline</span></a></p>
</div>
<section id="before-running-the-test">
<h3><a class="toc-backref" href="#id10" role="doc-backlink">Before running the test</a><a class="headerlink" href="#before-running-the-test" title="Permalink to this heading"></a></h3>
<p>Create a scratch directory:</p>
Expand Down Expand Up @@ -299,6 +257,10 @@ <h3><a class="toc-backref" href="#id12" role="doc-backlink">Testing AQS workflow
</section>
<section id="testing-airnow-workflow">
<h2><a class="toc-backref" href="#id13" role="doc-backlink">Testing Airnow workflow</a><a class="headerlink" href="#testing-airnow-workflow" title="Permalink to this heading"></a></h2>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="pipeline/airnow.html"><span class="doc std std-doc">Full EPA AirNow Processing Pipeline (including downloading shapefiles)</span></a></p>
</div>
<section id="id1">
<h3><a class="toc-backref" href="#id14" role="doc-backlink">Before running the test</a><a class="headerlink" href="#id1" title="Permalink to this heading"></a></h3>
<p>Create a scratch directory:</p>
Expand Down Expand Up @@ -344,7 +306,11 @@ <h3><a class="toc-backref" href="#id16" role="doc-backlink">Testing Airnow workf
</section>
<section id="testing-climate-workflow">
<h2><a class="toc-backref" href="#id17" role="doc-backlink">Testing Climate workflow</a><a class="headerlink" href="#testing-climate-workflow" title="Permalink to this heading"></a></h2>
<p>See also <a class="reference internal" href="Example-climate-workflow.html"><span class="doc std std-doc">Example of a workflow: aggregating a climate variable</span></a></p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="Example-climate-workflow.html"><span class="doc std std-doc">Example of a workflow: aggregating a climate variable</span></a>
<a class="reference internal" href="pipeline/gridmet.html"><span class="doc std std-doc">Pipeline to aggregate data from Climatology Lab</span></a></p>
</div>
<section id="id3">
<h3><a class="toc-backref" href="#id18" role="doc-backlink">Before running the test</a><a class="headerlink" href="#id3" title="Permalink to this heading"></a></h3>
<p>Create a scratch directory:</p>
Expand Down Expand Up @@ -386,7 +352,10 @@ <h3><a class="toc-backref" href="#id20" role="doc-backlink">Testing climate work
</section>
<section id="testing-exposure-workflow">
<h2><a class="toc-backref" href="#id21" role="doc-backlink">Testing Exposure workflow</a><a class="headerlink" href="#testing-exposure-workflow" title="Permalink to this heading"></a></h2>
<p>See also <a class="reference internal" href="Example-climate-workflow.html"><span class="doc std std-doc">Example of a workflow: aggregating a climate variable</span></a></p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="pipeline/pm25_yearly_download.html"><span class="doc std std-doc">Pipeline to aggregate data in NetCDF format over given geographies</span></a></p>
</div>
<section id="id5">
<h3><a class="toc-backref" href="#id22" role="doc-backlink">Before running the test</a><a class="headerlink" href="#id5" title="Permalink to this heading"></a></h3>
<p>Create a scratch directory:</p>
Expand Down
Binary file modified docs/_images/aggregate_one_file.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/_images/airnow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
35 changes: 20 additions & 15 deletions docs/_sources/TestingWorkflows.md.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,7 @@ local:

```{seealso}
[Database Testing Framework](DBT)
```

```{toctree}
---
maxdepth: 2

---
pipelines
pipeline/aqs
pipeline/airnow
pipeline/gridmet
pipeline/pm25_yearly_download
[](pipelines)
```

## Introduction to testing and prerequisites
Expand Down Expand Up @@ -78,6 +67,11 @@ you will need to replace `toil-cwl-runner` command with an appropriate alternati

## Testing AQS workflow

```{seealso}
[](pipeline/aqs)
```


### Before running the test

Create a scratch directory:
Expand Down Expand Up @@ -124,6 +118,11 @@ In the virtual environment that has Toil run the following command:

## Testing Airnow workflow

```{seealso}
[](pipeline/airnow)
```


### Before running the test

Create a scratch directory:
Expand Down Expand Up @@ -168,8 +167,11 @@ The command will be:


## Testing Climate workflow

See also [](Example-climate-workflow)

```{seealso}
[](Example-climate-workflow)
[](pipeline/gridmet)
```

### Before running the test

Expand Down Expand Up @@ -212,7 +214,10 @@ If you have installed dorieh locally, run the following command

## Testing Exposure workflow

See also [](Example-climate-workflow)
```{seealso}
[](pipeline/pm25_yearly_download)
```


### Before running the test

Expand Down
11 changes: 11 additions & 0 deletions docs/_sources/home.md.txt
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,17 @@ documentation for Dorieh platform
Documentation utilities to simplify creation of consistent
documentation for Dorieh platform

* [cwl2md](members/cwl2md) Generates [Markdown](https://www.markdownguide.org/basic-syntax/)
documentation for a CWL tool or workflow
* [collector](members/collector) Generates automatic
[reStructuredText](https://docutils.sourceforge.io/rst.html)
templates for all Python modules
* [copy_section](members/copy_section) Copies a specified section from one markdown document
to another. This way we can collect summaries in one file<!-- end of section documentation utilities overview from dorieh.docutils -->

Documentation utilities to simplify creation of consistent
documentation for Dorieh platform

* [cwl2md](members/cwl2md) Generates [Markdown](https://www.markdownguide.org/basic-syntax/)
documentation for a CWL tool or workflow
* [collector](members/collector) Generates automatic
Expand Down
11 changes: 11 additions & 0 deletions docs/home.html
Original file line number Diff line number Diff line change
Expand Up @@ -464,6 +464,17 @@ <h3><a class="toc-backref" href="#id6" role="doc-backlink">Dorieh Documentation
<li><p><a class="reference internal" href="members/copy_section.html"><span class="doc std std-doc">copy_section</span></a> Copies a specified section from one markdown document
to another. This way we can collect summaries in one file<!-- end of section documentation utilities overview from dorieh.docutils --></p></li>
</ul>
<p>Documentation utilities to simplify creation of consistent
documentation for Dorieh platform</p>
<ul class="simple">
<li><p><a class="reference internal" href="members/cwl2md.html"><span class="doc std std-doc">cwl2md</span></a> Generates <a class="reference external" href="https://www.markdownguide.org/basic-syntax/">Markdown</a>
documentation for a CWL tool or workflow</p></li>
<li><p><a class="reference internal" href="members/collector.html"><span class="doc std std-doc">collector</span></a> Generates automatic
<a class="reference external" href="https://docutils.sourceforge.io/rst.html">reStructuredText</a>
templates for all Python modules</p></li>
<li><p><a class="reference internal" href="members/copy_section.html"><span class="doc std std-doc">copy_section</span></a> Copies a specified section from one markdown document
to another. This way we can collect summaries in one file<!-- end of section documentation utilities overview from dorieh.docutils --></p></li>
</ul>
</section>
<section id="data-processing-and-loading-pipelines">
<h3><a class="toc-backref" href="#id7" role="doc-backlink">Data Processing and Loading Pipelines</a><a class="headerlink" href="#data-processing-and-loading-pipelines" title="Permalink to this heading"></a></h3>
Expand Down
2 changes: 1 addition & 1 deletion docs/searchindex.js

Large diffs are not rendered by default.

0 comments on commit 781fb38

Please sign in to comment.