0.4 RELEASE

sizhky · Mar 6, 2021 · 028528b · 028528b
1 parent 1862375
commit 028528b
Show file tree

Hide file tree

Showing 9 changed files with 192 additions and 273 deletions.
diff --git a/README.md b/README.md
@@ -2,73 +2,44 @@
 
 
 
-```python
-## Logging and plotting metrics
-```python
-from torch_snippets import Report
-
-n_epochs = 5
-log = Report(n_epochs)
-
-for epoch in range(n_epochs):
-    # No need to --> train_epoch_losses, train_epoch_accuracies = [], []
-    N = len(trn_dl)
-    for ix, batch in enumerate(iter(trn_dl)):
-        ...
-        pos = (epoch + (ix+1)/N) # a float between 0 - n_epochs
-        # give any number of kwargs that need to be reported and stored.
-        # args should be float
-        log.record(pos=pos, train_acc=np.mean(is_correct), train_loss=batch_loss, end='\r') # impersistent log
-
-    N = len(val_dl)
-    for ix, batch in enumerate(iter(val_dl)):
-        ...
-        pos = (epoch + (ix+1)/N) # a float between 0 - n_epochs
-        log.record(pos=pos, val_loss=batch_loss, end='\r') # impersistent log
-    log.report_avgs(epoch+1) # persist the report
+## Auxiliary Functions
+There are simple functions that are overloaded to take inputs and perform repetitive tasks that usually take a few lines to write
 
-```
-![](../assets/demo.gif)
-* Auto calculates time remaining
-* No need to preinitialize empty lists
-* Automatically stores metrics as collection of key words
-* Persistent vs Transient logging (use `end='\r'`)  
-```python
->>> print(log.logged) # get logged metric names
-# ['train_loss', 'train_acc', 'val_loss', 'val_acc']
->>> log.plot() # plot all the logged metrics
-```
+#### Images
+`show`, `inspect`, `Glob`, `read`, `resize`, `rotate`
 
+#### Files and Paths
+`stem`, `Glob`, `parent`, `name`, `fname`,
 
-* Auto calculates average of all metrics in an epoch
-* Plot entire training history with one command  
-![](../assets/avgs0.png)
 
+`makedir`, `zip_files`, `unzip_file`,   
 
-* selectively plot logged metrics using regex
-```python
->>> log.plot('*_loss')
-# or you can list out the metrics that need plotting
-# >>> log.plot(['train_acc','val_acc'])
-```
-![](assets/avgs1.png)
 
-## Auxiliary Functions
-There are simple functions that are overloaded to take inputs and perform repetitive tasks that usually take a few lines to write
-#### Images
-`show`, `inspect`, `Glob`, `read`
-#### FilePaths
-`stem`, `Glob`, `parent`, `name`
+`find`, `extn`,  
+
+
+`readlines`, `writelines`
+
+#### Lists
+`L`, `flatten`
+
 #### Dump and load python objects
 `loaddill`,`dumpdill`
+
 #### Misc 
-`Tqdm`, `Timer`, `randint`, `unique`, `diff`
+`Tqdm`, `Timer`, `randint`, `Logger`
+
+#### Sets
+`unique`, `diff`, `choose`, `common`
+
 #### Pytorch Modules
 `Reshape` and `Permute` (`nn.Modules`)
 
+#### Report as Pytorch Lightning Callback
+`LightningReport`
+
+and many more to come... 
 
-and many more... 
-
 ## Install
 `pip install torch_snippets`
 
@@ -78,4 +49,3 @@ import pytorch_snippets
 dir(pytorch_snippets)
 ```
 
-```
diff --git a/dist/torch_snippets-0.315-py3-none-any.whl b/dist/torch_snippets-0.315-py3-none-any.whl
diff --git a/docs/index.html b/docs/index.html
@@ -29,95 +29,29 @@
 </div>
     {% endraw %}
 
-    {% raw %}
-
-<div class="cell border-box-sizing code_cell rendered">
-<div class="input">
-
-<div class="inner_cell">
-    <div class="input_area">
-<div class=" highlight hl-ipython3"><pre><span></span><span class="err">```</span><span class="n">python</span>
-<span class="kn">from</span> <span class="nn">torch_snippets</span> <span class="kn">import</span> <span class="n">Report</span>
-
-<span class="n">n_epochs</span> <span class="o">=</span> <span class="mi">5</span>
-<span class="n">log</span> <span class="o">=</span> <span class="n">Report</span><span class="p">(</span><span class="n">n_epochs</span><span class="p">)</span>
-
-<span class="k">for</span> <span class="n">epoch</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">n_epochs</span><span class="p">):</span>
-    <span class="c1"># No need to --&gt; train_epoch_losses, train_epoch_accuracies = [], []</span>
-    <span class="n">N</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">trn_dl</span><span class="p">)</span>
-    <span class="k">for</span> <span class="n">ix</span><span class="p">,</span> <span class="n">batch</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="nb">iter</span><span class="p">(</span><span class="n">trn_dl</span><span class="p">)):</span>
-        <span class="o">...</span>
-        <span class="n">pos</span> <span class="o">=</span> <span class="p">(</span><span class="n">epoch</span> <span class="o">+</span> <span class="p">(</span><span class="n">ix</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span><span class="o">/</span><span class="n">N</span><span class="p">)</span> <span class="c1"># a float between 0 - n_epochs</span>
-        <span class="c1"># give any number of kwargs that need to be reported and stored.</span>
-        <span class="c1"># args should be float</span>
-        <span class="n">log</span><span class="o">.</span><span class="n">record</span><span class="p">(</span><span class="n">pos</span><span class="o">=</span><span class="n">pos</span><span class="p">,</span> <span class="n">train_acc</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">is_correct</span><span class="p">),</span> <span class="n">train_loss</span><span class="o">=</span><span class="n">batch_loss</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s1">&#39;</span><span class="se">\r</span><span class="s1">&#39;</span><span class="p">)</span> <span class="c1"># impersistent log</span>
-
-    <span class="n">N</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">val_dl</span><span class="p">)</span>
-    <span class="k">for</span> <span class="n">ix</span><span class="p">,</span> <span class="n">batch</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="nb">iter</span><span class="p">(</span><span class="n">val_dl</span><span class="p">)):</span>
-        <span class="o">...</span>
-        <span class="n">pos</span> <span class="o">=</span> <span class="p">(</span><span class="n">epoch</span> <span class="o">+</span> <span class="p">(</span><span class="n">ix</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span><span class="o">/</span><span class="n">N</span><span class="p">)</span> <span class="c1"># a float between 0 - n_epochs</span>
-        <span class="n">log</span><span class="o">.</span><span class="n">record</span><span class="p">(</span><span class="n">pos</span><span class="o">=</span><span class="n">pos</span><span class="p">,</span> <span class="n">val_loss</span><span class="o">=</span><span class="n">batch_loss</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s1">&#39;</span><span class="se">\r</span><span class="s1">&#39;</span><span class="p">)</span> <span class="c1"># impersistent log</span>
-    <span class="n">log</span><span class="o">.</span><span class="n">report_avgs</span><span class="p">(</span><span class="n">epoch</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># persist the report</span>
-
-<span class="err">```</span>
-<span class="o">![](</span>../assets/demo.gif<span class="o">)</span>
-<span class="o">*</span> <span class="n">Auto</span> <span class="n">calculates</span> <span class="n">time</span> <span class="n">remaining</span>
-<span class="o">*</span> <span class="n">No</span> <span class="n">need</span> <span class="n">to</span> <span class="n">preinitialize</span> <span class="n">empty</span> <span class="n">lists</span>
-<span class="o">*</span> <span class="n">Automatically</span> <span class="n">stores</span> <span class="n">metrics</span> <span class="k">as</span> <span class="n">collection</span> <span class="n">of</span> <span class="n">key</span> <span class="n">words</span>
-<span class="o">*</span> <span class="n">Persistent</span> <span class="n">vs</span> <span class="n">Transient</span> <span class="n">logging</span> <span class="p">(</span><span class="n">use</span> <span class="err">`</span><span class="n">end</span><span class="o">=</span><span class="s1">&#39;</span><span class="se">\r</span><span class="s1">&#39;</span><span class="err">`</span><span class="p">)</span>  
-<span class="err">```</span><span class="n">python</span>
-<span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">log</span><span class="o">.</span><span class="n">logged</span><span class="p">)</span> <span class="c1"># get logged metric names</span>
-<span class="c1"># [&#39;train_loss&#39;, &#39;train_acc&#39;, &#39;val_loss&#39;, &#39;val_acc&#39;]</span>
-<span class="o">&gt;&gt;&gt;</span> <span class="n">log</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span> <span class="c1"># plot all the logged metrics</span>
-<span class="err">```</span>
-
-
-<span class="o">*</span> <span class="n">Auto</span> <span class="n">calculates</span> <span class="n">average</span> <span class="n">of</span> <span class="nb">all</span> <span class="n">metrics</span> <span class="ow">in</span> <span class="n">an</span> <span class="n">epoch</span>
-<span class="o">*</span> <span class="n">Plot</span> <span class="n">entire</span> <span class="n">training</span> <span class="n">history</span> <span class="k">with</span> <span class="n">one</span> <span class="n">command</span>  
-<span class="o">![](</span>../assets/avgs0.png<span class="o">)</span>
-
-
-<span class="o">*</span> <span class="n">selectively</span> <span class="n">plot</span> <span class="n">logged</span> <span class="n">metrics</span> <span class="n">using</span> <span class="n">regex</span>
-<span class="err">```</span><span class="n">python</span>
-<span class="o">&gt;&gt;&gt;</span> <span class="n">log</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="s1">&#39;*_loss&#39;</span><span class="p">)</span>
-<span class="c1"># or you can list out the metrics that need plotting</span>
-<span class="c1"># &gt;&gt;&gt; log.plot([&#39;train_acc&#39;,&#39;val_acc&#39;])</span>
-<span class="err">```</span>
-<span class="o">![](</span>assets/avgs1.png<span class="o">)</span>
-
-<span class="c1">## Auxiliary Functions</span>
-<span class="n">There</span> <span class="n">are</span> <span class="n">simple</span> <span class="n">functions</span> <span class="n">that</span> <span class="n">are</span> <span class="n">overloaded</span> <span class="n">to</span> <span class="n">take</span> <span class="n">inputs</span> <span class="ow">and</span> <span class="n">perform</span> <span class="n">repetitive</span> <span class="n">tasks</span> <span class="n">that</span> <span class="n">usually</span> <span class="n">take</span> <span class="n">a</span> <span class="n">few</span> <span class="n">lines</span> <span class="n">to</span> <span class="n">write</span>
-<span class="c1">#### Images</span>
-<span class="err">`</span><span class="n">show</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">inspect</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">Glob</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">read</span><span class="err">`</span>
-<span class="c1">#### FilePaths</span>
-<span class="err">`</span><span class="n">stem</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">Glob</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">parent</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">name</span><span class="err">`</span>
-<span class="c1">#### Dump and load python objects</span>
-<span class="err">`</span><span class="n">loaddill</span><span class="err">`</span><span class="p">,</span><span class="err">`</span><span class="n">dumpdill</span><span class="err">`</span>
-<span class="c1">#### Misc </span>
-<span class="err">`</span><span class="n">Tqdm</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">Timer</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">randint</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">unique</span><span class="err">`</span><span class="p">,</span> <span class="err">`</span><span class="n">diff</span><span class="err">`</span>
-<span class="c1">#### Pytorch Modules</span>
-<span class="err">`</span><span class="n">Reshape</span><span class="err">`</span> <span class="ow">and</span> <span class="err">`</span><span class="n">Permute</span><span class="err">`</span> <span class="p">(</span><span class="err">`</span><span class="n">nn</span><span class="o">.</span><span class="n">Modules</span><span class="err">`</span><span class="p">)</span>
-
-
-<span class="ow">and</span> <span class="n">many</span> <span class="n">more</span><span class="o">...</span> 
-
-<span class="c1">## Install</span>
-<span class="err">`</span><span class="n">pip</span> <span class="n">install</span> <span class="n">torch_snippets</span><span class="err">`</span>
-
-<span class="c1">## Usage</span>
-<span class="err">```</span><span class="n">python</span>
-<span class="kn">import</span> <span class="nn">pytorch_snippets</span>
+<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h2 id="Auxiliary-Functions">Auxiliary Functions<a class="anchor-link" href="#Auxiliary-Functions"> </a></h2><p>There are simple functions that are overloaded to take inputs and perform repetitive tasks that usually take a few lines to write</p>
+<h4 id="Images">Images<a class="anchor-link" href="#Images"> </a></h4><p><code>show</code>, <code>inspect</code>, <code>Glob</code>, <code>read</code>, <code>resize</code>, <code>rotate</code></p>
+<h4 id="Files-and-Paths">Files and Paths<a class="anchor-link" href="#Files-and-Paths"> </a></h4><p><code>stem</code>, <code>Glob</code>, <code>parent</code>, <code>name</code>, <code>fname</code>,</p>
+<p><code>makedir</code>, <code>zip_files</code>, <code>unzip_file</code>,</p>
+<p><code>find</code>, <code>extn</code>,</p>
+<p><code>readlines</code>, <code>writelines</code></p>
+<h4 id="Lists">Lists<a class="anchor-link" href="#Lists"> </a></h4><p><code>L</code>, <code>flatten</code></p>
+<h4 id="Dump-and-load-python-objects">Dump and load python objects<a class="anchor-link" href="#Dump-and-load-python-objects"> </a></h4><p><code>loaddill</code>,<code>dumpdill</code></p>
+<h4 id="Misc">Misc<a class="anchor-link" href="#Misc"> </a></h4><p><code>Tqdm</code>, <code>Timer</code>, <code>randint</code>, <code>Logger</code></p>
+<h4 id="Sets">Sets<a class="anchor-link" href="#Sets"> </a></h4><p><code>unique</code>, <code>diff</code>, <code>choose</code>, <code>common</code></p>
+<h4 id="Pytorch-Modules">Pytorch Modules<a class="anchor-link" href="#Pytorch-Modules"> </a></h4><p><code>Reshape</code> and <code>Permute</code> (<code>nn.Modules</code>)</p>
+<h4 id="Report-as-Pytorch-Lightning-Callback">Report as Pytorch Lightning Callback<a class="anchor-link" href="#Report-as-Pytorch-Lightning-Callback"> </a></h4><p><code>LightningReport</code></p>
+<p>and many more to come...</p>
+<h2 id="Install">Install<a class="anchor-link" href="#Install"> </a></h2><p><code>pip install torch_snippets</code></p>
+<h2 id="Usage">Usage<a class="anchor-link" href="#Usage"> </a></h2><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pytorch_snippets</span>
 <span class="nb">dir</span><span class="p">(</span><span class="n">pytorch_snippets</span><span class="p">)</span>
-<span class="err">```</span>
 </pre></div>
 
-    </div>
 </div>
 </div>
-
 </div>
-    {% endraw %}
-
 </div>
 
 
diff --git a/docs/report.html b/docs/report.html
@@ -220,6 +220,31 @@
 </div>
     {% endraw %}
 
+<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h2 id="Features">Features<a class="anchor-link" href="#Features"> </a></h2><p><img src="https://i.imgur.com/5iZl2s3.gif" alt=""></p>
+<ul>
+<li>Auto calculates time remaining</li>
+<li>No need to preinitialize empty lists</li>
+<li>Automatically stores metrics as collection of key words</li>
+<li>Persistent vs Transient logging (use <code>end='\r'</code>)  <div class="highlight"><pre><span></span><span class="o">&gt;</span> <span class="o">&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">log</span><span class="o">.</span><span class="n">logged</span><span class="p">)</span> <span class="c1"># get logged metric names</span>
+<span class="c1"># [&#39;train_loss&#39;, &#39;train_acc&#39;, &#39;val_loss&#39;, &#39;val_acc&#39;]</span>
+<span class="o">&gt;&gt;&gt;</span> <span class="n">log</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span> <span class="c1"># plot all the logged metrics</span>
+</pre></div>
+</li>
+</ul>
+<ul>
+<li>Auto calculates average of all metrics in an epoch</li>
+<li><p>Plot entire training history with one command<br>
+<img src="https://i.imgur.com/BrQIaR7.png" alt=""></p>
+</li>
+<li><p>selectively plot logged metrics using regex</p>
+</li>
+</ul>
+
+</div>
+</div>
+</div>
 </div>