deploy: 1a74b88

_sources/experiment.rst.txt

@@ -478,15 +478,75 @@ translucent shape is added to the plot. By default, it is centered on
 the beam center, but may be moved by dragging the center of the shape
 and/or resized by dragging one of the shape edges. When the shape has
 the correct position and size, click on "Save Shape" for the shape to be
-added to the current list. After saving the shape, it is no longer
-draggable. However, the shape can still be modified by adjusting the
-shape parameters and then clicking on "Save Shape" again. A pull-down
-menu allows existing shapes to be selected for further edits or removal
-if necessary.
+added to the current list. A pull-down menu allows existing shapes to be
+selected for further edits or removal
+
+.. note:: After saving the shape, it is no longer draggable. However, 
+          the shape can still be modified by adjusting the shape 
+          parameters and then clicking on "Save Shape" again. 
 
 If a more complicated mask is required, it can be generated by an
 external image editor and imported using "Import Mask".
 
 When the mask is complete, click "Save" to save it to the configuration
 file.
 
+New Sample
+----------
+This dialog has the single purpose of creating a directory tree for a
+new sample. The dialog enables the creation of a sample directory within
+the requested experiment directory and a sub-directory with a unique
+label for each instance of that sample measured during an experiment.
+
+.. figure:: /images/new-sample.png
+   :align: center
+   :width: 60%
+
+New Scan
+--------
+This dialog is used to create a NeXus file in preparation for an
+experimental measurement. The file will be based on the selected
+configuration file and be saved in the specified sample/label directory.
+The name of the file will be "<sample>_<scan>.nxs", where <scan> is the
+Scan Label specified in the dialog ('300K' in the image below).
+
+.. figure:: /images/new-scan.png
+   :align: center
+   :width: 80%
+
+The NeXus file is left open in the NeXpy tree. Multiple files can be
+created within the dialog, with different scan labels and, typically,
+different temperatures, before the dialog is closed.
+
+External links to the raw data file are created in the NeXus file, even
+if the data does not yet exist. In the example above, the external link
+for the first detector position will be to ``f1.h5``, in the ``<scan>``
+subdirectory. Similarly, the external link for the second detector
+position would be to ``<scan>/f2.h5``, *etc*. This experimental layout
+is described in more detail in the `Experiment Layout`_ section above.
+
+Import Scans
+------------
+This dialog is for instruments in which the scans are already defined
+using different methods to those above. For example, on the QM2
+instrument at CHESS, the scans are defined in SPEC files, with the data
+stored separately in a separate read-only directory. With this dialog,
+the directories containing the raw images are associated with the
+corresponding SPEC scan, allowing NeXus files to be automatically
+generated. This customization is encoded in a QM2 sub-class of the
+``NXBeamLine`` class, which is installed separately as a NXRefine
+plugin. The process for customizing other beamlines is described later.
+
+Sum Scans
+---------
+This dialog allows data in NeXus files collected under identical
+conditions to be summed to produce a single NeXus file that can be
+processed using the usual workflow.
+
+Edit Settings
+-------------
+This dialog allows the settings, whose default values are defined in the
+server directory (see :ref:`default_settings`), to be customized for the
+data reduction performed in the selected experiment. The settings are
+stored in ``<experiment>/tasks/settings.ini``. The meanings of each
+setting are described in the next section.

_sources/installation.rst.txt

@@ -25,7 +25,7 @@ NeXpy              https://github.com/nexpy/nexpy/
 cctbx-base         https://cci.lbl.gov/cctbx_docs/
 pyfai              https://pyfai.readthedocs.io/
 sqlalchemy         https://docs.sqlalchemy.org/
-psutil             https://psutil.readthedocs.io/.io/
+psutil             https://psutil.readthedocs.io/
 persist-queue      https://github.com/peter-wangxu/persist-queue
 =================  =================================================
 

_sources/refine.rst.txt

@@ -1,2 +1,41 @@
-Data reduction
-**************
+Data Reduction
+**************
+In order to reduce raw data collected as images (or frames) on an area
+detector as a function of sample rotation angle and transform the
+results into reciprocal space maps, *i.e.*, S(**Q**), *NXRefine*
+performs the following steps:
+
+* combining the frames into a single three-dimensional array.
+* harvesting metadata collected during the sample rotations.
+* summing detector frames to facilitate absorption corrections.
+* searching for Bragg peaks embedded within the raw data.
+* defining an orientation matrix.
+* transforming the raw data into reciprocal space coordinates.
+
+When multiple sample rotations are performed to collect a single data
+set, these steps have to be applied to each rotation scan and the
+results merged to produce a single three-dimensional array representing
+S(**Q**). Optionally, *NXRefine* also transforms the data after applying
+masks that eliminate spurious signals caused by the scattering of Bragg
+peaks within the detector sensor layer.
+
+Once the data has been transformed into S(**Q**), it is possible to
+generate 3D-ΔPDF maps, which transform the data back into real space,
+producing difference Patterson maps, *i.e.*, maps of interatomic vector
+probabilities, which differ from the average crystalline structure. In
+this way, continuous distributions of diffuse scattering intensity are
+typically reduced to discrete peaks, with positive and negative
+intensities, representing these probability differences. *NXRefine*
+implements the "punch-and-fill" method, described by `Weber and Simonov
+<https:dx.doi.org.10.1524/zkri.2012.1504>`_.
+
+Nearly all of the steps in the *NXRefine* data reduction workflow can
+either be performed from the command line or launched from a NeXpy GUI.
+The exception is determining the crystal orientation, which must first
+be performed using the `Refine Lattice` dialog in NeXpy. Once the sample
+orientation has been determined from one of the measurements, *e.g.*, at
+room temperature, it can be copied and refined automatically when
+reducing the data from other measurements, provided the space group has
+not changed or is still compatible with the observed Bragg peaks. 
+
+

_sources/server.rst.txt

@@ -151,6 +151,8 @@ Here is the structure of the ``nxserver`` directory::
    :width: 90%
    :figwidth: 50%
 
+.. _default_settings:
+
 Default Settings
 ^^^^^^^^^^^^^^^^
 The file, ``settings.ini`` in the server directory contains the default

experiment.html

@@ -16,7 +16,7 @@
     <link rel="shortcut icon" href="_static/favicon.ico"/>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
-    <link rel="next" title="Data reduction" href="refine.html" />
+    <link rel="next" title="Data Reduction" href="refine.html" />
     <link rel="prev" title="Server Configuration" href="server.html" /> 
   </head><body>
     <div class="related" role="navigation" aria-label="related navigation">
@@ -26,7 +26,7 @@ <h3>Navigation</h3>
           <a href="genindex.html" title="General Index"
              accesskey="I">index</a></li>
         <li class="right" >
-          <a href="refine.html" title="Data reduction"
+          <a href="refine.html" title="Data Reduction"
              accesskey="N">next</a> |</li>
         <li class="right" >
           <a href="server.html" title="Server Configuration"
@@ -489,16 +489,75 @@ <h3>Create Mask<a class="headerlink" href="#create-mask" title="Permalink to thi
 the beam center, but may be moved by dragging the center of the shape
 and/or resized by dragging one of the shape edges. When the shape has
 the correct position and size, click on “Save Shape” for the shape to be
-added to the current list. After saving the shape, it is no longer
-draggable. However, the shape can still be modified by adjusting the
-shape parameters and then clicking on “Save Shape” again. A pull-down
-menu allows existing shapes to be selected for further edits or removal
-if necessary.</p>
+added to the current list. A pull-down menu allows existing shapes to be
+selected for further edits or removal</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>After saving the shape, it is no longer draggable. However,
+the shape can still be modified by adjusting the shape
+parameters and then clicking on “Save Shape” again.</p>
+</div>
 <p>If a more complicated mask is required, it can be generated by an
 external image editor and imported using “Import Mask”.</p>
 <p>When the mask is complete, click “Save” to save it to the configuration
 file.</p>
 </div>
+<div class="section" id="new-sample">
+<h3>New Sample<a class="headerlink" href="#new-sample" title="Permalink to this headline">¶</a></h3>
+<p>This dialog has the single purpose of creating a directory tree for a
+new sample. The dialog enables the creation of a sample directory within
+the requested experiment directory and a sub-directory with a unique
+label for each instance of that sample measured during an experiment.</p>
+<div class="figure align-center">
+<a class="reference internal image-reference" href="_images/new-sample.png"><img alt="_images/new-sample.png" src="_images/new-sample.png" style="width: 60%;" /></a>
+</div>
+</div>
+<div class="section" id="new-scan">
+<h3>New Scan<a class="headerlink" href="#new-scan" title="Permalink to this headline">¶</a></h3>
+<p>This dialog is used to create a NeXus file in preparation for an
+experimental measurement. The file will be based on the selected
+configuration file and be saved in the specified sample/label directory.
+The name of the file will be “&lt;sample&gt;_&lt;scan&gt;.nxs”, where &lt;scan&gt; is the
+Scan Label specified in the dialog (‘300K’ in the image below).</p>
+<div class="figure align-center">
+<a class="reference internal image-reference" href="_images/new-scan.png"><img alt="_images/new-scan.png" src="_images/new-scan.png" style="width: 80%;" /></a>
+</div>
+<p>The NeXus file is left open in the NeXpy tree. Multiple files can be
+created within the dialog, with different scan labels and, typically,
+different temperatures, before the dialog is closed.</p>
+<p>External links to the raw data file are created in the NeXus file, even
+if the data does not yet exist. In the example above, the external link
+for the first detector position will be to <code class="docutils literal notranslate"><span class="pre">f1.h5</span></code>, in the <code class="docutils literal notranslate"><span class="pre">&lt;scan&gt;</span></code>
+subdirectory. Similarly, the external link for the second detector
+position would be to <code class="docutils literal notranslate"><span class="pre">&lt;scan&gt;/f2.h5</span></code>, <em>etc</em>. This experimental layout
+is described in more detail in the <a class="reference internal" href="#experiment-layout">Experiment Layout</a> section above.</p>
+</div>
+<div class="section" id="import-scans">
+<h3>Import Scans<a class="headerlink" href="#import-scans" title="Permalink to this headline">¶</a></h3>
+<p>This dialog is for instruments in which the scans are already defined
+using different methods to those above. For example, on the QM2
+instrument at CHESS, the scans are defined in SPEC files, with the data
+stored separately in a separate read-only directory. With this dialog,
+the directories containing the raw images are associated with the
+corresponding SPEC scan, allowing NeXus files to be automatically
+generated. This customization is encoded in a QM2 sub-class of the
+<code class="docutils literal notranslate"><span class="pre">NXBeamLine</span></code> class, which is installed separately as a NXRefine
+plugin. The process for customizing other beamlines is described later.</p>
+</div>
+<div class="section" id="sum-scans">
+<h3>Sum Scans<a class="headerlink" href="#sum-scans" title="Permalink to this headline">¶</a></h3>
+<p>This dialog allows data in NeXus files collected under identical
+conditions to be summed to produce a single NeXus file that can be
+processed using the usual workflow.</p>
+</div>
+<div class="section" id="edit-settings">
+<h3>Edit Settings<a class="headerlink" href="#edit-settings" title="Permalink to this headline">¶</a></h3>
+<p>This dialog allows the settings, whose default values are defined in the
+server directory (see <a class="reference internal" href="server.html#default-settings"><span class="std std-ref">Default Settings</span></a>), to be customized for the
+data reduction performed in the selected experiment. The settings are
+stored in <code class="docutils literal notranslate"><span class="pre">&lt;experiment&gt;/tasks/settings.ini</span></code>. The meanings of each
+setting are described in the next section.</p>
+</div>
 </div>
 </div>
 
@@ -521,6 +580,11 @@ <h3><a href="index.html">Table of Contents</a></h3>
 <li><a class="reference internal" href="#new-configuration">New configuration</a></li>
 <li><a class="reference internal" href="#calibrate-powder">Calibrate Powder</a></li>
 <li><a class="reference internal" href="#create-mask">Create Mask</a></li>
+<li><a class="reference internal" href="#new-sample">New Sample</a></li>
+<li><a class="reference internal" href="#new-scan">New Scan</a></li>
+<li><a class="reference internal" href="#import-scans">Import Scans</a></li>
+<li><a class="reference internal" href="#sum-scans">Sum Scans</a></li>
+<li><a class="reference internal" href="#edit-settings">Edit Settings</a></li>
 </ul>
 </li>
 </ul>
@@ -536,7 +600,7 @@ <h4>Previous topic</h4>
   <div>
     <h4>Next topic</h4>
     <p class="topless"><a href="refine.html"
-                          title="next chapter">Data reduction</a></p>
+                          title="next chapter">Data Reduction</a></p>
   </div>
   <div role="note" aria-label="source link">
     <h3>This Page</h3>
@@ -566,7 +630,7 @@ <h3>Navigation</h3>
           <a href="genindex.html" title="General Index"
              >index</a></li>
         <li class="right" >
-          <a href="refine.html" title="Data reduction"
+          <a href="refine.html" title="Data Reduction"
              >next</a> |</li>
         <li class="right" >
           <a href="server.html" title="Server Configuration"

index.html

@@ -88,7 +88,7 @@ <h1>NXRefine<a class="headerlink" href="#nxrefine" title="Permalink to this head
 <li class="toctree-l2"><a class="reference internal" href="experiment.html#experiment-menu">Experiment Menu</a></li>
 </ul>
 </li>
-<li class="toctree-l1"><a class="reference internal" href="refine.html">Data reduction</a></li>
+<li class="toctree-l1"><a class="reference internal" href="refine.html">Data Reduction</a></li>
 </ul>
 </div>
 </div>

installation.html

@@ -82,7 +82,7 @@ <h2>Required Libraries<a class="headerlink" href="#required-libraries" title="Pe
 <td><p><a class="reference external" href="https://docs.sqlalchemy.org/">https://docs.sqlalchemy.org/</a></p></td>
 </tr>
 <tr class="row-even"><td><p>psutil</p></td>
-<td><p><a class="reference external" href="https://psutil.readthedocs.io/.io/">https://psutil.readthedocs.io/.io/</a></p></td>
+<td><p><a class="reference external" href="https://psutil.readthedocs.io/">https://psutil.readthedocs.io/</a></p></td>
 </tr>
 <tr class="row-odd"><td><p>persist-queue</p></td>
 <td><p><a class="reference external" href="https://github.com/peter-wangxu/persist-queue">https://github.com/peter-wangxu/persist-queue</a></p></td>

refine.html

@@ -5,7 +5,7 @@
   <head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>Data reduction &#8212; NXRefine 0.1.0 documentation</title>
+    <title>Data Reduction &#8212; NXRefine 0.1.0 documentation</title>
     <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
     <link rel="stylesheet" type="text/css" href="_static/sphinxdoc.css" />
     <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
@@ -27,7 +27,7 @@ <h3>Navigation</h3>
           <a href="experiment.html" title="Experiments"
              accesskey="P">previous</a> |</li>
         <li class="nav-item nav-item-0"><a href="index.html">NXRefine 0.1.0 documentation</a> &#187;</li>
-        <li class="nav-item nav-item-this"><a href="">Data reduction</a></li> 
+        <li class="nav-item nav-item-this"><a href="">Data Reduction</a></li> 
       </ul>
     </div>  
 
@@ -37,7 +37,41 @@ <h3>Navigation</h3>
           <div class="body" role="main">
 
   <div class="section" id="data-reduction">
-<h1>Data reduction<a class="headerlink" href="#data-reduction" title="Permalink to this headline">¶</a></h1>
+<h1>Data Reduction<a class="headerlink" href="#data-reduction" title="Permalink to this headline">¶</a></h1>
+<p>In order to reduce raw data collected as images (or frames) on an area
+detector as a function of sample rotation angle and transform the
+results into reciprocal space maps, <em>i.e.</em>, S(<strong>Q</strong>), <em>NXRefine</em>
+performs the following steps:</p>
+<ul class="simple">
+<li><p>combining the frames into a single three-dimensional array.</p></li>
+<li><p>harvesting metadata collected during the sample rotations.</p></li>
+<li><p>summing detector frames to facilitate absorption corrections.</p></li>
+<li><p>searching for Bragg peaks embedded within the raw data.</p></li>
+<li><p>defining an orientation matrix.</p></li>
+<li><p>transforming the raw data into reciprocal space coordinates.</p></li>
+</ul>
+<p>When multiple sample rotations are performed to collect a single data
+set, these steps have to be applied to each rotation scan and the
+results merged to produce a single three-dimensional array representing
+S(<strong>Q</strong>). Optionally, <em>NXRefine</em> also transforms the data after applying
+masks that eliminate spurious signals caused by the scattering of Bragg
+peaks within the detector sensor layer.</p>
+<p>Once the data has been transformed into S(<strong>Q</strong>), it is possible to
+generate 3D-ΔPDF maps, which transform the data back into real space,
+producing difference Patterson maps, <em>i.e.</em>, maps of interatomic vector
+probabilities, which differ from the average crystalline structure. In
+this way, continuous distributions of diffuse scattering intensity are
+typically reduced to discrete peaks, with positive and negative
+intensities, representing these probability differences. <em>NXRefine</em>
+implements the “punch-and-fill” method, described by <a class="reference external" href="https:dx.doi.org.10.1524/zkri.2012.1504">Weber and Simonov</a>.</p>
+<p>Nearly all of the steps in the <em>NXRefine</em> data reduction workflow can
+either be performed from the command line or launched from a NeXpy GUI.
+The exception is determining the crystal orientation, which must first
+be performed using the <cite>Refine Lattice</cite> dialog in NeXpy. Once the sample
+orientation has been determined from one of the measurements, <em>e.g.</em>, at
+room temperature, it can be copied and refined automatically when
+reducing the data from other measurements, provided the space group has
+not changed or is still compatible with the observed Bragg peaks.</p>
 </div>
 
 
@@ -83,7 +117,7 @@ <h3>Navigation</h3>
           <a href="experiment.html" title="Experiments"
              >previous</a> |</li>
         <li class="nav-item nav-item-0"><a href="index.html">NXRefine 0.1.0 documentation</a> &#187;</li>
-        <li class="nav-item nav-item-this"><a href="">Data reduction</a></li> 
+        <li class="nav-item nav-item-this"><a href="">Data Reduction</a></li> 
       </ul>
     </div>
     <div class="footer" role="contentinfo">