Skip to content

Commit

Permalink
deploy: 1e84aba
Browse files Browse the repository at this point in the history
  • Loading branch information
rayosborn committed Jun 21, 2024
1 parent e832c15 commit 781ee14
Show file tree
Hide file tree
Showing 5 changed files with 94 additions and 64 deletions.
73 changes: 43 additions & 30 deletions _sources/experiment.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,34 +16,38 @@ the storage of x-ray and neutron scattering data. There is a single
NeXus file associated with each measurement, which consists of one or
more rotation scans usually at a single temperature or other parametric
variable. This file contains all the information required for a complete
analysis, external links to the raw data, beamline monitors, powder
diffaction calibrations, and metadata generated by the data reduction workflow.
analysis, including external links to the raw data, beamline monitors,
powder diffaction calibrations, and metadata generated by the data
reduction workflow.

In this section, we will describe the *NXRefine* directory framework,
explaining where the NeXus files are stored and how they are linked to
the reduced data transformed into reciprocal space. At the beginning of
each new experiment, the GUI dialogs launched from the :ref:`Experiment
Menu` can be used to create the experiment directories, NeXus file
templates, and new NeXus files corresponding to the proposed scans.
templates, and new NeXus files based on those templates to contain scan
data.

Experiment Layout
=================
In *NXRefine*, it is assumed that all the files associated with a
particular experiment are stored in a single directory. At synchrotron
x-ray facilities, it is common to schedule all the measurements
associated with a particular proposal together, so they would be
labelled by the proposal number and/or run cycle. For example, on
beamline 6-ID-D at the Advanced Photon Source, measurements resulting
from Proposal No. GUP-75969 may be scheduled in a specific run cycle,
say 23-1, and stored in, *e.g.*, ``/data/6-id-d/GUP-75969-23-1``. This
directory should contain sub-directories that conform to a particular
layout, with calibration files stored in the directory ``calibrations``,
NeXus files containing measurement templates stored in
``configurations``, settings and log files stored in ``tasks``, and
experimental data from each sample stored in separate directories with a
descriptive name. These sample directories then contain sub-directories
for each crystal, measured during the experiment, containing the
measured data.
associated with a particular proposal together, associated with a
proposal number and/or run cycle. For example, on beamline 6-ID-D at the
Advanced Photon Source, measurements resulting from Proposal No.
GUP-75969 may be scheduled in a specific run cycle, say 23-1, and stored
in, *e.g.*, ``/data/6-id-d/GUP-75969-23-1``. This directory should
contain sub-directories that conform to a particular layout, with
calibration files stored in the directory ``calibrations``, NeXus files
containing measurement templates stored in ``configurations``, settings
and log files stored in ``tasks``, and experimental data from each
sample stored in separate directories with a name that is typically
derived from the chemical formula or a commonly used abbreviation. It is
common to test and/or measure multiple crystals with the same chemical
formula, so each of the sample directories contain sub-directories for
each measured crystal. These contain the NeXus files for each scan and
linked sub-directories containing the raw data.

Here is the structure of a possible experiment directory. Most of the
names in this example are chosen to be generic, *i.e.*, they will be
Expand Down Expand Up @@ -83,18 +87,18 @@ different for every experiment. The only exceptions are the files in the
├── sample2
├── sample3

The aim of this directory structure is for all the data and metadata
The goal of this directory structure is for all the data and metadata
required to analyze the results to be stored in an easily accessible
location, although not all files are required by *NXRefine* to be
present. For example, the powder calibration files can be imported from
any location.

If an instrument does not exclusively use *NXRefine* for its data
reduction, it is possible that there already exists a directory
containing the files that comprise an experiment as described above. To
avoid any interference with other instrument files, it is possible to
create the above directory structure within a sub-directory, usually
called ``nxrefine``, of the instrument's ``experiment`` directory.::
reduction, it is possible that an experiment directory already exists.
If so, in order to avoid any interference with other instrument files,
it is possible to create the above directory structure within a
sub-directory, called ``nxrefine``, contained within the existing
``experiment`` directory.::

experiment
└── nxrefine
Expand All @@ -109,9 +113,8 @@ called ``nxrefine``, of the instrument's ``experiment`` directory.::
.

.. note:: The name of this sub-directory is defined in the server
settings, which are described below. Although it can have any
name, it is strongly recommended that it be called
``nxrefine`` to simplify parsing of the directory tree.
settings, which are described below. It is strongly recommended that it be called by the default name, *i.e.*,
``nxrefine`` to facilitate parsing of the directory tree.

Experiment Sub-Directories
==========================
Expand All @@ -134,15 +137,25 @@ Experiment Sub-Directories
NeXus files (described below). The files don't have to be stored in
this sub-directory, but if they are in another location, it is
recommended to copy them here for completeness. If the calibrations
have been performed by another package, the PONI parameters can be
have been performed by another package, the parameters can be
imported directly from a PONI file.

**configurations**
The ``configurations`` sub-directory contains NeXus files that act
as templates when creating the files used to store the scan results.
These files contain scan parameters, such as the goniometer angles,
for one or more sample rotations, and are initialized by a *NeXpy*
GUI dialog.
as templates to be used when creating the files used to store the
scan results. There should be a separate template file for each new
experimental configuration, *.e.g.*, with a different wavelength or
detector distance. If multiple sample rotations are to be performed
with different detector translations and/or goniometer angles, the
corresponding template files will have entries for each scan
containing pre-defined values of the scan variables. These files are
initialized by a *NeXpy* GUI dialog.

.. note:: On QM2 at CHESS, it is usually only necessary
to create template files with a single entry, since the
number of rotation scans is not always pre-determined. When
the scans are imported, additional entries are automatically
added with the goniometer angles updated with the values in the corresponding scan SPEC file.

**scripts**
The ``scripts`` sub-directory is not used directly by *NXRefine*,
Expand Down
2 changes: 1 addition & 1 deletion _sources/installation.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -137,5 +137,5 @@ User Support
------------
If you are interested in using this package, please contact Ray Osborn
([email protected]). Please report any bugs as a
`Github issue <https://github.com/nxrefine/nxrefine/issues>`_, with
`Github issue <https://github.com/nexpy/nxrefine/issues>`_, with
relevant tracebacks.
79 changes: 48 additions & 31 deletions experiment.html
Original file line number Diff line number Diff line change
Expand Up @@ -57,31 +57,35 @@ <h1>Experiment Configuration<a class="headerlink" href="#experiment-configuratio
NeXus file associated with each measurement, which consists of one or
more rotation scans usually at a single temperature or other parametric
variable. This file contains all the information required for a complete
analysis, external links to the raw data, beamline monitors, powder
diffaction calibrations, and metadata generated by the data reduction workflow.</p>
analysis, including external links to the raw data, beamline monitors,
powder diffaction calibrations, and metadata generated by the data
reduction workflow.</p>
<p>In this section, we will describe the <em>NXRefine</em> directory framework,
explaining where the NeXus files are stored and how they are linked to
the reduced data transformed into reciprocal space. At the beginning of
each new experiment, the GUI dialogs launched from the <a class="reference internal" href="experiment-menu.html#experiment-menu"><span class="std std-ref">Experiment Menu</span></a> can be used to create the experiment directories, NeXus file
templates, and new NeXus files corresponding to the proposed scans.</p>
templates, and new NeXus files based on those templates to contain scan
data.</p>
<div class="section" id="experiment-layout">
<h2>Experiment Layout<a class="headerlink" href="#experiment-layout" title="Permalink to this headline"></a></h2>
<p>In <em>NXRefine</em>, it is assumed that all the files associated with a
particular experiment are stored in a single directory. At synchrotron
x-ray facilities, it is common to schedule all the measurements
associated with a particular proposal together, so they would be
labelled by the proposal number and/or run cycle. For example, on
beamline 6-ID-D at the Advanced Photon Source, measurements resulting
from Proposal No. GUP-75969 may be scheduled in a specific run cycle,
say 23-1, and stored in, <em>e.g.</em>, <code class="docutils literal notranslate"><span class="pre">/data/6-id-d/GUP-75969-23-1</span></code>. This
directory should contain sub-directories that conform to a particular
layout, with calibration files stored in the directory <code class="docutils literal notranslate"><span class="pre">calibrations</span></code>,
NeXus files containing measurement templates stored in
<code class="docutils literal notranslate"><span class="pre">configurations</span></code>, settings and log files stored in <code class="docutils literal notranslate"><span class="pre">tasks</span></code>, and
experimental data from each sample stored in separate directories with a
descriptive name. These sample directories then contain sub-directories
for each crystal, measured during the experiment, containing the
measured data.</p>
associated with a particular proposal together, associated with a
proposal number and/or run cycle. For example, on beamline 6-ID-D at the
Advanced Photon Source, measurements resulting from Proposal No.
GUP-75969 may be scheduled in a specific run cycle, say 23-1, and stored
in, <em>e.g.</em>, <code class="docutils literal notranslate"><span class="pre">/data/6-id-d/GUP-75969-23-1</span></code>. This directory should
contain sub-directories that conform to a particular layout, with
calibration files stored in the directory <code class="docutils literal notranslate"><span class="pre">calibrations</span></code>, NeXus files
containing measurement templates stored in <code class="docutils literal notranslate"><span class="pre">configurations</span></code>, settings
and log files stored in <code class="docutils literal notranslate"><span class="pre">tasks</span></code>, and experimental data from each
sample stored in separate directories with a name that is typically
derived from the chemical formula or a commonly used abbreviation. It is
common to test and/or measure multiple crystals with the same chemical
formula, so each of the sample directories contain sub-directories for
each measured crystal. These contain the NeXus files for each scan and
linked sub-directories containing the raw data.</p>
<p>Here is the structure of a possible experiment directory. Most of the
names in this example are chosen to be generic, <em>i.e.</em>, they will be
different for every experiment. The only exceptions are the files in the
Expand Down Expand Up @@ -120,17 +124,17 @@ <h2>Experiment Layout<a class="headerlink" href="#experiment-layout" title="Perm
├── sample3
</pre></div>
</div>
<p>The aim of this directory structure is for all the data and metadata
<p>The goal of this directory structure is for all the data and metadata
required to analyze the results to be stored in an easily accessible
location, although not all files are required by <em>NXRefine</em> to be
present. For example, the powder calibration files can be imported from
any location.</p>
<p>If an instrument does not exclusively use <em>NXRefine</em> for its data
reduction, it is possible that there already exists a directory
containing the files that comprise an experiment as described above. To
avoid any interference with other instrument files, it is possible to
create the above directory structure within a sub-directory, usually
called <code class="docutils literal notranslate"><span class="pre">nxrefine</span></code>, of the instrument’s <code class="docutils literal notranslate"><span class="pre">experiment</span></code> directory.:</p>
reduction, it is possible that an experiment directory already exists.
If so, in order to avoid any interference with other instrument files,
it is possible to create the above directory structure within a
sub-directory, called <code class="docutils literal notranslate"><span class="pre">nxrefine</span></code>, contained within the existing
<code class="docutils literal notranslate"><span class="pre">experiment</span></code> directory.:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>experiment
└── nxrefine
├── tasks
Expand All @@ -147,14 +151,13 @@ <h2>Experiment Layout<a class="headerlink" href="#experiment-layout" title="Perm
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The name of this sub-directory is defined in the server
settings, which are described below. Although it can have any
name, it is strongly recommended that it be called
<code class="docutils literal notranslate"><span class="pre">nxrefine</span></code> to simplify parsing of the directory tree.</p>
settings, which are described below. It is strongly recommended that it be called by the default name, <em>i.e.</em>,
<code class="docutils literal notranslate"><span class="pre">nxrefine</span></code> to facilitate parsing of the directory tree.</p>
</div>
</div>
<div class="section" id="experiment-sub-directories">
<h2>Experiment Sub-Directories<a class="headerlink" href="#experiment-sub-directories" title="Permalink to this headline"></a></h2>
<dl>
<dl class="simple">
<dt><strong>tasks</strong></dt><dd><p>The <code class="docutils literal notranslate"><span class="pre">tasks</span></code> sub-directory contains a number of files used by
<em>NXRefine</em> to store default settings, workflow logs, and a MySQL
database for recording the status of each workflow component. The
Expand All @@ -171,15 +174,29 @@ <h2>Experiment Sub-Directories<a class="headerlink" href="#experiment-sub-direct
NeXus files (described below). The files don’t have to be stored in
this sub-directory, but if they are in another location, it is
recommended to copy them here for completeness. If the calibrations
have been performed by another package, the PONI parameters can be
have been performed by another package, the parameters can be
imported directly from a PONI file.</p>
</dd>
<dt><strong>configurations</strong></dt><dd><p>The <code class="docutils literal notranslate"><span class="pre">configurations</span></code> sub-directory contains NeXus files that act
as templates when creating the files used to store the scan results.
These files contain scan parameters, such as the goniometer angles,
for one or more sample rotations, and are initialized by a <em>NeXpy</em>
GUI dialog.</p>
as templates to be used when creating the files used to store the
scan results. There should be a separate template file for each new
experimental configuration, <em>.e.g.</em>, with a different wavelength or
detector distance. If multiple sample rotations are to be performed
with different detector translations and/or goniometer angles, the
corresponding template files will have entries for each scan
containing pre-defined values of the scan variables. These files are
initialized by a <em>NeXpy</em> GUI dialog.</p>
</dd>
</dl>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>On QM2 at CHESS, it is usually only necessary
to create template files with a single entry, since the
number of rotation scans is not always pre-determined. When
the scans are imported, additional entries are automatically
added with the goniometer angles updated with the values in the corresponding scan SPEC file.</p>
</div>
<dl>
<dt><strong>scripts</strong></dt><dd><p>The <code class="docutils literal notranslate"><span class="pre">scripts</span></code> sub-directory is not used directly by <em>NXRefine</em>,
but is created by the <code class="docutils literal notranslate"><span class="pre">New</span> <span class="pre">Experiment</span></code> dialog described below. It
is designed to store macros for use during an experiment.</p>
Expand Down
2 changes: 1 addition & 1 deletion installation.html
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@ <h2>Initial Setup<a class="headerlink" href="#initial-setup" title="Permalink to
<h3>User Support<a class="headerlink" href="#user-support" title="Permalink to this headline"></a></h3>
<p>If you are interested in using this package, please contact Ray Osborn
(<a class="reference external" href="mailto:ROsborn&#37;&#52;&#48;anl&#46;gov">ROsborn<span>&#64;</span>anl<span>&#46;</span>gov</a>). Please report any bugs as a
<a class="reference external" href="https://github.com/nxrefine/nxrefine/issues">Github issue</a>, with
<a class="reference external" href="https://github.com/nexpy/nxrefine/issues">Github issue</a>, with
relevant tracebacks.</p>
</div>
</div>
Expand Down
Loading

0 comments on commit 781ee14

Please sign in to comment.