diff --git a/docs/source/experiment-menu.rst b/docs/source/experiment-menu.rst new file mode 100644 index 0000000..7821ba9 --- /dev/null +++ b/docs/source/experiment-menu.rst @@ -0,0 +1,284 @@ +Experiment Menu +=============== +The *NXRefine* plugin to *NeXpy* installs a top-level menu labelled +"Experiment". The sub-menus run operations to initialize the experiment +layout, create experimental data templates, calibrate powder data, and +initialize new data files. + +New Experiment +-------------- +This dialog initializes a new experiment directory layout using the +server settings to initialize default locations. When the dialog is +launched, click on "Choose Experiment Directory" to launch the system +file browser in order to select or create the new experiment directory. + +.. figure:: /images/new-experiment-CHESS.png + :align: center + :width: 80% + +There are two scenarios. + +1. If ``raw_home`` is not blank in the server settings, the file browser + will default to the ``raw_home`` directory, in which an experiment + directory, containing the raw image files, should already exist. This + experiment directory is then selected, after which the dialog above + is created, with the experiment name (*i.e.*, the base name of the + experiment directory path) already filled in, along with the path to + analysis home directory (``analysis_home`` in the server settings) + and the name of the analysis sub-directory if required. When the + "Save" button is pressed, the new experiment directory is created + within the analysis home directory if it does not already exist, and + the experiment directory tree is initialized with the + ``calibrations``, ``configurations``, ``scripts`` and ``tasks`` + sub-directories. + +2. If ``raw_home`` is blank, the file browser will default to the + ``analysis_home`` directory, but another location can be selected if + required. The file browser can be used either to select an existing + experiment directory or to create a new one. The above dialog is then + created with the experiment name given by the base name of the + selected experiment directory path, and the analysis home directory + defined by its parent. When the "Save" button is pressed, the + experiment directory tree is initialized with the ``calibrations``, + ``configurations``, ``scripts`` and ``tasks`` sub-directories. + +A new ``settings.ini`` file is created in the ``tasks`` sub-directory, +with values copied from the equivalent file in the server directory, +excluding the "Server" section. This allows the refinement parameters to +be customized for each experiment. + +New configuration +----------------- +This dialog creates NeXus files that are used as templates for the +experimental files that are used to store all the data and metadata +associated with a particular set of rotation scans. The initial metadata +is defined by parameters in the settings file in the ``tasks`` +sub-directory, which can be modified by the "Edit Settings" sub-menu +described below. However, some of the metadata will be refined using a +powder calibration, whose results are then stored in this file. + +After selecting the experiment directory, the following dialog is created. + +.. figure:: /images/new-configuration-CHESS.png + :align: center + :width: 80% + +This allows the settings used in subsequent analysis to be initialized, +the parameters defining the rotation scans (range, step size, frame +rate) to be set, the detector configuration to be defined, and the +angles and/or detector positions to be used in one or more rotation +scans. These are all saved to the NeXus template. The wavelength and +detector distance can be nominal values at this stage, since they are +updated by performing a powder calibration. Similarly, the instrument +angles, :math:`\theta`, :math:`\omega`, and :math:`\chi` are set to the +angles set by the motors, but will usually be refined when the sample +orientation is determined. + +It is possible to create more than one configuration template, if, for +example, different angles and/or detector positions are used in +different phases of an experiment. *NXRefine* allows the appropriate +template to be selected when setting up the scan. A separate template +should be created for each configuration that requires a change in the +instrument calibration (wavelength, detector distance, detector +translation) or scan angles. + +The detector is chosen from a pull-down menu that contains all the +detectors defined in the *PyFAI* package. This defines the number of +pixels, their size, and a mask array used to exclude all the pixels +within gaps between the detector chips. + +Calibrate Powder +---------------- +This dialog will import a TIFF or CBF file containing measurements of a +powder calibrant and refine the detector position and coordinates, using +the *PyFAI* API. Alternatively, if the calibration parameters are +already available in a PONI file, they can be directly imported. The +resulting powder data and calbration parameters are then saved to the +configuration template previously created using the *New Configuration* +dialog. + +.. figure:: /images/calibrate-powder.png + :align: center + :width: 80% + +After launching the dialog, select the entry in the configuration file +to be calibrated by the powder measurement, *i.e.*, the one with the +correct wavelength, detector distance and translations. This expands the +dialog with the default parameters defined by the settings file. The +checkboxes at the side of each parameter specify whether the parameter +is to be refined. By default, the wavelength checkbox is de-selected, +since this is normally defined accurately by other means. It is too +highly correlated to the detector distance for both to be refined +simultaneously. + +Then click on "Import Powder Data" to select the powder calibration +file. This will generate a plot containing the powder data on a log +scale. Select the approprate powder calibrant from those specified in +the Calibrant pull-down menu. + +If a PONI file already exists from a prior calibration, it can be +imported using the "Import Calibration" button. If this is sufficiently +accurate, it is not necessary to perform further calibrations. Instead +the calibration parameters can be saved to the configuration file by +clicking on "Save" and the dialog can be closed. + +To obtain an initial calibration, zoom into this plot to display +the first few rings. + +.. figure:: /images/select-ring.png + :align: center + :width: 80% + + *Points generated for the innermost ring after manually selecting + four points* + +After clicking on "Select Points", click somewhere on the innermost +ring. This triggers the PyFAI Massif module, which automatically detects +other points on the Debye-Scherrer ring that are contiguous to the +selected point. Because of the gaps between detector chips, the Massif +detection is confined to pixels within a single chip, so it is normally +necessary to select other points on neighboring chips to complete a +single ring. In the above ring, four selections, corresponding to the +brighter red circles, were made. + +It is only necessary to do this for a single ring. De-select the "Select +Points" button and click "Calibrate" to perform an initial calibration. +After this, it is possible to generate points automatically on the other +rings using the "Autogenerate Rings" button. Select how many rings to +generate, using the ring pull-down menu. + +.. figure:: /images/autogenerate-rings.png + :align: center + :width: 80% + + *Autogenerated rings after selecting "Ring6" on the pull-down menu* + +When enough rings have been defined, click "Calibrate" again to produce +a more accurate refinement. + +The "Plot Cake" button can be used to generate a "cake" plot, in which +all the powder rings, which are plotted against polar angle, should fall +on vertical lines. + +.. figure:: /images/cake-plot.png + :align: center + :width: 80% + + *Cake Plot which allows a comparison of the powder data, plotted as a + function of polar angle, with the theoretical powder lines (dotted + red lines).* + +This can be used to determine whether the calibration is sufficiently +good over the entire angular range of the detector. If there is evidence +of distortions at higher polar angle, it may be necessary to +autogenerate more rings before an additional calibration. + +When the calibration is satisfactory, click "Save" to save both the +powder calibration data and parameters to the configuration file. The +calibration parameters can also be saved to a PONI file, using the +"Export Calibration" button. This process should be repeated for each +entry, after which the dialog can be closed. + +Create Mask +----------- +This dialog creates a pixel mask that is used to exclude bad pixels from +further analysis. As described above, when a new configuration file is +created, a pixel mask that excludes gaps between detector chips is +automatically added. Additional pixels can be excluded using this +dialog, either by adding editable shapes that are constructively added +to the existing mask or by importing the mask from an external file, +which can store the mask in any image format. The latter is useful if a +beamline regularly updates a particular detector's mask as bad pixels are identified. + +.. warning:: If an external mask is input using "Import Mask", it will + overwrite the existing mask. It is important therefore that + the external pixel mask also excludes the detector gaps. + +After launching the dialog, the current mask is automatically plotted, +as an overlay on the powder diffraction data to enable the center of the +beam and other features of the data to be identified. + +.. figure:: /images/create-mask.png + :align: center + :width: 80% + + *Create Mask dialog. The translucent shape shows the rectangle + created by clicking "Add Shape".* + +By clicking on "Add Shape" with either a rectangle or circle selected, a +translucent shape is added to the plot. By default, it is centered on +the beam center, but may be moved by dragging the center of the shape +and/or resized by dragging one of the shape edges. When the shape has +the correct position and size, click on "Save Shape" for the shape to be +added to the current list. A pull-down menu allows existing shapes to be +selected for further edits or removal + +.. note:: After saving the shape, it is no longer draggable. However, + the shape can still be modified by adjusting the shape + parameters and then clicking on "Save Shape" again. + +If a more complicated mask is required, it can be generated by an +external image editor and imported using "Import Mask". + +When the mask is complete, click "Save" to save it to the configuration +file. + +New Sample +---------- +This dialog has the single purpose of creating a directory tree for a +new sample. The dialog enables the creation of a sample directory within +the requested experiment directory and a sub-directory with a unique +label for each instance of that sample measured during an experiment. + +.. figure:: /images/new-sample.png + :align: center + :width: 60% + +New Scan +-------- +This dialog is used to create a NeXus file in preparation for an +experimental measurement. The file will be based on the selected +configuration file and be saved in the specified sample/label directory. +The name of the file will be "_.nxs", where is the +Scan Label specified in the dialog ('300K' in the image below). + +.. figure:: /images/new-scan.png + :align: center + :width: 80% + +The NeXus file is left open in the NeXpy tree. Multiple files can be +created within the dialog, with different scan labels and, typically, +different temperatures, before the dialog is closed. + +External links to the raw data file are created in the NeXus file, even +if the data does not yet exist. In the example above, the external link +for the first detector position will be to ``f1.h5``, in the ```` +subdirectory. Similarly, the external link for the second detector +position would be to ``/f2.h5``, *etc*. This experimental layout +is described in more detail in the `Experiment Layout`_ section above. + +Import Scans +------------ +This dialog is for instruments in which the scans are already defined +using different methods to those above. For example, on the QM2 +instrument at CHESS, the scans are defined in SPEC files, with the data +stored separately in a separate read-only directory. With this dialog, +the directories containing the raw images are associated with the +corresponding SPEC scan, allowing NeXus files to be automatically +generated. This customization is encoded in a QM2 sub-class of the +``NXBeamLine`` class, which is installed separately as a NXRefine +plugin. The process for customizing other beamlines is described later. + +Sum Scans +--------- +This dialog allows data in NeXus files collected under identical +conditions to be summed to produce a single NeXus file that can be +processed using the usual workflow. + +Edit Settings +------------- +This dialog allows the settings, whose default values are defined in the +server directory (see :ref:`default_settings`), to be customized for the +data reduction performed in the selected experiment. The settings are +stored in ``/tasks/settings.ini``. The meanings of each +setting are described in the next section. diff --git a/docs/source/experiment.rst b/docs/source/experiment.rst index c7ffe85..9c092b4 100644 --- a/docs/source/experiment.rst +++ b/docs/source/experiment.rst @@ -1,43 +1,49 @@ -Experiments -*********** +Experiment Configuration +************************ +*NXRefine* uses a hierarchical directory structure for each experiment +to store both the raw data generated on an x-ray beamline and processed +data generated by the data reduction workflow. An 'experiment' in +*NXRefine* comprises measurements on a set of samples that are logically +grouped together, often because they are performed within a specific +time period and/or share calibration files. Plugins to the *NeXpy* GUI +are used to facilitate the creation of both the directory hierarchy and +the associated data files. + Data that are processed by the *NXRefine* package are stored as HDF5 -files stored according to the `NeXus format +files that conform to the `NeXus format `__, which is an international standard for -the storage of x-ray and neutron scattering data. *NXRefine* uses a -hierarchical directory structure to store both the experimental data, -ingested from the raw data generated on an x-ray beamline, and processed -data generated by the data reduction workflow. Scripts and plugins to -the *NeXpy* GUI are used to facilitate the creation of both the -directory hierarchy and the NeXus files themselves. - -In the next section, we will describe the workflow used to both ingest -the raw data and transform it into reciprocal space coordinates and -pair-distribution-functions. In this section, we will describe the -directory framework, within which these processes function, and how -*NeXpy* can be used to initialize it. +the storage of x-ray and neutron scattering data. There is a single +NeXus file associated with each measurement, which consists of one or +more rotation scans usually at a single temperature or other parametric +variable. This file contains all the information required for a complete +analysis, external links to the raw data, beamline monitors, powder +diffaction calibrations, and metadata generated by the data reduction workflow. + +In this section, we will describe the *NXRefine* directory framework, +explaining where the NeXus files are stored and how they are linked to +the reduced data transformed into reciprocal space. At the beginning of +each new experiment, the GUI dialogs launched from the :ref:`Experiment +Menu` can be used to create the experiment directories, NeXus file +templates, and new NeXus files corresponding to the proposed scans. Experiment Layout ================= -An 'experiment' in *NXRefine* comprises measurements on a set of samples -that are logically grouped together, often because they are performed -within a specific time period and/or share calibration files. At -synchrotron x-ray facilities, it is common to schedule all the -measurements associated with a particular proposal together, so they -would be labelled by the proposal number and/or run cycle. For example, -on beamline 6-ID-D at the Advanced Photon Source, measurements resulting -from Proposal No. GUP-75969 may be scheduled in a specific run cycle, -say 23-1, and stored in, *e.g.*, ``/data/6-id-d/GUP-75969-23-1``. - In *NXRefine*, it is assumed that all the files associated with a -particular experiment are stored in a single directory, *i.e.*, in the -APS example above, ``GUP-75969-23-1``. This directory should contain -sub-directories that conform to a particular layout, with calibration -files stored in the directory ``calibrations``, NeXus files containing -measurement templates stored in ``configurations``, settings and log -files stored in ``tasks``, and experimental data from each sample stored -in separate directories with a descriptive name. These sample -directories then contain sub-directories for each crystal, measured -during the experiment, containing the measured data. +particular experiment are stored in a single directory. At synchrotron +x-ray facilities, it is common to schedule all the measurements +associated with a particular proposal together, so they would be +labelled by the proposal number and/or run cycle. For example, on +beamline 6-ID-D at the Advanced Photon Source, measurements resulting +from Proposal No. GUP-75969 may be scheduled in a specific run cycle, +say 23-1, and stored in, *e.g.*, ``/data/6-id-d/GUP-75969-23-1``. This +directory should contain sub-directories that conform to a particular +layout, with calibration files stored in the directory ``calibrations``, +NeXus files containing measurement templates stored in +``configurations``, settings and log files stored in ``tasks``, and +experimental data from each sample stored in separate directories with a +descriptive name. These sample directories then contain sub-directories +for each crystal, measured during the experiment, containing the +measured data. Here is the structure of a possible experiment directory. Most of the names in this example are chosen to be generic, *i.e.*, they will be @@ -65,9 +71,15 @@ different for every experiment. The only exceptions are the files in the ├── f3_transform.nxs └── transform.nxs ├── sample1_200K.nxs + └── 200K + ├── ... └── sample1_300K.nxs + └── 300K + ├── ... └── label2 └── sample1_300K.nxs + └── 300K + ├── ... ├── sample2 ├── sample3 diff --git a/docs/source/glossary.rst b/docs/source/glossary.rst new file mode 100644 index 0000000..17d6b10 --- /dev/null +++ b/docs/source/glossary.rst @@ -0,0 +1,54 @@ +Glossary of Terms +***************** +This section contains a list of terms used in the preceding sections of the documentation, along with brief descriptions and links to where they are explained in more detail. + +**Experiment** + Each experiment consists of measurements performed within a specific + time period, which share a common set of instrument configurations and + calibration files. The measurements could be performed on multiple + samples. + + .. seealso:: :ref:`Experiment Layout` + +**Experiment Directory** + This is the directory containing all the files and sub-directories associated with a single experiment. + +**Sample** + Each experiment typcially contains measurements on one or more + samples, which are labelled by their chemical formula or a descriptive + name, *e.g.,* "TiSe2" or "LBCO". It is common to perform measurements + on multiple crystals that are nominally the same material, each with a + unique label, *e.g.*, defined by the crystal grower, before deciding which crystal to use. + +**Sample Directory** + In the "Experiment Directory", each sample has its own sub-directory, within which are one or more sub-directories for each measured crystal. For example, in the following directory tree, there are two possible "Sample Directories", either ``experiment/sample/label1`` or ``experiment/sample/label2``:: + + experiment + └── sample + └── label1 + └── label2 + +**Scan File** + This is the NeXus file that contains all the information required to reduce, analyze, and interpret the data measured at a single temperature or other instrumental setting. This is stored in one of the "Sample Directories", with a name given by combining the sample name and scan parameter, *e.g.* for measurements at 100K, the following NeXus file is defined.:: + + experiment + └── sample + └── label + └── sample_100K.nxs + +**Scan Directory** + Each NeXus file contains external links to HDF5 files containing the raw data and larger processed data files. These are stored in "Scan Directories", whose names must match the corresponding "Scan File":: + + experiment + └── sample + └── label + ├── sample_100K.nxs + └── 100K + ├── f1.h5 + ... + +**Server Directory** + This directory is defined when *NXRefine* is first installed in order to store log files, task queues, and default settings. On systems with multiple users, this must be in a shared location. It is recommended that all users are members of the same group, which has read/write privileges in the "Server Directory". + + .. seealso:: :ref:`Server Management` + diff --git a/docs/source/images/choose-parameters.png b/docs/source/images/choose-parameters.png new file mode 100644 index 0000000..64640e1 Binary files /dev/null and b/docs/source/images/choose-parameters.png differ diff --git a/docs/source/images/scan-file.png b/docs/source/images/scan-file.png new file mode 100644 index 0000000..526397b Binary files /dev/null and b/docs/source/images/scan-file.png differ diff --git a/docs/source/index.rst b/docs/source/index.rst index b4e0e24..ac333c4 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -18,13 +18,16 @@ arrays and transformed from detector coordinates to reciprocal space coordinates, using an orientation matrix derived from the measured Bragg peaks. -NXRefine implements a complete workflow for both data acquisition and -reduction of single crystal x-ray scattering. Advanced workflows exist -for the generation of Bragg peak intensities used by crystallographers -to solve the average crystalline structure. However, the goal of -NXRefine is to generate a three-dimensional mesh of scattering intensity -that includes both Bragg peaks and the diffuse scattering that arises -from deviations from the average structure. +*NXRefine* implements a complete workflow for both data acquisition and +reduction of single crystal x-ray scattering. Workflows developed for +crystallography typically have the goal of generating Bragg peak +intensities used to determine the average crystalline structure, but the +goal of *NXRefine* is to produce a three-dimensional mesh of scattering +intensity in reciprocal space that includes both Bragg peaks *and* the +diffuse scattering that arises from deviations from the average +structure. The workflow also generates three-dimensional +pair-distribution-function maps (PDFs), both total PDF and 3D-ΔPDF, to +determine interatomic vector probabilities in real space. The workflow is written as a set of Python modules that can either be run from the command line, launched from a GUI that is implemented as a @@ -40,9 +43,11 @@ temperature, are complete. introduction installation - server experiment - refine + sample + reduction + server + glossary Indices and tables ================== diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 01553ab..cbc6cfb 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -1,5 +1,5 @@ Installation -============ +************ Currently, *NXRefine* must be installed from source by cloning the `NXRefine Git repository `_:: @@ -15,7 +15,7 @@ In the near future, *NXRefine* will be uploaded to the PyPI server so that it can be installed without downloading the source code. Required Libraries ------------------- +================== The following packages are listed as dependencies. ================= ================================================= @@ -31,7 +31,7 @@ persist-queue https://github.com/peter-wangxu/persist-queue ================= ================================================= NeXpy ------ +===== Although much of the *NXRefine* workflow can be performed using command-line scripts, it is recommended that they are used in conjunction with the Python-based GUI, `NeXpy @@ -70,17 +70,72 @@ items to the *NeXpy* menu bar. The menus will be described in more detail in subsequent sections. CCTW ----- +==== `CCTW `_ (Crystal Coordination Transformation Workflow) is a C++ package written by Guy Jennings. It is launched as a separate process by *NXRefine*, which uses the experimental metadata to define the settings file used to define the input and output grids. It has to be separately installed. +Initial Setup +============= +In order to allow *NXRefine* to be used by multiple users on a single +machine or cluster, a common directory is defined to store log files, +task queues, and default settings. The location of this directory should +be defined immediately after installing *NXRefine* for the first time. +Since the files in this directory are modified by *NXRefine* commands +that could be run by multiple users, it is recommended that all such +users are members of the same group. When initialized by a member of +that group, the files in the server directory have group read/write +permissions by default. + +The location of the server directory is initialized on the command line +by the 'nxserver' command:: + + $ nxserver -d /path/to/parent + +This will create a directory at ``/path/to/parent/nxserver`` containing +the files that are required by *NXRefine* server. + +.. note:: If the supplied path already ends in ``nxserver``, it will not + be appended. + +Once the server directory has been initialized, it is necessary for its +location to be defined for other users. This can be done in one of two +ways. + +1. If *NXRefine* is being configured by a system administrator, it is + possible to use a system-wide environment variable, ``NX_SERVER``, to + to define the path to the server directory. Alternatively, this + environment variable could be added to each user's login script. + +2. The ``nxserver`` command used to initialize the server directory also + adds a hidden file to the user's home directory, + ``~/.nxserver/settings.ini``, which contains the server directory + path. If the server directory already exists, the command can be run + again by other users without affecting the initial directory. In + principle, it only needs to be run once by each user, although it + could also be added to a login script if preferred. + + .. note:: If the ``NX_SERVER`` environment variable is defined, it + takes precedence over the path in + ``~/.nxserver/settings.ini``. + +*NXRefine* uses file-based locking to prevent corruption of data files. +This system is provided by the +`nexusformat package `_, which defines +the directory to contain the lock files using the ``NX_LOCKDIRECTORY`` +environment variable. It is recommended that this directory be placed +within the server directory. + +.. note:: The *NeXpy* GUI has a settings file that can be used to define + the lock directory, but it is overridden by the environment + variable if it is defined. This allows system administrators + to set up a unique lock file directory for all their users. + User Support ------------ If you are interested in using this package, please contact Ray Osborn (ROsborn@anl.gov). Please report any bugs as a `Github issue `_, with relevant tracebacks. - diff --git a/docs/source/introduction.rst b/docs/source/introduction.rst index f78ef4b..33a8416 100644 --- a/docs/source/introduction.rst +++ b/docs/source/introduction.rst @@ -13,8 +13,8 @@ those probabilities that deviate from the average crystalline structure are retained. The final stage of the *NXRefine* workflow is to generate such 3D-ΔPDF maps. -The uncompressed raw data from such measurements comprise tens (and -sometimes hundreds) of gigabytes, which are often collected in under 30 +The uncompressed raw data from such measurements comprise tens, and +sometimes hundreds, of gigabytes, which can be collected in under 30 minutes. The speed of data collection allows such measurements to be repeated multiple times as a function of a parametric variable, such as temperature. Such data needs to be transformed into reciprocal space as @@ -63,7 +63,7 @@ of the merged data, for reasons that are explained in a later section. defined by H. You [see Fig. 1 in J. Appl. Cryst. **32**, 614 (1999)], with θ and ω corresponding to η and μ, respectively. At present, *NXRefine* assumes that the two angles coupled to - the detector (δ and ν in You's paper), are fixed to 0°, with + the detector (δ and ν in You's paper), are fixed at 0°, with detector misalignments handled by the yaw and pitch angles refined in powder calibrations. diff --git a/docs/source/reduction.rst b/docs/source/reduction.rst new file mode 100644 index 0000000..1e0f9b2 --- /dev/null +++ b/docs/source/reduction.rst @@ -0,0 +1,289 @@ +Data Reduction +************** +In order to reduce raw data collected as images (or frames) on an area +detector as a function of sample rotation angle and transform the +results into reciprocal space maps, *i.e.*, S(**Q**), *NXRefine* +performs the following steps: + +* combining the frames into a single three-dimensional array. +* harvesting metadata collected during the sample rotations. +* summing detector frames to facilitate absorption corrections. +* searching for Bragg peaks embedded within the raw data. +* defining an orientation matrix. +* transforming the raw data into reciprocal space coordinates. + +When multiple sample rotations are performed to collect a single data +set, these steps have to be applied to each rotation scan and the +results merged to produce a single three-dimensional array representing +S(**Q**). Optionally, *NXRefine* also transforms the data after applying +masks that eliminate spurious signals caused by the scattering of Bragg +peaks within the detector sensor layer. + +Once the data has been transformed into S(**Q**), it is possible to +generate 3D-ΔPDF maps, which transform the data back into real space, +producing difference Patterson maps, *i.e.*, maps of interatomic vector +probabilities, which differ from the average crystalline structure. In +this way, continuous distributions of diffuse scattering intensity are +typically reduced to discrete peaks, with positive and negative +intensities, representing these probability differences. *NXRefine* +implements the "punch-and-fill" method, described by `Weber and Simonov +`_. + +Nearly all of the steps in the *NXRefine* data reduction workflow can +either be performed from the command line or launched from a NeXpy GUI. +The exception is determining the crystal orientation, which must first +be performed using the `Refine Lattice` dialog in NeXpy. Once the sample +orientation has been determined from one of the measurements, *e.g.*, at +room temperature, it can be copied and refined automatically when +reducing the data from other measurements, provided the space group has +not changed or is still compatible with the observed Bragg peaks. + +nxload +------ +Load data + +.. code-block:: + + usage: nxload [-h] -d DIRECTORY [-e ENTRIES [ENTRIES ...]] [-o] [-q] + + Load raw data + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -e ENTRIES [ENTRIES ...], --entries ENTRIES [ENTRIES ...] + names of entries to be loaded + -o, --overwrite overwrite existing peaks + -q, --queue add to server task queue + +nxlink +------ +Link metadata + +.. code-block:: + + usage: nxlink [-h] -d DIRECTORY [-e ENTRIES [ENTRIES ...]] [-o] [-q] + + Link data and metadata to NeXus file + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -e ENTRIES [ENTRIES ...], --entries ENTRIES [ENTRIES ...] + names of entries to be searched + -o, --overwrite overwrite existing peaks + -q, --queue add to server task queue + +nxcopy +------ +Copy data + +.. code-block:: + + usage: nxcopy [-h] -d DIRECTORY [-e ENTRIES [ENTRIES ...]] [-p PARENT] [-o] [-q] + + Copy instrument parameters from a parent file + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -e ENTRIES [ENTRIES ...], --entries ENTRIES [ENTRIES ...] + names of entries to be searched + -p PARENT, --parent PARENT + file name of file to copy from + -o, --overwrite overwrite existing peaks + -q, --queue add to server task queue + +nxmax +----- +.. code-block:: + + usage: nxmax [-h] -d DIRECTORY [-e ENTRIES [ENTRIES ...]] [-f FIRST] [-l LAST] [-o] [-m] [-q] + + Find maximum counts of the signal in the specified path + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -e ENTRIES [ENTRIES ...], --entries ENTRIES [ENTRIES ...] + names of entries to be processed + -f FIRST, --first FIRST + first frame + -l LAST, --last LAST last frame + -o, --overwrite overwrite existing maximum + -m, --monitor monitor progress in the command line + -q, --queue add to server task queue + +nxfind +------ + +.. code-block:: + + usage: nxfind [-h] -d DIRECTORY [-e ENTRIES [ENTRIES ...]] [-t THRESHOLD] [-f FIRST] [-l LAST] [-P PIXELS] [-o] [-p PARENT] [-m] [-q] + + Find peaks within the NeXus data + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -e ENTRIES [ENTRIES ...], --entries ENTRIES [ENTRIES ...] + names of entries to be searched + -t THRESHOLD, --threshold THRESHOLD + peak threshold + -f FIRST, --first FIRST + first frame + -l LAST, --last LAST last frame + -P PIXELS, --pixels PIXELS + minimum pixels between peaks + -o, --overwrite overwrite existing peaks + -p PARENT, --parent PARENT + The parent .nxs file to use + -m, --monitor monitor progress in the command line + -q, --queue add to server task queue + + +nxrefine +-------- + +.. code-block:: + + usage: nxrefine [-h] -d DIRECTORY [-e ENTRIES [ENTRIES ...]] [-l] [-p POLAR_MAX] [-T HKL_TOLERANCE] [-o] [-q] + + Refine lattice parameters and goniometer angles + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -e ENTRIES [ENTRIES ...], --entries ENTRIES [ENTRIES ...] + names of entries to be processed + -l, --lattice refine lattice parameters + -p POLAR_MAX, --polar_max POLAR_MAX + maximum polar angle in degrees + -T HKL_TOLERANCE, --hkl_tolerance HKL_TOLERANCE + tolerance for including peak in Å-1 + -o, --overwrite overwrite existing maximum + -q, --queue add to server task queue + +nxprepare +--------- + +.. code-block:: + + usage: nxprepare [-h] -d DIRECTORY [-e ENTRIES [ENTRIES ...]] [--t1 T1] [--h1 H1] [--t2 T2] [--h2 H2] [-o] [-m] [-q] + + Prepare 3D mask around Bragg peaks + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -e ENTRIES [ENTRIES ...], --entries ENTRIES [ENTRIES ...] + names of entries to be processed + --t1 T1 threshold for smaller convolution + --h1 H1 size of smaller convolution + --t2 T2 threshold for larger convolution + --h2 H2 size of larger convolution + -o, --overwrite overwrite existing mask + -m, --monitor monitor progress in the command line + -q, --queue add to server task queue + +nxtransform +----------- + +.. code-block:: + + usage: nxtransform [-h] -d DIRECTORY [-e ENTRIES [ENTRIES ...]] [-qh QH QH QH] [-qk QK QK QK] [-ql QL QL QL] [-R] [-M] [-o] [-q] + + Perform CCTW transform + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -e ENTRIES [ENTRIES ...], --entries ENTRIES [ENTRIES ...] + names of entries to be processed + -qh QH QH QH Qh - min, step, max + -qk QK QK QK Qk - min, step, max + -ql QL QL QL Ql - min, step, max + -R, --regular perform regular transform + -M, --mask perform transform with 3D mask + -o, --overwrite overwrite existing transforms + -q, --queue add to server task queue + + +nxcombine +--------- + +.. code-block:: + + usage: nxcombine [-h] [-d DIRECTORY] [-e ENTRIES [ENTRIES ...]] [-R] [-M] [-o] [-q] + + Combine CCTW transforms + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -e ENTRIES [ENTRIES ...], --entries ENTRIES [ENTRIES ...] + names of entries to be combined. + -R, --regular combine transforms + -M, --mask combine transforms with 3D mask + -o, --overwrite overwrite existing transform + -q, --queue add to server task queue + + +nxpdf +----- + +.. code-block:: + + usage: nxpdf [-h] -d DIRECTORY [-l [LAUE]] [-r RADIUS] [-Q QMAX] [-R] [-M] [-o] [-q] + + Calculate PDF transforms + + optional arguments: + -h, --help show this help message and exit + -d DIRECTORY, --directory DIRECTORY + scan directory + -l [LAUE], --laue [LAUE] + Laue group to be used if different from file + -r RADIUS, --radius RADIUS + radius of punched holes in Å-1 + -Q QMAX, --Qmax QMAX Maximum Q in Å-1 used in PDF tapers + -R, --regular Calculate using regular transforms + -M, --mask Calculate using masked transforms + -o, --overwrite overwrite existing transforms + -q, --queue add to server task queue + + +nxserver +-------- + +.. code-block:: + + usage: nxserver [-h] [-d [DIRECTORY]] [-t TYPE] [-n NODES [NODES ...]] [-c CORES] [-r REMOVE [REMOVE ...]] [command] + + Launch server for data reduction workflow + + positional arguments: + command status|start|stop|list|clear|kill + + optional arguments: + -h, --help show this help message and exit + -d [DIRECTORY], --directory [DIRECTORY] + Start the server in this directory + -t TYPE, --type TYPE Server type: multicore|multinode + -n NODES [NODES ...], --nodes NODES [NODES ...] + Add nodes + -c CORES, --cores CORES + Number of cores + -r REMOVE [REMOVE ...], --remove REMOVE [REMOVE ...] + Remove nodes + + diff --git a/docs/source/refine-menu.rst b/docs/source/refine-menu.rst new file mode 100644 index 0000000..ce8ac58 --- /dev/null +++ b/docs/source/refine-menu.rst @@ -0,0 +1,91 @@ +Refine Menu +=========== +The *NXRefine* plugin to *NeXpy* installs a top-level menu labelled +"Refine", which initializes parameters required for the data reduction workflow, determines . Some o. + +Choose Parameters +----------------- +This dialog initializes a new experiment directory layout using the +server settings to initialize default locations. When the dialog is +launched, click on "Choose Experiment Directory" to launch the system +file browser in order to select or create the new experiment directory. + +Copy Parameters +--------------- +This dialog creates NeXus files that are used as templates for the +experimental files that are used to store all the data and metadata +associated with a particular set of rotation scans. The initial metadata +is defined by parameters in the settings file in the ``tasks`` +sub-directory, which can be modified by the "Edit Settings" sub-menu +described below. However, some of the metadata will be refined using a +powder calibration, whose results are then stored in this file. + +Find Maximum +------------ +This dialog will import a TIFF or CBF file containing measurements of a +powder calibrant and refine the detector position and coordinates, using +the *PyFAI* API. Alternatively, if the calibration parameters are +already available in a PONI file, they can be directly imported. The +resulting powder data and calbration parameters are then saved to the +configuration template previously created using the *New Configuration* +dialog. + +.. figure:: /images/calibrate-powder.png + :align: center + :width: 80% + +Find Peaks +---------- +This dialog creates a pixel mask that is used to exclude bad pixels from +further analysis. As described above, when a new configuration file is +created, a pixel mask that excludes gaps between detector chips is +automatically added. Additional pixels can be excluded using this +dialog, either by adding editable shapes that are constructively added +to the existing mask or by importing the mask from an external file, +which can store the mask in any image format. The latter is useful if a +beamline regularly updates a particular detector's mask as bad pixels are identified. + +Prepare 3D Mask +--------------- +This dialog has the single purpose of creating a directory tree for a +new sample. The dialog enables the creation of a sample directory within +the requested experiment directory and a sub-directory with a unique +label for each instance of that sample measured during an experiment. + +.. figure:: /images/new-sample.png + :align: center + :width: 60% + +Calculate Angles +---------------- +This dialog is used to create a NeXus file in preparation for an +experimental measurement. The file will be based on the selected +configuration file and be saved in the specified sample/label directory. +The name of the file will be "_.nxs", where is the +Scan Label specified in the dialog ('300K' in the image below). + +Define Lattice +-------------- +This dialog is for instruments in which the scans are already defined +using different methods to those above. For example, on the QM2 +instrument at CHESS, the scans are defined in SPEC files, with the data +stored separately in a separate read-only directory. With this dialog, +the directories containing the raw images are associated with the +corresponding SPEC scan, allowing NeXus files to be automatically +generated. This customization is encoded in a QM2 sub-class of the +``NXBeamLine`` class, which is installed separately as a NXRefine +plugin. The process for customizing other beamlines is described later. + +Refine Lattice +-------------- +This dialog allows data in NeXus files collected under identical +conditions to be summed to produce a single NeXus file that can be +processed using the usual workflow. + +Transform Data +-------------- +This dialog allows the settings, whose default values are defined in the +server directory (see :ref:`default_settings`), to be customized for the +data reduction performed in the selected experiment. The settings are +stored in ``/tasks/settings.ini``. The meanings of each +setting are described in the next section. diff --git a/docs/source/refine.rst b/docs/source/refine.rst deleted file mode 100644 index a4c0272..0000000 --- a/docs/source/refine.rst +++ /dev/null @@ -1,41 +0,0 @@ -Data Reduction -************** -In order to reduce raw data collected as images (or frames) on an area -detector as a function of sample rotation angle and transform the -results into reciprocal space maps, *i.e.*, S(**Q**), *NXRefine* -performs the following steps: - -* combining the frames into a single three-dimensional array. -* harvesting metadata collected during the sample rotations. -* summing detector frames to facilitate absorption corrections. -* searching for Bragg peaks embedded within the raw data. -* defining an orientation matrix. -* transforming the raw data into reciprocal space coordinates. - -When multiple sample rotations are performed to collect a single data -set, these steps have to be applied to each rotation scan and the -results merged to produce a single three-dimensional array representing -S(**Q**). Optionally, *NXRefine* also transforms the data after applying -masks that eliminate spurious signals caused by the scattering of Bragg -peaks within the detector sensor layer. - -Once the data has been transformed into S(**Q**), it is possible to -generate 3D-ΔPDF maps, which transform the data back into real space, -producing difference Patterson maps, *i.e.*, maps of interatomic vector -probabilities, which differ from the average crystalline structure. In -this way, continuous distributions of diffuse scattering intensity are -typically reduced to discrete peaks, with positive and negative -intensities, representing these probability differences. *NXRefine* -implements the "punch-and-fill" method, described by `Weber and Simonov -`_. - -Nearly all of the steps in the *NXRefine* data reduction workflow can -either be performed from the command line or launched from a NeXpy GUI. -The exception is determining the crystal orientation, which must first -be performed using the `Refine Lattice` dialog in NeXpy. Once the sample -orientation has been determined from one of the measurements, *e.g.*, at -room temperature, it can be copied and refined automatically when -reducing the data from other measurements, provided the space group has -not changed or is still compatible with the observed Bragg peaks. - - diff --git a/docs/source/sample.rst b/docs/source/sample.rst new file mode 100644 index 0000000..35cbfb8 --- /dev/null +++ b/docs/source/sample.rst @@ -0,0 +1,177 @@ +Sample Refinement +***************** +When a new sample has been mounted and the first scan collected, +*NXRefine* provides a set of tools to prepare the data for +transformation into S(**Q**). These are normally run using *NeXpy* +dialogs accessible from the :ref:`Refine Menu` described below, which +are used to select parameters for use in the data reduction, perform an +analysis of all the collected frames in order to enable, for example, +absorption corrections for each frame and other diagnostic information, +launch a peak search function to identify all the Bragg peaks embedded +in the data, define the sample space group, determine and optimize the +sample orientation based on the Bragg peak assignments, and generate the +Q-mesh used when transforming the data to reciprocal space. Typically, +these steps are performed after the first sample rotation scan, often at +room temperature or while the sample is cooling. The results of this +process are stored in the associated NeXus scan file, which then is +designated the "parent" file, from which all the other scans copy their +initial orientation before the automated refinement. This allows the +data from the remaining scans to be reduced by the automated workflow. + +The only requirement is that the parent shares the same space group as +the scans that are reduced by the automated workflow. The unit cell +parameters and orientation matrix are refined by a least-squares +optimization of the Bragg peak locations identified in each new scan. +If there is a significant change in the space group at a structural +phase transition, it may be necessary to define a different scan file as +the parent for scans performed above or below the transition. + +In this section, we will describe the structure of the NeXus files as well as details of how the *NeXpy* GUI dialogs in the :ref:`Refine +Menu` can be used to prepare the files for subsequent analyis. + +.. figure:: /images/scan-file.png + :align: right + :width: 90% + :figwidth: 40% + +NeXus files +=========== +The scan files are stored using the hierarchical `NeXus format +`__, in which the data for each scan are stored in groups, or entries, conforming to the `NXentry` base class. There is one entry for each sample rotation scan, usually labelled `f1`, `f2`, `f3`, *etc.*, although the number of such scans can vary. There is also a top-level entry (called 'entry'), which contains the metadata that is common to all the rotation scans, as well as the results of merging the reduced data from each one. + +In the example on the right, most of the items are also groups corresponding to different base classes, that contain either raw data, reduced data, metadata, or information resulting from each component of the workflow. When the NeXus file is loaded into NeXpy, its contents can be inspected in a tree view, such as the one shown here. Here are a few examples. + +:instrument: This is a group that contains instrumental parameters, such + as the incident wavelength, detector distance, goniometer + angles, and attenuators. It also stores the powder + calibration data and parameters. + +:sample: This group contains the sample information, including the + chemical formula, unit cell parameters, space and Laue groups, + and sample environment parameters, such as temperature. + *NXRefine* assumes that the sample parameters are independent + of the particular rotation scan, so all the sample groups are + linked to the one stored in the 'entry' group. + +There are a number of groups that contain the results of some of the analysis. + +:peaks: This group contains the results of all the Bragg peaks + identified by the peak search, such as their pixel coordinates + on the detector, their polar and azimuthal angles, and + intensities. These are used to determine the sample orientation + matrix. The group + + + + +Refine Menu +=========== +The *NXRefine* plugin to *NeXpy* installs a top-level menu labelled +"Refine", which initializes parameters required for the data reduction workflow, determines + +Choose Parameters +----------------- +This dialog allows the parameters used in the data reduction workflow to +be specified for a particular scan file. + +.. figure:: /images/choose-parameters.png + :align: center + :width: 80% + +The following parameters are defined. + +:Peak Threshold: This defines the minimum intensity used to identify a + scattering feature as a potential Bragg peak. In the + :ref:`Find Peaks` algorithm, a first-moment analysis is + performed on peaks that exceed this threshold, with + Bragg peaks on successive frames merged to form one + peak. + +:First Frame: This is the first frame in the rotation scan to be + included in subsequent analyses. The default is 10. + +:Last Frame: This is the last frame in the rotation scan to be included + in subsequent analyses. The default is 3640, based on the assumption that a complete rotation contains 3650 frames. + +:Max. Polar Angle: This is the maximum scattering angle that is to be + included in refinments of the orientation matrix. + +:analysis_path: This is the path within the experiment directory to the + *NXRefine* sub-directories. In the above example, this + would be ``nxrefine``. + + + +Copy Parameters +--------------- +This dialog allows parameters to be copied from a parent scan file. + +Find Maximum +------------ +This dialog will import a TIFF or CBF file containing measurements of a +powder calibrant and refine the detector position and coordinates, using +the *PyFAI* API. Alternatively, if the calibration parameters are +already available in a PONI file, they can be directly imported. The +resulting powder data and calbration parameters are then saved to the +configuration template previously created using the *New Configuration* +dialog. + +.. figure:: /images/calibrate-powder.png + :align: center + :width: 80% + +Find Peaks +---------- +This dialog creates a pixel mask that is used to exclude bad pixels from +further analysis. As described above, when a new configuration file is +created, a pixel mask that excludes gaps between detector chips is +automatically added. Additional pixels can be excluded using this +dialog, either by adding editable shapes that are constructively added +to the existing mask or by importing the mask from an external file, +which can store the mask in any image format. The latter is useful if a +beamline regularly updates a particular detector's mask as bad pixels are identified. + +Prepare 3D Mask +--------------- +This dialog has the single purpose of creating a directory tree for a +new sample. The dialog enables the creation of a sample directory within +the requested experiment directory and a sub-directory with a unique +label for each instance of that sample measured during an experiment. + +.. figure:: /images/new-sample.png + :align: center + :width: 60% + +Calculate Angles +---------------- +This dialog is used to create a NeXus file in preparation for an +experimental measurement. The file will be based on the selected +configuration file and be saved in the specified sample/label directory. +The name of the file will be "_.nxs", where is the +Scan Label specified in the dialog ('300K' in the image below). + +Define Lattice +-------------- +This dialog is for instruments in which the scans are already defined +using different methods to those above. For example, on the QM2 +instrument at CHESS, the scans are defined in SPEC files, with the data +stored separately in a separate read-only directory. With this dialog, +the directories containing the raw images are associated with the +corresponding SPEC scan, allowing NeXus files to be automatically +generated. This customization is encoded in a QM2 sub-class of the +``NXBeamLine`` class, which is installed separately as a NXRefine +plugin. The process for customizing other beamlines is described later. + +Refine Lattice +-------------- +This dialog allows data in NeXus files collected under identical +conditions to be summed to produce a single NeXus file that can be +processed using the usual workflow. + +Transform Data +-------------- +This dialog allows the settings, whose default values are defined in the +server directory (see :ref:`default_settings`), to be customized for the +data reduction performed in the selected experiment. The settings are +stored in ``/tasks/settings.ini``. The meanings of each +setting are described in the next section. diff --git a/docs/source/server-menu.rst b/docs/source/server-menu.rst new file mode 100644 index 0000000..a529b45 --- /dev/null +++ b/docs/source/server-menu.rst @@ -0,0 +1,32 @@ +Server Menu +=========== +The *NXRefine* plugin to *NeXpy* installs a top-level menu labelled +"Server", which is used to launch and monitor data reduction operations performed as part of the workflow. + +Manage Workflows +---------------- +This dialog initializes a new experiment directory layout using the +server settings to initialize default locations. When the dialog is +launched, click on "Choose Experiment Directory" to launch the system +file browser in order to select or create the new experiment directory. + + +Copy Parameters +--------------- +This dialog creates NeXus files that are used as templates for the +experimental files that are used to store all the data and metadata +associated with a particular set of rotation scans. The initial metadata +is defined by parameters in the settings file in the ``tasks`` +sub-directory, which can be modified by the "Edit Settings" sub-menu +described below. However, some of the metadata will be refined using a +powder calibration, whose results are then stored in this file. + +Manage Server +------------- +This dialog will import a TIFF or CBF file containing measurements of a +powder calibrant and refine the detector position and coordinates, using +the *PyFAI* API. Alternatively, if the calibration parameters are +already available in a PONI file, they can be directly imported. The +resulting powder data and calbration parameters are then saved to the +configuration template previously created using the *New Configuration* +dialog. diff --git a/docs/source/server.rst b/docs/source/server.rst index bbea5c5..4576c28 100644 --- a/docs/source/server.rst +++ b/docs/source/server.rst @@ -1,5 +1,5 @@ -Server Configuration -==================== +Server Management +***************** *NXRefine* implements a data reduction workflow, which can be run as a series of line commands in the terminal. However, since some of the processes can take a long time to complete (from a few minutes to an @@ -7,66 +7,10 @@ hour, depending on the process and system being used), it is possible to queue these operations using the *NXRefine*'s queue manager, to be run locally using multiple cores or distributed to other nodes. The *NXRefine* queue manager can be configured to submit jobs to another job -queue manager if one is available. - -Initial Setup -------------- -In order to allow *NXRefine* to be used by multiple users on a single -machine or cluster, a common directory is defined to store log files, -task queues, and default settings. The location of this directory should -be defined immediately after installing *NXRefine* for the first time. -Since the files in this directory are modified by *NXRefine* commands -that could be run by multiple users, it is recommended that all such -users are members of the same group. When initialized by a member of -that group, the files in the server directory have group read/write -permissions by default. - -The location of the server directory is initialized on the command line -by the 'nxserver' command:: - - $ nxserver -d /path/to/parent - -This will create a directory at ``/path/to/parent/nxserver`` containing -the files that are required by *NXRefine* server. - -.. note:: If the supplied path already ends in ``nxserver``, it will not - be appended. - -Once the server directory has been initialized, it is necessary for its -location to be defined for other users. This can be done in one of two -ways. - -1. If *NXRefine* is being configured by a system administrator, it is - possible to use a system-wide environment variable, ``NX_SERVER``, to - to define the path to the server directory. Alternatively, this - environment variable could be added to each user's login script. - -2. The ``nxserver`` command used to initialize the server directory also - adds a hidden file to the user's home directory, - ``~/.nxserver/settings.ini``, which contains the server directory - path. If the server directory already exists, the command can be run - again by other users without affecting the initial directory. In - principle, it only needs to be run once by each user, although it - could also be added to a login script if preferred. - - .. note:: If the ``NX_SERVER`` environment variable is defined, it - takes precedence over the path in - ``~/.nxserver/settings.ini``. - -*NXRefine* uses file-based locking to prevent corruption of data files. -This system is provided by the -`nexusformat package `_, which defines -the directory to contain the lock files using the ``NX_LOCKDIRECTORY`` -environment variable. It is recommended that this directory be placed -within the server directory. - -.. note:: The *NeXpy* GUI has a settings file that can be used to define - the lock directory, but it is overridden by the environment - variable if it is defined. This allows system administrators - to set up a unique lock file directory for all their users. +queue manager if one is available. Server Directory -^^^^^^^^^^^^^^^^ +================ Here is the structure of the ``nxserver`` directory:: nxserver @@ -154,7 +98,7 @@ Here is the structure of the ``nxserver`` directory:: .. _default_settings: Default Settings -^^^^^^^^^^^^^^^^ +================ The file, ``settings.ini`` in the server directory contains the default settings for the server, the beamline, and the workflow. These values can be changed, either by opening the "Edit Settings" dialog in the @@ -169,7 +113,7 @@ the data and default values of the data reduction parameters. They will be described later. Server Settings -^^^^^^^^^^^^^^^ +=============== The server settings are used by the workflow server, which is described in a later section. They define the server configuration, such as the number of simultaneous jobs that may be run, the command required to @@ -208,3 +152,22 @@ be wrapped in a shell script. :cctw: This is the path to the CCTW executable used to transform data from instrumental coordinates to reciprocal space. + +Server Menu +=========== +The *NXRefine* plugin to *NeXpy* installs a top-level menu labelled +"Server", which is used to launch and monitor data reduction operations performed as part of the workflow. + +Manage Workflows +---------------- +This dialog shows the workflow status for all the scans stored in a +particular sample directory. When the dialog is launched, click on +"Choose Sample Directory" to launch the system file browser in order to +select a directory containing a set of NeXus scan files. This opens the +"Manage Workflows" dialog, in which the status of every component of the +*NXRefine* workflow is listed for all the scan files in the sample +directory. + +Manage Server +------------- +This dialog allows the status of the server to be monitored.