-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update install notes for Mizuki #65
Merged
Merged
Changes from 7 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
e578fd4
DOC: Mention scripts and conda envs.
danielballan 73aff74
Fix typo
danielballan 2a8cdb7
Update suggested insert API
danielballan b7ec47d
typo
danielballan f604c14
typo
danielballan 457040a
Udpate conda explanation
danielballan b07fbde
Update fresh-installation.rst
stuartcampbell 323397f
Update fresh-installation.rst
stuartcampbell 7ebe859
Added sentence about conda on other machines
stuartcampbell 1ba3b23
BLD: Install doctr from PR branch instead of specific commit on that …
danielballan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -17,6 +17,43 @@ one at least one machine, skip to the next section. | |
class that installs mongo 3.x some dedicated server and start the mongo | ||
daemon. | ||
|
||
Create Conda Environments | ||
+++++++++++++++++++++++++ | ||
|
||
A puppet class should install conda into ``/opt/conda`` on machines designated | ||
``*-srv*`` ("server") or ``*-ws*`` ("workstation") as soon as they are on the | ||
NSLS-II network and have a puppet client configured and running. It should also | ||
create an empty directory, ``/opt/conda_envs`` for root-controlled conda environments. | ||
The configuration in ``/opt/conda/.condarc`` designates this location as the second | ||
place conda should look for environments, after ``~/conda_envs``. | ||
|
||
Environments for data collection and data anlysis should be installed into | ||
``/opt/conda_envs``. In the second operating cycle of 2017, the commands to do | ||
this were: | ||
|
||
.. code-block:: bash | ||
|
||
sudo conda create -p /opt/conda_envs/collection-17Q2.0 -c nsls2-tag -y collection | ||
sudo conda create -p /opt/conda_envs/analysis-17Q2.0 -c nsls2-tag -y analysis | ||
fix_conda_privileges.sh | ||
|
||
where ``collection-17Q2.0`` and ``analysis-17Q2.0`` are the names of conda | ||
metapackages and the names of the environments (under ``/opt/conda_envs``) | ||
where these metapackages are installed. | ||
|
||
Install Scripts | ||
+++++++++++++++ | ||
|
||
Two convenience scripts, ``fix_conda_privileges.sh`` and ``bsui`` should be | ||
installed in ``/usr/local/bin``, also by puppet. | ||
|
||
* ``fix_conda_privileges.sh`` works around a bug in conda wherein users cannot | ||
properly access root-installed conda packages. It should be run after | ||
creating or updating and conda environments in ``/opt/conda_envs``, as shown | ||
above. | ||
* ``bsui`` is a shortcut script that activates a conda environment and starts | ||
IPython with the 'collection' profile. | ||
|
||
Create New IPython Profile | ||
++++++++++++++++++++++++++ | ||
|
||
|
@@ -66,14 +103,14 @@ server where the mongo daemon is running. | |
'port': 27017, | ||
'database': 'filestore-production-v1', | ||
mds = MDS(mds_config) | ||
mds_readonly = MDS(mds_config) | ||
mds_readonly = MDSRO(mds_config) | ||
fs_readonly = FileStoreRO(fs_config) | ||
db = Broker(mds_readonly, fs_readonly) | ||
|
||
# Subscribe metadatastore to documents. | ||
# If this is removed, data is not saved to metadatastore. | ||
from bluesky.global_state import gs | ||
gs.RE.subscribe('all', mds.insert) | ||
gs.RE.subscribe('all', db.insert) | ||
|
||
# Import matplotlib and put it in interactive mode. | ||
import matplotlib.pyplot as plt | ||
|
@@ -101,7 +138,7 @@ server where the mongo daemon is running. | |
# ophyd.logger.setLevel(logging.DEBUG) | ||
# logging.basicConfig(level=logging.DEBUG) | ||
|
||
Create a Beamling GitHub Organization | ||
Create a Beamline GitHub Organization | ||
+++++++++++++++++++++++++++++++++++++ | ||
|
||
1. Create a username on github.com if you don't have one. Create a new | ||
|
@@ -124,8 +161,8 @@ Create a Beamling GitHub Organization | |
|
||
.. code-block:: bash | ||
|
||
git remote add https://github.com/NSLS-II-XXX/profile_collection.git | ||
git push -u origin master | ||
git remote add upstream https://github.com/NSLS-II-XXX/profile_collection.git | ||
git push -u upstream master | ||
|
||
|
||
Configure the Olog | ||
|
@@ -217,7 +254,8 @@ New Workstation for Data Collection or Analysis | |
puppet. | ||
|
||
2. Create configuration files for metadatastore and filestore. As root user, | ||
compose two new files: | ||
compose two new files. The ``hostname`` should be the host where the mongo | ||
service running (conventionally, the ``*-ca1`` machine, as noted above). | ||
|
||
.. code-block:: bash | ||
|
||
|
@@ -247,52 +285,52 @@ Add the following to the user's ``~/.bashrc`` file. | |
export no_proxy=cs.nsls2.local | ||
export PATH=/opt/conda/bin:$PATH | ||
|
||
The first three lines are local NSLS-II controls network configuration. | ||
The first three lines are local NSLS-II controls network configuration. They | ||
should already be set at the system level but in practice they are often not. | ||
|
||
Conda has already been installed on all NSLS-II workstations (ws) and servers | ||
(srv) in a shared location. The last line adds conda to the user's PATH so that | ||
it overrides any system-installed Python, IPython, etc. | ||
|
||
Convenience Script ``bsui`` | ||
+++++++++++++++++++++++++++ | ||
|
||
The script ``bsui`` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we want to add a link to a copy of this ? |
||
|
||
Custom User Environments | ||
++++++++++++++++++++++++ | ||
|
||
Any user can create a conda environment, a set of binaries and Python packages | ||
completely under their control. User conda environments are stored under | ||
``~/conda_envs/<environment-name>``. | ||
|
||
This command creates a new environment called ``collection`` with the latest | ||
"tagged" (i.e., stable) versions of ophyd, bluesky, pyolog, and xray_vision. | ||
This command creates a new environment called ``my-env`` with all the versions | ||
of the collection software used for the second operating cycle of 2017. | ||
|
||
.. code-block:: bash | ||
|
||
conda create -c nsls2-tag -n collection ophyd bluesky pyolog xray_vision | ||
conda create -c nsls2-tag -n my-env collection-17Q2 | ||
|
||
To test the new environment, activate it: | ||
|
||
.. code-block:: bash | ||
|
||
source activate collection | ||
source activate my-env | ||
|
||
Check that ``which ipython`` point to a path with the word ``collection`` it | ||
in (not ``/usr/bin/python``, as a counterexample). To troubleshoot, you | ||
might need to refresh bash with the command ``hash -r``. | ||
Troubleshooting: Check that ``which ipython`` point to a path with the word | ||
``my-env`` it in (not ``/usr/bin/python``, as a counterexample). To | ||
troubleshoot, you might need to refresh bash with the command ``hash -r``. | ||
|
||
To get "development" versions that are maybe less stable but contain the latest | ||
bug fixes and features, use the ``nsls2-dev`` channel in place of | ||
``nsls2-tag``. | ||
|
||
|
||
Create or Updating Shared (Root) Environments | ||
+++++++++++++++++++++++++++++++++++++++++++++ | ||
Creating or Updating Shared (Root) Environments | ||
+++++++++++++++++++++++++++++++++++++++++++++++ | ||
|
||
Administrators with sudo access can create or update conda environments that | ||
users can use ("activate") but only administrators can edit. These environments | ||
are located in ``/opt/conda_envs``. All beamlines that use JupyterHub | ||
(notebook.nsls2.bnl.gov) have an environment at ``/opt/conda_envs/analysis``. | ||
Some beamlines also have a shared collection environment at | ||
``/opt/conda_envs/collection``. Others prefer to install the collection | ||
software individually per user. In either case, it always possible for users to | ||
create custom environments as desired. | ||
are located in ``/opt/conda_envs``. | ||
|
||
.. note:: | ||
|
||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it worth mentioning the few exceptions that conda lives on ca or gpu machines?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a sentence