Bash and SSH

.bashrc

Your bash environment is configured via the file ~/.bashrc (or sometimes ~/.bash_profile).

On NAF machines the default is a zsh shell. If you want to change to bash by default, add exec bash to ~/.zprofile

If these files to not exist, you can just create them. It is recommended to make use of shared configurations in artus-analysis/bashrc.

screen

The window manager screen is a very useful tool for multiplexing terminals. Start a screen session with

screen

If a screen screen session already exists, you can reattach it with

screen -r [-d]

Inside a screen session you need the following commands:

Ctrl-a c # create a new window
Ctrl-a n # change to next window
Ctrl-a p # change to previous window
Ctrl-a d # detach session, which continues running in the background

screen and NAF:

For logging in at the NAF you need an afs token which is created while you log in and which is valid only 24h (without other tricks). Shells become invalid/unuseable after this time. Another issue is, that this token is destroyed after logging out. So if you create a screen session on the NAF, detach ich and log out, the screen session becomes invalid. Solution: Start the screen session on another machine (e.g. desktop in Aachen) and log in to the NAF inside this session. This login will still be valid only 24h.

sed

Replace all strings matching search in file file with replace:

sed -i -e "s/search/replace/g" file

The delimiter (here /) can be (almost) freely choosen, but must not be part of search or replace.

Delete lines matching search in file file:

sed -i -e "/search/d" file

SSH Keys

You can generate your own key for SSH authentification using one of the two commands

ssh-keygen -t rsa
ssh-keygen -t dsa

This has to be done only once. Then the public key is then installed on any the remote machine with the corresponding command

ssh-copy-id -i ~/.ssh/id_rsa.pub <user>@<address of remote machine>
ssh-copy-id -i ~/.ssh/id_dsa.pub <user>@<address of remote machine>

in order to simplify the authentification at this machine. A passwort is then only needed to unlock the key.

SSH Config

If not already existing you create a text file in ~/.ssh/config. In this file you can then define settings for connections to remote machines. The Host defines an alias for your SSH connections, such that you can easily connect to a remote machine via ssh naf, for example. Here is a list of settings that is usually needed in our group:

• NAF@DESY

   Host naf
   	Hostname = naf-cms.desy.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  
   Host naf1
   	Hostname = nafhh-cms01.desy.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  
   Host naf2
   	Hostname = nafhh-cms02.desy.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  
   Host naf3
   	Hostname = nafhh-cms03.desy.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  
   Host naf4
   	Hostname = nafhh-cms04.desy.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  
   Host naf5
   	Hostname = nafhh-cms05.desy.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  
   Host naf6
   	Hostname = nafhh-cms06.desy.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  

• EKP@KIT

   Host ekplx*
   	Hostname = %h.physik.uni-karlsruhe.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  
   Host ekpcms*
   	Hostname = %h.physik.uni-karlsruhe.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  
   Host ekpams*
   	Hostname = %h.physik.uni-karlsruhe.de
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  

• RWTH Aachen

    Host *.physik.rwth-aachen.de
            User <user>
            IdentityFile ~/.ssh/keys/physik_rsa
  
    Host physik
            User <user>
            IdentityFile ~/.ssh/keys/physik_rsa
            HostName lxportal1.physik.rwth-aachen.de
  
    Host lx3*
            User <user>
            IdentityFile ~/.ssh/keys/physik_rsa
            ProxyCommand bash -c "if nc -z %h %p &> /dev/null; then exec nc %h %p; else exec ssh tunnel.physik.rwth-aachen.de -W %h:%p 2> /dev/null; fi"
            ForwardX11 yes
            ForwardX11Trusted yes
  
    Host *
            Port 22
            StrictHostKeyChecking no
            Compression no
            Cipher blowfish-cbc
  

Additional steps:

1. • cd ~
   • mkdir .ssh/keys
   • cd .ssh/keys
   • ssh-keygen -t rsa
   • name the file as in the config: ~/.ssh/keys/physik_rsa
2. • ONLY FOR MAC: brew install ssh-copy-id
   • copy public key to ssh target: ssh-copy-id -i ~/.ssh/keys/physik_rsa physik
3. • connect to portal: ssh physik
   • connect to machine: e.g.: ssh lx3b87

This setup is way waster than the one above and does not require to enter your password all the time!

• LXPLUS@CERN

   Host lxplus	
   	Hostname = lxplus.cern.ch
   	Compression = yes
   	User = <user>
   	ForwardX11 = yes
  

Access to Remote Directories in File Browser

• Nautilus

Go to menu File and then Connect to server and type one of the following lines in the dialog that opens.

ssh://<user>@<remote machine>
ssh://<user>@<remote machine>:/<path>
ssh://<host alias from SSH config>

Then create a bookmark and you can easily edit your files with your favourite local and graphical editor.

• Finder

In order to mount a remote folder on your Mac, you need to install OSXFuse and sshfs at first. Both programmes can be downloaded here:

https://osxfuse.github.io/

Create a small script in your ~/bin/ directory on your Mac

vim ~/bin/mount_lx3b

And write there something similar to the example below, changing the pathes accordingly

#mount my shared_folder at portal.physik.rwth-aachen.de
sudo umount -f /Volumes/lx3b
sudo mkdir /Volumes/lx3b
#mount it
sudo sshfs -o volname=lx3b -o IdentityFile=/Users/greyxray/.ssh/id_rsa -o follow_symlinks -o allow_other,defer_permissions [email protected]:/home/home2/institut_3b/hlushchenko/ /Volumes/lx3b/

In order to mount the remote drive you now simply have to type:

sudo ~/bin/mount_remote_drive

Explanations

sshfs will find your ssh-keys only if you specify the full path to your homedirectory where your /.ssh folder is located. The usual "~" is not working here!

After that you need to chmod +x the file to make it executable.

• -o volname: specifies the name of the remote drive in your Finder.
• -o follow_symlinks: Enables you to use symlinks created on the remote machine.
• -o IdentityFile identifies you with your public rsa key.

Find your remote drive in the Finder and add it to the sidebar.

• Please note: When you reboot your Mac, you need to mount the folder again.
• Sometimes it is necessary to unmount the folder to be able to mount it again. For this you can add

sudo umount /Volumes/lx3b

in your script at the beginning. This will unmount and mount your drive each time you call the script.

• Working with a local copy and automized synchronization

For Mac Users it is quite easy to work with a local copy of your favorite repositories and automize them triggering changes to be pushed to naf/lx3b each time you save local changes. Clone the repositories you want to work on to one place on your local pc. Watching your repositories for changes can be done with fswatch (https://github.com/emcrisostomo/fswatch). In case you use homebrew as your package manager, you can install it by simply executing bash brew install fswatch in your terminal. To push the changes to the remote we will use rsync.
The following functions need to be added to your bash profile and adapted to the path where your code is and where to which location you want to push the changes.

function vsync_higgs {
    higgs_repo='HiggsAnalysis/KITHiggsToTauTau'
    rsync -azvP ~/code/aachen-3b/$higgs_repo <username>[email protected]:~/cms_analysis/CMSSW_8_1_0/src/HiggsAnalysis
}

function vsync_artus {
    artus_repo='Artus'
    rsync -azvP ~/code/aachen-3b/$artus_repo <username>@nafhh-cms02.desy.de:~/cms_analysis/CMSSW_8_1_0/src/
}
function autovsync {
    aachen3b_repo='code/aachen-3b/'
    echo "Syncing $aachen3b_repo"
    vsync_higgs && vsync_artus && fswatch -0 -r $aachen3b_repo -e index.lock ".git" | xargs -0 -n 1 -I{} rsync -azvP $aachen3b_repo <username>@nafhh-cms02.desy.de:~/cms_analysis/CMSSW_8_1_0/src/ 
}

The function autovsync will envoke the filewatching and push the changes every time you save changes in the watched repositories.