PhotoPipe is a pipeline for automated reduction, photometry and astrometry, written in Python (2.7), and designed to process imaging data from the following instruments:
- RATIR: the Reionization and Transients Infrared/Optical Project.
- RIMAS: the Rapid infrared IMAger Spectrometer.
This project is based on the previous versions of the pipeline by cenko and vickitoy.
Built at the NASA Goddard Space Flight Center, in collaboration with the University of Maryland.
See the Wiki for full documentation, examples, operational details and other information.
The pipeline can be installed with very little additional software. However, depending on the installation, different libraries and software may be required. Please see the Installation section below, and the dedicated wiki page for a full list of prerequisites.
The easiest way to get up and running with the pipeline is to download the ready-to-use virtual machine box and run it with Vagrant.
Alternatively, it can be installed on Linux or macOS either with pip, or with the provided installation scripts.
(WARNING: compiling the dependencies can take more than 6 hours.)
Please refer to the virtual machine repository.
-
Run
sudo -H pip install photopipeto install the latest stable version from PyPI. -
Or, clone from
git:
$ git clone [email protected]:maxperry/photometrypipeline.git
$ cd photometrypipeline
$ sudo python setup.py install
NOTE (macOS): If the installation fails with sudo: port: command not found make sure that MacPorts is installed and /opt/local/bin is in the $PATH (e.g. export PATH=/opt/local/bin:/opt/local/sbin:$PATH).
If you to run the pipeline from the Python enviroment rather than using the photopipe command as described in the Usage section, please follow the step by step instructions in the wiki pages below to install all the dependencies manually.
The following steps can be reproduced using the test data downloadable here, either from your host machine or from the virtual machine.
NOTE for PhotoPipe-VM: Create the imdata directory in the VM's shared data dir (should be ./photopipe-vm/data)
imdata
│
└───bias
│ │ 20160628T032914C0b.fits
│ │ 20160628T032914C1b.fits
│ │ ...
│
└───dark
│ │ 20160628T040211C0d.fits
│ │ 20160628T040211C1d.fits
│ │ ...
│
└───flat
│ │ 20160628T024207C0f.fits
│ │ 20160628T024207C1f.fits
│ │ ...
│
└───science
│ │ 20160628T043940C0o.fits
│ │ 20160628T043940C1o.fits
│ │ ...
│
└───science_selected
│
└───reduced
If running on your Host Machine: Open the terminal and cd ./imdata/reduced
If running on PhotoPipe-VM: Launch the VM first (see instructions), and cd /vagrant_data/imdata/reduced
NOTE: It's important to perform the steps below from the reduced folder, otherwise make sure to move the master frames there after running mkmaster (it saves to the current dir!).
- Enter Python environment:
$ python - Run the following script:
from photopipe.reduction import preproc
# Bias frames calibration
bias_calib = preproc.choose_calib( 'ratir',
'bias',
workdir='/vagrant_data/imdata/bias/',
cams=[0,1],
auto=True,
amin=0.0, amax=1.0,
reject_sat=False,
save_select=True,
noplot=False )
# Dark frames calibration
dark_calib = preproc.choose_calib( 'ratir',
'dark',
workdir='/vagrant_data/imdata/dark/',
cams=[0,1],
auto=True,
amin=0.0, amax=1.0,
reject_sat=False,
save_select=True,
noplot=False )
# Flat frames calibration
flat_calib = preproc.choose_calib( 'ratir',
'flat',
workdir='/vagrant_data/imdata/flat/',
cams=[0,1,2,3],
auto=True,
amin=0.2, amax=0.8,
reject_sat=False,
save_select=True,
noplot=False )
# WARNING: RATIR flats are often bad even when
# the median value is in the acceptable range.
# Auto mode is only recommended for bias frame
# selection.
# Select science frames
# (selected frames will be copied to target_dir)
science_dict = preproc.choose_science( 'ratir',
workdir='/vagrant_data/imdata/science,
targetdir='/vagrant_data/imdata/science_selected',
cams=[0,1,2,3],
auto=True,
save_select=True,
calibrate=False,
noplot=False )
# WARNING: When auto is True, all science frames
# are selected. Since the telescope occasionally
# has tracking issues, it is recommended to check
# all frames.
# Make master frames
# (saves to the current dir)
preproc.mkmaster('ratir', bias_calib, 'bias')
preproc.mkmaster('ratir', dark_calib, 'dark')
preproc.mkmaster('ratir', flat_calib, 'flat') See wiki page for preproc functions reference.
WARNING:
- Always use absolute paths terminated by a
/when settingdirparameters. - Make sure to set the correct number of cameras where required (e.g.
cams[0,1,2,3). amin/amaxset the min and max saturation values. Make sure to select the appropriate values for each type of frame, only frames with median values in this range will be selected.
- Start a new python environment
- Execute the script below:
from photopipe.reduction.auto.autoproc import autoproc
autoproc( datadir='/vagrant_data/imdata/science_selected/',
imdir='/vagrant_data/imdata/reduced/',
redo=1 )- Reduced frames will be saved to
imdirusingcoaddas prefix in the filename.
See wiki page for autoproc function reference.
- Move the
coaddfiles to a new dir calledphotometry, and cd into it - Start a new python environment, and execute the script below:
from photopipe.photometry.autoredux import autoredux
autoredux()NOTE: It's important to start python from the photometry dir containing the coadd files.
autoredux will run the following scripts:
- photom.py: Samples and crops all files, creates a multicolor image used to find all sources, then finds the aperture photometry for each of those sources (resampled) using sextractor (with corrected zeropoint).
- Output: *.am (absolute magnitude) files for w/ RA and DEC identified by sextractor
- plotphotom.py: Plots photometry results to a HTML page.
For bugs, questions and discussions please use the Github Issues.
This project is licensed under the GNU GPLv3 License - see the LICENSE file for details.