Installation and usage issues

[ppissias]

Hi!
I am trying to install and test A-Track and I have encountered some issues:

I used an Ubuntu 20 image with python 3.8.5 and run the installation script and the installation finished successfully.
I am running Ubuntu in VirtualBox and have allocated 4 CPUs to it and 2 Gigs of memory.

Then on trying to run it I encountered the following issues:

osboxes@osboxes:~/soft/atrack$ python3 atrack.py -a /home/osboxes/soft/images/test/
Python cannot import sources.py. Make sure sources.py is in the same folder as atrack.py.

In order to understand what was the issue, I created this simple file to see what is the problem

osboxes@osboxes:~/soft/atrack$ cat test.py 
import sources

and then I run it and I got thie exception

osboxes@osboxes:~/soft/atrack$ python3 test.py 
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import sources
  File "/home/osboxes/soft/atrack/sources.py", line 24, in <module>
    from astrolib import astronomy
ModuleNotFoundError: No module named 'astrolib'

after copying astrolib from https://github.com/yucelkilic/astrolib

osboxes@osboxes:~/soft/atrack$ python3 test.py 
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import sources
  File "/home/osboxes/soft/atrack/sources.py", line 24, in <module>
    from astrolib import astronomy
  File "/home/osboxes/soft/atrack/astrolib/astronomy.py", line 12, in <module>
    from ccdproc import ImageFileCollection
ModuleNotFoundError: No module named 'ccdproc'

after sudo pip3 instlal ccdproc

osboxes@osboxes:~/soft/atrack$ python3 test.py 
WARNING: AstropyDeprecationWarning: block_reduce was moved to the astropy.nddata.blocks module.  Please update your import statement. [astropy.nddata.utils]
WARNING: AstropyDeprecationWarning: block_replicate was moved to the astropy.nddata.blocks module.  Please update your import statement. [astropy.nddata.utils]
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import sources
  File "/home/osboxes/soft/atrack/sources.py", line 24, in <module>
    from astrolib import astronomy
  File "/home/osboxes/soft/atrack/astrolib/astronomy.py", line 24, in <module>
    import sep
ModuleNotFoundError: No module named 'sep'

after sudo pip3 install sep

sboxes@osboxes:~/soft/atrack$ python3 test.py 
WARNING: AstropyDeprecationWarning: block_reduce was moved to the astropy.nddata.blocks module.  Please update your import statement. [astropy.nddata.utils]
WARNING: AstropyDeprecationWarning: block_replicate was moved to the astropy.nddata.blocks module.  Please update your import statement. [astropy.nddata.utils]
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import sources
  File "/home/osboxes/soft/atrack/sources.py", line 24, in <module>
    from astrolib import astronomy
  File "/home/osboxes/soft/atrack/astrolib/astronomy.py", line 25, in <module>
    import sewpy
ModuleNotFoundError: No module named 'sewpy'

after a manual installation of
https://github.com/megalut/sewpy

osboxes@osboxes:~/soft/sewpy$ sudo python3 setup.py install
running install
running build
running build_py
running install_lib
creating /usr/local/lib/python3.8/dist-packages/sewpy
copying build/lib/sewpy/sewpy.py -> /usr/local/lib/python3.8/dist-packages/sewpy
copying build/lib/sewpy/__init__.py -> /usr/local/lib/python3.8/dist-packages/sewpy
byte-compiling /usr/local/lib/python3.8/dist-packages/sewpy/sewpy.py to sewpy.cpython-38.pyc
byte-compiling /usr/local/lib/python3.8/dist-packages/sewpy/__init__.py to __init__.cpython-38.pyc
running install_egg_info
Writing /usr/local/lib/python3.8/dist-packages/sewpy-1.0dev.egg-info

I received no more errors.

But then when I try to run A-track I get the following exception :

osboxes@osboxes:~/soft/atrack$ ls /home/osboxes/soft/images2
atrack                                      L_2020-06-06_22-32-12_Bin1x1_120s__10C.fit
L_2020-06-06_22-26-43_Bin1x1_120s__10C.fit  L_2020-06-06_22-34-35_Bin1x1_120s__10C.fit
L_2020-06-06_22-29-33_Bin1x1_120s__10C.fit  L_2020-06-06_22-37-35_Bin1x1_120s__10C.fit
osboxes@osboxes:~/soft/atrack$ python3 atrack.py -a /home/osboxes/soft/images2
WARNING: AstropyDeprecationWarning: block_reduce was moved to the astropy.nddata.blocks module.  Please update your import statement. [astropy.nddata.utils]
WARNING: AstropyDeprecationWarning: block_replicate was moved to the astropy.nddata.blocks module.  Please update your import statement. [astropy.nddata.utils]

Creating catalog files... Complete!
Catalog files are saved as *affineremap.pysexcat.
Elapsed time: 0 min 0 sec.

Building master catalog file... Complete!
Master catalog file is saved as master.pysexcat.
Elapsed time: 0 min 0 sec.

Detecting candidates... Complete!
Candidates for each image are saved as *affineremap.cnd.
Elapsed time: 0 min 0 sec.

Detecting moving objects...

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/osboxes/soft/atrack/asteroids.py", line 356, in detect_segments
    partition = partitions(workload)[int(processor)]
IndexError: list index out of range
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/osboxes/soft/atrack/asteroids.py", line 511, in detect_lines
    pool.map(detect_segments, cmds, 1)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "atrack.py", line 170, in <module>
    lines = asteroids.detect_lines(outdir, fitsdir)
  File "/home/osboxes/soft/atrack/asteroids.py", line 514, in detect_lines
    detect_segments((catdir, fitsdir, 0))
  File "/home/osboxes/soft/atrack/asteroids.py", line 356, in detect_segments
    partition = partitions(workload)[int(processor)]
IndexError: list index out of range

Any help would be appreciated.
Thanks!

[yucelkilic]

Dear @ppissias,

First of all, thank you for your great effort. Can you send me 3-4 sample fits files to [email protected] to repeat the problem and to help you better?

Best wishes.

[ppissias]

Hi @yucelkilic
thanks a lot for your reply,
Finally I saw that sextractor was not installed correctly. It seems that using the package manager i.e. sudo apt-get install sextractor does not install it correctly (it seems to install just some documentation and no executable... not sure what the isssue is). I manually installed it and then installed some other missing python packages and finally got it to working correctly!
I have 2 questions:

1. I noticed that it cannot process color images, do you confirm that this is the case? (the initial images I had the error I sent you were from color images, now I got it working with monochrome images).
2. One other question, if the files already have the WCS elements on the FITS header, does the software read them from the FITS header?

Many thanks,
Petros

[yucelkilic]

Dear @ppissias,

Glad you solved the problem. When you using A-Track, I suggest you run it with the "--skip-mpcreport" argument for now. For example;

python3 atrack.py /path/fits_files/ --skip-mpcreport

Otherwise, you may have some problems with the mpcreport part, so you can skip it. On the other hand, A-Track only supports monochrome images, you are right. Actually, the A-Track runs on catalog files produced by the sextractor and can work with all input files that SExtractor accepts. I suggest you set "solve_field = False" and "reject_area = False" in "atrack.config" file.

If you already have WCS in your images, 'ALPHA_J2000', 'DELTA_J2000' information will be extracted by the source extractor and A-Track will be able to read it easily and you should also see this information in the result list of detected moving objects.

I hope I could help. If you have any other problems or suggestions, please write at any time.

Best wishes,
Yucel

[ppissias]

Thanks Yucel,

Regarding WCS data, the images are already "plate solved" using a local astrometry engine or by nova.astrometry.net
and have a header like

SIMPLE  =                    T / Standard FITS file                             
BITPIX  =                    8 / ASCII or bytes array                           
NAXIS   =                    0 / Minimal header                                 
EXTEND  =                    T / There may be FITS ext                          
WCSAXES =                    2 / no comment                                     
CTYPE1  = 'RA---TAN-SIP' / TAN (gnomic) projection + SIP distortions            
CTYPE2  = 'DEC--TAN-SIP' / TAN (gnomic) projection + SIP distortions            
EQUINOX =               2000.0 / Equatorial coordinates definition (yr)         
LONPOLE =                180.0 / no comment                                     
LATPOLE =                  0.0 / no comment                                     
CRVAL1  =        312.584785922 / RA  of reference point                         
CRVAL2  =        44.3796425302 / DEC of reference point                         
CRPIX1  =        739.564693451 / X reference pixel                              
CRPIX2  =        412.081944227 / Y reference pixel                              
CUNIT1  = 'deg     ' / X pixel scale units                                      
CUNIT2  = 'deg     ' / Y pixel scale units                                      
CD1_1   =   -0.000129378038268 / Transformation matrix                          
CD1_2   =    0.000693935921799 / no comment                                     
CD2_1   =    -0.00069486223599 / no comment                                     
CD2_2   =   -0.000129152873537 / no comment                                     
IMAGEW  =                 1522 / Image width,  in pixels.                       
IMAGEH  =                 1125 / Image height, in pixels.                       
A_ORDER =                    2 / Polynomial order, axis 1                       
A_0_0   =                    0 / no comment                                     
A_0_1   =                    0 / no comment                                     
A_0_2   =   -1.26499559714E-06 / no comment                                     
A_1_0   =                    0 / no comment                                     
A_1_1   =   -2.39062436835E-07 / no comment                                     
A_2_0   =   -8.12762702006E-08 / no comment                                     
B_ORDER =                    2 / Polynomial order, axis 2                       
B_0_0   =                    0 / no comment                                     
B_0_1   =                    0 / no comment                                     
B_0_2   =     1.3610757751E-06 / no comment                                     
B_1_0   =                    0 / no comment                                     
B_1_1   =     8.8428680197E-08 / no comment                                     
B_2_0   =     2.5731858321E-07 / no comment                                     
AP_ORDER=                    2 / Inv polynomial order, axis 1                   
AP_0_0  =    0.000105004787597 / no comment                                     
AP_0_1  =    -5.4694811938E-07 / no comment                                     
AP_0_2  =    1.26357249829E-06 / no comment                                     
AP_1_0  =   -3.97035003821E-08 / no comment                                     
AP_1_1  =    2.38950114567E-07 / no comment                                     
AP_2_0  =    8.11809968058E-08 / no comment                                     
BP_ORDER=                    2 / Inv polynomial order, axis 2                   
BP_0_0  =   -0.000114472684206 / no comment                                     
BP_0_1  =    6.04213905525E-07 / no comment                                     
BP_0_2  =   -1.35946156203E-06 / no comment                                     
BP_1_0  =   -3.87567298792E-08 / no comment                                     
BP_1_1  =   -8.84976210267E-08 / no comment                                     
BP_2_0  =   -2.57232933876E-07 / no comment      

Apologies, since I am not an expert on this but would this be enough ?

Thanks!
Petros

[yucelkilic]

Dear @ppissias,

For the astrometric solution of FITS files, I also use local astrometry.net (solve_field). This header will be enough. You can use ds9 to check your WCS header.

Best wishes,
Yucel