CDFTracker

This is a python package that helps keep track of both Raw Binary and CDF Files in a Relational Database.

• Github repository: https://github.com/HERMES-SOC/CDFTracker/

Features

• Track both Raw Binary and CDF Files
• Supports multiple instrument configurations
• Support for multiple RDBMS via SQLAlchemy connection strings

Requirements

• Python 3.8 or higher
• SQL Database (via connection string)
• Make
• Poetry (optional)

Installation and Usage

To install the library and its dependencies, follow these steps:

1. Install with dependency manager of your choice:

   # Install with pip
   pip install git+https://github.com/HERMES-SOC/CDFTracker.git
   
   # Install with poetry
   poetry add git+https://github.com/HERMES-SOC/CDFTracker.git

2. Create both the engine and the session, with your RDBMS of choice connection string. For example, to create a session with a SQLite database, you can do the following:

   from cdftracker.database import create_engine, create_session
   
   # Create a database engine and session
   engine = create_engine("sqlite:///test.db")
   session = create_session(engine)

3. If this is your first time using the library, you will need to create the database tables. To do so, run the following command:

   from cdftracker.database.tables import set_up_tables
   
   # Set up tables
   set_up_tables(engine, session)

4. Define a science file name parser function which parses the file Path object and returns the following information in a dictionary. This is the formart the dictionary outputted by the function should have:

   # def science_file_name_parser():
   #    return {
     #      "instrument": str,
     #      "mode": str,
     #      "test": bool,
     #      "time": str,
     #      "level": str,
     #      "version": str,
     #      "descriptor": str,
   #    }
   
   # For this example we will be using a predefined science file parser defined in the 
   # hermes_core package (https://github.com/HERMES-SOC/hermes_core.git).
   from hermes_core.util import util
   
   parser = util.parse_science_filename

5. Now you can instantiate a CDFTracker object with the engine and science file parser function you defined:

   from cdftracker.tracker import tracker
   from pathlib import Path
   
   # Instantiate the tracker
   tracker = tracker.CDFTracker(engine, parser)
   
   # Path of the science file to be tracked (for example)
   file = Path("./hermes_MAG_l0_2022259-030002_v01.bin")

6. You can also track the file which adds the appropriate entries to the database. To do so, run the following command:

   # Track the file
   tracker.track(file)

Database Schema

This is the database schema for the CDFTracker database. The database schema is defined in the cdftracker.database.tables module.

Contributing

How to set-up Development Environment

This project makes use of Poetry to manage dependencies and virtual environments. Also included is a Make file to set-up your development environment. To set-up your development environment, follow these steps:

1. Clone the repository

   git clone https://github.com/HERMES-SOC/CDFTracker.git
   
   cd CDFTracker

2. Set-up your development environment

   make install

How to run tests

This project uses pytest to run tests and exports an HTML report of the code coverage. To run tests, follow these steps:

1. Inside the project directory, run the following command:

   make test

How to run linter

This project uses black and ruff to lint the code. To run the linter, follow these steps:

1. Inside the project directory, run the following command:

   make check

License

This project is licensed under the terms of the MIT license.