- 📒 Table of Contents
- 📍 Overview
- ⚙️ Features
- 📂 Project Structure
- 🧩 Modules
- 🚀 Getting Started
- 🗺 Roadmap
This repository is a Flask backend application that provides data analytics and insights for a music sharing platform. It utilizes caching to improve performance and retrieves information such as top tracks, artists, listeners, and global music statistics from a PostgreSQL database using SQLAlchemy. The project aims to help users understand their music consumption habits and provides valuable insights to drive better human-centeric music recommendations.
| Feature | Description |
|---|---|
| ⚙️ Architecture | The system is built using the Flask framework, following a modular design pattern with blueprints for different functionalities. It integrates with a database using SQLAlchemy for efficient data handling. |
| 📖 Documentation | Documentation is available for environment configuration and some code files. In the future, an automated API documentation tool like Swagger or APIFairy will be used to generate documentation for the API endpoints. |
| 🔗 Dependencies | The main dependency is the Flask framework for building the web application. Additional dependencies include Flask-Caching and SQLAlchemy for caching and database handling respectively. |
| 🧩 Modularity | Files are organized into different modules and blueprints based on their functionalities: API endpoints, data retrieval, etc. This separation allows for code reuse, testing, and easier maintenance. |
| ✔️ Testing | Testing is performed via a local Postman application. An automated test suite will be implemented in the future. |
| ⚡️ Performance | Flask-Caching improves performance by reducing the both number of database queries and amount of data retrieved from AWS. This improves both page load speeds and AWS RDS usage costs. |
| 🔐 Security | This API is available only to the songswap-app frontend. For the time being, it will not be made publicly available. |
| 🔌 Integrations | This API is used by the songswap-app frontend application. It is responsible for retrieving data to be visualized on users' "Insights" pages. |
| 📶 Scalability | There is no database sharding or replication implemented at this time. However, the application is designed to be scalable and can be deployed on multiple instances. If deployed on multiple instances, it is necessary to re-design the API to use a central caching server. |
repo
├── Dockerfile
├── app
│ ├── __init__.py
│ ├── cache
│ │ ├── __init__.py
│ │ └── decorators.py
│ ├── models
│ │ ├── __init__.py
│ │ └── history.py
│ ├── routes
│ │ ├── __init__.py
│ │ ├── cache.py
│ │ ├── insights.py
│ │ └── insights_global.py
│ └── services
│ ├── __init__.py
│ ├── insights_global_service.py
│ └── insights_service.py
├── buildspec.yml
├── config.py
├── requirements.txt
├── run.py
└── tests
└── __init__.py
6 directories, 18 files
Root
| File | Summary |
|---|---|
| run.py | Creates and runs the API by calling the create_app() function from the app module. The app runs on the specified port and in debug mode if the DEBUG flag is set. |
| config.py | Loads Flask-specific environment variables from a .env file. It retrieves values for DEBUG, SECRET_KEY, FLASK_RUN_PORT, SQLALCHEMY_DATABASE_URI, and CACHE_TYPE, among others, from the environment. It also sets up cache configurations and disables SQLAlchemy track modifications. |
| Dockerfile | Sets up a Python Flask application in a Docker container. It installs necessary dependencies, sets environment variables, configures the working directory, exposes a port, and runs the application using Gunicorn. The application can be accessed externally on port 5001. |
Cache
| File | Summary |
|---|---|
| decorators.py | A decorator function that caches an endpoint's response and adds logging functionality. It allows for setting a timeout for cache expiration, a key prefix for the cache key, and a condition for bypassing caching if necessary. The function decorates the view function, checks if the response is already cached, and returns the cached value if it exists. If the response is not cached, it calls the view function, stores the result in the cache, and returns the result. |
Models
| File | Summary |
|---|---|
| history.py | Defines various models for a database schema for Spotify's tracks, artists, and their relationships, using SQLAlchemy. It includes functionalities for managing cascading deletion of records within these tables. |
Routes
| File | Summary |
|---|---|
| cache.py | Defines a Flask blueprint that defines two endpoints for managing a simple cache. The first endpoint clears the cache when a secret key is provided, and the second endpoint returns the current size of the cache. This code is used to test and validate the cache's functionality. |
| insights_global.py | A Flask blueprint that defines endpoints for retrieving insights about global listening history data. These endpoints include retrieving the total number of listens, distinct tracks and artists, total listen time, and top tracks, artists, and listeners based on a specified limit. The code uses caching for performance optimization and handles validation of the limit parameter. The response is returned in JSON format with appropriate status codes. |
| insights.py | A Flask Blueprint that defines endpoints related to user-specific insights. It includes functions to retrieve top tracks, top artists, and top primary artists for a given user. The code also includes helper functions for parsing and validating query parameters. Caching is applied to the API routes using decorators. |
Services
| File | Summary |
|---|---|
| insights_global_service.py | Functions to query a database using SQLAlchemy. It retrieves top tracks, top artists, top primary artists, top listeners, total listens, distinct tracks, distinct artists, distinct primary artists, and total listen time. |
| insights_service.py | Functions for retrieving top tracks and artists based on user history, counting the total tracks, and distinct tracks for a given user. It utilizes SQLAlchemy queries to join tables, apply filters, and group data. There are also helper functions for printing SQL queries and handling image URLs. |
Before you begin, ensure that you have the following prerequisites installed:
ℹ️ dockerℹ️ PostgreSQL
- Clone the songswap-insights repository:
git clone [email protected]:SongSwap-social/songswap-insights.git- Change to the project directory:
cd songswap-insights- Install the dependencies:
pip install -r requirements.txt- Run the application locally, for testing or development purposes:
python main.py- Run the application in a Docker container:
docker build -t songswap-insights .
docker run songswap-insights
ℹ️ Create user-specific insights endpointsℹ️ Create global insights endpointsℹ️ Add caching to endpointsℹ️ Add logging to endpoints- []
ℹ️ Add documentation for endpoints (Swagger, APIFairy)- []
ℹ️ Add automated testing