Skip to content

Create database out of public IMDB dataset

Notifications You must be signed in to change notification settings

Totto16/IMDB2sql

Repository files navigation

IMDB2SQL

The project IMDB2SQL intended to create local postgres database with data from open source IMDB datasets (https://datasets.imdbws.com). The second part of the project exposes database via GraphQL

Prerequisites:
  • Python 3.6+
  • docker-compose
Usage:
  • run.py
  -h, --help            show this help message and exit
  --root ROOT, -r ROOT  Directory where data sets will be downloaded
  --download, -d
  --extract, -x
  --parse, -p
  --load, -l
  --dburi DBURI, -db DBURI
                        Database URI
  --resume {name,principal,rating}
                        Start parsing not from first table
  --debug, -dd
  --quiet, -q

Download, extract, parse and copy dataset to postgres database

python3 run.py -r ~/ -d -x -p -l 

  • app.py - Flask application which exposes GraphQL endpoint
http://127.0.0.1:5000/graphql
Testing:
./run_tests.sh

About

Create database out of public IMDB dataset

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 52.1%
  • JavaScript 41.5%
  • HTML 4.6%
  • Other 1.8%