Esper: a framework for end-to-end video analytics

Esper is a development environment that provides out-of-the-box integration for a variety of tools useful for analyzing videos, including:

• Scanner: high-performance parallel processing of videos
• Rekall: data representation and operations for spatiotemporal data in Python
• VGrid: visualizing metadata on videos using Python and Javascript
• Frameserver: dynamically extracting frames from a video
• Django: managing SQL models and serving web pages
• Spark: parallel analysis of metadata

Esper is kind of like Ruby on Rails but for video analytics. It provides a skeleton of a data science workbench that you can customize for your particular project, along with a set of useful utility functions, models, preinstalled dependencies, and so on.

⚠️ WARNING ⚠️: Esper is a tool for programmers. It uses a programmatic query interface, which means we use REMOTE CODE EXECUTION to run queries. DO NOT expose this interface publicly, or else risk having a hacker trash your computer, data, and livelihood.

Setup

First, install Docker CE, Python 3.5, jq, and pip. If you're on Ubuntu, you can install Python/pip/jq as follows:

sudo apt-get install python3 python3-pip jq

Ensure that you have Docker version >= 17.12, which you can check by running:

$ docker --version
Docker version 17.12.0-ce, build c97c6d6

Note: If you have a GPU and are running on Linux, then install nvidia-docker2. Set device = "gpu-9.1-cudnn7 in config/local.toml.

Next, you will need to configure your Esper installation.

$ git clone https://github.com/scanner-research/esper
$ cd esper
$ pip3 install -r docker/requirements.host.txt
$ python3 docker/configure.py --config config/local.toml
$ docker-compose up -d
$ docker-compose run app bash --login -c "cd ui && npm install && npm run prepublishOnly"

Esper developers

If you're developing the Esper core platform or otherwise want to stay up to date with our dependencies, then you should clone and link our submodules.

$ git submodule update --init --recursive
$ docker-compose run app bash --login -c "./deps/install.sh"