We present a new approach that augments existing codecs with a small, content-adaptive super-resolution model that significantly boosts video quality. Our method, SRVC, encodes video into two bitstreams:
- a
content stream, produced by compressing downsampled low-resolution video with the existing codec, - a
model stream, which encodes periodic updates to a lightweight super-resolution neural network customized for short segments of the video.
SRVC decodes the video by passing the decompressed low-resolution video frames through the (time-varying) super-resolution model to reconstruct high-resolution video frames.
For installing the required packages using Conda, use:
git clone https://github.com/AdaptiveVC/SRVC.git
cd SRVC
conda env create -f environment.yml
For running SRVC, check python srvc.py --help.
We use 28 short films with direct download buttons on Vimeo. These videos are in high-resolution, have realistic scene changes from movie-makers, and have 10min+ duration.
Vimeo video_ids are located here, and you can view/download them at vimeo.com/video_id, e.g., vimeo.com/441417334.
Once downloaded, you can use the scripts and instructions located here to pre-process the data into the low-resolution and high-resolution formats needed by SRVC.
We use the following four long video sequences from the Xiph video dataset:
You may cite this work using:
@InProceedings{Khani_2021_ICCV,
author = {Khani, Mehrdad and Sivaraman, Vibhaalakshmi and Alizadeh, Mohammad},
title = {Efficient Video Compression via Content-Adaptive Super-Resolution},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {4521-4530}
}