This repository contains the MATLAB implementation of the BabelCalib calibration framework.
Method overview and result. (left) BabelCalib pipeline: the camera model proposal step ensures a good initialization (right) example result showing residuals of reprojected corners of test images.
Projection of calibration target from estimated calibration. Detected corners are red crosses, target projected using initial calibration are blue squares and using the final calibration are cyan circles.
BabelCalib is a calibration framework that can estimate camera models for all types of central projection cameras. Calibration is robust and fully automatic. BabelCalib provides models for pinhole cameras with additive distortion as well as omni-directional cameras and catadioptric rigs. The supported camera models are listed under the solvers directory. BabelCalib supports calibration targets made of a collection of calibration boards, i.e., multiple planar targets. The method is agnostic to the pattern type on the calibration boards. It is robust to inaccurately localized corners, outlying detections and occluded targets.
You need to clone the repository. The required library Visual Geometry Toolkit is added as a submodule. Please clone the repository with submodules:
git clone --recurse-submodules https://github.com/ylochman/babelcalibIf you already cloned the project without submodules, you can run
git submodule update --init --recursive Calibration is performed by the function calibrate.m. The user provides the 2D<->3D correspondence of the corner detections in the captured images as well as the coordinates of the calibration board fiducials and the absolute poses of the calibration boards. Any calibration board of the target may be partially or fully occluded in a calibration image. The camera model is returned as well as diagnostics about the calibration.
function [model, res, corners, boards] = calibrate(corners, boards, imgsize, varargin)Parameters:
corners: type cornersboards: type boardsimgsize: 1x2 array specifying the height and width of the images; all images in a capture are assumed to have the same dimensions.varargin: optional arguments
Returns
BabelCalib adopts the train-test set methodology for fitting and evaluation. The training set contains the images used for calibration, and the test set contains held-out images for evaluation. Evaluating a model on test-set images demonstrates how well a calibration generalizes to unseen imagery. During testing, the intriniscs are kept fixed and only the poses of the camera are regressed. The RMS re-projection error is used to assess calibration quality. The poses are estimated by get_poses.m:
function [model, res, corners, boards] = get_poses(intrinsics, corners, boards, imgsize, varargin)Parameters:
intrinsics: type modelcorners: type cornersboards: type boardsimgsize: 1x2 array specifies the height and width of the images; all the images are assumed to have the same dimensionsvarargin: optional arguments
Returns
Contains the set of 2D<->3D correspondences of the calibration board fiducials to the detected corners in each image. Here, we let N be the number of images; Kn be the number of detected corners in the n-th image, where (n=1,...,N); and B be the number of planar calibration boards.
| field | data type | description |
|---|---|---|
x |
2xKn array | 2D coordinates specifying the detected corners |
cspond |
2xKn array | correspondences, where each column is a correspondence and the first row contains the indices to points and the second row contains indices to calibration board fiducials |
Contains the set of absolute poses for each of the B calibration boards of the target, where (b=1,...,B) indexes the calibration boards. Also specifies the coordinates of the fiducials on each of the calibration boards.
| field | data type | description |
|---|---|---|
Rt |
3x4 array | absolute orientation of each pose is encoded in the 3x4 pose matrix |
X |
2xKb array | 2D coordinates of the fiducials on board b of the target. The coordinates are specified with respect to the 2D coordinate system attached to each board |
Contains the intrinsics and extrinsics of the regressed camera model. The number of parameters of the back-projection or projection model, denoted C, depends on the chosen camera model and model complexity.
| field | data type | description |
|---|---|---|
proj_model |
str | name of the target projection model |
proj_params |
1xC array | parameters of the projection/back-projection function |
K |
3x3 array | camera calibration matrix (relating to A in the paper: K = inv(A)) |
Rt |
3x4xN array | camera poses stacked along the array depth |
Contains the information about the residuals, loss and initialization (minimal solution). Here, we let K be the total number of corners in all the images.
| field | data type | description |
|---|---|---|
loss |
double | loss value |
ir |
double | inlier ratio |
reprojerrs |
1xK array | reprojection errors |
rms |
double | root mean square reprojection error |
wrms |
double | root mean square weighted reprojection error (Huber weights) |
info |
type info |
Contains additional information about the residuals, loss and initialization (minimal solution).
| field | data type | description |
|---|---|---|
dx |
2xK array | re-projection difference vectors: dx = x - x_hat |
w |
1xK array | Huber weights on the norms of dx |
residual |
2xK array | residuals: residual = w .* dx |
cs |
1xK array (boolean) | consensus set indicators (1 if inlier, 0 otherwise) |
min_model |
type model | model corresponding to the minimal solution |
min_res |
type res | residual info corresponding to the minimal solution |
cfg contains the optional configurations. Default values for the optional parameters are loaded from parse_cfg.m. These values can be changed by using the varargin parameter. Parameters values passed in by varargin take precedence. The varargin format is 'param_1', value_1, 'param_2', value_2, .... The parameter descriptions are grouped by which component of BabelCalib they change.
Solver configurations:
final_model- the selected camera model (default: 'kb')final_complexity- a degree of the polynomial if the final model is polynomial, otherwise ignored (default: 4)
Sampler configurations:
min_trial_count- minimum number of iterations (default: 20)max_trial_count- maximum number of iterations (default: 50)max_num_retries- maximum number of sampling tries in the case of a solver failure (default: 50)confidence- confidence rate (default: 0.995)sample_size- the number of 3D<->2D correspondences that are sampled for each RANSAC iteration (default: 14)
RANSAC configurations:
display- toggles the display of verbose output of intermediate steps (default: true)display_freq- frequency of output during the iterations of robust sampling. (default: 1)irT- minimum inlier ratio to perform refinement (default: 0)
Refinement configurations:
reprojT- reprojection error threshold (default: 1.5)max_iter- maximum number of iterations on the refinement (default: 50)
BabelCalib provides a convenience wrapper (see calib_run_plane.m and calib_run_cube.m) for running the calibration calibrate.m with a training set and evaluating get_poses.m with a test set.
The Deltille detector is a robust deltille and checkerboard detector. It comes with detector library, example detector code, and MATLAB bindings. BabelCalib provides functions for calibration and evaluation using the Deltille software's outputs. Calibration from Deltille detections requires format conversion which is peformed by import_ODT.m. A complete example of using calibrate and get_poses with import_ODT is provided in calib_run_plane_deltille.m and calib_run_cube_deltille.m.
If you find this work useful in your research, please consider citing:
@InProceedings{Lochman-ICCV21,
title = {BabelCalib: A Universal Approach to Calibrating Central Cameras},
author = {Lochman, Yaroslava and Liepieshov, Kostiantyn and Chen, Jianhui and Perdoch, Michal and Zach, Christopher and Pritts, James},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
year = {2021},
pages = {15253-15262}
}The software is licensed under the MIT license. Please see LICENSE for details.