- Initial version, main functionality:
- Body keypoint detection and rendering in Ubuntu 14 and 16.
- It can read an image directory, video or webcam.
- It can display the results or storing them on disk.
- Main improvements:
- Rendering max resolution from 720p to >32k images.
- Highly improved documentation.
- Functions or parameters renamed:
- Demo renamed from rtpose to openpose.
- Main bugs fixed:
- Demo uses exec instead of start, so it works with more OpenCV custom compiled versions.
- Main improvements:
- Added face keypoint detection.
- Added Windows 10 compatibility.
- Auto-detection of the number of GPUs.
- MPI visualization more similar to COCO one.
- Rendering max resolution from 720p to >32k images.
- GUI info adder working when the worker TDatum has more than 1 Datum.
- It prints out the error description before throwing the exception (so that it is written on the Windows cmd).
- Highly improved documentation.
- Functions or parameters renamed:
- Flag
--write_pose
renamed as--write_keypoint
and it also applies to face and/or hands. - Flag
--write_pose_json
renamed as--write_keypoint_json
and it also applies to face and/or hands. - Flag
--write_pose_format
renamed as--write_keypoint_format
and it also applies to face and/or hands. - PoseSaver and its JSON variant renamed as KeypointSaver.
- PoseJsonCocoSaver renamed as CocoJsonSaver.
- Flag
- Main bugs fixed:
- All visualization functions moved to same thread, so it works with most OpenCV custom compiled versions.
- Fixed error on debug mode:
Too many resources requested for launch
.
- Main improvements:
- Added hand keypoint detection.
- Windows branch merged to master branch.
- Face and hands use
Maximum
instead ofNms
, since there is only 1 person / detection. - Increased accuracy on multi-scale (added
Datum::scaleRatios
to save the relative scale ratio when multi-scale). - Increased speed ~5% by adding CPU rendering (but GPU is the default rendering).
- Rendering colors modified, visually better results.
- Rendering threshold for pose, face and hands becomes user-configurable.
- Check() functions give more feedback.
- WCocoJsonSaver finished and removed its 3599-image limit.
- Added
--camera_fps
so generated video (--write_video
) will use that frame rate. - Reduced the number of printed information messages. Default logging priority threshold increased to Priority::Max.
- GFlags to OpenPose configuration parameters reader moved from each demo to utilities/flagsToOpenPose.
- Nms classes do not use
numberParts
forReshape
, they deduce the value. - Improved documentation.
- Functions or parameters renamed:
- Render flags renamed in the demo in order to incorporate the CPU/GPU rendering.
- Keypoints saved in JSON files (
--write_keypoint_json
) are now saved aspose_keypoints
,face_keypoints
,hand_left_keypoints
, andhand_right_keypoints
. They all were previously saved asbody_parts
. - Flag
--num_scales
renamed as--scale_number
. - All hand and pose flags renamed such as they start by
--hand_
and--face_
respectively.
- Main bugs fixed:
- Fixed bug in Array::getConstCvMat() if mVolume=0, now returning empty cv::Mat.
- Fixed bug:
--process_real_time
threw error with webcam. - Fixed bug: Face not working when input and output resolutions are different.
- Fixed some bugs that prevented debug version to run.
- Face saved in JSON files were called
--body_parts
. Now they are called--face_keypoints
.
- Main improvements:
- Windows library turned into DLL dynamic library (i.e., portable).
- Improved documentation.
- Functions or parameters renamed:
openpose/utilities/macros.hpp
moved toopenpose/utilities/macros.hpp
.
- Main improvements:
- Added OpenCV 3.3 compatibility.
- Caffe turned into DLL library.
- OpenPose is now completely portable across Windows 10 computers (with Nvidia graphic card).
- Added OpenPose 1.0.1 portable demo.
- Removed Python and some unnecessary boost dependencies on the VS project.
- Replaced all double quotes by angle brackets in include statements (issue #61).
- Added 3-D reconstruction demo.
- Auto-detection of the camera index.
- Speed up of ~30% in op::floatPtrToUCharCvMat.
- COCO extractor now extracts image ID from the image name itslef (format "string_%d"). Before, only working with validation test, now applicable to e.g., test sets.
- Changed display texts, added
OpenPose
name.
- Main bugs fixed:
- Pycaffe can now be imported from Python.
- Fixed
Tutorial/Wrapper
VS linking errors.
- Main improvements:
- Added CMake installer for Ubuntu.
- Added how to use keypoint data in
examples/tutorial_wrapper/
. - Added flag for warnings of type
-Wsign-compare
and removed in code. - Slightly improved accuracy by considering ears-shoulder connection (e.g., +0.4 mAP for 1 scale in validation set).
- Main bugs fixed:
- Windows version crashing with std::map copy.
- Main improvements:
- Speed increase when processing images with different aspect ratios. E.g., ~20% increase over 3.7k COCO validation images on 1 scale.
- Huge speed increase and memory reduction when processing multi-scale. E.g., over 3.7k COCO validation images on 4 scales: ~40% (~770 to ~450 sec) speed increase, ~25% memory reduction (from ~8.9 to ~6.7 GB / GPU).
- Slightly increase of accuracy given the fixed mini-bugs.
- Added IP camera support.
- Output images can have the input size, OpenPose able to change its size for each image and not required fixed size anymore.
- FrameDisplayer accepts variable size images by rescaling every time a frame with bigger width or height is displayed (gui module).
- OpOutputToCvMat & GuiInfoAdder does not require to know the output size at construction time, deduced from each image.
- CvMatToOutput and Renderers allow to keep input resolution as output for images (core module).
- New standalone face keypoint detector based on OpenCV face detector: much faster if body keypoint detection is not required but much less accurate.
- Face and hand keypoint detectors now can return each keypoint heatmap.
- The flag
USE_CUDNN
is no longer required;USE_CAFFE
andUSE_CUDA
(replacing the oldCPU_ONLY
) are no longer required to use the library, only to build it. In addition, Boost, Caffe, and its dependencies have been removed from the OpenPose header files. Only OpenCV include and lib folders are required when building a project using OpenPose. - OpenPose successfully compiles if the flags
USE_CAFFE
and/orUSE_CUDA
are not enabled, although it will give an error saying they are required. - COCO JSON file outputs 0 as score for non-detected keypoints.
- Added example for OpenPose for user asynchronous output and cleaned all
tutorial_wrapper/
examples. - Added
-1
option for--net_resolution
in order to auto-select the best possible aspect ratio given the user input. - Net resolution can be dynamically changed (e.g., for images with different size).
- Added example to add functionality/modules to OpenPose.
- Added
--disable_multi_thread
flag in order to allow debug and/or highly reduce the latency (e.g., when using webcam in real-time). - Allowed to output images without any rendering.
- Functions or parameters renamed:
- OpenPose able to change its size and initial size dynamically:
- Flag
--resolution
renamed as--output_resolution
. - FrameDisplayer, GuiInfoAdder and Gui constructors arguments modified (gui module).
- OpOutputToCvMat constructor removed (core module).
- New Renders classes to split GpuRenderers from CpuRenderers.
- Etc.
- Flag
- OpenPose able to change its net resolution size dynamically:
- Changed several functions on
core/
,pose/
,face/
, andhand/
modules.
- Changed several functions on
CPU_ONLY
changed byUSE_CUDA
to keep format.
- OpenPose able to change its size and initial size dynamically:
- Main bugs fixed:
- Scaling resize issue fixed: ~1-pixel offset due to not considering 0-based indexes.
- Ubuntu installer script now works even if Python pip was not installed previously.
- Flags to set first and last frame as well as jumping frames backward and forward now works on image directory reader.
- Main improvements:
- Heatmaps can be saved in floating format.
- More efficient non-processing version (i.e., if all keypoint extractors are disabled, and only image extraction and display/saving operations are performed).
- Heat maps scaling: Added
--heatmaps_scale
to OpenPoseDemo, added option not to scale the heatmaps, and added customfloat
format to save heatmaps in floating format. - Detector of the number of GPU also considers the initial GPU index given by the user.
- Added
--write_json
as new version of--write_keypoint_json
. It includes the body part candidates (if enabled), as well as any extra information added in the future (e.g., person ID). - Body part candidates can be retrieved in op::Datum and saved with
--write_json
.
- Functions or parameters renamed:
PoseParameters
splitted intoPoseParameters
andPoseParametersRender
and const parameters turned into functions for more clarity.
- Main bugs fixed:
- Render working on images > 4K (#324).
- Cleaned redundant arguments on
getAverageScore
andgetKeypointsArea
. - Slight speed up when heatmaps must be returned to the user (not doing a double copy anymore).
- Main improvements:
- Output of
--write_json
uses less hard disk space (enters and tabs removed). - Removed Boost dependencies.
- Caffe added as submodule.
- CMake installer compatible with Windows.
- Added freeglut download script (3-D reconstruction demo for Windows).
- Added Debug version for Windows (CMake).
- Runtime verbose about average speed configurable by user with
PROFILER_ENABLED
option (CMake/Makefile.config) and--profile_speed
flag. - Lighter Caffe version compiled by CMake in Ubuntu: disabled Caffe extra support (e.g., OpenCV, Python) and doc.
- Renamed CMake binaries (Ubuntu) to match old Makefile format:
_bin
by.bin
. - 3-D reconstruction demo cleaned, implemented in Ubuntu too, and now defined as module of OpenPose rather than just a demo.
- CMake as default installer in documentation.
- Added flag: number_people_max to optionally select the maximum number of people to be detected.
- 3-D reconstruction module forces the user to set
number_people_max 1
to avoid errors (as it assumes only 1 person per image). - Removed old
windows/
version. CMake is the only Windows version available. - Camera parameters (flir camera) are read from disk at runtime rather than being compiled.
- 3-D reconstruction module can be implemented with different camera brands or custom image sources.
- Flag
--write_json
includes 3-D keypoints. - 3-D reconstruction module can be used with images and videos. Flag
--3d_views
added to allow--image_dir
and--video
allow loading stereo images. - Flag
--camera_resolution
applicable to--flir_camera
. - Throw error message if requested GPU IDs does not exist (e.g., asking for 2 GPUs starting in ID 1 if there is only 2 GPUs in total).
- VideoSaver (
--write_video
) compatible with multi-camera setting. It will save all the different views concatenated. - OpenPose small GUI rescale the verbose text to the displayed image, to avoid the text to be either too big or small.
- OpenPose small GUI shows the frame number w.r.t. the original producer, rather than the frame id. E.g., if video is started at frame 30, OpenPose will display 30 rather than 0 in the first frame.
- OpenPose GUI: 'l' and 'k' functionality swapped.
- 3-D reconstruction module: Added flag
--3d_min_views
to select minimum number of cameras required for 3-D reconstruction. - Flir camera producer
n
times faster forn
cameras (multi-threaded). If the number of cameras is greater than the number of the computer threads, the speed up might not be exactlyn
times.
- Output of
- Functions or parameters renamed:
- Flag
no_display
renamed asdisplay
, able to select betweenNoDisplay
,Display2D
,Display3D
, andDisplayAll
. - 3-D reconstruction demo is now inside the OpenPose demo binary.
- Renamed
*_keypoints
by*_keypoints_2d
to avoid confusion with 3d ones in--write_json
output file. - CvMatToOpInput requires PoseModel to know the normalization to be performed.
- Created
net/
module in order to reducecore/
number of classes and files and for future scalability.
- Flag
- Main bugs fixed:
- Slight speed up (~1%) for performing the non-maximum suppression stage only in the body part heatmaps channels, and not also in the PAF channels.
- Fixed core-dumped in PoseRenderer with GUI when changed element to be rendered to something else than skeleton.
- 3-D visualizer does not crash on exit anymore.
- Fake pause ('m' key pressed) works again.
- Main improvements:
- Model BODY_25 released, that includes the 17 COCO keypoints + neck + midhip + 6 foot keypoints. It is also about 3% more accurate and 30% faster than the original
COCO
model. - New calibration module: Intrinsic and extrinsic camera calibration toolbox based on OpenCV.
- Improvements involving Flir cameras:
- Added software trigger and a dedicated thread to keep reading images so latency is removed and runtime is faster (analogously to webcamReader).
- Undistortion of the images is x3.5 faster per camera, i.e., x3.5 Flir camera producer reading w.r.t previous multi-threaded version, which was x number_cameras faster than the original version.
- Added flag
flir_camera_index
to allow running on all the cameras at once, or only on 1 camera at the time. - Added flag
frame_keep_distortion
not to undistort the images. E.g., useful when recording images for camera calibration. - Changed Spinnaker::DEFAULT image extraction mode by Spinnaker::IPP, which does not show a pixelated image while keeping very similar runtime.
- 3-D reconstruction:
- Added non-linear minimization to further improve 3-D triangulation accuracy by ~5% (Ubuntu only).
- It is only run if reprojction error is more than a minimum threshold (improve speed with already good quality results) and also less than another outlier threshold.
- Outliers are removed from final result if >= 3 camera views.
- Applied RANSAC if >=4 camera views.
- Latency highly reduced in multi-GPU setting. Each GPU process a different camera view, instead of a different time-instant sequence.
- CMake: All libraries as single variable (simpler to add/remove libraries).
- Averaged latency reduced to half.
- 15% speed up for default CMake version. CMake was not setting
Release
mode by default. - Light speed up, and body approach much more invariant to number of people. Removed
checkEQ
from tight loop in bodyPartConnectorBase, which took a huge time exponential to the number of people. - Datum includes extrinsic and intrinsic camera parameters.
- Function
scaleKeypoints(Array<float>& keypoints, const float scale)
also accepts 3D keypoints. - 3D keypoints and camera parameters in meters (instead of millimeters) in order to reduce numerical errors.
- New
PoseExtractor
class to contain future ID and tracking algorithms as well as the current OpenPose keypoint detection algorithm. - Added initial alpha versions of the
tracking
andidentification
modules (for now disabled but available in the source code), includingPersonIdExtractor
andPersonTracker
.PersonIdExtractor
includes greedy matrix OP-LK matching. - Added catchs to all demos for higher debug information.
- GUI includes the capability of dynamically enable/disable the face, hand, and 3-D rendering, as well as more clear visualization for skeleton, background, heatmap addition, and PAF addition channels.
- When GUI changes some parameter from PoseExtractorNet, there is a log to notify the user of the change.
- Deprecated flag
--write_keypoint_json
removed (--write_json
is the equivalent since version 1.2.1). - Speed up of cvMatToOpOutput and opOutputToCvMat: op::Datum::outputData is now H x W x C instead of C x H x W, making it much faster to be copied to/from op::Datum::cvOutputData.
- Much faster GUI display by adding the
WITH_OPENCV_WITH_OPENGL
flag to tell whether to use OpenGL support for OpenCV. - Turned sanity check error into warning when using dynamic
net_resolution
forimage_dir
in CPU/OpenCL versions. - Minimized CPU usage when queues are empty or full, in order to prevent problems such as general computer slow down, overheating, or excesive power usage.
- Model BODY_25 released, that includes the 17 COCO keypoints + neck + midhip + 6 foot keypoints. It is also about 3% more accurate and 30% faster than the original
- Functions or parameters renamed:
- Removed scale parameter from hand and face rectangle extractor (causing wrong results if custom
--output_resolution
). - Functions
scaleKeypoints
, other thanscaleKeypoints(Array<float>& keypoints, const float scale)
, renamed asscaleKeypoints2d
. (W)PoseExtractor
renamed to(W)PoseExtractorNet
to distinguish from newPoseExtractor
. Analogously with(W)FaceExtractorNet
and(W)HandExtractorNet
.- Experimental module removed and internal
tracking
folder moved to main openpose folder. - Switched GUI shortcuts for the kind of channel to render (skeleton, heatmap, PAF, ...) in order to make it more intuitive: 1 for skeleton, 1 for background heatmap, 2 for adding all heatmaps, 3 for adding all PAFs, and 4 to 0 for the initial heatmaps.
- Removed scale parameter from hand and face rectangle extractor (causing wrong results if custom
- Main bugs fixed:
- Fixed hand and face extraction and rendering scaling issues when
--output_resolution
is not the default one. - Part candidates (
--part_candidates
) are saved with the same scale than the final keypoints itself. - Fixed bug in keepTopNPeople.hpp (
--number_people_max
) that provoked core dumped if lots of values equal to the threshold. - Flir cameras: Cameras sorted by serial number. Video and images recorded from flir cameras were (and are) assigned the camera parameters based on serial number order, so it would fail if the cameras order was not the same than if sorted by serial number.
- CPU version working in non-Nvidia Windows machines.
- Fixed hand and face extraction and rendering scaling issues when
- Main improvements:
- Added initial single-person tracker for further speed up or visual smoothing (
--tracking
flag). - Greedy body part connector implemented in CUDA: +~30% speed up in Nvidia (CUDA) version with default flags and +~10% in maximum accuracy configuration. In addition, it provides a small 0.5% boost in accuracy (default flags).
- OpenPose can be built as Unity plugin: Added flag
BUILD_UNITY_SUPPORT
and special Unity code. - If camera is unplugged, OpenPose GUI and command line will display a warning and try to reconnect it.
- Wrapper classes simplified and renamed. Wrapper renamed as WrapperT, and created Wrapper as the non-templated class equivalent.
- API and examples improved:
- New header file
flags.hpp
that includes all OpenPose flags, removing the need to copy them repeatedly on each OpenPose example file. tutorial_wrapper
renamed astutorial_api_cpp
as well as new examples were added.tutorial_python
renamed astutorial_api_python
as well as new examples were added.tutorial_pose
andtutorial_thread
renamed astutorial_developer
, not meant to be used by users, but rather for OpenPose developers.
- New header file
- Added a virtual destructor to almost all clases, so they can be inherited. Exceptions (for performance reasons): Array, Point, Rectangle, CvMatToOpOutput, OpOutputToCvMat.
- Auxiliary classes in errorAndLog turned into namespaces (Profiler must be kept as class to allow static parameters).
- Added flag
--frame_step
to allow the user to select the step or gap between processed frames. E.g.,--frame_step 5
would read and process frames 0, 5, 10, etc. - Added sanity checks to avoid
--frame_last
to be smaller than--frame_first
or higher than the number of total frames. - Array improvements for Pybind11 compatibility:
- Array::getStride() to get step size of each dimension of the array.
- Array::getPybindPtr() to get an editable const pointer.
- Array::pData as binding of spData.
- Array::Array that takes as input a pointer, so it does not re-allocate memory.
- Producer defined inside Wrapper rather than being defined on each example.
- Reduced many Visual Studio warnings (e.g., uncontrolled conversions between types).
- Added new keypoint-related auxiliary functions in
utilities/keypoints.hpp
. - Function
resizeFixedAspectRatio
can take already allocated memory (e.g., faster if target is an Array object, no intermediate cv::Mat required). - Added compatibility for OpenCV 4.0, while preserving 2.4.X and 3.X compatibility.
- Improved and added several functions to
utilities/keypoints.hpp
and Array to simplify keypoint post-processing. - Removed warnings from Spinnaker SDK at compiling time.
- All bash scripts incorporate
#!/bin/bash
to tell the terminal that they are bash scripts. - Added flag
--verbose
to plot the progress. - Added find_package(Protobuf) to allow specific versions of Protobuf.
- Examples do not end in core dumped if an OpenPose exception occurred during initialization, but it is rather closed returning -1. However, it will still results in core dumped if the exception occurs during multi-threading execution.
- Video (
--write_video
) can be generated from images (--image_dir
), as long as they maintain the same resolution. - Added
--fps_max
flag to limit the maximum processing frame rate of OpenPose (useful to display results at a maximum desired speed).
- Added initial single-person tracker for further speed up or visual smoothing (
- Functions or parameters renamed:
- By default, python example
tutorial_developer/python_2_pose_from_heatmaps.py
was using 2 scales starting at -1x736, changed to 1 scale at -1x368. - WrapperStructPose default parameters changed to match those of the OpenPose demo binary.
- WrapperT.configure() changed from 1 function that requries all arguments to individual functions that take 1 argument each.
- Added
Forward
to all net classes that automatically selects between CUDA, OpenCL, or CPU-only version depending on the defines. - Previously hardcoded
COCO_CHALLENGE
variable turned into user configurable flag--maximize_positives
. - Removed old COCO 2014 validation scripts.
- WrapperStructOutput split into WrapperStructOutput and WrapperStructGui.
- Replaced
--camera_fps
flag by--write_video_fps
, given that it was a confusing name: It did not affect the webcam FPS, but only the FPS of the output video. In addition, default value changed from 30 to -1.
- By default, python example
- Main bugs fixed:
- CMake-GUI was forcing to Release mode, allowed Debug modes too.
- NMS returns in index 0 the number of found peaks. However, while the number of peaks was truncated to a maximum of 127, this index 0 was saving the real number instead of the truncated one.
- Template functions could not be imported in Windows for projects using the OpenPose library DLL.
- Function
scaleKeypoints2d
was not working if any of the scales was 1 (e.g., fail if scaleX = 1 but scaleY != 1, or if any offset was not 0). - Fixed bug in
KeepTopNPeople
that could provoke segmentation fault fornumber_people_max
> 1.
Download and/or check any OpenPose version from https://github.com/CMU-Perceptual-Computing-Lab/openpose/releases.