Array adapted for Pybind11 and NMS bug fixed

README.md

@@ -185,4 +185,4 @@ Please cite these papers in your publications if it helps your research (the fac
 
 
 ## License
-OpenPose is freely available for free non-commercial use, and may be redistributed under these conditions. Please, see the [license](LICENSE) for further details. [Interested in a commercial license? Check this link](https://flintbox.com/public/project/47343/). For commercial queries, contact [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/).
+OpenPose is freely available for free non-commercial use, and may be redistributed under these conditions. Please, see the [license](LICENSE) for further details. Interested in a commercial license? Check this [FlintBox link](https://flintbox.com/public/project/47343/). For commercial queries, check the contact information provided in the [FlintBox link](https://flintbox.com/public/project/47343/) and also cc [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/) in that email.

doc/faq.md

@@ -13,6 +13,7 @@ OpenPose - Frequently Asked Question (FAQ)
     8. [Source Directory does not Contain CMakeLists.txt (Windows)](#source-directory-does-not-contain-cmakelists.txt-windows)
     9. [How Should I Link my IP Camera?](#how-should-i-link-my-ip-camera)
     10. [Difference between BODY_25 vs. COCO vs. MPI](#difference-between-body_25-vs.-coco-vs.-mpi)
+    11. [How to Measure the Latency Time?](#how-to-measure-the-latency-time)
 
 
 
@@ -87,11 +88,18 @@ Note: OpenPose library is not an executable, but a library. So instead clicking
 
 
 ### How Should I Link my IP Camera?
-**Q: How Should I Link my IP Camera with http protocol?.**
+**Q: How Should I Link my IP Camera with http protocol?**
 
 **A**: Usually with `http://CamIP:PORT_NO./video?x.mjpeg`.
 
 
 
 ### Difference between BODY_25 vs. COCO vs. MPI
 COCO model will eventually be removed. BODY_25 model is faster, more accurate, and it includes foot keypoints. However, COCO requires less memory on GPU (being able to fit into 2GB GPUs with the default settings) and it runs faster on CPU-only mode. MPI model is only meant for people requiring the MPI-keypoint structure. It is also slower than BODY_25 and far less accurate.
+
+
+
+### How to Measure the Latency Time?
+**Q: How to measure/calculate/estimate the latency/lag time?**
+
+**A**: [Profile](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/installation.md#profiling-speed) the OpenPose speed. For 1-GPU or CPU-only systems (use `--disable_multi_thread` for simplicity in multi-GPU systems for latency measurement), the latency will be roughly the sum of all the reported measurements.

doc/release_notes.md

@@ -275,13 +275,18 @@ OpenPose Library - Release Notes
     8. Auxiliary classes in errorAndLog turned into namespaces (Profiler must be kept as class to allow static parameters).
     9. Added flag `--frame_step` to allow the user to select the step or gap between processed frames. E.g., `--frame_step 5` would read and process frames 0, 5, 10, etc.
     10. Added sanity checks to avoid `--frame_last` to be smaller than `--frame_first` or higher than the number of total frames.
-    11. Added Array::getStride() to get step size of each dimension of the array.
+    11. Array improvements for Pybind11 compatibility:
+        1. Array::getStride() to get step size of each dimension of the array.
+        2. Array::getPybindPtr() to get an editable const pointer.
+        3. Array::pData as binding of spData.
+        4. Array::Array that takes as input a pointer, so it does not re-allocate memory.
 2. Functions or parameters renamed:
     1. By default, python example `tutorial_developer/python_2_pose_from_heatmaps.py` was using 2 scales starting at -1x736, changed to 1 scale at -1x368.
     2. WrapperStructPose default parameters changed to match those of the OpenPose demo binary.
     3. WrapperT.configure() changed from 1 function that requries all arguments to individual functions that take 1 argument each.
 3. Main bugs fixed:
     1. CMake-GUI was forcing to Release mode, allowed Debug modes too.
+    2. NMS returns in index 0 the number of found peaks. However, while the number of peaks was truncated to a maximum of 127, this index 0 was saving the real number instead of the truncated one.
 
 
 

include/openpose/core/array.hpp

@@ -55,6 +55,24 @@ namespace op
          */
         Array(const std::vector<int>& sizes, const T value);
 
+        /**
+         * Array constructor.
+         * Equivalent to default constructor, but it does not allocate memory, but rather use dataPtr.
+         * @param size Integer with the number of T element to be allocated. E.g., size = 5 is internally similar to
+         * `new T[5]`.
+         * @param dataPtr Pointer to the memory to be used by the Array.
+         */
+        Array(const int size, T* const dataPtr);
+
+        /**
+         * Array constructor.
+         * Equivalent to default constructor, but it does not allocate memory, but rather use dataPtr.
+         * @param sizes Vector with the size of each dimension. E.g., size = {3, 5, 2} is internally similar to:
+         * `new T[3*5*2]`.
+         * @param dataPtr Pointer to the memory to be used by the Array.
+         */
+        Array(const std::vector<int>& sizes, T* const dataPtr);
+
         /**
          * Copy constructor.
          * It performs `fast copy`: For performance purpose, copying a Array<T> or Datum or cv::Mat just copies the
@@ -236,7 +254,7 @@ namespace op
          */
         inline T* getPtr()
         {
-            return spData.get();
+            return pData; // spData.get()
         }
 
         /**
@@ -245,7 +263,17 @@ namespace op
          */
         inline const T* getConstPtr() const
         {
-            return spData.get();
+            return pData; // spData.get()
+        }
+
+        /**
+         * Similar to getConstPtr(), but it allows the data to be edited.
+         * This function is only implemented for Pybind11 usage.
+         * @return A raw pointer to the data.
+         */
+        inline T* getPybindPtr() const
+        {
+            return pData; // spData.get()
         }
 
         /**
@@ -282,7 +310,7 @@ namespace op
         inline T& operator[](const int index)
         {
             #ifdef NDEBUG
-                return spData.get()[index];
+                return pData[index]; // spData.get()[index]
             #else
                 return at(index);
             #endif
@@ -298,7 +326,7 @@ namespace op
         inline const T& operator[](const int index) const
         {
             #ifdef NDEBUG
-                return spData.get()[index];
+                return pData[index]; // spData.get()[index]
             #else
                 return at(index);
             #endif
@@ -395,6 +423,7 @@ namespace op
         std::vector<int> mSize;
         size_t mVolume;
         std::shared_ptr<T> spData;
+        T* pData; // pData is a wrapper of spData. Used for Pybind11 binding.
         std::pair<bool, cv::Mat> mCvMatData;
 
         /**
@@ -422,11 +451,7 @@ namespace op
          */
         T& commonAt(const int index) const;
 
-        /**
-         * Private auxiliar function that sets the cv::Mat wrapper and makes it point to the same data than
-         * std::shared_ptr points to.
-         */
-        void setCvMatFromSharedPtr();
+        void resetAuxiliary(const std::vector<int>& sizes, T* const dataPtr = nullptr);
     };
 
     // Static methods

include/openpose/pose/enumClasses.hpp

@@ -25,6 +25,7 @@ namespace op
         CAR_12,         /**< Experimental. Do not use. */
         BODY_25D,       /**< Experimental. Do not use. */
         BODY_23,        /**< Experimental. Do not use. */
+        CAR_22,         /**< Experimental. Do not use. */
         Size,
     };
 

include/openpose/pose/poseParametersRender.hpp

@@ -322,6 +322,43 @@ namespace op
         255.f,     0.f,   255.f, \
         255.f,     0.f,   255.f
 
+    // CAR_22
+    #define POSE_CAR_22_PAIRS_RENDER_GPU \
+            0,1,1,3,3,2,2,0,   6,7,7,16,16,17,17,6,   12,13,13,14,14,15,15,12,   6,8,7,8,6,9,7,9,6,4,7,5,   12,11,13,10, \
+            16,18,17,18,16,19,17,19,   6,21,7,20
+    #define POSE_CAR_22_SCALES_RENDER_GPU 0.625
+    #define POSE_CAR_22_COLORS_RENDER_GPU \
+        255.f,   128.f,   128.f, \
+        255.f,     0.f,     0.f, \
+         64.f,     0.f,     0.f, \
+        255.f,     0.f,     0.f, \
+        \
+          0.f,   255.f,     0.f, \
+          0.f,   255.f,     0.f, \
+        \
+          0.f,     0.f,    64.f, \
+        128.f,   128.f,   255.f, \
+        \
+          0.f,   255.f,     0.f, \
+          0.f,   255.f,     0.f, \
+        \
+          0.f,   255.f,     0.f, \
+          0.f,   255.f,     0.f, \
+        \
+         64.f,     0.f,     0.f, \
+        255.f,   128.f,   128.f, \
+        255.f,     0.f,     0.f, \
+        255.f,     0.f,     0.f, \
+        \
+          0.f,     0.f,   255.f, \
+          0.f,     0.f,   255.f, \
+        \
+          0.f,   255.f,     0.f, \
+          0.f,   255.f,     0.f, \
+        \
+          0.f,     0.f,   255.f, \
+          0.f,     0.f,    64.f
+
     // Rendering functions
     OP_API const std::vector<float>& getPoseScales(const PoseModel poseModel);
     OP_API const std::vector<float>& getPoseColors(const PoseModel poseModel);