Using saved rgb and depth images

[sabrinazuraimi]

Hi, I'm trying to do some offline meshing with my own rgb and depth images. I converted the raw rgb and depth streams from Kinect v2 to png images. I saw that Infinitam works with pgm and ppm files, so I used imagemagick to convert my RGB and depth images to ppm and pgm files. I've already tested my built InfiniTAM with the Teddy dataset and it worked fine. I think my problem is with the pgm files(the depth images). Has anyone got this to work with their own images/frames??

[sgolodetz]

What unit are the depths expressed in, and is your calib.txt set up for that appropriately?

[sabrinazuraimi]

I first read the raw depth data from kinect v2(which has dimension of 512 by 424 and depth values ranging from 0 to 8000mm). I then convert them to depth images using this code. (BS is the depth stream)

gray = np.frombuffer(fp.read(BS), dtype=np.uint16).reshape((H,W,1))
gray = cv2.flip(gray, 1)
gray = np.minimum(gray * (255/8000) , 255).astype(np.uint8)
depth = cv2.applyColorMap(gray, cv2.COLORMAP_HSV)
cv2.imwrite("Depthdata\Depth{}.png".format(j-1), depth)

Afterwards, I use imagemagick to convert it to pgm

[sgolodetz]

So you have depth values stored as 16-bit unsigned ints (i.e. unsigned shorts) ranging from 0-8000, which are in mm. You initially quantize them into 8-bit unsigned ints (i.e. unsigned chars), losing some depth precision. You then map them to colours, which to some extent correspond to depth, but such that the pixel values themselves no longer represent depth directly, and later convert your colour image to greyscale using ImageMagick. The end result is a greyscale version of your colour-mapped image, whose pixel values you then try to treat as depth - that's why it's not working.

What you want to do is store the original 16-bit depth values in the .pgm, and then make sure that your calib.txt file specifies affine 0.001 0 to tell InfiniTAM that you're using depths in mm (i.e. 0.001 of a m).

[sabrinazuraimi]

Thanks for the reply. Store original 16-bit depth values in the .pgm...

            gray = np.frombuffer(fp.read(BS), dtype=np.uint16).reshape((H,W,1))
            depth = cv2.flip(gray, 1)
            cv2.imwrite('Depth{}.pgm'.format(j-1), depth)

This is how I did it. But I'm still getting errors reading the depth images

using depth images: /home/Desktop/InfiniTAM-master/InfiniTAM/Files/OwnTable/Frames/%04i.pgm
error reading file '/home/Desktop/InfiniTAM-master/InfiniTAM/Files/OwnTable/Frames/0000.pgm'
Cannot open input file