Video Stitching is not working for Gear360(R210) 2017 model video

[arupcsedu]

Image Stitching is working fine. During executing command for video Stitching, it is failing:
Here is the command output :

~/StitchingPoC/gear360pano$ ./gear360pano.cmd 360_7690.MP4File not found: 360_7690.MP4
Processing panofile: 360_7690.MP4
Processing input images (nona)
nona: using graphics card: NVIDIA Corporation GeForce GTX 1050 Ti/PCIe/SSE2
caught exception:
Precondition violation!
Unable to open file '360_7690.MP4'.
(/build/libvigraimpex-dp3J2t/libvigraimpex-1.10.0+dfsg/src/impex/codecmanager.cxx:203)

Error while running nona

Please help.

[ultramango]

Could you please try with the latest version?

[Skygaze]

Hi, the image stitch works fine (thanks!), but I too am having problem with the video (nona image sizes not consistent). My unit is a SM-R210. In the camera settings the video is a fixed 2560x1280. This may be a problem since you said it had to be 1290. In the phone Gear-app, the video size is grayed out, so I cannot change it. Presumably I can go into the pto and modify it?

The output to the frames sub-directory works. However, at the start of the decompression, ffmpeg output the following:
[swscaler @ 0000019358be65c0] deprecated pixel format used, make sure you did set range correctly]

Regards,
Alister.

[Skygaze]

Good news everyone! I got it working on the video from the SM-R210. I guessed correctly that it was the "wrong" numbers in the pto file because the image size was different. I just scaled all the X,Y, displacement numbers appropriately , in my case divide by 2.2625 (instead of 5792x2896 it is 2560x1280) , loaded up the image and modified pto in Hugin: adjusted a few control points, recalculated, stitched, check that it looked the same (it does), then hard-coded gear360video.cmd to point to the modded pto, and it ran fine.

I will post the new pto when I figure out where I can attach it.
Regards,
Alister.

[Skygaze]

Oh, it says it right there at the bottom of the box. Doh. I just added the .log at the end so github would attach the .pto file.

More than one way for changing the scripts, but a simple one is adding a command-line parameter while calling gear360pano.cmd to let it know to load the different pto. Or running an if structure based on the height/width.

One thing I did notice is that the ffmpeg, when writing out the frames, in the exif, the camera model is not one of the parameters, so that's why gear360pano.cmd does a default assign (line 150).

Regards,
Alister.

gear360sm-r210_vid.pto.log

[Boris911]

I have a question, please? Do you change contain of gear360video.cmd (for Gear 360 2017), please?

I changed:
1.) PTO file:
from "set PTOTMPL=gear360video3840.pto"

to "set PTOTMPL=gear360video4096.pto"

2.) Frame rate and resolution for Gear 360 2017 videos:
from ""%FFMPEGPATH%/ffmpeg.exe" -y -f image2 -i %OUTTEMPDIR%/%IMAGETMPLENC% -r 30 -s 3840:1920 -vcodec libx264 %OUTTEMPDIR%/%TMPVIDEO%"

to ""%FFMPEGPATH%/ffmpeg.exe" -y -f image2 -i %OUTTEMPDIR%/%IMAGETMPLENC% -r 24 -s 4096:2048 -vcodec libx264 %OUTTEMPDIR%/%TMPVIDEO%"

What not working for me is use Intel Quick Sync (in Windows 10 1709), on Skylike Intel Core i5-6200U CPU/Intel HD Graphic 520, in ffmpeg (when I changed in gear360video.cmd: from "-vcodec libx264" to "-vcodec h264_qsv"). Well, only one way is use very slow rendering via CPU, on my laptop without dedicated GPU, when ffmpeg can't make a video file from stitched images with Quick Sync Hardware Acceleration. But it is not part of this project (software).

[Boris911]

Otherwise I had to "repair" (add 360 metadata) created stitched videos, from Gear360Pano, via "Spatial Media" https://github.com/google/spatial-media, because Youtube and Facebook don't recognized that video files like 360 videos.

[Skygaze]

Hi Boris, I just tried a 4096x2048 video on Gear 360 (2017) SM-R210 and was successful with gear360video.cmd.

Changes to gear360video.cmd, same as you:
ln 15 set PTOTMPL=gear360video3840.pto to set PTOTMPL=gear360video4096.pto
ln 51 3840:1920 to 4096:2048 Oops, I see I forgot to change the frame rate to 24.

I have no knowledge about hardware acceleration, so I cannot help you with your QuickSync and codec problem. Sorry. I also need to learn about posting to Facebook Youtube for the 360 videos.

[ultramango]

I see you are using video stitching script for Windows which is pretty limited (compared to Linux version). I'll spend some time to match functionality with Linux (Batch programming is a challenge).

Problems you are having is from the script not recognising correct video side.

BTW: @Skygaze thanks for the contribution, I'll compare this file with the existing one gear360video2560.pto to see what gives better results.

[Skygaze]

No problem, I get a kick out of working on puzzles. It should be obvious that my change to the cmd was a hardcode that should have a if branch depending on the imagesize. I did not want to try and write that bit of code because I am not familiar with windows batch (just enough to reasonably understand). For the pto, there are a couple of control points high up that I left alone because I only had blank sky. Remember that it is possible that the hardware has slight assembly differences from one camera to the other that prevents having an absolute pto across all cameras. But as a first guess it should be very close.

Thank you again for all the work you have put in so far! It certainly made my life easier. If you wish, I can look at (on the windows side) incorporating the spatial media exif writer so that when uploaded to YouTube, the browsers recognize the 360 tag. [I am away a few days so can't start until next weekend].

[Skygaze]

Hi Boris and ultramango. I came across this link regarding video size and encoding parameters -perhaps you will find it useful.

I have not completed reading it, but it may explain why the 4096x2048 video works on YouTube, but not my laptop's Windows Media Player.

Regards,
Alister.

[Skygaze]

https://purplepill.io/blog/best-encoding-settings-resolution-for-4k-360-3d-vr-videos/

The recommendation was to use the following settings, and that the obvious vertical reduction has no impact, the 360 player seems to understand to play it correctly in the vertical:
"%FFMPEGPATH%/ffmpeg.exe" -y -f image2 -i %OUTTEMPDIR%/%IMAGETMPLENC% -r 30 -s 3840:2160 -vcodec libx264 -level:v 4.2 -crf 18 %OUTTEMPDIR%/%TMPVIDEO%

Here is the result:
https://youtu.be/GA_LU-1aSHI

[kszere]

You can update to script files?

[Skygaze]

Hi kszere, the owner of the script, ultramango, may be on summer holidays, so it could take a while before he responds. I hope you can wait. I might be able to help when I return from my holidays, in the middle of August.

Regards,
Alister.

[ultramango]

Hi,

I remember reading this article a while ago and I had some problems with understanding it (a lot of content, very little specifics). I read it the second time now but got similar feelings. What I understood:

• first part of the article, that you also refer to, is about stereoscopic (3D) 360 videos; to have a better quality video and support older hardware they do some trickery with the video (reduce vertical video size, part of the video /up, down/ are static images), so this is not applicable to Gear360,
• second part refers to 2D 360 videos, still, couldn't find anything concrete.

Note: article is from late 2015; since then hardware got more powerful. Also I recon they might care about size as they want to stream videos, this is not the priority for gear360 script (for now :) ).

So far I don't see anything valuable, from that article, that I could improve in the gear360 script. Actually current quality parameter crf is already pretty "high": 18 - see FFmpeg wiki entry (18 for h264 is considered, visually, lossless, for h265 quality will be even better).

Also let's not forget that the source material from Samsung 360 camera is not of the highest quality, which then makes some high quality settings for output video coded an overkill.

The only thing that I would be tempted to do is to revert the default to h264 as it is faster and size is not that important nowadays. Or perhaps add more quality options (presets).

Cheers,
L.

P.S.
Holidays maybe later this year.