Issues on a Nvidia Shield K1 tablet and UE4 4.16

[hectorC]

Thank you for making this available! I hope a community grows around it.

I just tested it on an Nvidia Shield K1 tablet and ran into some issues (running the sample project):

1. Black screen at start that I fixed by going into the app's permissions and enabling camera access
2. After the target is detected, the screen goes black (unrelated to above issue) but the cone appears. Loosing tracking doesn't restore the video background and the screen remains black (same issue in portrait and landscape)
3. On portrait the tracking is wrong and the axis are reversed
4. On landscape the tracking seems closer to what it should be but the positioning of the cone seems strange

I wonder if some of these issues are related to Vuforia not having access to camera calibration data since it appears as if the projection matrix is wrong.

07-14 14:20:32.984 23401 23417 D UE4 : [2017.07.14-18.20.32:983][ 0]CFVuforiaLog: Successfully initialized ObjectTracker.
07-14 14:20:33.048 23401 23417 D UE4 : [2017.07.14-18.20.33:048][ 0]CFVuforiaLog: Successfully loaded and activated data set.
07-14 14:20:33.054 23401 23417 D UE4 : CFVuforiaLog:Warning: AVuforiaActor::BeginPlay
07-14 14:20:33.054 23401 23417 D UE4 : [2017.07.14-18.20.33:054][ 0]CFVuforiaLog:Warning: AVuforiaActor::BeginPlay
07-14 14:20:33.612 23401 23417 D UE4 : CFVuforiaLog:Warning: Viewport size = 842,1280
07-14 14:20:33.612 23401 23417 D UE4 : [2017.07.14-18.20.33:612][ 0]CFVuforiaLog:Warning: Viewport size = 842,1280
07-14 14:20:33.613 23401 23417 D UE4 : CFVuforiaLog:Warning: CameraActive 61.217457
07-14 14:20:33.613 23401 23417 D UE4 : [2017.07.14-18.20.33:613][ 0]CFVuforiaLog:Warning: CameraActive 61.217457
07-14 14:20:33.617 23401 23417 D UE4 : CFVuforiaLog:Warning: UVuforiaVideoPlaneComponent::Init 37862.652344, 57558.425781
07-14 14:20:33.618 23401 23417 D UE4 : [2017.07.14-18.20.33:617][ 0]CFVuforiaLog:Warning: UVuforiaVideoPlaneComponent::Init 37862.652344, 57558.425781
07-14 14:20:36.486 23401 23580 D UE4 : CFVuforiaLog:Warning: setVideoBackgroundTexture succeded
07-14 14:20:36.486 23401 23417 D UE4 : CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.486 23401 23417 D UE4 : [2017.07.14-18.20.36:486][ 0]CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.491 23401 23417 D UE4 : [2017.07.14-18.20.36:491][ 0]CFVuforiaLog:Warning: setVideoBackgroundTexture succeded
07-14 14:20:36.502 23401 23417 D UE4 : CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.503 23401 23417 D UE4 : [2017.07.14-18.20.36:503][ 0]CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.532 23401 23417 D UE4 : CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.533 23401 23417 D UE4 : [2017.07.14-18.20.36:533][ 0]CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.562 23401 23417 D UE4 : CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.562 23401 23417 D UE4 : [2017.07.14-18.20.36:562][ 0]CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.576 23401 23417 D UE4 : CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.580 23401 23417 D UE4 : [2017.07.14-18.20.36:576][ 0]CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.622 23401 23417 D UE4 : CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.623 23401 23417 D UE4 : [2017.07.14-18.20.36:623][ 0]CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.630 23401 23417 D UE4 : CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.630 23401 23417 D UE4 : [2017.07.14-18.20.36:630][ 0]CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.653 23401 23417 D UE4 : CFVuforiaLog:Warning: updateVideoBackgroundTexture failed
07-14 14:20:36.654 23401 23417 D UE4 : [2017.07.14-18.20.36:654][ 0]CFVuforiaLog:Warning: updateVideoBackgroundTexture failed

[geovie]

Hi, thanks for testing our Plugin and your feedback!

1. Which version of Android are you using?
2. So your saying you can see the camera stream before the target gets tracked? What is the number of frames per second? Are they dropping as soon as the cone appears?
3. This is a known issue, we currently only support Landscape as the orientation usually does not matter in an AR app... If you need this, feel free to open another issue!
4. Could you attach a screenshot of this? Have you tried other Vuforia Apps? Is the tracking there working as expected?

[hectorC]

Thank you for your reply.

1. It's Android 7.0. This might be UE4 related. Unity triggers the system permission request dialog for the camera use when you start an app for the fist time (same device) but UE4 doesn't.
2. Yes, the camera stream is visible but when the target is detected it disappears and it doesn't come back after tracking is lost. I think at some point I saw it floating somewhere else below and perpendicular to the view as if the plane had the wrong orientation respect to the view camera. There are no frame rate drops and it stays at around 60FPS.
3. It's OK, landscape should be enough for our purposes.
4. Yes, I've done quite a few Vuforia projects using Unity (and also created a WebGL-Vuforia interface for native iOS apps) and the tracking seemed off to me but without the video background is hard to tell. I can confirm this point once we get the video background disappearance sorted out.

[elitap]

Hi hectorC,
thank you for testing and bringing that up.

1. That might be it and that should be further investigated. For now, can you manually trigger the camera permission, in the app settings after the installation?
2. I assume that the camera stream, that you see is the one rendered by Vuforia. In order to get the stream with a high frame rate, they render it in a somewhat deep surface view (that normally isn't visible unless something on the startup goes wrong). But that stream usually does not have the same aspect as your screen, so it appears distorted. Maybe you can have a look at that? This behaviour would actually also explain the behaviour of 4).

Hth and keep us posted, we try to do our best to help.

[geovie]

In my opinion the problem is in this line, could you try and comment it out (The cone will not appear, but the camera stream should remain).
I think the problem is related to the mounting of the hardware camera (tablet vs. phone). Maybe someone could test it with another tablet...
BTW which version of the Unreal Engine are you using?

[hectorC]

That is correct. By commenting out that line the cone didn't appear and the camera stream remained. I'm using UE4 4.16.2. Could it be that the cone and the video background plane are for some reason not aligned properly in the world space?

[geovie]

Hi,
could you try the changes in branch fix/i1.

Maybe it solves the issue...

[hectorC]

Thank you! Yes, that fixed it.

Two notes:

1. Tracking seems to lag a little. The frame count shows a steady ~60 fps but when moving the tablet around, the cone in the sample scene doesn't appear firmly attached to the target and seems to be constantly catching up. Maybe there's something in the rendering order of the video plane and the scene or the synchronization with the Vuforia data frame that needs to be fixed? The same tablet with Vuforia running in Unity shows solid object placement.
2. A minor improvement but I think important for first time user experience: The included sample level needs the cone object to be moved up by 50cm, otherwise it is not sitting flat on the zero plane and it appears as if it's drifting in relationship to the target when viewed in the device.