Republishing multiple cameras has unexpected results

[andrewbest-tri]

This issue is probably a few separate issues and can be cleaned up, but I need to get the facts down and we can create smaller issues from this one.

Setup:
Foxy on ubuntu, latest apt packages
3 cameras:

• 1 and 2 using https://github.com/anqixu/ueye_cam
• 3 using https://github.com/clydemcqueen/gscam2

My desire is to run a single compressor if possible which remaps the /cameraX/image_raw to its compressed version automatically without complex topic remapping.

I have 3 problems which manifest in strange ways depending on my configuration.

Problem 1: image_republish doesn't detect that gscam cameras are the kinds of things it can compress automatically
If i run with ros2 run image_transport republish raw, Camera1 and Camera2 get the correct new topics (they don;t work but they get the correct remapping, see below). However, Camera3 does not. It is as if image_transport does not recognize that it can compress the gscamera feed? So, I run 2 compressors, one with no args and one with ros2 run image_transport republish raw --ros-args -r in:=/camera3/image_raw -r out/compressed:=/camera3/image_raw/compressed

Which leads to

Problem 2: either image_transport doesn't forward its "subscription" or the camera driver ueye_cam reads its subscribers wrong. Basically, if only the raw compressor is running, the driver never turns on the camera. I think this may be related to these issues:

• Enable "lazy" publishers and subscribers #111
• [ROS2] image_transport::CameraPublisher::getNumSubscribers not working #114

So, with default node... i get the right topic but no data in them since the camera doesn't know it has a subscriber.

Basically, I cannot use the default. So I run 3 compressors

ros2 run image_transport republish raw  --ros-args -r in:=/camera1/image_raw -r out/compressed:=/camera1/image_raw/compressed
ros2 run image_transport republish raw  --ros-args -r in:=/camera2/image_raw -r out/compressed:=/camera2/image_raw/compressed
ros2 run image_transport republish raw  --ros-args -r in:=/camera3/image_raw -r out/compressed:=/camera3/image_raw/compressed

which causes

Problem 3: The /out topic exists after remapping and contains ALL camera feeds

Once I run the command above... i get /out with the feed of all cameras transmitting simultaneously. This affects ros bag -a because it will subscribe to an extremely large and non-nonsensical camera topic.

[gbiggs]

Reading through the issue, I think I need some bag files so I can debug what's going wrong.

• I don't think I can reproduce problem 1 without a bag as I don't have a gscam2 camera, and I think it depends on the output of that driver
• Problem two is probably in the ueye_cam driver rather than in image_transport.
• Problem 3 needs a bag to reproduce with

If you can get some bags for me that contain the topics published by the cameras, I think I can dig into how image_transport is handling the camera outputs and find what's going wrong.

[gbiggs]

I think you might be providing the parameters to the image_transport command incorrectly. My reading of the documentation and source is that you need to run one instance per camera topic you want to republish. I think that you also need to provide the parameters for the in and out topics like this:

ros2 run image_transport republish raw in:=/camera_overview/image_raw out:=/camera_overview/compressed

Despite the warning that this is old remapping syntax, it's how the node wants its parameters.

Try these three commands:

ros2 run image_transport republish raw in:=/camera_overview/image_raw out:=/camera_overview/compressed
ros2 run image_transport republish raw in:=/camera_ueye_facing/driver_facing/image_raw out:=/camera_ueye_facing/driver_facing/compressed
ros2 run image_transport republish raw in:=/camera_ueye_shoulder/driver_shoulder/image_raw out:=/camera_ueye_shoulder/driver_shoulder/compressed

[clydemcqueen]

@gbiggs That looks correct to me.

FYI that gscam2 does not use image_transport::CameraPublisher, which might be causing some confusion. https://github.com/ros-drivers/gscam does use image_transport::CameraPublisher.

[andrewbest-tri]

So, that was the syntax we used before. Here is the output from the node

ros2 run image_transport republish raw in:=/camera_overview/image_raw out:=/camera_overview/compressed
\[WARN] [1670274023.378750025] [rcl]: Found remap rule 'in:=/camera_overview/image_raw'. This syntax is deprecated. Use '--ros-args --remap in:=/camera_overview/image_raw' instead.
[WARN] [1670274023.378772289] [rcl]: Found remap rule 'out:=/camera_overview/compressed'. This syntax is deprecated. Use '--ros-args --remap out:=/camera_overview/compressed' instead.
[WARN] [1670274023.378973833] [rcl]: Found remap rule 'in:=/camera_overview/image_raw'. This syntax is deprecated. Use '--ros-args --remap in:=/camera_overview/image_raw' instead.
[WARN] [1670274023.379016213] [rcl]: Found remap rule 'out:=/camera_overview/compressed'. This syntax is deprecated. Use '--ros-args --remap out:=/camera_overview/compressed' instead.

So it warns to not use this syntax.

Also, with the compressor installed, doing nothing at all, I still get this topic when i launch a camera
terminal1:

ros2 run ueye_cam standalone_node --ros-args --params-file src/motion_simulator/camera_launcher/config/camera_ueye_driver_facing.yaml 

terminal 2:
ros2 topic list | grep driver_facing
/ueye_cam/driver_facing/image_raw/compressed

ros2 topic list | grep driver_facing
/ueye_cam/driver_facing/camera_info
/ueye_cam/driver_facing/image_raw
/ueye_cam/driver_facing/image_raw/compressed
/ueye_cam/driver_facing/image_raw/compressedDepth
/ueye_cam/driver_facing/image_raw/theora

The compressor started automatically. /out is not present. there is no data on the compressed channel because that camera driver doesn't understand that it should start emiting data to the compressor by default.

Now, if I run the compressor.
terminal 2

ros2 run image_transport republish raw in:=/ueye_cam/driver_facing/image_raw out:=/ueye_cam/driver_facing/compressed
[WARN] [1670275188.025448030] [rcl]: Found remap rule 'in:=/ueye_cam/driver_facing/image_raw'. This syntax is deprecated. Use '--ros-args --remap in:=/ueye_cam/driver_facing/image_raw' instead.
[WARN] [1670275188.025467685] [rcl]: Found remap rule 'out:=/ueye_cam/driver_facing/compressed'. This syntax is deprecated. Use '--ros-args --remap out:=/ueye_cam/driver_facing/compressed' instead.
[WARN] [1670275188.029797129] [rcl]: Found remap rule 'in:=/ueye_cam/driver_facing/image_raw'. This syntax is deprecated. Use '--ros-args --remap in:=/ueye_cam/driver_facing/image_raw' instead.
[WARN] [1670275188.029838493] [rcl]: Found remap rule 'out:=/ueye_cam/driver_facing/compressed'. This syntax is deprecated. Use '--ros-args --remap out:=/ueye_cam/driver_facing/compressed' instead.

i list topics:

out/compressed
/out/compressedDepth
/out/theora
/parameter_events
/rosout
/ueye_cam/driver_facing/camera_info
/ueye_cam/driver_facing/compressed
/ueye_cam/driver_facing/image_raw
/ueye_cam/driver_facing/image_raw/compressed
/ueye_cam/driver_facing/image_raw/compressedDepth
/ueye_cam/driver_facing/image_raw/theora

out now exists, as does /out/compressed. /out has no data. /out/compressed does. So does /ueye_cam/driver_facing/compressed

Manually running the node started publishing out and outCompressed.

Now with gscam.. i had to muddle the config because its a pan to use.

Terminal 1

GSCAM_CONFIG="REDACTED" ros2 run gscam gscam_node --ros-args -r image_raw:=camera_overview/image_raw -r camera_info:=camera_overview/camera_info --params-file src/motion_simulator/camera_launcher/config/gscam_axis_overview.yaml 

Terminal 2 (no compressor running)

ros2 topic list
/camera/camera_info
/camera/image_raw
/camera/image_raw/compressed
/camera/image_raw/compressedDepth
/camera/image_raw/theora
/parameter_events
/rosout

The compressed topic in this case HAS data.

If I run the compressor.

Terminal 2

ros2 run image_transport republish raw in:=/camera/image_raw out:=/camera/compresed
[WARN] [1670275664.602530288] [rcl]: Found remap rule 'in:=/camera/image_raw'. This syntax is deprecated. Use '--ros-args --remap in:=/camera/image_raw' instead.
[WARN] [1670275664.602551122] [rcl]: Found remap rule 'out:=/camera/compresed'. This syntax is deprecated. Use '--ros-args --remap out:=/camera/compresed' instead.
[WARN] [1670275664.606922924] [rcl]: Found remap rule 'in:=/camera/image_raw'. This syntax is deprecated. Use '--ros-args --remap in:=/camera/image_raw' instead.
[WARN] [1670275664.606958315] [rcl]: Found remap rule 'out:=/camera/compresed'. This syntax is deprecated. Use '--ros-args --remap out:=/camera/compresed' instead.

Terminal 3 now sees

/camera/camera_info
/camera/compresed
/camera/image_raw
/camera/image_raw/compressed
/camera/image_raw/compressedDepth
/camera/image_raw/theora
/out/compressed
/out/compressedDepth
/out/theora
/parameter_events
/rosout

Notice that out appears again... and out/compressed has data

So, if I ran both compressors they would both publish to /out/compressed replicating my issue.

[andrewbest-tri]

TLDR: I can replicate the issue using the recommended syntax across two different camera drivers.

[clydemcqueen]

You may want to check out GSCam. It is officially supported and uses image_transport, so it supports compression without using a republish node. The only feature GSCam2 provides that GSCam does not is intra-process communication.

It looks like ueye_cam for Foxy also uses image_transport, so it should also support compression without a republish node.