The Calculation Method of "latency_ms" in Node "multi_object_tracker" Seems Unreasonable

[cyn-liu]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I'm convinced that this is not my fault but a bug.

Description

pipeline_latency_ms represents the time it takes for the entire pipeline from point cloud publishing to the completion of execution at the current node.

We found node /perception/object_recognition/tracking/multi_object_tracker's debug/pipeline_latency_ms is large and the data fluctuates greatly.

pipeline_latency.mp4

We believe that there is a problem with the calculation method of /perception/object_recognition/tracking/multi_object_tracker/debug/pipeline_latency_ms , which cannot actually reflect the pipeline latency from publishing the point cloud to completing the operation of node multi_object_tracker .

Expected behavior

/perception/object_recognition/tracking/multi_object_tracker/debug/pipeline_latency_ms is slightly large than /perception/object_recognition/detection/object_lanelet_filter/debug/pipeline_latency_ms.

Actual behavior

/perception/object_recognition/tracking/multi_object_tracker/debug/pipeline_latency_ms is much large than /perception/object_recognition/detection/object_lanelet_filter/debug/pipeline_latency_ms.

Steps to reproduce

1. Launch Autoware logging_simulator
2. Play ROS2 bag
3. Use rqt_plot to view /perception/object_recognition/tracking/multi_object_tracker/debug/pipeline_latency_ms

Versions

• OS: ubuntu22.04
• ROS2: humble
• Autoware: Latest version

Possible causes

1. Perhaps debug time should not be published in onTime.
2. When calculating time, use the timestamp of the oldest data in the input data queue, the amount of data in the queue often changes, resulting in significant fluctuations in the time calculation results.

Additional context

None

[technolojin]

@cyn-liu
What is your configuration of enable_delay_compensation?
If this value is true, it can extrapolate the tracking result (depends on your system situation), and the pipeline latency becomes large.

By turn enable_delay_compensation false, you can test if the pipeline-latency becomes stable.

[technolojin]

Related PR

#6710
#8657

[cyn-liu]

@cyn-liu What is your configuration of enable_delay_compensation? If this value is true, it can extrapolate the tracking result (depends on your system situation), and the pipeline latency becomes large.

By turn enable_delay_compensation false, you can test if the pipeline-latency becomes stable.

1. when enable_delay_compensation=true
   

2. when enable_delay_compensation=false
   

After testing, I believe that enable_delay_compensation=true is more stable, and enable_delay_compensation=false has a smaller vibration range.

As stated in the PR you provided, the reason for the instability of pipeline_latency_ms when enable_delay_compensation=false is the input (merged objects from multiple detectors) frequency is unstable. But as shown in the above figure, the range of variation of this value cannot be ignored.

[technolojin]

@cyn-liu
One of function of the enable_delay_compensation in the multi object tracker is to absorb the fluctuation of detection latency.

When the detection is not reached (over than 100 ms + margin), It publishes estimated tracking result. Since the estimation is done by old measurements, the pipeline latency is enlarged one additional cycle.
If this extrapolation is not done, the map based prediction and the planning module will not get any update.

Because of the extrapolation function and the fluctuating detection latency, the pipeline latency will fluctuating. In my understand, this is by design for now.

[technolojin]

@cyn-liu You may can have an experiment that disabling the extrapolation.

autoware.universe/perception/autoware_multi_object_tracker/src/multi_object_tracker_node.cpp

Line 242 in 5372403

should_publish = should_publish || elapsed_time > maximum_publish_interval;