Fix sensors system parallel updates #2201
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🦟 Bug fix
depends on #2200
Summary
While profiling the Sensors system, I noticed that the render updates are no longer happening in parallel but instead it's blocking the main thread.
The remotery graph below illustrate the issue. The
SimulationRunner
is being blocked for the entire duration of the sensors system's updateWith changes in this PR, the
SimulationRunner
will now continue to step while sensor updates take place:More info:
It came down to this lock that blocks the
PostUpdate
thread longer than it should. The fix is to use a new mutex (renderUtilMutex
) that's specificRenderUtil
updates.After reducing the scope of the lock, I noticed a race condition that sometimes the RenderUtil's states are overriden (in the
PostUpdate
thread) before the changes are applied to the scene (in the rendering thread). AnupdateTimeCv
condition variable is added to make sure that does not happen.To test:
Run the
INTEGRATION_sensors_system_update_rate
test multiple times to verify the sensors' timestamps are correct.On my desktop machine, the RTF values (displayed in the GUI) for the
sensors_demo.sdf
world are:Here is another test world consisting of 2 1980x1280 cameras @ 10Hz.
The raw RTF values over 2000 iterations are plotted in the graphs below.
Before:
RTF dips on every camera update
After:
There is still jitter but less than before
Checklist
codecheck
passed (See contributing)Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining
Signed-off-by
messages.