Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(control_evaluator): add a trigger to choice whether to output metrics to log folder #9478

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

xtk8532704
Copy link
Contributor

@xtk8532704 xtk8532704 commented Nov 26, 2024

Description

  1. Add a bool parameter output_metrics, if it's true, the control_evaluator node writes the statics of the metrics measured during its lifetime to <ros2_logging_directory>/autoware_metrics/<node_name>-<time_stamp>.json when shut down.
    This feature will be used to collect and analyze test results in real vehicle tests and evaluator tests.
  2. Refactored control_evaluator's launch method to make it clearer: it now launches from autoware_control_evaluator/launch/control_evaluator.launch.xml, and can read param/control_evaluator.defaults.yaml correctly.

This feature will be used to collect and analyze test results in real vehicle tests and evaluator tests.

Related links

Parent Issue:

  • Link

How was this PR tested?

Psim with output_metrics=true:

$ cat /home/temkeikem/.ros/log/autoware_metrics/autoware_control_evaluator-2024-11-27-00-28-32.json 
{
    "goal_lateral_deviation/count": 0,
    "goal_lateral_deviation/description": "Lateral deviation from the goal point[m]",
    "goal_lateral_deviation/max": -1.7976931348623157e+308,
    "goal_lateral_deviation/mean": 0.0,
    "goal_lateral_deviation/min": 1.7976931348623157e+308,
    # ignore many outputs.
    "yaw_deviation/count": 227,
    "yaw_deviation/description": "Yaw deviation from the reference trajectory[rad]",
    "yaw_deviation/max": 0.06566750430257973,
    "yaw_deviation/mean": 0.031098377450999377,
    "yaw_deviation/min": 4.124327548060691e-05
}

Notes for reviewers

None.

Interface changes

None.

Effects on system behavior

None.

@github-actions github-actions bot added type:documentation Creating or refining documentation. (auto-assigned) component:launch Launch files, scripts and initialization tools. (auto-assigned) component:common Common packages from the autoware-common repository. (auto-assigned) component:evaluator Evaluation tools for planning, localization etc. (auto-assigned) labels Nov 26, 2024
Copy link

github-actions bot commented Nov 26, 2024

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

@xtk8532704 xtk8532704 force-pushed the xtk/metric-log-control-evaluator branch from c040dc5 to 542230a Compare November 26, 2024 15:44
@xtk8532704 xtk8532704 added the tag:run-build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Nov 26, 2024
Copy link

codecov bot commented Nov 26, 2024

Codecov Report

Attention: Patch coverage is 68.62745% with 32 lines in your changes missing coverage. Please review.

Project coverage is 29.44%. Comparing base (e5ad03a) to head (542230a).
Report is 8 commits behind head on main.

Files with missing lines Patch % Lines
...e_control_evaluator/src/control_evaluator_node.cpp 31.91% 30 Missing and 2 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #9478   +/-   ##
=======================================
  Coverage   29.44%   29.44%           
=======================================
  Files        1442     1445    +3     
  Lines      108558   108590   +32     
  Branches    41532    41545   +13     
=======================================
+ Hits        31967    31979   +12     
- Misses      73479    73497   +18     
- Partials     3112     3114    +2     
Flag Coverage Δ *Carryforward flag
differential 31.78% <68.62%> (?)
total 29.41% <ø> (-0.04%) ⬇️ Carriedforward from e5ad03a

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:common Common packages from the autoware-common repository. (auto-assigned) component:evaluator Evaluation tools for planning, localization etc. (auto-assigned) component:launch Launch files, scripts and initialization tools. (auto-assigned) tag:run-build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) type:documentation Creating or refining documentation. (auto-assigned)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant