Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add page for statistical flight log analysis #3534

Merged
merged 2 commits into from
Jan 23, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions en/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -752,6 +752,7 @@
- [Binary Size Profiling](debug/binary_size_profiling.md)
- [Logging](dev_log/logging.md)
- [Flight Log Analysis](dev_log/flight_log_analysis.md)
- [Statistical Analysis](dev_log/flight_log_analysis_statistical.md)
- [ULog File Format](dev_log/ulog_file_format.md)
- [Log Encryption](dev_log/log_encryption.md)
- [Advanced Topics](advanced/index.md)
Expand Down
1 change: 1 addition & 0 deletions en/_sidebar.md
Original file line number Diff line number Diff line change
Expand Up @@ -741,6 +741,7 @@
- [Binary Size Profiling](/debug/binary_size_profiling.md)
- [Logging](/dev_log/logging.md)
- [Flight Log Analysis](/dev_log/flight_log_analysis.md)
- [Statistical Analysis](/dev_log/flight_log_analysis_statistical.md)
- [ULog File Format](/dev_log/ulog_file_format.md)
- [Log Encryption](dev_log/log_encryption.md)
- [Advanced Topics](/advanced/index.md)
Expand Down
1 change: 1 addition & 0 deletions en/dev_log/flight_log_analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ Information about collecting and analysing flight logs is covered in:
- [Flight Reporting](../getting_started/flight_reporting.md) - How to download a log and report/discuss issues about a flight.
- [Log Analysis using Flight Review](../log/flight_review.md) - How to analyse many common vehicle problems using the [Flight Review](https://logs.px4.io/) online tool.
- [Flight Log Analysis](../log/flight_log_analysis.md) - Introduction to flight analysis and links to a number of analysis tools.
- [Statistical Analysis](flight_log_analysis_statistical.md) - Information & resources for statistical analysis (including Flight Review logs).
23 changes: 23 additions & 0 deletions en/dev_log/flight_log_analysis_statistical.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Statistical Flight Log Analysis
Copy link
Collaborator

@hamishwillee hamishwillee Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bkueng It seems to me that this topic is really "how to download flight review logs", which can then be used for any purpose.
A topic named Statistical Flight Log analysis would seem to me to be about "how you go about doing such an analysis", and it might include information such as where to get public data sets, such as the ones in Flight review.

So proposing that we move this content into Log Analysis using Flight Review. This would be a section at the bottom "Downloading Public Flight Logs". In the introduction perhaps a link or note `Public Flight Review logs can be downloaded for statistical analysis or machine learning: See XXx.

What do you think? I can do this if you think my point is reasonable?

Suggested change
# Statistical Flight Log Analysis
# Statistical Flight Log Analysis

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it's a better description for what it is about.
My reasoning was that I wanted to maximize the chance for someone searching in general to find it (maybe not even knowing about flight review), and that the statistical analysis is the primary use-case for downloading those logs in bulk.
I also thought about adding it to the Log Analysis page, but concluded it's not a good fit as that page is specifically about analyzing a single (or specific set of) logs with the flight review plotting page.
But I'm also not specifically against adding it there if you still think it makes the most sense.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good points. I've left as is, but added an introduction to the page that makes it clear that this page could grow with other resources when we have them. Still discoverable, but now the title and content match better.


This topic contains information and resources related to statistical flight log analysis.

## Flight Review Public Logs

[Flight Review](../log/flight_log_analysis.md#flight-review-online-tool) hosts a large set of publicly available log files that can be used for statistical analysis, machine learning, or other purposes.

The dataset contains a set of different:

- vehicle types
- PX4 versions (including development versions)
- boards
- flight modes

The logs are accessible on [logs.px4.io/browse](https://logs.px4.io/browse) and are licensed under [CC-BY PX4](https://creativecommons.org/licenses/by/4.0/).

Log files can also be downloaded in bulk with the [download_logs.py](https://github.com/PX4/flight_review/blob/main/app/download_logs.py) script.
The script allows to filter by different attributes (like flight modes, airframe name or type).
Use the `--help` flag for a full list.
The newest logs will be downloaded first, and downloads can be interrupted and resumed later on.

There are different parsing libraries, for example [pyulog](../log/flight_log_analysis.md#pyulog) can be used to read logs with Python.
Loading