Skip to content

Commit

Permalink
Merge pull request #94 from PennyLaneAI/wae_reporting
Browse files Browse the repository at this point in the history
Warnings as error reporting
  • Loading branch information
mlxd authored Nov 13, 2024
2 parents 9560865 + 7cba722 commit 3ecca7d
Show file tree
Hide file tree
Showing 3 changed files with 111 additions and 0 deletions.
25 changes: 25 additions & 0 deletions .github/workflows/scripts/unique_warning_reporter.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash

# This file determines and counts the unique warnings that appear in the test suite using the
# warnings-as-errors action runs. The volume of each reported warning can be used to indicate
# the severity or importance of rectification.

JOBID=$(gh run list -w "Test-suite with Python warnings as errors" -L 1 --json databaseId -q '.[0].databaseId')
echo "View latest job at https://github.com/PennyLaneAI/pennylane/actions/runs/$JOBID"
gh run view $JOBID --log-failed >/tmp/job_$JOBID.out
cat /tmp/job_$JOBID.out | grep "Warning:" | awk '{split($0,a,"Warning:"); print a[1]"Warning"}' | awk '{split($0,a," - "); print a[2]}' | sort -u >unique_wae.txt

declare -A waeCounts

while read -r line; do
[[ -n "$line" && "$line" != [[:blank:]]* ]] && waeCounts["$line"]=$(cat /tmp/job_$JOBID.out | grep "$line" | wc -l)
done <unique_wae.txt
echo $waeCounts

jsonOut="[{"
for x in "${!waeCounts[@]}"; do
echo $x :: ${waeCounts[$x]}
jsonOut+="\"$x\":${waeCounts[$x]},"
done
jsonOut="${jsonOut::-1}}]"
echo $jsonOut >unique_wae.json
38 changes: 38 additions & 0 deletions .github/workflows/workflow-wae-reporting.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 48 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,54 @@ The following table shows the current Catalyst compatibility with the Lightning
</tr>
</table>

## Latest reported warnings

### PennyLane

<table>
<tr>
<td><code>UserWarning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24%5B0%5D.UserWarning&label=&color=9c6700></td>
</tr>
<tr>
<td><code>DeprecationWarning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24%5B0%5D.DeprecationWarning&label=&color=9c6700></td>
</tr>
<tr>
<td><code>pytest.PytestRemovedIn9Warning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24.%5B0%5D.%5B%22pytest.PytestRemovedIn9Warning%22%5D&label=&color=9c6700></td>
</tr>
<tr>
<td><code>PendingDeprecationWarning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24.%5B0%5D.%5B%22PendingDeprecationWarning%22%5D&label=&color=9c6700></td>
</tr>
<tr>
<td><code>pytest.PytestUnraisableExceptionWarning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24.%5B0%5D.%5B%22pytest.PytestUnraisableExceptionWarning%22%5D&label=&color=9c6700></td>
</tr>
<tr>
<td><code>numpy.exceptions.ComplexWarning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24.%5B0%5D.%5B%22numpy.exceptions.ComplexWarning%22%5D&label=&color=9c6700></td>
</tr>
<tr>
<td><code>RuntimeWarning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24.%5B0%5D.%5B%22RuntimeWarning%22%5D&label=&color=9c6700></td>
</tr>
<tr>
<td><code>pytest.PytestCollectionWarning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24.%5B0%5D.%5B%22pytest.PytestCollectionWarning%22%5D&label=&color=9c6700></td>
</tr>
<tr>
<td><code>FutureWarning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24.%5B0%5D.%5B%22FutureWarning%22%5D&label=&color=9c6700></td>
</tr>
<tr>
<td><code>pennylane.PennyLaneDeprecationWarning</code></td>
<td><a href=https://github.com/PennyLaneAI/pennylane/actions/workflows/package_warnings_as_errors.yml><img src=https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FPennyLaneAI%2Fplugin-test-matrix%2Frefs%2Ftags%2Fwae_data_tag%2Funique_wae.json&query=%24.%5B0%5D.%5B%22pennylane.PennyLaneDeprecationWarning%22%5D&label=&color=9c6700></td>
</tr>
</table>


## Interpreting the test matrix

* **Stable/stable**: This reflects a scenario where PennyLane and the plugin are both installed
Expand Down

0 comments on commit 3ecca7d

Please sign in to comment.