Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Replace the CLI interface, `pbench-report-status` with a change to use
the command line `logger` program to send to `/dev/log` directly. This
has the unfortunate effect of causing a change to every legacy gold
file.
The change is broken down into a number of pieces described below:
1. Removal of the CLI command itself
This involves the expected changes to packaging:
server/Makefile
server/rpm/pbench-server.spec.j2
and its invocation from the various cron jobs entry points:
pbench-dispatch.sh
pbench-satellite-cleanup.sh
pbench-sync-satellite.sh
pbench-unpack-tarballs.sh
2. Removal of the `server/lib/pbench/report.py` module
A future commit will remove the indexing sub-system entirely, but in
order to avoid too many changes to that sub-system in this commit,
the `Report` class is simply moved to the `pbench-index.py` CLI
interface which becomes its only consumer with the removal of
`pbench-report-status`.
3. Removal of related test infrastructure
This includes `test-23` (`server/bin/unittests`), which was designed
to test the `syslog` behavior of `pbench-report-status`, along with
the tests setup, configuration, and gold files
(`server/bin/gold/test-23.txt` and `server/bin/state/test-23*`).
The mock `curl` command is also removed both because it is related
to `pbench-report-status` and it was unused (along with the
`_testcurlpayload` variable references).
4. Removal of `pbench-pp-status` which leveraged the stored reports
5. Use of `logger` to send records to `syslog` directly
This last piece of the puzzle is being done to enhance the cron jobs to
make their behaviors "observable" via a central log collection system
based on `rsyslog`. The reports were never that useful, but we take the
opportunity with these changes to enhance the final log messages to use
embedded JSON so that `rsyslog`'s `mmjsonparse` can pull data fields
from the logs and `mmelasticsearch` can index them into an Elastcsearch,
or OpenSearch, instance.
The `pbench-bash.sh` module no longer accepts a file name argument for
the `log_info`, `log_error`, and `log_exit` shell functions. The
`log_info` and `log_error` shell functions still `printf` to the current
log file, but also send its argument to `syslog` via the `logger -t
${PROG} -p daemon.<info|err>` command.
Each of the four cron jobs generate report JSON data based on the
following skeleton:
{
"pbench": {
"report": {
"summary": {
"end_ts": "<ending timestamp>",
"prog": "<cron job program name>",
"start_ts": "<starting timestamp>",
.
.
.
}
}
}
}
Each of the four cron jobs left add their own data files to the
"summary" report as follows (no attempt is taken to document their
meaning outside of review the code that emits them):
1. `pbench-dispatch`
{
"summary": {
...
"errors": ...,
"ndups": ...,
"npartialsucc": ...,
"nquarantined": ...,
"ntbs": ...,
"ntotal": ...
}
}
2. `pbench-satellite-cleanup`
{
"summary": {
"errors": ...,
"nhosts": ...,
"nincomingerrs": ...,
"nmd5errs": ...,
"nresultserrs": ...,
"nstateerrs": ...,
"ntbs": ...,
"ntberrs": ...
}
}
3. `pbench-sync-satellite`
{
"summary": {
"duration": ...,
"errors": ...,
"nhosts": ...,
"nprocessed": ...,
"ntotal": ...,
"remote_prefix": ...
}
}
4. `pbench-unpack-tarballs`
{
"summary": {
"errors": ...,
"nrechecks": ...,
"ntbs": ...,
"ntotal": ...,
"nwarn": ...
}
}
The changes to the legacy server test gold files are numerous because
most tests require the use of `pbench-unpack-tarballs` to operate.
Many of the changes stem from the use of `logger` output, including the
new use of embedded JSON.
The mock'd up `logger` command is removed, along with the
`_testloggerpayload` references, and a `bash` `alias` is setup for
`logger` to cause it to emit to `stderr` instead of `syslog` for testing
purposes.
A majority of the gold file changes are for the `test-7.*.txt` files.
Since `pbench-index.py` is still issuing reports, the typical changes
are the removal of the `pbench-unpack-tarballs` report, the addition of
the `pbench-unpack-tarballs` syslog summary entry, and slight change in
the syslog entries for the `Report` class moved into the
`pbench-index.py` CLI interface.
Most of the remaining gold files are pure removals of the reports no
longer posted, and the removal of gratuitous log entries.- Loading branch information
