From deabf08e999bc47844148181f33704d1373c95e2 Mon Sep 17 00:00:00 2001 From: Peter Portante Date: Mon, 6 Mar 2023 17:38:13 -0500 Subject: [PATCH] Remove use of `pbench-report-status` Replace the CLI interface, `pbench-report-status` with a change to use the command line `logger` program to send to `/dev/log` directly. This has the unfortunate effect of causing a change to every legacy gold file. The change is broken down into a number of pieces described below: 1. Removal of the CLI command itself This involves the expected changes to packaging: server/Makefile server/rpm/pbench-server.spec.j2 and its invocation from the various cron jobs entry points: pbench-dispatch.sh pbench-satellite-cleanup.sh pbench-sync-satellite.sh pbench-unpack-tarballs.sh 2. Removal of the `server/lib/pbench/report.py` module A future commit will remove the indexing sub-system entirely, but in order to avoid too many changes to that sub-system in this commit, the `Report` class is simply moved to the `pbench-index.py` CLI interface which becomes its only consumer with the removal of `pbench-report-status`. 3. Removal of related test infrastructure This includes `test-23` (`server/bin/unittests`), which was designed to test the `syslog` behavior of `pbench-report-status`, along with the tests setup, configuration, and gold files (`server/bin/gold/test-23.txt` and `server/bin/state/test-23*`). The mock `curl` command is also removed both because it is related to `pbench-report-status` and it was unused (along with the `_testcurlpayload` variable references). 4. Removal of `pbench-pp-status` which leveraged the stored reports 5. Use of `logger` to send records to `syslog` directly This last piece of the puzzle is being done to enhance the cron jobs to make their behaviors "observable" via a central log collection system based on `rsyslog`. The reports were never that useful, but we take the opportunity with these changes to enhance the final log messages to use embedded JSON so that `rsyslog`'s `mmjsonparse` can pull data fields from the logs and `mmelasticsearch` can index them into an Elastcsearch, or OpenSearch, instance. The `pbench-bash.sh` module no longer accepts a file name argument for the `log_info`, `log_error`, and `log_exit` shell functions. The `log_info` and `log_error` shell functions still `printf` to the current log file, but also send its argument to `syslog` via the `logger -t ${PROG} -p daemon.` command. Each of the four cron jobs generate report JSON data based on the following skeleton: { "pbench": { "report": { "summary": { "end_ts": "", "prog": "", "start_ts": "", . . . } } } } Each of the four cron jobs left add their own data files to the "summary" report as follows (no attempt is taken to document their meaning outside of review the code that emits them): 1. `pbench-dispatch` { "summary": { ... "errors": ..., "ndups": ..., "npartialsucc": ..., "nquarantined": ..., "ntbs": ..., "ntotal": ... } } 2. `pbench-satellite-cleanup` { "summary": { "errors": ..., "nhosts": ..., "nincomingerrs": ..., "nmd5errs": ..., "nresultserrs": ..., "nstateerrs": ..., "ntbs": ..., "ntberrs": ... } } 3. `pbench-sync-satellite` { "summary": { "duration": ..., "errors": ..., "nhosts": ..., "nprocessed": ..., "ntotal": ..., "remote_prefix": ... } } 4. `pbench-unpack-tarballs` { "summary": { "errors": ..., "nrechecks": ..., "ntbs": ..., "ntotal": ..., "nwarn": ... } } The changes to the legacy server test gold files are numerous because most tests require the use of `pbench-unpack-tarballs` to operate. Many of the changes stem from the use of `logger` output, including the new use of embedded JSON. The mock'd up `logger` command is removed, along with the `_testloggerpayload` references, and a `bash` `alias` is setup for `logger` to cause it to emit to `stderr` instead of `syslog` for testing purposes. A majority of the gold file changes are for the `test-7.*.txt` files. Since `pbench-index.py` is still issuing reports, the typical changes are the removal of the `pbench-unpack-tarballs` report, the addition of the `pbench-unpack-tarballs` syslog summary entry, and slight change in the syslog entries for the `Report` class moved into the `pbench-index.py` CLI interface. Most of the remaining gold files are pure removals of the reports no longer posted, and the removal of gratuitous log entries. --- server/Makefile | 2 - server/bin/gold/test-10.txt | 38 +-- server/bin/gold/test-11.txt | 39 +-- server/bin/gold/test-12.txt | 35 +-- server/bin/gold/test-16.txt | 35 +-- server/bin/gold/test-17.txt | 35 +-- server/bin/gold/test-18.txt | 34 +-- server/bin/gold/test-19.txt | 35 +-- server/bin/gold/test-2.txt | 72 +---- server/bin/gold/test-21.txt | 34 +-- server/bin/gold/test-22.txt | 35 +-- server/bin/gold/test-23.txt | 134 --------- server/bin/gold/test-28.txt | 7 +- server/bin/gold/test-3.txt | 72 +---- server/bin/gold/test-4.txt | 77 +---- server/bin/gold/test-5.1.txt | 145 +--------- server/bin/gold/test-5.2.txt | 155 +---------- server/bin/gold/test-5.txt | 145 +--------- server/bin/gold/test-7.1.txt | 41 +-- server/bin/gold/test-7.10.txt | 47 +--- server/bin/gold/test-7.11.txt | 47 +--- server/bin/gold/test-7.12.txt | 47 +--- server/bin/gold/test-7.13.txt | 47 +--- server/bin/gold/test-7.14.txt | 47 +--- server/bin/gold/test-7.15.txt | 47 +--- server/bin/gold/test-7.16.txt | 47 +--- server/bin/gold/test-7.17.txt | 47 +--- server/bin/gold/test-7.18.txt | 41 +-- server/bin/gold/test-7.19.txt | 47 +--- server/bin/gold/test-7.2.0.txt | 41 +-- server/bin/gold/test-7.2.1.txt | 41 +-- server/bin/gold/test-7.20.txt | 47 +--- server/bin/gold/test-7.21.txt | 47 +--- server/bin/gold/test-7.22.txt | 47 +--- server/bin/gold/test-7.23.txt | 47 +--- server/bin/gold/test-7.24.txt | 47 +--- server/bin/gold/test-7.25.txt | 47 +--- server/bin/gold/test-7.26.txt | 6 +- server/bin/gold/test-7.3.txt | 41 +-- server/bin/gold/test-7.4.txt | 47 +--- server/bin/gold/test-7.5.txt | 47 +--- server/bin/gold/test-7.6.txt | 47 +--- server/bin/gold/test-7.7.txt | 47 +--- server/bin/gold/test-7.8.txt | 47 +--- server/bin/gold/test-7.9.txt | 47 +--- server/bin/pbench-base.sh | 21 +- server/bin/pbench-dispatch.sh | 66 +++-- server/bin/pbench-index.py | 262 +++++++++++++++++- server/bin/pbench-pp-status | 59 ---- server/bin/pbench-report-status | 1 - server/bin/pbench-report-status.py | 89 ------ server/bin/pbench-satellite-cleanup.sh | 81 +++--- server/bin/pbench-sync-satellite.sh | 103 ++++--- server/bin/pbench-unpack-tarballs.sh | 118 ++++---- .../state/test-23.config/pbench-server.cfg | 51 ---- server/bin/state/test-23.setup | 1 - server/bin/test-bin/curl | 18 -- server/bin/test-bin/logger | 6 - server/bin/unittests | 21 +- server/lib/pbench/report.py | 246 ---------------- server/rpm/pbench-server.spec.j2 | 6 +- 61 files changed, 688 insertions(+), 2788 deletions(-) delete mode 100644 server/bin/gold/test-23.txt delete mode 100755 server/bin/pbench-pp-status delete mode 120000 server/bin/pbench-report-status delete mode 100755 server/bin/pbench-report-status.py delete mode 100644 server/bin/state/test-23.config/pbench-server.cfg delete mode 120000 server/bin/state/test-23.setup delete mode 100755 server/bin/test-bin/curl delete mode 100755 server/bin/test-bin/logger delete mode 100644 server/lib/pbench/report.py diff --git a/server/Makefile b/server/Makefile index 66e8122df0..962272c667 100644 --- a/server/Makefile +++ b/server/Makefile @@ -25,7 +25,6 @@ binbase = \ # satellites only, but we install them everywhere. bintools = \ pbench-trampoline\ - pbench-pp-status\ pbench-server-activate-create-crontab\ pbench-remote-satellite-state-change\ pbench-remote-sync-package-tarballs\ @@ -43,7 +42,6 @@ binlinks = \ pbench-dispatch\ pbench-index\ pbench-reindex\ - pbench-report-status\ pbench-satellite-cleanup\ pbench-satellite-state-change\ pbench-sync-package-tarballs\ diff --git a/server/bin/gold/test-10.txt b/server/bin/gold/test-10.txt index a881339a55..21ee19c674 100644 --- a/server/bin/gold/test-10.txt +++ b/server/bin/gold/test-10.txt @@ -1,32 +1,4 @@ +++ Running pbench-sync-satellite satellite-one -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-sync-satellite", - "text": "pbench-sync-satellite.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nRemote ONE: processed 0 files and 0 errors.\n\nrun-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: duration (secs): 0\nrun-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors\n", - "total_chunks": 1, - "total_size": 528 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-sync-satellite (status=0) +++ Running unit test audit --- Finished unit test audit (status=1) @@ -105,7 +77,7 @@ drwxrwxr-x - logs/pbench-sync-satellite drwxrwxr-x - logs/pbench-sync-satellite/ONE -rw-rw-r-- 0 logs/pbench-sync-satellite/ONE/change_state.log -rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.error --rw-rw-r-- 849 logs/pbench-sync-satellite/pbench-sync-satellite.log +-rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -174,14 +146,6 @@ end-1970-01-01T00:00:42-UTC: archive hierarchy: /var/tmp/pbench-test-server/test +++++ pbench-sync-satellite/pbench-sync-satellite.error ----- pbench-sync-satellite/pbench-sync-satellite.error +++++ pbench-sync-satellite/pbench-sync-satellite.log -run-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: duration (secs): 0 -run-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-sync-satellite/pbench-sync-satellite.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-11.txt b/server/bin/gold/test-11.txt index 7b4834f485..86db963748 100644 --- a/server/bin/gold/test-11.txt +++ b/server/bin/gold/test-11.txt @@ -1,32 +1,4 @@ +++ Running pbench-sync-satellite satellite-one -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-sync-satellite", - "text": "pbench-sync-satellite.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 1 errors\nRemote ONE: processed 1 files and 1 errors.\n\nrun-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC\npbench-sync-satellite: completing satellite state changes ... (/var/tmp/pbench-test-server/test-11/pbench-local/logs/pbench-sync-satellite/ONE/change_state.log)\npbench-sync-satellite: completed satellite state changes\nrun-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: duration (secs): 0\nrun-1970-01-01T00:00:42-UTC: Total 1 files processed and 1 errors\n", - "total_chunks": 1, - "total_size": 746 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-sync-satellite (status=0) +++ Running unit test audit --- Finished unit test audit (status=0) @@ -94,7 +66,7 @@ drwxrwxr-x - logs/pbench-audit-server drwxrwxr-x - logs/pbench-sync-satellite drwxrwxr-x - logs/pbench-sync-satellite/ONE -rw-rw-r-- 508 logs/pbench-sync-satellite/pbench-sync-satellite.error --rw-rw-r-- 1021 logs/pbench-sync-satellite/pbench-sync-satellite.log +-rw-rw-r-- 547 logs/pbench-sync-satellite/pbench-sync-satellite.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - pbench-move-results-receive/fs-version-002/ONE::controller @@ -168,15 +140,8 @@ pbench-sync-satellite: completing satellite state changes ... (/var/tmp/pbench-t pbench-sync-satellite: completed satellite state changes ----- pbench-sync-satellite/pbench-sync-satellite.error +++++ pbench-sync-satellite/pbench-sync-satellite.log -run-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC mv: failed on /var/tmp/pbench-test-server/test-11/pbench-local/tmp/pbench-sync-satellite.XXXXX/unpack.ONE/controller/pbench-user-benchmark_38_2016-05-18_19:36:32.tar.xz ./ -run-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: duration (secs): 0 -run-1970-01-01T00:00:42-UTC: Total 1 files processed and 1 errors -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): ONE Processed 1 files (from 1 hosts, 2 total tar balls), with 1 errors, in 0 seconds -- @cee:{"pbench": {"report": {"summary": {"duration": 0, "end_ts": "1970-01-01T00:00:42-UTC", "errors": 1, "nhosts": 1, "nprocessed": 1, "ntotal": 2, "prog": "pbench-sync-satellite", "remote_prefix": "ONE", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-sync-satellite/pbench-sync-satellite.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-12.txt b/server/bin/gold/test-12.txt index 7b77e89587..1173759994 100644 --- a/server/bin/gold/test-12.txt +++ b/server/bin/gold/test-12.txt @@ -1,32 +1,4 @@ +++ Running pbench-dispatch -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-dispatch", - "text": "run-1970-01-01T00:00:42-UTC: SAT::controller/tarball_for-put-api_1970.01.01T00.00.00: success\nrun-1970-01-01T00:00:42-UTC: controller/tarball_for-put-api_1970.01.01T00.00.00: success\npbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 2 result tar balls, 2 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors.\n", - "total_chunks": 1, - "total_size": 372 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-dispatch (status=0) +++ Running unit test audit --- Finished unit test audit (status=0) @@ -131,7 +103,7 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-dispatch -rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.error --rw-rw-r-- 931 logs/pbench-dispatch/pbench-dispatch.log +-rw-rw-r-- 694 logs/pbench-dispatch/pbench-dispatch.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - pbench-move-results-receive/fs-version-002/SAT::controller @@ -189,14 +161,11 @@ drwxrwxr-x - tmp +++++ pbench-dispatch/pbench-dispatch.error ----- pbench-dispatch/pbench-dispatch.error +++++ pbench-dispatch/pbench-dispatch.log -run-1970-01-01T00:00:42-UTC tarball_for-put-api_1970.01.01T00.00.00.tar.xz: OK run-1970-01-01T00:00:42-UTC: SAT::controller/tarball_for-put-api_1970.01.01T00.00.00: success tarball_for-put-api_1970.01.01T00.00.00.tar.xz: OK run-1970-01-01T00:00:42-UTC: controller/tarball_for-put-api_1970.01.01T00.00.00: success -pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 2 result tar balls, 2 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors. -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 2 result tar balls, 2 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors. -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "ndups": 0, "npartialsucc": 0, "nquarantined": 0, "ntbs": 2, "ntotal": 2, "prog": "pbench-dispatch", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-dispatch/pbench-dispatch.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-16.txt b/server/bin/gold/test-16.txt index 22a1b6b7a1..fcb7784451 100644 --- a/server/bin/gold/test-16.txt +++ b/server/bin/gold/test-16.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs small -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs-small", - "text": "pbench-unpack-tarballs-small.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 1 errors\nProcessed 4 result tar balls, 3 successfully, 0 warnings, 1 errors, and 0 duplicates\n\nrun-1970-01-01T00:00:42-UTC: symlink target for /var/tmp/pbench-test-server/test-16/pbench/archive/fs-version-001/controller00/TO-UNPACK/benchmark-doesnotexist_1970-01-01T00:00:00.tar.xz does not exist\n", - "total_chunks": 1, - "total_size": 370 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running unit test audit --- Finished unit test audit (status=1) @@ -167,7 +139,7 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 359 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-unpack-tarballs-small -rw-rw-r-- 202 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error --rw-rw-r-- 1962 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log +-rw-rw-r-- 1783 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -235,7 +207,6 @@ end-1970-01-01T00:00:42-UTC: results hierarchy: /var/tmp/pbench-test-server/test run-1970-01-01T00:00:42-UTC: symlink target for /var/tmp/pbench-test-server/test-16/pbench/archive/fs-version-001/controller00/TO-UNPACK/benchmark-doesnotexist_1970-01-01T00:00:00.tar.xz does not exist ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error +++++ pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-16/pbench/public_html/incoming/controller01/benchmark-result-medium_1970-01-01T00:00:00 /var/tmp/pbench-test-server/test-16/pbench/public_html/results/controller01/prefix01/benchmark-result-medium_1970-01-01T00:00:00 ln -s /var/tmp/pbench-test-server/test-16/pbench/public_html/incoming/controller01/benchmark-result-medium_1970-01-01T00:00:00 /var/tmp/pbench-test-server/test-16/pbench/public_html/users/user01/controller01/prefix01/benchmark-result-medium_1970-01-01T00:00:00 run-1970-01-01T00:00:42-UTC: controller01/benchmark-result-medium_1970-01-01T00:00:00: success - elapsed time (secs): 0 - size (bytes): 3180 @@ -243,9 +214,7 @@ ln -s /var/tmp/pbench-test-server/test-16/pbench/public_html/incoming/controller run-1970-01-01T00:00:42-UTC: controller00/benchmark-result-large_1970-01-01T00:00:00: success - elapsed time (secs): 0 - size (bytes): 10240 ln -s /var/tmp/pbench-test-server/test-16/pbench/public_html/incoming/controller02/benchmark-result-small_1970-01-01T00:00:00 /var/tmp/pbench-test-server/test-16/pbench/public_html/results/controller02/benchmark-result-small_1970-01-01T00:00:00 run-1970-01-01T00:00:42-UTC: controller02/benchmark-result-small_1970-01-01T00:00:00: success - elapsed time (secs): 0 - size (bytes): 848 -run-1970-01-01T00:00:42-UTC: Processed 3 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 4 result tar balls, 3 successfully, with 0 rechecks, 0 warnings, and 1 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 1, "nrechecks": 0, "ntbs": 3, "ntotal": 4, "nwarn": 0, "prog": "pbench-unpack-tarballs-small", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-17.txt b/server/bin/gold/test-17.txt index dd74ba3737..7778cd5526 100644 --- a/server/bin/gold/test-17.txt +++ b/server/bin/gold/test-17.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs small -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs-small", - "text": "pbench-unpack-tarballs-small.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 1 errors\nProcessed 4 result tar balls, 3 successfully, 0 warnings, 1 errors, and 0 duplicates\n\nrun-1970-01-01T00:00:42-UTC: symlink target for /var/tmp/pbench-test-server/test-17/pbench/archive/fs-version-001/controller00/TO-UNPACK/benchmark-doesnotexist_1970-01-01T00:00:00.tar.xz does not exist\n", - "total_chunks": 1, - "total_size": 370 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running unit test audit --- Finished unit test audit (status=1) @@ -167,7 +139,7 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 359 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-unpack-tarballs-small -rw-rw-r-- 202 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error --rw-rw-r-- 1961 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log +-rw-rw-r-- 1782 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -235,7 +207,6 @@ end-1970-01-01T00:00:42-UTC: results hierarchy: /var/tmp/pbench-test-server/test run-1970-01-01T00:00:42-UTC: symlink target for /var/tmp/pbench-test-server/test-17/pbench/archive/fs-version-001/controller00/TO-UNPACK/benchmark-doesnotexist_1970-01-01T00:00:00.tar.xz does not exist ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error +++++ pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-17/pbench/public_html/incoming/controller01/benchmark-result-medium_1970-01-01T00:00:00 /var/tmp/pbench-test-server/test-17/pbench/public_html/results/controller01/prefix01/benchmark-result-medium_1970-01-01T00:00:00 ln -s /var/tmp/pbench-test-server/test-17/pbench/public_html/incoming/controller01/benchmark-result-medium_1970-01-01T00:00:00 /var/tmp/pbench-test-server/test-17/pbench/public_html/users/user01/controller01/prefix01/benchmark-result-medium_1970-01-01T00:00:00 run-1970-01-01T00:00:42-UTC: controller01/benchmark-result-medium_1970-01-01T00:00:00: success - elapsed time (secs): 0 - size (bytes): 3176 @@ -243,9 +214,7 @@ ln -s /var/tmp/pbench-test-server/test-17/pbench/public_html/incoming/controller run-1970-01-01T00:00:42-UTC: controller00/benchmark-result-large_1970-01-01T00:00:00: success - elapsed time (secs): 0 - size (bytes): 5920 ln -s /var/tmp/pbench-test-server/test-17/pbench/public_html/incoming/controller02/benchmark-result-small_1970-01-01T00:00:00 /var/tmp/pbench-test-server/test-17/pbench/public_html/results/controller02/benchmark-result-small_1970-01-01T00:00:00 run-1970-01-01T00:00:42-UTC: controller02/benchmark-result-small_1970-01-01T00:00:00: success - elapsed time (secs): 0 - size (bytes): 848 -run-1970-01-01T00:00:42-UTC: Processed 3 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 4 result tar balls, 3 successfully, with 0 rechecks, 0 warnings, and 1 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 1, "nrechecks": 0, "ntbs": 3, "ntotal": 4, "nwarn": 0, "prog": "pbench-unpack-tarballs-small", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-18.txt b/server/bin/gold/test-18.txt index 5275913a98..a964c7e302 100644 --- a/server/bin/gold/test-18.txt +++ b/server/bin/gold/test-18.txt @@ -32,34 +32,6 @@ File 4 of 4 (3 restored, 1 existing, so far), exists: /var/tmp/pbench-test-serve Restored 3 tar balls from backup volume, /var/tmp/pbench-test-server/test-18/pbench-local/archive.backup (ended at: 42.0, elapsed: 0.00) --- Finished pbench-restore-and-unpack-tarballs (status=0) +++ Running pbench-unpack-tarballs none re-unpack -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-re-unpack-tarballs", - "text": "pbench-re-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 0 result tar balls, 0 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 165 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running unit test audit --- Finished unit test audit (status=0) @@ -141,7 +113,7 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-re-unpack-tarballs -rw-rw-r-- 0 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error --rw-rw-r-- 528 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log +-rw-rw-r-- 0 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -197,10 +169,6 @@ drwxrwxr-x - tmp +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: Processed 0 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-19.txt b/server/bin/gold/test-19.txt index bd5053983a..cd06df13b0 100644 --- a/server/bin/gold/test-19.txt +++ b/server/bin/gold/test-19.txt @@ -33,34 +33,6 @@ File 4 of 4 (3 restored, 1 existing, so far), exists: /var/tmp/pbench-test-serve Restored 3 tar balls from backup volume, /var/tmp/pbench-test-server/test-19/pbench-local/archive.backup (ended at: 42.0, elapsed: 0.00) --- Finished pbench-restore-and-unpack-tarballs (status=0) +++ Running pbench-unpack-tarballs none re-unpack -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-re-unpack-tarballs", - "text": "pbench-re-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 3 result tar balls, 3 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 165 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running unit test audit --- Finished unit test audit (status=0) @@ -188,7 +160,7 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-re-unpack-tarballs -rw-rw-r-- 0 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error --rw-rw-r-- 1590 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log +-rw-rw-r-- 1414 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -244,16 +216,13 @@ drwxrwxr-x - tmp +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-19/pbench/public_html/incoming/ctrl-00/tarball_config-01_1970.01.01T00.00.42 /var/tmp/pbench-test-server/test-19/pbench/public_html/results/ctrl-00/tarball_config-01_1970.01.01T00.00.42 run-1970-01-01T00:00:42-UTC: ctrl-00/tarball_config-01_1970.01.01T00.00.42: success - elapsed time (secs): 0 - size (bytes): 228 ln -s /var/tmp/pbench-test-server/test-19/pbench/public_html/incoming/ctrl-01/tarball_config-02_1970.01.01T00.00.41 /var/tmp/pbench-test-server/test-19/pbench/public_html/results/ctrl-01/tarball_config-02_1970.01.01T00.00.41 run-1970-01-01T00:00:42-UTC: ctrl-01/tarball_config-02_1970.01.01T00.00.41: success - elapsed time (secs): 0 - size (bytes): 228 ln -s /var/tmp/pbench-test-server/test-19/pbench/public_html/incoming/ctrl-00/tarball_config-00_1970.01.01T00.00.40 /var/tmp/pbench-test-server/test-19/pbench/public_html/results/ctrl-00/tarball_config-00_1970.01.01T00.00.40 run-1970-01-01T00:00:42-UTC: ctrl-00/tarball_config-00_1970.01.01T00.00.40: success - elapsed time (secs): 0 - size (bytes): 224 -run-1970-01-01T00:00:42-UTC: Processed 3 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 3 result tar balls, 3 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 3, "ntotal": 3, "nwarn": 0, "prog": "pbench-re-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-2.txt b/server/bin/gold/test-2.txt index 1a754b00c0..971ba14e72 100644 --- a/server/bin/gold/test-2.txt +++ b/server/bin/gold/test-2.txt @@ -1,62 +1,6 @@ +++ Running pbench-sync-satellite satellite-one -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-sync-satellite", - "text": "pbench-sync-satellite.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nRemote ONE: processed 0 files and 0 errors.\n\nrun-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: duration (secs): 0\nrun-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors\n", - "total_chunks": 1, - "total_size": 528 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-sync-satellite (status=0) +++ Running pbench-dispatch -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-dispatch", - "text": "pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors.\n", - "total_chunks": 1, - "total_size": 189 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-dispatch (status=0) +++ Running pbench-unpack-tarballs small pbench-unpack-tarballs-small: Bad INCOMING=/var/tmp/pbench-test-server/test-2/pbench/public_html/incoming @@ -152,7 +96,7 @@ drwxrwxr-x - logs/pbench-cull-unpacked-tarballs -rw-rw-r-- 225 logs/pbench-cull-unpacked-tarballs/pbench-cull-unpacked-tarballs.log drwxrwxr-x - logs/pbench-dispatch -rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.error --rw-rw-r-- 646 logs/pbench-dispatch/pbench-dispatch.log +-rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.log drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 177 logs/pbench-index-tool-data/pbench-index-tool-data.log @@ -161,7 +105,7 @@ drwxrwxr-x - logs/pbench-sync-satellite drwxrwxr-x - logs/pbench-sync-satellite/ONE -rw-rw-r-- 0 logs/pbench-sync-satellite/ONE/change_state.log -rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.error --rw-rw-r-- 849 logs/pbench-sync-satellite/pbench-sync-satellite.log +-rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -221,10 +165,6 @@ drwxrwxr-x - tmp +++++ pbench-dispatch/pbench-dispatch.error ----- pbench-dispatch/pbench-dispatch.error +++++ pbench-dispatch/pbench-dispatch.log -run-1970-01-01T00:00:42-UTC -pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors. -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-dispatch/pbench-dispatch.log +++++ pbench-index-tool-data/pbench-index-tool-data.log 1970-01-01T00:00:42.000000 ERROR pbench-index-tool-data.pbench-index main -- pbench-index-tool-data: Bad INCOMING=/var/tmp/pbench-test-server/test-2/pbench/public_html/incoming @@ -237,14 +177,6 @@ pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar b +++++ pbench-sync-satellite/pbench-sync-satellite.error ----- pbench-sync-satellite/pbench-sync-satellite.error +++++ pbench-sync-satellite/pbench-sync-satellite.log -run-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: duration (secs): 0 -run-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-sync-satellite/pbench-sync-satellite.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-21.txt b/server/bin/gold/test-21.txt index 776cd4f8c6..b27f2674be 100644 --- a/server/bin/gold/test-21.txt +++ b/server/bin/gold/test-21.txt @@ -1,32 +1,4 @@ +++ Running pbench-dispatch -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-dispatch", - "text": "pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors.\n", - "total_chunks": 1, - "total_size": 189 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-dispatch (status=0) +++ Running unit test audit --- Finished unit test audit (status=0) @@ -93,7 +65,7 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-dispatch -rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.error --rw-rw-r-- 646 logs/pbench-dispatch/pbench-dispatch.log +-rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -149,10 +121,6 @@ drwxrwxr-x - tmp +++++ pbench-dispatch/pbench-dispatch.error ----- pbench-dispatch/pbench-dispatch.error +++++ pbench-dispatch/pbench-dispatch.log -run-1970-01-01T00:00:42-UTC -pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors. -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-dispatch/pbench-dispatch.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-22.txt b/server/bin/gold/test-22.txt index 4d7de22495..2a5a77e24b 100644 --- a/server/bin/gold/test-22.txt +++ b/server/bin/gold/test-22.txt @@ -1,32 +1,4 @@ +++ Running pbench-satellite-cleanup -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-satellite-cleanup", - "text": "pbench-satellite-cleanup.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 total errors\nTotal 6 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors.\n\n", - "total_chunks": 1, - "total_size": 257 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-satellite-cleanup (status=0) +++ Running unit test audit --- Finished unit test audit (status=3) @@ -116,7 +88,7 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 1230 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-satellite-cleanup -rw-rw-r-- 0 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.error --rw-rw-r-- 742 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log +-rw-rw-r-- 522 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -207,10 +179,7 @@ end-1970-01-01T00:00:42-UTC: results hierarchy: /var/tmp/pbench-test-server/test +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.error ----- pbench-satellite-cleanup/pbench-satellite-cleanup.error +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.log -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup starting -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup ends: Total 6 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors. -1970-01-01T00:00:42.000000 DEBUG pbench-satellite-cleanup.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-satellite-cleanup.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Total 6 tar balls cleaned up (for 3 hosts), with 0 tar ball removal errors, 0 md5 file removal errors, 0 state change errors, 0 incoming directory removal errors, and 0 result directory removal errors. -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nhosts": 3, "nincomingerrs": 0, "nmd5errs": 0, "nresultserrs": 0, "nstateerrs": 0, "ntbs": 6, "ntberrs": 0, "prog": "pbench-satellite-cleanup", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-satellite-cleanup/pbench-satellite-cleanup.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-23.txt b/server/bin/gold/test-23.txt deleted file mode 100644 index 2bbc6c3c09..0000000000 --- a/server/bin/gold/test-23.txt +++ /dev/null @@ -1,134 +0,0 @@ -+++ Running pbench-dispatch ---- Finished pbench-dispatch (status=0) -+++ Running unit test audit ---- Finished unit test audit (status=0) -+++ var/www/html tree state (/var/tmp/pbench-test-server/test-23/var-www-html) -lrwxrwxrwx 63 incoming -> /var/tmp/pbench-test-server/test-23/pbench/public_html/incoming -drwxrwxr-x - pbench-results-host-info.versioned -lrwxrwxrwx 38 pbench-results-host-info.versioned/pbench-results-host-info.URL002 -> pbench-results-host-info.URL002.active --rw-rw-r-- 118 pbench-results-host-info.versioned/pbench-results-host-info.URL002.active --rw-rw-r-- 95 pbench-results-host-info.versioned/pbench-results-host-info.URL002.maint -lrwxrwxrwx 62 results -> /var/tmp/pbench-test-server/test-23/pbench/public_html/results -lrwxrwxrwx 61 static -> /var/tmp/pbench-test-server/test-23/pbench/public_html/static -lrwxrwxrwx 60 users -> /var/tmp/pbench-test-server/test-23/pbench/public_html/users ---- var/www/html tree state -+++ results host info (/var/tmp/pbench-test-server/test-23/var-www-html/pbench-results-host-info.versioned) -/var/tmp/pbench-test-server/test-23/var-www-html/pbench-results-host-info.versioned/pbench-results-host-info.URL002.active:pbench@pbench.example.com:/var/tmp/pbench-test-server/test-23/pbench-local/pbench-move-results-receive/fs-version-002 -/var/tmp/pbench-test-server/test-23/var-www-html/pbench-results-host-info.versioned/pbench-results-host-info.URL002.maint:MESSAGE===System Under Maintenance - please retry at a later time (unit-test-user@example.com) ---- results host info -+++ var/www/html-satellite tree state (/var/tmp/pbench-test-server/test-23/var-www-html-satellite) -lrwxrwxrwx 73 incoming -> /var/tmp/pbench-test-server/test-23/pbench-satellite/public_html/incoming -drwxrwxr-x - pbench-results-host-info.versioned -lrwxrwxrwx 38 pbench-results-host-info.versioned/pbench-results-host-info.URL002 -> pbench-results-host-info.URL002.active --rw-rw-r-- 138 pbench-results-host-info.versioned/pbench-results-host-info.URL002.active --rw-rw-r-- 95 pbench-results-host-info.versioned/pbench-results-host-info.URL002.maint -lrwxrwxrwx 72 results -> /var/tmp/pbench-test-server/test-23/pbench-satellite/public_html/results -lrwxrwxrwx 71 static -> /var/tmp/pbench-test-server/test-23/pbench-satellite/public_html/static -lrwxrwxrwx 70 users -> /var/tmp/pbench-test-server/test-23/pbench-satellite/public_html/users ---- var/www/html-satellite tree state -+++ results host info (/var/tmp/pbench-test-server/test-23/var-www-html-satellite/pbench-results-host-info.versioned) -/var/tmp/pbench-test-server/test-23/var-www-html-satellite/pbench-results-host-info.versioned/pbench-results-host-info.URL002.active:pbench@pbench-satellite.example.com:/var/tmp/pbench-test-server/test-23/pbench-satellite-local/pbench-move-results-receive/fs-version-002 -/var/tmp/pbench-test-server/test-23/var-www-html-satellite/pbench-results-host-info.versioned/pbench-results-host-info.URL002.maint:MESSAGE===System Under Maintenance - please retry at a later time (unit-test-user@example.com) ---- results host info -+++ pbench tree state (/var/tmp/pbench-test-server/test-23/pbench) -drwxrwxr-x - archive -drwxrwxr-x - archive/fs-version-001 -drwxrwxr-x - public_html -drwxrwxr-x - public_html/incoming -drwxrwxr-x - public_html/results -drwxrwxr-x - public_html/static -drwxrwxr-x - public_html/static/css -drwxrwxr-x - public_html/static/css/v0.2 -drwxrwxr-x - public_html/static/css/v0.2/css --rw-rw-r-- 308 public_html/static/css/v0.2/css/pbench_utils.css -drwxrwxr-x - public_html/static/css/v0.3 -drwxrwxr-x - public_html/static/css/v0.3/css --rw-rw-r-- 11798 public_html/static/css/v0.3/css/LICENSE.TXT --rw-rw-r-- 3663 public_html/static/css/v0.3/css/jschart.css -drwxrwxr-x - public_html/static/js -drwxrwxr-x - public_html/static/js/v0.2 -drwxrwxr-x - public_html/static/js/v0.2/js --rw-rw-r-- 9415 public_html/static/js/v0.2/js/app.js --rw-rw-r-- 5556 public_html/static/js/v0.2/js/pbench_utils.js -drwxrwxr-x - public_html/static/js/v0.3 -drwxrwxr-x - public_html/static/js/v0.3/js --rw-rw-r-- 11798 public_html/static/js/v0.3/js/LICENSE.TXT --rw-rw-r-- 143934 public_html/static/js/v0.3/js/jschart.js -drwxrwxr-x - public_html/users ---- pbench tree state -+++ pbench-local tree state (/var/tmp/pbench-test-server/test-23/pbench-local) -drwxrwxr-x - archive.backup -drwxrwxr-x - logs -drwxrwxr-x - logs/pbench-audit-server --rw-rw-r-- 0 logs/pbench-audit-server/pbench-audit-server.error --rw-rw-r-- 0 logs/pbench-audit-server/pbench-audit-server.log --rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt -drwxrwxr-x - logs/pbench-dispatch --rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.error --rw-rw-r-- 762 logs/pbench-dispatch/pbench-dispatch.log -drwxrwxr-x - pbench-move-results-receive -drwxrwxr-x - pbench-move-results-receive/fs-version-002 -drwxrwxr-x - quarantine -drwxrwxr-x - quarantine/duplicates-002 -drwxrwxr-x - quarantine/errors-002 -drwxrwxr-x - quarantine/md5-002 -drwxrwxr-x - tmp ---- pbench-local tree state -+++ pbench-satellite tree state (/var/tmp/pbench-test-server/test-23/pbench-satellite) -drwxrwxr-x - archive -drwxrwxr-x - archive/fs-version-001 -drwxrwxr-x - public_html -drwxrwxr-x - public_html/incoming -drwxrwxr-x - public_html/results -drwxrwxr-x - public_html/static -drwxrwxr-x - public_html/static/css -drwxrwxr-x - public_html/static/css/v0.2 -drwxrwxr-x - public_html/static/css/v0.2/css --rw-rw-r-- 308 public_html/static/css/v0.2/css/pbench_utils.css -drwxrwxr-x - public_html/static/css/v0.3 -drwxrwxr-x - public_html/static/css/v0.3/css --rw-rw-r-- 11798 public_html/static/css/v0.3/css/LICENSE.TXT --rw-rw-r-- 3663 public_html/static/css/v0.3/css/jschart.css -drwxrwxr-x - public_html/static/js -drwxrwxr-x - public_html/static/js/v0.2 -drwxrwxr-x - public_html/static/js/v0.2/js --rw-rw-r-- 9415 public_html/static/js/v0.2/js/app.js --rw-rw-r-- 5556 public_html/static/js/v0.2/js/pbench_utils.js -drwxrwxr-x - public_html/static/js/v0.3 -drwxrwxr-x - public_html/static/js/v0.3/js --rw-rw-r-- 11798 public_html/static/js/v0.3/js/LICENSE.TXT --rw-rw-r-- 143934 public_html/static/js/v0.3/js/jschart.js -drwxrwxr-x - public_html/users ---- pbench-satellite tree state -+++ pbench-satellite-local tree state (/var/tmp/pbench-test-server/test-23/pbench-satellite-local) -drwxrwxr-x - logs -drwxrwxr-x - pbench-move-results-receive -drwxrwxr-x - pbench-move-results-receive/fs-version-002 -drwxrwxr-x - quarantine -drwxrwxr-x - quarantine/duplicates-002 -drwxrwxr-x - quarantine/errors-002 -drwxrwxr-x - quarantine/md5-002 -drwxrwxr-x - tmp ---- pbench-satellite-local tree state -+++ pbench log file contents -++++ pbench-local/logs -+++++ pbench-audit-server/pbench-audit-server.error ------ pbench-audit-server/pbench-audit-server.error -+++++ pbench-audit-server/pbench-audit-server.log ------ pbench-audit-server/pbench-audit-server.log -+++++ pbench-audit-server/report.latest.txt ------ pbench-audit-server/report.latest.txt -+++++ pbench-dispatch/pbench-dispatch.error ------ pbench-dispatch/pbench-dispatch.error -+++++ pbench-dispatch/pbench-dispatch.log -run-1970-01-01T00:00:42-UTC -pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors. -1970-01-01T00:00:42.000000 INFO pbench-dispatch.report post_status -- @cee:{"@generated-by": {"commit_id": "unit-test", "group_id": 43, "hostname": "example.com", "pid": 42, "user_id": 44, "version": ""}, "@timestamp": "1970-01-01T00:00:42", "chunk_id": 1, "doctype": "status", "name": "pbench-dispatch", "text": "pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors.\n", "total_chunks": 1, "total_size": 189} ------ pbench-dispatch/pbench-dispatch.log ----- pbench-local/logs -++++ pbench-satellite-local/logs ----- pbench-satellite-local/logs ---- pbench log file contents -+++ test-execution.log file contents -hostname -f ---- test-execution.log file contents diff --git a/server/bin/gold/test-28.txt b/server/bin/gold/test-28.txt index f8dc31ca04..17c421e896 100644 --- a/server/bin/gold/test-28.txt +++ b/server/bin/gold/test-28.txt @@ -65,8 +65,8 @@ drwxrwxr-x - logs/pbench-audit-server drwxrwxr-x - logs/pbench-sync-satellite drwxrwxr-x - logs/pbench-sync-satellite/ONE -rw-rw-r-- 0 logs/pbench-sync-satellite/ONE/change_state.log --rw-rw-r-- 105 logs/pbench-sync-satellite/pbench-sync-satellite.error --rw-rw-r-- 61 logs/pbench-sync-satellite/pbench-sync-satellite.log +-rw-rw-r-- 136 logs/pbench-sync-satellite/pbench-sync-satellite.error +-rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -122,10 +122,9 @@ drwxrwxr-x - tmp +++++ pbench-sync-satellite/ONE/change_state.log ----- pbench-sync-satellite/ONE/change_state.log +++++ pbench-sync-satellite/pbench-sync-satellite.error -pbench-sync-satellite: FAILED: ssh: connect to host pbench-satellite-error.example.com: No route to host +pbench-sync-satellite: run-1970-01-01T00:00:42-UTC: FAILED -- ssh: connect to host pbench-satellite-error.example.com: No route to host ----- pbench-sync-satellite/pbench-sync-satellite.error +++++ pbench-sync-satellite/pbench-sync-satellite.log -run-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC ----- pbench-sync-satellite/pbench-sync-satellite.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-3.txt b/server/bin/gold/test-3.txt index fb420eaf21..a55d77b37e 100644 --- a/server/bin/gold/test-3.txt +++ b/server/bin/gold/test-3.txt @@ -1,62 +1,6 @@ +++ Running pbench-sync-satellite satellite-one -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-sync-satellite", - "text": "pbench-sync-satellite.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nRemote ONE: processed 0 files and 0 errors.\n\nrun-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: duration (secs): 0\nrun-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors\n", - "total_chunks": 1, - "total_size": 528 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-sync-satellite (status=0) +++ Running pbench-dispatch -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-dispatch", - "text": "pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors.\n", - "total_chunks": 1, - "total_size": 189 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-dispatch (status=0) +++ Running pbench-unpack-tarballs small pbench-unpack-tarballs-small: Bad RESULTS=/var/tmp/pbench-test-server/test-3/pbench/public_html/results @@ -150,7 +94,7 @@ drwxrwxr-x - logs/pbench-cull-unpacked-tarballs -rw-rw-r-- 223 logs/pbench-cull-unpacked-tarballs/pbench-cull-unpacked-tarballs.log drwxrwxr-x - logs/pbench-dispatch -rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.error --rw-rw-r-- 646 logs/pbench-dispatch/pbench-dispatch.log +-rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.log drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log @@ -159,7 +103,7 @@ drwxrwxr-x - logs/pbench-sync-satellite drwxrwxr-x - logs/pbench-sync-satellite/ONE -rw-rw-r-- 0 logs/pbench-sync-satellite/ONE/change_state.log -rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.error --rw-rw-r-- 849 logs/pbench-sync-satellite/pbench-sync-satellite.log +-rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -215,10 +159,6 @@ drwxrwxr-x - tmp +++++ pbench-dispatch/pbench-dispatch.error ----- pbench-dispatch/pbench-dispatch.error +++++ pbench-dispatch/pbench-dispatch.log -run-1970-01-01T00:00:42-UTC -pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors. -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-dispatch/pbench-dispatch.log +++++ pbench-index-tool-data/pbench-index-tool-data.log 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: starting @@ -233,14 +173,6 @@ pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar b +++++ pbench-sync-satellite/pbench-sync-satellite.error ----- pbench-sync-satellite/pbench-sync-satellite.error +++++ pbench-sync-satellite/pbench-sync-satellite.log -run-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: duration (secs): 0 -run-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-sync-satellite/pbench-sync-satellite.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-4.txt b/server/bin/gold/test-4.txt index 6cbe5d39d6..cf43af4d42 100644 --- a/server/bin/gold/test-4.txt +++ b/server/bin/gold/test-4.txt @@ -1,62 +1,6 @@ +++ Running pbench-sync-satellite satellite-one -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-sync-satellite", - "text": "pbench-sync-satellite.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nRemote ONE: processed 0 files and 0 errors.\n\nrun-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: duration (secs): 0\nrun-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors\n", - "total_chunks": 1, - "total_size": 528 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-sync-satellite (status=0) +++ Running pbench-dispatch -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-dispatch", - "text": "pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors.\n", - "total_chunks": 1, - "total_size": 189 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-dispatch (status=0) +++ Running pbench-unpack-tarballs small pbench-unpack-tarballs-small: Bad USERS=/var/tmp/pbench-test-server/test-4/pbench/public_html/users @@ -151,7 +95,7 @@ drwxrwxr-x - logs/pbench-cull-unpacked-tarballs -rw-rw-r-- 219 logs/pbench-cull-unpacked-tarballs/pbench-cull-unpacked-tarballs.log drwxrwxr-x - logs/pbench-dispatch -rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.error --rw-rw-r-- 646 logs/pbench-dispatch/pbench-dispatch.log +-rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.log drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log @@ -160,7 +104,7 @@ drwxrwxr-x - logs/pbench-sync-satellite drwxrwxr-x - logs/pbench-sync-satellite/ONE -rw-rw-r-- 0 logs/pbench-sync-satellite/ONE/change_state.log -rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.error --rw-rw-r-- 849 logs/pbench-sync-satellite/pbench-sync-satellite.log +-rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -199,7 +143,7 @@ drwxrwxr-x - public_html/static/js/v0.3/js drwxrwxr-x - logs drwxrwxr-x - logs/pbench-satellite-cleanup -rw-rw-r-- 0 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.error --rw-rw-r-- 928 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log +-rw-rw-r-- 0 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log drwxrwxr-x - logs/pbench-sync-package-tarballs -rw-rw-r-- 0 logs/pbench-sync-package-tarballs/pbench-sync-package-tarballs.error -rw-rw-r-- 0 logs/pbench-sync-package-tarballs/pbench-sync-package-tarballs.log @@ -223,10 +167,6 @@ drwxrwxr-x - tmp +++++ pbench-dispatch/pbench-dispatch.error ----- pbench-dispatch/pbench-dispatch.error +++++ pbench-dispatch/pbench-dispatch.log -run-1970-01-01T00:00:42-UTC -pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors. -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-dispatch/pbench-dispatch.log +++++ pbench-index-tool-data/pbench-index-tool-data.log 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: starting @@ -241,23 +181,12 @@ pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar b +++++ pbench-sync-satellite/pbench-sync-satellite.error ----- pbench-sync-satellite/pbench-sync-satellite.error +++++ pbench-sync-satellite/pbench-sync-satellite.log -run-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: duration (secs): 0 -run-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-sync-satellite/pbench-sync-satellite.log ---- pbench-local/logs ++++ pbench-satellite-local/logs +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.error ----- pbench-satellite-cleanup/pbench-satellite-cleanup.error +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.log -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup starting -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup ends: Total 0 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors. -1970-01-01T00:00:42.000000 INFO pbench-satellite-cleanup.report post_status -- @cee:{"@generated-by": {"commit_id": "unit-test", "group_id": 43, "hostname": "example.com", "pid": 42, "user_id": 44, "version": ""}, "@timestamp": "1970-01-01T00:00:42", "chunk_id": 1, "doctype": "status", "name": "pbench-satellite-cleanup", "text": "pbench-satellite-cleanup.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 total errors\nTotal 0 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors.\n\n", "total_chunks": 1, "total_size": 257} ----- pbench-satellite-cleanup/pbench-satellite-cleanup.log +++++ pbench-sync-package-tarballs/pbench-sync-package-tarballs.error ----- pbench-sync-package-tarballs/pbench-sync-package-tarballs.error diff --git a/server/bin/gold/test-5.1.txt b/server/bin/gold/test-5.1.txt index 0517a5a915..ebea7fc305 100644 --- a/server/bin/gold/test-5.1.txt +++ b/server/bin/gold/test-5.1.txt @@ -1,126 +1,14 @@ +++ Running pbench-sync-satellite satellite-one -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-sync-satellite", - "text": "pbench-sync-satellite.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nRemote ONE: processed 0 files and 0 errors.\n\nrun-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: duration (secs): 0\nrun-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors\n", - "total_chunks": 1, - "total_size": 528 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-sync-satellite (status=0) +++ Running pbench-dispatch -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-dispatch", - "text": "pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors.\n", - "total_chunks": 1, - "total_size": 189 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-dispatch (status=0) +++ Running pbench-unpack-tarballs small -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs-small", - "text": "pbench-unpack-tarballs-small.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 0 result tar balls, 0 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 168 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index --- Finished pbench-index (status=0) +++ Running pbench-index --tool-data --- Finished pbench-index (status=0) +++ Running pbench-unpack-tarballs none re-unpack -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-re-unpack-tarballs", - "text": "pbench-re-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 0 result tar balls, 0 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 165 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-clean-up-dangling-results-links --- Finished pbench-clean-up-dangling-results-links (status=0) @@ -208,22 +96,22 @@ drwxrwxr-x - logs/pbench-cull-unpacked-tarballs -rw-rw-r-- 174 logs/pbench-cull-unpacked-tarballs/pbench-cull-unpacked-tarballs.log drwxrwxr-x - logs/pbench-dispatch -rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.error --rw-rw-r-- 646 logs/pbench-dispatch/pbench-dispatch.log +-rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.log drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log -rw-rw-r-- 224 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-re-unpack-tarballs -rw-rw-r-- 0 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error --rw-rw-r-- 528 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log +-rw-rw-r-- 0 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log drwxrwxr-x - logs/pbench-sync-satellite drwxrwxr-x - logs/pbench-sync-satellite/ONE -rw-rw-r-- 0 logs/pbench-sync-satellite/ONE/change_state.log -rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.error --rw-rw-r-- 849 logs/pbench-sync-satellite/pbench-sync-satellite.log +-rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.log drwxrwxr-x - logs/pbench-unpack-tarballs-small -rw-rw-r-- 0 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error --rw-rw-r-- 534 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log +-rw-rw-r-- 0 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -262,7 +150,7 @@ drwxrwxr-x - public_html/users drwxrwxr-x - logs drwxrwxr-x - logs/pbench-satellite-cleanup -rw-rw-r-- 0 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.error --rw-rw-r-- 928 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log +-rw-rw-r-- 0 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log drwxrwxr-x - logs/pbench-sync-package-tarballs -rw-rw-r-- 0 logs/pbench-sync-package-tarballs/pbench-sync-package-tarballs.error -rw-rw-r-- 0 logs/pbench-sync-package-tarballs/pbench-sync-package-tarballs.log @@ -292,10 +180,6 @@ drwxrwxr-x - tmp +++++ pbench-dispatch/pbench-dispatch.error ----- pbench-dispatch/pbench-dispatch.error +++++ pbench-dispatch/pbench-dispatch.log -run-1970-01-01T00:00:42-UTC -pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors. -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-dispatch/pbench-dispatch.log +++++ pbench-index-tool-data/pbench-index-tool-data.log 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: starting @@ -308,41 +192,22 @@ pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar b +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: Processed 0 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log +++++ pbench-sync-satellite/ONE/change_state.log ----- pbench-sync-satellite/ONE/change_state.log +++++ pbench-sync-satellite/pbench-sync-satellite.error ----- pbench-sync-satellite/pbench-sync-satellite.error +++++ pbench-sync-satellite/pbench-sync-satellite.log -run-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: duration (secs): 0 -run-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-sync-satellite/pbench-sync-satellite.log +++++ pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error +++++ pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log -run-1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: Processed 0 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log ---- pbench-local/logs ++++ pbench-satellite-local/logs +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.error ----- pbench-satellite-cleanup/pbench-satellite-cleanup.error +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.log -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup starting -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup ends: Total 0 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors. -1970-01-01T00:00:42.000000 INFO pbench-satellite-cleanup.report post_status -- @cee:{"@generated-by": {"commit_id": "unit-test", "group_id": 43, "hostname": "example.com", "pid": 42, "user_id": 44, "version": ""}, "@timestamp": "1970-01-01T00:00:42", "chunk_id": 1, "doctype": "status", "name": "pbench-satellite-cleanup", "text": "pbench-satellite-cleanup.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 total errors\nTotal 0 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors.\n\n", "total_chunks": 1, "total_size": 257} ----- pbench-satellite-cleanup/pbench-satellite-cleanup.log +++++ pbench-sync-package-tarballs/pbench-sync-package-tarballs.error ----- pbench-sync-package-tarballs/pbench-sync-package-tarballs.error diff --git a/server/bin/gold/test-5.2.txt b/server/bin/gold/test-5.2.txt index 0ff070db12..4274c0e47f 100644 --- a/server/bin/gold/test-5.2.txt +++ b/server/bin/gold/test-5.2.txt @@ -1,92 +1,8 @@ +++ Running pbench-sync-satellite satellite-one -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-sync-satellite", - "text": "pbench-sync-satellite.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nRemote ONE: processed 3 files and 0 errors.\n\nrun-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC\npbench-sync-satellite: completing satellite state changes ... (/var/tmp/pbench-test-server/test-5.2/pbench-local/logs/pbench-sync-satellite/ONE/change_state.log)\npbench-sync-satellite: completed satellite state changes\nrun-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: duration (secs): 0\nrun-1970-01-01T00:00:42-UTC: Total 3 files processed and 0 errors\n", - "total_chunks": 1, - "total_size": 747 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-sync-satellite (status=0) +++ Running pbench-dispatch -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-dispatch", - "text": "run-1970-01-01T00:00:42-UTC: ONE::controllerA/tarball-simple1_1970-01-01T00:42:00: success\nrun-1970-01-01T00:00:42-UTC: ONE::controllerB/tarball-simple2_1970-01-01T00:41:00: success\nrun-1970-01-01T00:00:42-UTC: ONE::controllerC/tarball-simple0-prefix_1970-01-01T00:42:00: success\nrun-1970-01-01T00:00:42-UTC: Quarantined: /var/tmp/pbench-test-server/test-5.2/pbench-local/pbench-move-results-receive/fs-version-002/controller-a-missing-tb/tarball_have-md5-no-tb_1970.01.01T00.00.00.tar.xz failed MD5 check\nrun-1970-01-01T00:00:42-UTC: controller-b-with-prefixes/tarball-0_1970.01.01T00.42.00: success\nrun-1970-01-01T00:00:42-UTC: controller-b-with-prefixes/tarball-w-dot-prefix_1970.01.01T00.42.00: success\nrun-1970-01-01T00:00:42-UTC: controller-b-with-prefixes/tarball-w-prefix-dot_1970.01.01T00.42.00: success\nrun-1970-01-01T00:00:42-UTC: Duplicate: /var/tmp/pbench-test-server/test-5.2/pbench-local/pbench-move-results-receive/fs-version-002/controller-d-duplicate/tarball-duplicate_1970.01.01T00.42.00.tar.xz duplicate name\nrun-1970-01-01T00:00:42-UTC: Quarantined: /var/tmp/pbench-test-server/test-5.2/pbench-local/pbench-move-results-receive/fs-version-002/controller-e-bad-md5/tarball-bad-md5_1970.01.01T00.42.00.tar.xz failed MD5 check\nrun-1970-01-01T00:00:42-UTC: Quarantined: /var/tmp/pbench-test-server/test-5.2/pbench-local/pbench-move-results-receive/fs-version-002/controller-f-bad-md5/tarball-bad-md5-1_1970.01.01T00.42.00.tar.xz failed MD5 check\nrun-1970-01-01T00:00:42-UTC: controller-g-normal/tarball-normal_1970.01.01T00.42.00: success\npbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 11 result tar balls, 7 successfully (0 partial), with 3 quarantined tar balls, 1 duplicately-named tar balls, and 0 errors.\n", - "total_chunks": 1, - "total_size": 1746 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-dispatch (status=0) +++ Running pbench-unpack-tarballs small -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs-small", - "text": "pbench-unpack-tarballs-small.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 1 errors\nProcessed 8 result tar balls, 7 successfully, 0 warnings, 1 errors, and 0 duplicates\n\nrun-1970-01-01T00:00:42-UTC: symlink target for /var/tmp/pbench-test-server/test-5.2/pbench/archive/fs-version-001/controller-d-duplicate/TO-UNPACK/tarball-that-does-not-exist_1970.01.01T00.42.00.tar.xz does not exist\n", - "total_chunks": 1, - "total_size": 386 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -163,34 +79,6 @@ len(actions) = 1 +++ Running pbench-index --tool-data --- Finished pbench-index (status=0) +++ Running pbench-unpack-tarballs none re-unpack -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-re-unpack-tarballs", - "text": "pbench-re-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 0 result tar balls, 0 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 165 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-clean-up-dangling-results-links --- Finished pbench-clean-up-dangling-results-links (status=0) @@ -463,21 +351,21 @@ drwxrwxr-x - logs/pbench-cull-unpacked-tarballs -rw-rw-r-- 174 logs/pbench-cull-unpacked-tarballs/pbench-cull-unpacked-tarballs.log drwxrwxr-x - logs/pbench-dispatch -rw-rw-r-- 876 logs/pbench-dispatch/pbench-dispatch.error --rw-rw-r-- 2081 logs/pbench-dispatch/pbench-dispatch.log +-rw-rw-r-- 1845 logs/pbench-dispatch/pbench-dispatch.log drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 7305 logs/pbench-index/pbench-index.log +-rw-rw-r-- 7317 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-re-unpack-tarballs -rw-rw-r-- 0 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error --rw-rw-r-- 528 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log +-rw-rw-r-- 0 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log drwxrwxr-x - logs/pbench-sync-satellite drwxrwxr-x - logs/pbench-sync-satellite/ONE -rw-rw-r-- 219 logs/pbench-sync-satellite/pbench-sync-satellite.error --rw-rw-r-- 849 logs/pbench-sync-satellite/pbench-sync-satellite.log +-rw-rw-r-- 375 logs/pbench-sync-satellite/pbench-sync-satellite.log drwxrwxr-x - logs/pbench-unpack-tarballs-small -rw-rw-r-- 218 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error --rw-rw-r-- 3302 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log +-rw-rw-r-- 3123 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - pbench-move-results-receive/fs-version-002/ONE::controllerA @@ -557,7 +445,7 @@ drwxrwxr-x - public_html/users drwxrwxr-x - logs drwxrwxr-x - logs/pbench-satellite-cleanup -rw-rw-r-- 0 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.error --rw-rw-r-- 928 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log +-rw-rw-r-- 522 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log drwxrwxr-x - logs/pbench-sync-package-tarballs -rw-rw-r-- 0 logs/pbench-sync-package-tarballs/pbench-sync-package-tarballs.error -rw-rw-r-- 0 logs/pbench-sync-package-tarballs/pbench-sync-package-tarballs.log @@ -591,7 +479,6 @@ run-1970-01-01T00:00:42-UTC: Quarantined: /var/tmp/pbench-test-server/test-5.2/p run-1970-01-01T00:00:42-UTC: Quarantined: /var/tmp/pbench-test-server/test-5.2/pbench-local/pbench-move-results-receive/fs-version-002/controller-f-bad-md5/tarball-bad-md5-1_1970.01.01T00.42.00.tar.xz failed MD5 check ----- pbench-dispatch/pbench-dispatch.error +++++ pbench-dispatch/pbench-dispatch.log -run-1970-01-01T00:00:42-UTC tarball-simple1_1970-01-01T00:42:00.tar.xz: OK run-1970-01-01T00:00:42-UTC: ONE::controllerA/tarball-simple1_1970-01-01T00:42:00: success ./tarball-simple2_1970-01-01T00:41:00.tar.xz: OK @@ -613,9 +500,7 @@ tarball-bad-md5-1_1970.01.01T00.42.00.tar.xz: FAILED md5sum: WARNING: 1 computed checksum did NOT match tarball-normal_1970.01.01T00.42.00.tar.xz: OK run-1970-01-01T00:00:42-UTC: controller-g-normal/tarball-normal_1970.01.01T00.42.00: success -pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 11 result tar balls, 7 successfully (0 partial), with 3 quarantined tar balls, 1 duplicately-named tar balls, and 0 errors. -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 11 result tar balls, 7 successfully (0 partial), with 3 quarantined tar balls, 1 duplicately-named tar balls, and 0 errors. -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "ndups": 1, "npartialsucc": 0, "nquarantined": 3, "ntbs": 7, "ntotal": 11, "prog": "pbench-dispatch", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-dispatch/pbench-dispatch.log +++++ pbench-index-tool-data/pbench-index-tool-data.log 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: starting @@ -627,7 +512,7 @@ pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 11 result tar 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-5.2/pbench/archive/fs-version-001/controller-b-with-prefixes/TO-INDEX/tarball-0_1970.01.01T00.42.00.tar.xz (size 212) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -659,35 +544,23 @@ pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 11 result tar 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-5.2/pbench/archive/fs-version-001/ONE::controllerC/TO-INDEX/tarball-simple0-prefix_1970-01-01T00:42:00.tar.xz (size 228) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 0 (skipped 7) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: Processed 0 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log +++++ pbench-sync-satellite/pbench-sync-satellite.error pbench-sync-satellite: completing satellite state changes ... (/var/tmp/pbench-test-server/test-5.2/pbench-local/logs/pbench-sync-satellite/ONE/change_state.log) pbench-sync-satellite: completed satellite state changes ----- pbench-sync-satellite/pbench-sync-satellite.error +++++ pbench-sync-satellite/pbench-sync-satellite.log -run-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: duration (secs): 0 -run-1970-01-01T00:00:42-UTC: Total 3 files processed and 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): ONE Processed 3 files (from 3 hosts, 3 total tar balls), with 0 errors, in 0 seconds -- @cee:{"pbench": {"report": {"summary": {"duration": 0, "end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nhosts": 3, "nprocessed": 3, "ntotal": 3, "prog": "pbench-sync-satellite", "remote_prefix": "ONE", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-sync-satellite/pbench-sync-satellite.log +++++ pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error run-1970-01-01T00:00:42-UTC: symlink target for /var/tmp/pbench-test-server/test-5.2/pbench/archive/fs-version-001/controller-d-duplicate/TO-UNPACK/tarball-that-does-not-exist_1970.01.01T00.42.00.tar.xz does not exist ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error +++++ pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-5.2/pbench/public_html/incoming/controller-b-with-prefixes/tarball-0_1970.01.01T00.42.00 /var/tmp/pbench-test-server/test-5.2/pbench/public_html/results/controller-b-with-prefixes/tarball-0_1970.01.01T00.42.00 run-1970-01-01T00:00:42-UTC: controller-b-with-prefixes/tarball-0_1970.01.01T00.42.00: success - elapsed time (secs): 0 - size (bytes): 212 ln -s /var/tmp/pbench-test-server/test-5.2/pbench/public_html/incoming/controller-b-with-prefixes/tarball-w-prefix-dot_1970.01.01T00.42.00 /var/tmp/pbench-test-server/test-5.2/pbench/public_html/results/controller-b-with-prefixes/tarball-w-prefix-dot_1970.01.01T00.42.00 @@ -702,18 +575,14 @@ ln -s /var/tmp/pbench-test-server/test-5.2/pbench/public_html/incoming/ONE::cont run-1970-01-01T00:00:42-UTC: ONE::controllerC/tarball-simple0-prefix_1970-01-01T00:42:00: success - elapsed time (secs): 0 - size (bytes): 228 ln -s /var/tmp/pbench-test-server/test-5.2/pbench/public_html/incoming/ONE::controllerB/tarball-simple2_1970-01-01T00:41:00 /var/tmp/pbench-test-server/test-5.2/pbench/public_html/results/ONE::controllerB/tarball-simple2_1970-01-01T00:41:00 run-1970-01-01T00:00:42-UTC: ONE::controllerB/tarball-simple2_1970-01-01T00:41:00: success - elapsed time (secs): 0 - size (bytes): 220 -run-1970-01-01T00:00:42-UTC: Processed 7 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 8 result tar balls, 7 successfully, with 0 rechecks, 0 warnings, and 1 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 1, "nrechecks": 0, "ntbs": 7, "ntotal": 8, "nwarn": 0, "prog": "pbench-unpack-tarballs-small", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log ---- pbench-local/logs ++++ pbench-satellite-local/logs +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.error ----- pbench-satellite-cleanup/pbench-satellite-cleanup.error +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.log -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup starting -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup ends: Total 3 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors. -1970-01-01T00:00:42.000000 INFO pbench-satellite-cleanup.report post_status -- @cee:{"@generated-by": {"commit_id": "unit-test", "group_id": 43, "hostname": "example.com", "pid": 42, "user_id": 44, "version": ""}, "@timestamp": "1970-01-01T00:00:42", "chunk_id": 1, "doctype": "status", "name": "pbench-satellite-cleanup", "text": "pbench-satellite-cleanup.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 total errors\nTotal 3 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors.\n\n", "total_chunks": 1, "total_size": 257} +run-1970-01-01T00:00:42-UTC(unit-test): Total 3 tar balls cleaned up (for 3 hosts), with 0 tar ball removal errors, 0 md5 file removal errors, 0 state change errors, 0 incoming directory removal errors, and 0 result directory removal errors. -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nhosts": 3, "nincomingerrs": 0, "nmd5errs": 0, "nresultserrs": 0, "nstateerrs": 0, "ntbs": 3, "ntberrs": 0, "prog": "pbench-satellite-cleanup", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-satellite-cleanup/pbench-satellite-cleanup.log +++++ pbench-sync-package-tarballs/pbench-sync-package-tarballs.error ----- pbench-sync-package-tarballs/pbench-sync-package-tarballs.error diff --git a/server/bin/gold/test-5.txt b/server/bin/gold/test-5.txt index f194884ec1..4de17efe91 100644 --- a/server/bin/gold/test-5.txt +++ b/server/bin/gold/test-5.txt @@ -1,126 +1,14 @@ +++ Running pbench-sync-satellite satellite-one -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-sync-satellite", - "text": "pbench-sync-satellite.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nRemote ONE: processed 0 files and 0 errors.\n\nrun-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC\nrun-1970-01-01T00:00:42-UTC: duration (secs): 0\nrun-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors\n", - "total_chunks": 1, - "total_size": 528 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-sync-satellite (status=0) +++ Running pbench-dispatch -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-dispatch", - "text": "pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors.\n", - "total_chunks": 1, - "total_size": 189 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-dispatch (status=0) +++ Running pbench-unpack-tarballs small -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs-small", - "text": "pbench-unpack-tarballs-small.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 0 result tar balls, 0 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 168 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index --- Finished pbench-index (status=0) +++ Running pbench-index --tool-data --- Finished pbench-index (status=0) +++ Running pbench-unpack-tarballs none re-unpack -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-re-unpack-tarballs", - "text": "pbench-re-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 0 result tar balls, 0 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 165 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-clean-up-dangling-results-links --- Finished pbench-clean-up-dangling-results-links (status=0) @@ -208,22 +96,22 @@ drwxrwxr-x - logs/pbench-cull-unpacked-tarballs -rw-rw-r-- 174 logs/pbench-cull-unpacked-tarballs/pbench-cull-unpacked-tarballs.log drwxrwxr-x - logs/pbench-dispatch -rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.error --rw-rw-r-- 646 logs/pbench-dispatch/pbench-dispatch.log +-rw-rw-r-- 0 logs/pbench-dispatch/pbench-dispatch.log drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log -rw-rw-r-- 224 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-re-unpack-tarballs -rw-rw-r-- 0 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error --rw-rw-r-- 528 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log +-rw-rw-r-- 0 logs/pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log drwxrwxr-x - logs/pbench-sync-satellite drwxrwxr-x - logs/pbench-sync-satellite/ONE -rw-rw-r-- 0 logs/pbench-sync-satellite/ONE/change_state.log -rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.error --rw-rw-r-- 849 logs/pbench-sync-satellite/pbench-sync-satellite.log +-rw-rw-r-- 0 logs/pbench-sync-satellite/pbench-sync-satellite.log drwxrwxr-x - logs/pbench-unpack-tarballs-small -rw-rw-r-- 0 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error --rw-rw-r-- 534 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log +-rw-rw-r-- 0 logs/pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -262,7 +150,7 @@ drwxrwxr-x - public_html/users drwxrwxr-x - logs drwxrwxr-x - logs/pbench-satellite-cleanup -rw-rw-r-- 0 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.error --rw-rw-r-- 928 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log +-rw-rw-r-- 0 logs/pbench-satellite-cleanup/pbench-satellite-cleanup.log drwxrwxr-x - logs/pbench-sync-package-tarballs -rw-rw-r-- 0 logs/pbench-sync-package-tarballs/pbench-sync-package-tarballs.error -rw-rw-r-- 0 logs/pbench-sync-package-tarballs/pbench-sync-package-tarballs.log @@ -292,10 +180,6 @@ drwxrwxr-x - tmp +++++ pbench-dispatch/pbench-dispatch.error ----- pbench-dispatch/pbench-dispatch.error +++++ pbench-dispatch/pbench-dispatch.log -run-1970-01-01T00:00:42-UTC -pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar balls, 0 successfully (0 partial), with 0 quarantined tar balls, 0 duplicately-named tar balls, and 0 errors. -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-dispatch.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-dispatch/pbench-dispatch.log +++++ pbench-index-tool-data/pbench-index-tool-data.log 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: starting @@ -308,41 +192,22 @@ pbench-dispatch.run-1970-01-01T00:00:42-UTC(unit-test): Processed 0 result tar b +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.error +++++ pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: Processed 0 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-re-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-re-unpack-tarballs/pbench-re-unpack-tarballs.log +++++ pbench-sync-satellite/ONE/change_state.log ----- pbench-sync-satellite/ONE/change_state.log +++++ pbench-sync-satellite/pbench-sync-satellite.error ----- pbench-sync-satellite/pbench-sync-satellite.error +++++ pbench-sync-satellite/pbench-sync-satellite.log -run-1970-01-01T00:00:42-UTC: start - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs fetched, unpacking ... - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: remote tarballs unpacked - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: end - 1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: duration (secs): 0 -run-1970-01-01T00:00:42-UTC: Total 0 files processed and 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-sync-satellite.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-sync-satellite/pbench-sync-satellite.log +++++ pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.error +++++ pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log -run-1970-01-01T00:00:42-UTC -run-1970-01-01T00:00:42-UTC: Processed 0 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs-small.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-unpack-tarballs-small/pbench-unpack-tarballs-small.log ---- pbench-local/logs ++++ pbench-satellite-local/logs +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.error ----- pbench-satellite-cleanup/pbench-satellite-cleanup.error +++++ pbench-satellite-cleanup/pbench-satellite-cleanup.log -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup starting -run-1970-01-01T00:00:42-UTC: pbench-satellite-cleanup ends: Total 0 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors. -1970-01-01T00:00:42.000000 INFO pbench-satellite-cleanup.report post_status -- @cee:{"@generated-by": {"commit_id": "unit-test", "group_id": 43, "hostname": "example.com", "pid": 42, "user_id": 44, "version": ""}, "@timestamp": "1970-01-01T00:00:42", "chunk_id": 1, "doctype": "status", "name": "pbench-satellite-cleanup", "text": "pbench-satellite-cleanup.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 total errors\nTotal 0 tarballs cleaned up, with 0 tarball removal errors, 0 md5 file remove errors, 0 state change errors, 0 incoming removal errors, and 0 result removal errors errors.\n\n", "total_chunks": 1, "total_size": 257} ----- pbench-satellite-cleanup/pbench-satellite-cleanup.log +++++ pbench-sync-package-tarballs/pbench-sync-package-tarballs.error ----- pbench-sync-package-tarballs/pbench-sync-package-tarballs.error diff --git a/server/bin/gold/test-7.1.txt b/server/bin/gold/test-7.1.txt index 158829a1f3..30a4b356fd 100644 --- a/server/bin/gold/test-7.1.txt +++ b/server/bin/gold/test-7.1.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -195,10 +167,10 @@ drwxrwxr-x - logs/pbench-audit-server drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 2195 logs/pbench-index/pbench-index.log +-rw-rw-r-- 2207 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 861 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 688 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -261,7 +233,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.1/pbench/archive/fs-version-001/controller/TO-INDEX/test_7.1_1970.01.01T00.00.00.tar.xz (size 1256) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -269,17 +241,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.1/pbench/archive/fs-version-001/controller/TO-INDEX/test_7.1_1970.01.01T00.00.00.tar.xz (size 1256) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 0 (skipped 1) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.1/pbench/public_html/incoming/controller/test_7.1_1970.01.01T00.00.00 /var/tmp/pbench-test-server/test-7.1/pbench/public_html/results/controller/test_7.1_1970.01.01T00.00.00 run-1970-01-01T00:00:42-UTC: controller/test_7.1_1970.01.01T00.00.00: success - elapsed time (secs): 0 - size (bytes): 1256 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.10.txt b/server/bin/gold/test-7.10.txt index ce3218d364..db6d6a7123 100644 --- a/server/bin/gold/test-7.10.txt +++ b/server/bin/gold/test-7.10.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -10285,11 +10257,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 10172 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3500 logs/pbench-index/pbench-index.log +-rw-rw-r-- 10184 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3512 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 884 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 711 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -10348,7 +10320,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.10/pbench/archive/fs-version-001/dhcp31-44/TO-INDEX-TOOL/uperf_uperftest_2018.02.02T20.58.00.tar.xz (size 2360408) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -10393,7 +10365,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.10/pbench/archive/fs-version-001/dhcp31-44/TO-INDEX-TOOL/uperf_uperftest_2018.02.02T20.58.00.tar.xz (size 2360408) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -10401,7 +10373,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.10/pbench/archive/fs-version-001/dhcp31-44/TO-INDEX/uperf_uperftest_2018.02.02T20.58.00.tar.xz (size 2360408) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -10424,17 +10396,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.10/pbench/archive/fs-version-001/dhcp31-44/TO-INDEX/uperf_uperftest_2018.02.02T20.58.00.tar.xz (size 2360408) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.10/pbench/public_html/incoming/dhcp31-44/uperf_uperftest_2018.02.02T20.58.00 /var/tmp/pbench-test-server/test-7.10/pbench/public_html/results/dhcp31-44/uperf_uperftest_2018.02.02T20.58.00 run-1970-01-01T00:00:42-UTC: dhcp31-44/uperf_uperftest_2018.02.02T20.58.00: success - elapsed time (secs): 0 - size (bytes): 2360408 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.11.txt b/server/bin/gold/test-7.11.txt index 81fb27179f..37b1f50831 100644 --- a/server/bin/gold/test-7.11.txt +++ b/server/bin/gold/test-7.11.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -5616,11 +5588,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 5815 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3473 logs/pbench-index/pbench-index.log +-rw-rw-r-- 5827 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3485 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 857 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 684 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -5679,7 +5651,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.11/pbench/archive/fs-version-001/dhcp31-44/TO-INDEX-TOOL/fio_rw_2018.02.01T22.40.57.tar.xz (size 2166868) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -5707,7 +5679,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.11/pbench/archive/fs-version-001/dhcp31-44/TO-INDEX-TOOL/fio_rw_2018.02.01T22.40.57.tar.xz (size 2166868) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -5715,7 +5687,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.11/pbench/archive/fs-version-001/dhcp31-44/TO-INDEX/fio_rw_2018.02.01T22.40.57.tar.xz (size 2166868) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -5738,17 +5710,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.11/pbench/archive/fs-version-001/dhcp31-44/TO-INDEX/fio_rw_2018.02.01T22.40.57.tar.xz (size 2166868) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.11/pbench/public_html/incoming/dhcp31-44/fio_rw_2018.02.01T22.40.57 /var/tmp/pbench-test-server/test-7.11/pbench/public_html/results/dhcp31-44/fio_rw_2018.02.01T22.40.57 run-1970-01-01T00:00:42-UTC: dhcp31-44/fio_rw_2018.02.01T22.40.57: success - elapsed time (secs): 0 - size (bytes): 2166868 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.12.txt b/server/bin/gold/test-7.12.txt index d61eb62a58..3266a3c3ce 100644 --- a/server/bin/gold/test-7.12.txt +++ b/server/bin/gold/test-7.12.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -4967,11 +4939,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 9229 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3558 logs/pbench-index/pbench-index.log +-rw-rw-r-- 9241 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3570 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 941 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 768 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -5030,7 +5002,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.12/pbench/archive/fs-version-001/EC2::ip-172-31-52-154/TO-INDEX-TOOL/pbench-user-benchmark__2018.02.05T20.35.36.tar.xz (size 5258824) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -5069,7 +5041,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.12/pbench/archive/fs-version-001/EC2::ip-172-31-52-154/TO-INDEX-TOOL/pbench-user-benchmark__2018.02.05T20.35.36.tar.xz (size 5258824) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -5077,7 +5049,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.12/pbench/archive/fs-version-001/EC2::ip-172-31-52-154/TO-INDEX/pbench-user-benchmark__2018.02.05T20.35.36.tar.xz (size 5258824) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -5100,17 +5072,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.12/pbench/archive/fs-version-001/EC2::ip-172-31-52-154/TO-INDEX/pbench-user-benchmark__2018.02.05T20.35.36.tar.xz (size 5258824) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.12/pbench/public_html/incoming/EC2::ip-172-31-52-154/pbench-user-benchmark__2018.02.05T20.35.36 /var/tmp/pbench-test-server/test-7.12/pbench/public_html/results/EC2::ip-172-31-52-154/pbench-user-benchmark__2018.02.05T20.35.36 run-1970-01-01T00:00:42-UTC: EC2::ip-172-31-52-154/pbench-user-benchmark__2018.02.05T20.35.36: success - elapsed time (secs): 0 - size (bytes): 5258824 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.13.txt b/server/bin/gold/test-7.13.txt index e492c78efc..fec24db398 100644 --- a/server/bin/gold/test-7.13.txt +++ b/server/bin/gold/test-7.13.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -11554,11 +11526,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 4477 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3576 logs/pbench-index/pbench-index.log +-rw-rw-r-- 4489 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3588 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 959 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 786 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -11617,7 +11589,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.13/pbench/archive/fs-version-001/b03-h01-1029p/TO-INDEX-TOOL/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19.tar.xz (size 3323572) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -11638,7 +11610,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.13/pbench/archive/fs-version-001/b03-h01-1029p/TO-INDEX-TOOL/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19.tar.xz (size 3323572) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -11646,7 +11618,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.13/pbench/archive/fs-version-001/b03-h01-1029p/TO-INDEX/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19.tar.xz (size 3323572) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -11669,17 +11641,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.13/pbench/archive/fs-version-001/b03-h01-1029p/TO-INDEX/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19.tar.xz (size 3323572) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.13/pbench/public_html/incoming/b03-h01-1029p/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19 /var/tmp/pbench-test-server/test-7.13/pbench/public_html/results/b03-h01-1029p/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19 run-1970-01-01T00:00:42-UTC: b03-h01-1029p/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19: success - elapsed time (secs): 0 - size (bytes): 3323572 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.14.txt b/server/bin/gold/test-7.14.txt index 9489b27d9f..fb55251f50 100644 --- a/server/bin/gold/test-7.14.txt +++ b/server/bin/gold/test-7.14.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -8705,11 +8677,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 38987 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3576 logs/pbench-index/pbench-index.log +-rw-rw-r-- 38999 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3588 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 959 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 786 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -8768,7 +8740,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.14/pbench/archive/fs-version-001/b03-h01-1029p/TO-INDEX-TOOL/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19.tar.xz (size 3324496) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -8917,7 +8889,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.14/pbench/archive/fs-version-001/b03-h01-1029p/TO-INDEX-TOOL/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19.tar.xz (size 3324496) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -8925,7 +8897,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.14/pbench/archive/fs-version-001/b03-h01-1029p/TO-INDEX/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19.tar.xz (size 3324496) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -8948,17 +8920,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.14/pbench/archive/fs-version-001/b03-h01-1029p/TO-INDEX/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19.tar.xz (size 3324496) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.14/pbench/public_html/incoming/b03-h01-1029p/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19 /var/tmp/pbench-test-server/test-7.14/pbench/public_html/results/b03-h01-1029p/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19 run-1970-01-01T00:00:42-UTC: b03-h01-1029p/pbench-user-benchmark_mbruzek-test-2_2018.04.10T19.01.19: success - elapsed time (secs): 0 - size (bytes): 3324496 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.15.txt b/server/bin/gold/test-7.15.txt index 784207e56f..27705bf303 100644 --- a/server/bin/gold/test-7.15.txt +++ b/server/bin/gold/test-7.15.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -9266,11 +9238,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 30001 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3481 logs/pbench-index/pbench-index.log +-rw-rw-r-- 30013 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3493 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 863 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 690 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -9329,7 +9301,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.15/pbench/archive/fs-version-001/master_40gb/TO-INDEX-TOOL/uperf__2016-10-06_16:34:03.tar.xz (size 5807176) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -9454,7 +9426,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.15/pbench/archive/fs-version-001/master_40gb/TO-INDEX-TOOL/uperf__2016-10-06_16:34:03.tar.xz (size 5807176) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -9462,7 +9434,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.15/pbench/archive/fs-version-001/master_40gb/TO-INDEX/uperf__2016-10-06_16:34:03.tar.xz (size 5807176) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -9485,17 +9457,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.15/pbench/archive/fs-version-001/master_40gb/TO-INDEX/uperf__2016-10-06_16:34:03.tar.xz (size 5807176) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.15/pbench/public_html/incoming/master_40gb/uperf__2016-10-06_16:34:03 /var/tmp/pbench-test-server/test-7.15/pbench/public_html/results/master_40gb/uperf__2016-10-06_16:34:03 run-1970-01-01T00:00:42-UTC: master_40gb/uperf__2016-10-06_16:34:03: success - elapsed time (secs): 0 - size (bytes): 5807176 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.16.txt b/server/bin/gold/test-7.16.txt index 63a350975c..1637e07cea 100644 --- a/server/bin/gold/test-7.16.txt +++ b/server/bin/gold/test-7.16.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -10932,11 +10904,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 19364 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3554 logs/pbench-index/pbench-index.log +-rw-rw-r-- 19376 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3566 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 938 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 765 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -10995,7 +10967,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.16/pbench/archive/fs-version-001/rhel8-4/TO-INDEX-TOOL/uperf_rhel8_4.18.0-18.el8_40gb_pass_2018.10.04T06.53.43.tar.xz (size 1584556) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -11072,7 +11044,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.16/pbench/archive/fs-version-001/rhel8-4/TO-INDEX-TOOL/uperf_rhel8_4.18.0-18.el8_40gb_pass_2018.10.04T06.53.43.tar.xz (size 1584556) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -11080,7 +11052,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.16/pbench/archive/fs-version-001/rhel8-4/TO-INDEX/uperf_rhel8_4.18.0-18.el8_40gb_pass_2018.10.04T06.53.43.tar.xz (size 1584556) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -11103,17 +11075,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.16/pbench/archive/fs-version-001/rhel8-4/TO-INDEX/uperf_rhel8_4.18.0-18.el8_40gb_pass_2018.10.04T06.53.43.tar.xz (size 1584556) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.16/pbench/public_html/incoming/rhel8-4/uperf_rhel8_4.18.0-18.el8_40gb_pass_2018.10.04T06.53.43 /var/tmp/pbench-test-server/test-7.16/pbench/public_html/results/rhel8-4/uperf_rhel8_4.18.0-18.el8_40gb_pass_2018.10.04T06.53.43 run-1970-01-01T00:00:42-UTC: rhel8-4/uperf_rhel8_4.18.0-18.el8_40gb_pass_2018.10.04T06.53.43: success - elapsed time (secs): 0 - size (bytes): 1584556 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.17.txt b/server/bin/gold/test-7.17.txt index 3db2c968a3..bcb2e745ea 100644 --- a/server/bin/gold/test-7.17.txt +++ b/server/bin/gold/test-7.17.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -3681,11 +3653,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 8863 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3569 logs/pbench-index/pbench-index.log +-rw-rw-r-- 8875 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3581 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 983 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 810 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -3744,7 +3716,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.17/pbench/archive/fs-version-001/ansible-host/TO-INDEX-TOOL/pbench-user-benchmark_example-vmstat_2018.10.24T14.38.18.tar.xz (size 31384) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -3782,7 +3754,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.17/pbench/archive/fs-version-001/ansible-host/TO-INDEX-TOOL/pbench-user-benchmark_example-vmstat_2018.10.24T14.38.18.tar.xz (size 31384) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -3790,7 +3762,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.17/pbench/archive/fs-version-001/ansible-host/TO-INDEX/pbench-user-benchmark_example-vmstat_2018.10.24T14.38.18.tar.xz (size 31384) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -3813,17 +3785,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.17/pbench/archive/fs-version-001/ansible-host/TO-INDEX/pbench-user-benchmark_example-vmstat_2018.10.24T14.38.18.tar.xz (size 31384) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.17/pbench/public_html/incoming/ansible-host/pbench-user-benchmark_example-vmstat_2018.10.24T14.38.18 /var/tmp/pbench-test-server/test-7.17/pbench/public_html/results/ansible-host/test_glusterfs-storage-block/pbench-user-benchmark_example-vmstat_2018.10.24T14.38.18 run-1970-01-01T00:00:42-UTC: ansible-host/pbench-user-benchmark_example-vmstat_2018.10.24T14.38.18: success - elapsed time (secs): 0 - size (bytes): 31384 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.18.txt b/server/bin/gold/test-7.18.txt index f3722d2014..469588652f 100644 --- a/server/bin/gold/test-7.18.txt +++ b/server/bin/gold/test-7.18.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 2 result tar balls, 2 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -200,10 +172,10 @@ drwxrwxr-x - logs/pbench-audit-server drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 2519 logs/pbench-index/pbench-index.log +-rw-rw-r-- 2531 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 1272 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 1099 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -268,7 +240,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.18/pbench/archive/fs-version-001/bad-controller/TO-INDEX/test_7.18_2018.02.05T15.31.08.tar.xz (size 1512) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -276,19 +248,16 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.18/pbench/archive/fs-version-001/bad-controller/TO-INDEX/test_7.18_2018.02.05T15.31.08.tar.xz (size 1512) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 0 (skipped 1) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.18/pbench/public_html/incoming/bad-controller/test_7.18_2018.02.05T15.31.08 /var/tmp/pbench-test-server/test-7.18/pbench/public_html/results/bad-controller/test_7.18_2018.02.05T15.31.08 run-1970-01-01T00:00:42-UTC: bad-controller/test_7.18_2018.02.05T15.31.08: success - elapsed time (secs): 0 - size (bytes): 1512 ln -s /var/tmp/pbench-test-server/test-7.18/pbench/public_html/incoming/bad-controller/pbench-user-benchmark__2018.02.05T20.35.36 /var/tmp/pbench-test-server/test-7.18/pbench/public_html/results/bad-controller/pbench-user-benchmark__2018.02.05T20.35.36 run-1970-01-01T00:00:42-UTC: bad-controller/pbench-user-benchmark__2018.02.05T20.35.36: success - elapsed time (secs): 0 - size (bytes): 232 -run-1970-01-01T00:00:42-UTC: Processed 2 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 2 result tar balls, 2 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 2, "ntotal": 2, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.19.txt b/server/bin/gold/test-7.19.txt index e0dbf79e70..7cd083e4a6 100644 --- a/server/bin/gold/test-7.19.txt +++ b/server/bin/gold/test-7.19.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -2425,11 +2397,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 3468 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 4099 logs/pbench-index/pbench-index.log +-rw-rw-r-- 3480 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 4111 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 1094 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 921 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -2488,7 +2460,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.19/pbench/archive/fs-version-001/perf122/TO-INDEX-TOOL/trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38.tar.xz (size 1030152) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -2505,7 +2477,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.19/pbench/archive/fs-version-001/perf122/TO-INDEX-TOOL/trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38.tar.xz (size 1030152) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -2513,7 +2485,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.19/pbench/archive/fs-version-001/perf122/TO-INDEX/trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38.tar.xz (size 1030152) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -2537,17 +2509,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 WARNING pbench-index.indexer dump_opctx -- ** Errors encountered while indexing: [{"counters": {"sample_missing_timeseries": 27}, "object": "ResultData", "tbname": "/var/tmp/pbench-test-server/test-7.19/pbench/archive/fs-version-001/perf122/trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38.tar.xz"}] 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.19/pbench/public_html/incoming/perf122/trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38 /var/tmp/pbench-test-server/test-7.19/pbench/public_html/results/perf122/trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38 run-1970-01-01T00:00:42-UTC: perf122/trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38: success - elapsed time (secs): 0 - size (bytes): 1030152 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.2.0.txt b/server/bin/gold/test-7.2.0.txt index 69ce4c0fe2..cf8b2ac5cf 100644 --- a/server/bin/gold/test-7.2.0.txt +++ b/server/bin/gold/test-7.2.0.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -194,10 +166,10 @@ drwxrwxr-x - logs/pbench-audit-server drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 2192 logs/pbench-index/pbench-index.log +-rw-rw-r-- 2204 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 871 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 698 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -260,7 +232,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.2.0/pbench/archive/fs-version-001/controller/TO-INDEX/test_7.2.0_1970.01.01T00.42.00.tar.xz (size 1224) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -268,17 +240,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.2.0/pbench/archive/fs-version-001/controller/TO-INDEX/test_7.2.0_1970.01.01T00.42.00.tar.xz (size 1224) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 0 (skipped 1) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.2.0/pbench/public_html/incoming/controller/test_7.2.0_1970.01.01T00.42.00 /var/tmp/pbench-test-server/test-7.2.0/pbench/public_html/results/controller/test_7.2.0_1970.01.01T00.42.00 run-1970-01-01T00:00:42-UTC: controller/test_7.2.0_1970.01.01T00.42.00: success - elapsed time (secs): 0 - size (bytes): 1224 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.2.1.txt b/server/bin/gold/test-7.2.1.txt index 8323fa538c..38913a5881 100644 --- a/server/bin/gold/test-7.2.1.txt +++ b/server/bin/gold/test-7.2.1.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -938,10 +910,10 @@ drwxrwxr-x - logs/pbench-audit-server drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 2247 logs/pbench-index/pbench-index.log +-rw-rw-r-- 2259 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 862 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 689 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -1004,7 +976,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.2.1/pbench/archive/fs-version-001/controller/TO-INDEX/uperf__2016-10-06_16:34:03.tar.xz (size 5809020) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -1012,17 +984,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.2.1/pbench/archive/fs-version-001/controller/TO-INDEX/uperf__2016-10-06_16:34:03.tar.xz (size 5809020) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 0 (skipped 1) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.2.1/pbench/public_html/incoming/controller/uperf__2016-10-06_16:34:03 /var/tmp/pbench-test-server/test-7.2.1/pbench/public_html/results/controller/uperf__2016-10-06_16:34:03 run-1970-01-01T00:00:42-UTC: controller/uperf__2016-10-06_16:34:03: success - elapsed time (secs): 0 - size (bytes): 5809020 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.20.txt b/server/bin/gold/test-7.20.txt index 0ca1c2e0de..6da578da59 100644 --- a/server/bin/gold/test-7.20.txt +++ b/server/bin/gold/test-7.20.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -1471,11 +1443,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 3223 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3469 logs/pbench-index/pbench-index.log +-rw-rw-r-- 3235 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3481 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 1103 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 930 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -1534,7 +1506,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.20/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/fio_mock_2020.02.27T22.16.14.tar.xz (size 828112) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -1551,7 +1523,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.20/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/fio_mock_2020.02.27T22.16.14.tar.xz (size 828112) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -1559,7 +1531,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.20/pbench/archive/fs-version-001/ctlrA/TO-INDEX/fio_mock_2020.02.27T22.16.14.tar.xz (size 828112) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -1582,18 +1554,15 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.20/pbench/archive/fs-version-001/ctlrA/TO-INDEX/fio_mock_2020.02.27T22.16.14.tar.xz (size 828112) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.20/pbench/public_html/incoming/ctlrA/fio_mock_2020.02.27T22.16.14 /var/tmp/pbench-test-server/test-7.20/pbench/public_html/results/ctlrA/prb-test-sync/fio_mock_2020.02.27T22.16.14 ln -s /var/tmp/pbench-test-server/test-7.20/pbench/public_html/incoming/ctlrA/fio_mock_2020.02.27T22.16.14 /var/tmp/pbench-test-server/test-7.20/pbench/public_html/users/janedoe@example.com/ctlrA/prb-test-sync/fio_mock_2020.02.27T22.16.14 run-1970-01-01T00:00:42-UTC: ctlrA/fio_mock_2020.02.27T22.16.14: success - elapsed time (secs): 0 - size (bytes): 828112 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.21.txt b/server/bin/gold/test-7.21.txt index 18a39d40d4..1e6cf4047e 100644 --- a/server/bin/gold/test-7.21.txt +++ b/server/bin/gold/test-7.21.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 2 result tar balls, 2 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -5693,11 +5665,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 5432 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 6529 logs/pbench-index/pbench-index.log +-rw-rw-r-- 5444 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 6541 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 1789 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 1616 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -5756,7 +5728,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.21/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/trafficgen_mock_2020.02.28T19.49.39.tar.xz (size 53736) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -5788,7 +5760,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.21/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/trafficgen_mock_2020.02.28T20.04.29.tar.xz (size 724228) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 2 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -5796,7 +5768,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.21/pbench/archive/fs-version-001/ctlrA/TO-INDEX/trafficgen_mock_2020.02.28T19.49.39.tar.xz (size 53736) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -5841,21 +5813,18 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 WARNING pbench-index.indexer dump_opctx -- ** Errors encountered while indexing: [{"counters": {"sample_missing_timeseries": 27}, "object": "ResultData", "tbname": "/var/tmp/pbench-test-server/test-7.21/pbench/archive/fs-version-001/ctlrA/trafficgen_mock_2020.02.28T19.49.39.tar.xz"}, {"counters": {"sample_missing_timeseries": 27}, "object": "ResultData", "tbname": "/var/tmp/pbench-test-server/test-7.21/pbench/archive/fs-version-001/ctlrA/trafficgen_mock_2020.02.28T20.04.29.tar.xz"}] 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 2 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.21/pbench/public_html/incoming/ctlrA/trafficgen_mock_2020.02.28T20.04.29 /var/tmp/pbench-test-server/test-7.21/pbench/public_html/results/ctlrA/prb-trafficgen-testing/trafficgen_mock_2020.02.28T20.04.29 ln -s /var/tmp/pbench-test-server/test-7.21/pbench/public_html/incoming/ctlrA/trafficgen_mock_2020.02.28T20.04.29 /var/tmp/pbench-test-server/test-7.21/pbench/public_html/users/janedoe@example.com/ctlrA/prb-trafficgen-testing/trafficgen_mock_2020.02.28T20.04.29 run-1970-01-01T00:00:42-UTC: ctlrA/trafficgen_mock_2020.02.28T20.04.29: success - elapsed time (secs): 0 - size (bytes): 724228 ln -s /var/tmp/pbench-test-server/test-7.21/pbench/public_html/incoming/ctlrA/trafficgen_mock_2020.02.28T19.49.39 /var/tmp/pbench-test-server/test-7.21/pbench/public_html/results/ctlrA/prb-trafficgen-testing/trafficgen_mock_2020.02.28T19.49.39 ln -s /var/tmp/pbench-test-server/test-7.21/pbench/public_html/incoming/ctlrA/trafficgen_mock_2020.02.28T19.49.39 /var/tmp/pbench-test-server/test-7.21/pbench/public_html/users/janedoe@example.com/ctlrA/prb-trafficgen-testing/trafficgen_mock_2020.02.28T19.49.39 run-1970-01-01T00:00:42-UTC: ctlrA/trafficgen_mock_2020.02.28T19.49.39: success - elapsed time (secs): 0 - size (bytes): 53736 -run-1970-01-01T00:00:42-UTC: Processed 2 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 2 result tar balls, 2 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 2, "ntotal": 2, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.22.txt b/server/bin/gold/test-7.22.txt index 38599e0e25..38ee3f170e 100644 --- a/server/bin/gold/test-7.22.txt +++ b/server/bin/gold/test-7.22.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -1194,11 +1166,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 3233 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3780 logs/pbench-index/pbench-index.log +-rw-rw-r-- 3245 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3792 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 1119 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 946 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -1257,7 +1229,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.22/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/linpack_mock_2020.02.28T19.10.55.tar.xz (size 20848) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -1274,7 +1246,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.22/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/linpack_mock_2020.02.28T19.10.55.tar.xz (size 20848) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -1282,7 +1254,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.22/pbench/archive/fs-version-001/ctlrA/TO-INDEX/linpack_mock_2020.02.28T19.10.55.tar.xz (size 20848) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -1306,18 +1278,15 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 WARNING pbench-index.indexer dump_opctx -- ** Errors encountered while indexing: [{"counters": {"sample_missing_timeseries": 6}, "object": "ResultData", "tbname": "/var/tmp/pbench-test-server/test-7.22/pbench/archive/fs-version-001/ctlrA/linpack_mock_2020.02.28T19.10.55.tar.xz"}] 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.22/pbench/public_html/incoming/ctlrA/linpack_mock_2020.02.28T19.10.55 /var/tmp/pbench-test-server/test-7.22/pbench/public_html/results/ctlrA/linpack-dev2/linpack_mock_2020.02.28T19.10.55 ln -s /var/tmp/pbench-test-server/test-7.22/pbench/public_html/incoming/ctlrA/linpack_mock_2020.02.28T19.10.55 /var/tmp/pbench-test-server/test-7.22/pbench/public_html/users/janedo@example.com/ctlrA/linpack-dev2/linpack_mock_2020.02.28T19.10.55 run-1970-01-01T00:00:42-UTC: ctlrA/linpack_mock_2020.02.28T19.10.55: success - elapsed time (secs): 0 - size (bytes): 20848 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.23.txt b/server/bin/gold/test-7.23.txt index f4de606d86..fd384c9538 100644 --- a/server/bin/gold/test-7.23.txt +++ b/server/bin/gold/test-7.23.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -1997,11 +1969,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 3223 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3469 logs/pbench-index/pbench-index.log +-rw-rw-r-- 3235 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3481 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 1076 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 903 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -2060,7 +2032,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.23/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/fio_mock_2020.01.19T00.18.06.tar.xz (size 492296) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -2077,7 +2049,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.23/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/fio_mock_2020.01.19T00.18.06.tar.xz (size 492296) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -2085,7 +2057,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.23/pbench/archive/fs-version-001/ctlrA/TO-INDEX/fio_mock_2020.01.19T00.18.06.tar.xz (size 492296) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -2108,18 +2080,15 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.23/pbench/archive/fs-version-001/ctlrA/TO-INDEX/fio_mock_2020.01.19T00.18.06.tar.xz (size 492296) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.23/pbench/public_html/incoming/ctlrA/fio_mock_2020.01.19T00.18.06 /var/tmp/pbench-test-server/test-7.23/pbench/public_html/results/ctlrA/378/fio_mock_2020.01.19T00.18.06 ln -s /var/tmp/pbench-test-server/test-7.23/pbench/public_html/incoming/ctlrA/fio_mock_2020.01.19T00.18.06 /var/tmp/pbench-test-server/test-7.23/pbench/public_html/users/virt-perf-ci/ctlrA/378/fio_mock_2020.01.19T00.18.06 run-1970-01-01T00:00:42-UTC: ctlrA/fio_mock_2020.01.19T00.18.06: success - elapsed time (secs): 0 - size (bytes): 492296 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.24.txt b/server/bin/gold/test-7.24.txt index 6221cd97cf..8e93854b28 100644 --- a/server/bin/gold/test-7.24.txt +++ b/server/bin/gold/test-7.24.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 2 result tar balls, 2 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -1316,11 +1288,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 5195 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 5791 logs/pbench-index/pbench-index.log +-rw-rw-r-- 5207 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 5803 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 1445 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 1272 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -1379,7 +1351,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.24/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/pbench-user-benchmark_1000_80_latency_perf_2021.01.28T17.22.57.tar.xz (size 109188) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -1409,7 +1381,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.24/pbench/archive/fs-version-001/ctlrA/TO-INDEX-TOOL/pbench-user-benchmark_Maridb_tuned_TP_HTon_40P_256Gmem_with_csv_2020.02.06T15.26.14.tar.xz (size 884748) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 2 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -1417,7 +1389,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.24/pbench/archive/fs-version-001/ctlrA/TO-INDEX/pbench-user-benchmark_1000_80_latency_perf_2021.01.28T17.22.57.tar.xz (size 109188) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -1459,19 +1431,16 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.24/pbench/archive/fs-version-001/ctlrA/TO-INDEX/pbench-user-benchmark_Maridb_tuned_TP_HTon_40P_256Gmem_with_csv_2020.02.06T15.26.14.tar.xz (size 884748) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 2 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.24/pbench/public_html/incoming/ctlrA/pbench-user-benchmark_1000_80_latency_perf_2021.01.28T17.22.57 /var/tmp/pbench-test-server/test-7.24/pbench/public_html/results/ctlrA/pbench-user-benchmark_1000_80_latency_perf_2021.01.28T17.22.57 run-1970-01-01T00:00:42-UTC: ctlrA/pbench-user-benchmark_1000_80_latency_perf_2021.01.28T17.22.57: success - elapsed time (secs): 0 - size (bytes): 109188 ln -s /var/tmp/pbench-test-server/test-7.24/pbench/public_html/incoming/ctlrA/pbench-user-benchmark_Maridb_tuned_TP_HTon_40P_256Gmem_with_csv_2020.02.06T15.26.14 /var/tmp/pbench-test-server/test-7.24/pbench/public_html/results/ctlrA/pbench-user-benchmark_Maridb_tuned_TP_HTon_40P_256Gmem_with_csv_2020.02.06T15.26.14 run-1970-01-01T00:00:42-UTC: ctlrA/pbench-user-benchmark_Maridb_tuned_TP_HTon_40P_256Gmem_with_csv_2020.02.06T15.26.14: success - elapsed time (secs): 0 - size (bytes): 884748 -run-1970-01-01T00:00:42-UTC: Processed 2 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 2 result tar balls, 2 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 2, "ntotal": 2, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.25.txt b/server/bin/gold/test-7.25.txt index 2b520d5b58..5bf5f53a3d 100644 --- a/server/bin/gold/test-7.25.txt +++ b/server/bin/gold/test-7.25.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -1788,11 +1760,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 3339 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3583 logs/pbench-index/pbench-index.log +-rw-rw-r-- 3351 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3595 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 965 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 792 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -1851,7 +1823,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.25/pbench/archive/fs-version-001/rhel8-1/TO-INDEX-TOOL/uperf_rhel8.1_4.18.0-107.el8_snap4_25gb_virt_2019.06.21T01.28.57.tar.xz (size 8423968) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -1868,7 +1840,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.25/pbench/archive/fs-version-001/rhel8-1/TO-INDEX-TOOL/uperf_rhel8.1_4.18.0-107.el8_snap4_25gb_virt_2019.06.21T01.28.57.tar.xz (size 8423968) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -1876,7 +1848,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.25/pbench/archive/fs-version-001/rhel8-1/TO-INDEX/uperf_rhel8.1_4.18.0-107.el8_snap4_25gb_virt_2019.06.21T01.28.57.tar.xz (size 8423968) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -1899,17 +1871,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.25/pbench/archive/fs-version-001/rhel8-1/TO-INDEX/uperf_rhel8.1_4.18.0-107.el8_snap4_25gb_virt_2019.06.21T01.28.57.tar.xz (size 8423968) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.25/pbench/public_html/incoming/rhel8-1/uperf_rhel8.1_4.18.0-107.el8_snap4_25gb_virt_2019.06.21T01.28.57 /var/tmp/pbench-test-server/test-7.25/pbench/public_html/results/rhel8-1/uperf_rhel8.1_4.18.0-107.el8_snap4_25gb_virt_2019.06.21T01.28.57 run-1970-01-01T00:00:42-UTC: rhel8-1/uperf_rhel8.1_4.18.0-107.el8_snap4_25gb_virt_2019.06.21T01.28.57: success - elapsed time (secs): 0 - size (bytes): 8423968 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.26.txt b/server/bin/gold/test-7.26.txt index 9afec621bc..78763f1219 100644 --- a/server/bin/gold/test-7.26.txt +++ b/server/bin/gold/test-7.26.txt @@ -1063,7 +1063,7 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/pbench-audit-server.log -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index-re --rw-rw-r-- 3882 logs/pbench-index-re/pbench-index-re.log +-rw-rw-r-- 3894 logs/pbench-index-re/pbench-index-re.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -1122,7 +1122,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-re.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-re.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-re.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-re.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-re.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-re.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-re.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.26/pbench/archive/fs-version-001/ctlrA/TO-RE-INDEX/linpack_mock_2020.02.28T19.10.55.tar.xz (size 20848) 1970-01-01T00:00:42.000000 DEBUG pbench-index-re.pbench-index main -- open tar ball @@ -1146,7 +1146,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 WARNING pbench-index-re.indexer dump_opctx -- ** Errors encountered while indexing: [{"counters": {"sample_missing_timeseries": 6}, "object": "ResultData", "tbname": "/var/tmp/pbench-test-server/test-7.26/pbench/archive/fs-version-001/ctlrA/linpack_mock_2020.02.28T19.10.55.tar.xz"}] 1970-01-01T00:00:42.000000 DEBUG pbench-index-re.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-re.pbench-index main -- pbench-index-re.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-re.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-re.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-re/pbench-index-re.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.3.txt b/server/bin/gold/test-7.3.txt index d6085f2ebc..531a5ddef8 100644 --- a/server/bin/gold/test-7.3.txt +++ b/server/bin/gold/test-7.3.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -195,10 +167,10 @@ drwxrwxr-x - logs/pbench-audit-server drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data -rw-rw-r-- 254 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 2197 logs/pbench-index/pbench-index.log +-rw-rw-r-- 2209 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 861 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 688 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -261,7 +233,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.3/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.3_2015.09.21T15.31.08.tar.xz (size 1492) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -269,17 +241,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.3/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.3_2015.09.21T15.31.08.tar.xz (size 1492) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 0 (skipped 1) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.3/pbench/public_html/incoming/alphaville/test_7.3_2015.09.21T15.31.08 /var/tmp/pbench-test-server/test-7.3/pbench/public_html/results/alphaville/test_7.3_2015.09.21T15.31.08 run-1970-01-01T00:00:42-UTC: alphaville/test_7.3_2015.09.21T15.31.08: success - elapsed time (secs): 0 - size (bytes): 1492 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.4.txt b/server/bin/gold/test-7.4.txt index cc0a422b23..0536f2e6f6 100644 --- a/server/bin/gold/test-7.4.txt +++ b/server/bin/gold/test-7.4.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -432,11 +404,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 3232 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3473 logs/pbench-index/pbench-index.log +-rw-rw-r-- 3244 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3485 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 861 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 688 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -495,7 +467,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.4/pbench/archive/fs-version-001/alphaville/TO-INDEX-TOOL/test_7.4_2015.09.21T15.31.08.tar.xz (size 1492) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -512,7 +484,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.4/pbench/archive/fs-version-001/alphaville/TO-INDEX-TOOL/test_7.4_2015.09.21T15.31.08.tar.xz (size 1492) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -520,7 +492,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.4/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.4_2015.09.21T15.31.08.tar.xz (size 1492) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -543,17 +515,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.4/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.4_2015.09.21T15.31.08.tar.xz (size 1492) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.4/pbench/public_html/incoming/alphaville/test_7.4_2015.09.21T15.31.08 /var/tmp/pbench-test-server/test-7.4/pbench/public_html/results/alphaville/test_7.4_2015.09.21T15.31.08 run-1970-01-01T00:00:42-UTC: alphaville/test_7.4_2015.09.21T15.31.08: success - elapsed time (secs): 0 - size (bytes): 1492 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.5.txt b/server/bin/gold/test-7.5.txt index fa9f0f72ee..f5e8e03333 100644 --- a/server/bin/gold/test-7.5.txt +++ b/server/bin/gold/test-7.5.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -432,11 +404,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 3232 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3473 logs/pbench-index/pbench-index.log +-rw-rw-r-- 3244 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3485 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 861 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 688 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -495,7 +467,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.5/pbench/archive/fs-version-001/alphaville/TO-INDEX-TOOL/test_7.5_2015.09.21T15.31.08.tar.xz (size 1508) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -512,7 +484,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.5/pbench/archive/fs-version-001/alphaville/TO-INDEX-TOOL/test_7.5_2015.09.21T15.31.08.tar.xz (size 1508) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -520,7 +492,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.5/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.5_2015.09.21T15.31.08.tar.xz (size 1508) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -543,17 +515,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.5/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.5_2015.09.21T15.31.08.tar.xz (size 1508) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.5/pbench/public_html/incoming/alphaville/test_7.5_2015.09.21T15.31.08 /var/tmp/pbench-test-server/test-7.5/pbench/public_html/results/alphaville/test_7.5_2015.09.21T15.31.08 run-1970-01-01T00:00:42-UTC: alphaville/test_7.5_2015.09.21T15.31.08: success - elapsed time (secs): 0 - size (bytes): 1508 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.6.txt b/server/bin/gold/test-7.6.txt index 93522154db..c5dbbbb066 100644 --- a/server/bin/gold/test-7.6.txt +++ b/server/bin/gold/test-7.6.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -433,11 +405,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 3232 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3473 logs/pbench-index/pbench-index.log +-rw-rw-r-- 3244 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3485 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 861 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 688 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -496,7 +468,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.6/pbench/archive/fs-version-001/alphaville/TO-INDEX-TOOL/test_7.6_2015.09.21T15.31.08.tar.xz (size 1504) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -513,7 +485,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.6/pbench/archive/fs-version-001/alphaville/TO-INDEX-TOOL/test_7.6_2015.09.21T15.31.08.tar.xz (size 1504) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -521,7 +493,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.6/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.6_2015.09.21T15.31.08.tar.xz (size 1504) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -544,17 +516,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.6/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.6_2015.09.21T15.31.08.tar.xz (size 1504) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.6/pbench/public_html/incoming/alphaville/test_7.6_2015.09.21T15.31.08 /var/tmp/pbench-test-server/test-7.6/pbench/public_html/results/alphaville/test_7.6_2015.09.21T15.31.08 run-1970-01-01T00:00:42-UTC: alphaville/test_7.6_2015.09.21T15.31.08: success - elapsed time (secs): 0 - size (bytes): 1504 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.7.txt b/server/bin/gold/test-7.7.txt index d63ce87be7..e7df6031a3 100644 --- a/server/bin/gold/test-7.7.txt +++ b/server/bin/gold/test-7.7.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -433,11 +405,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 3232 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3473 logs/pbench-index/pbench-index.log +-rw-rw-r-- 3244 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3485 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 861 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 688 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -496,7 +468,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.7/pbench/archive/fs-version-001/alphaville/TO-INDEX-TOOL/test_7.7_2015.09.21T15.31.08.tar.xz (size 1512) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -513,7 +485,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.7/pbench/archive/fs-version-001/alphaville/TO-INDEX-TOOL/test_7.7_2015.09.21T15.31.08.tar.xz (size 1512) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -521,7 +493,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.7/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.7_2015.09.21T15.31.08.tar.xz (size 1512) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -544,17 +516,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.7/pbench/archive/fs-version-001/alphaville/TO-INDEX/test_7.7_2015.09.21T15.31.08.tar.xz (size 1512) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.7/pbench/public_html/incoming/alphaville/test_7.7_2015.09.21T15.31.08 /var/tmp/pbench-test-server/test-7.7/pbench/public_html/results/alphaville/test_7.7_2015.09.21T15.31.08 run-1970-01-01T00:00:42-UTC: alphaville/test_7.7_2015.09.21T15.31.08: success - elapsed time (secs): 0 - size (bytes): 1512 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.8.txt b/server/bin/gold/test-7.8.txt index 4a969ce63f..b87c53bda8 100644 --- a/server/bin/gold/test-7.8.txt +++ b/server/bin/gold/test-7.8.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -9266,11 +9238,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 29999 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3479 logs/pbench-index/pbench-index.log +-rw-rw-r-- 30011 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3491 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 861 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 688 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -9329,7 +9301,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.8/pbench/archive/fs-version-001/master_40gb/TO-INDEX-TOOL/uperf__2016-10-06_16:34:03.tar.xz (size 5472928) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -9454,7 +9426,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.8/pbench/archive/fs-version-001/master_40gb/TO-INDEX-TOOL/uperf__2016-10-06_16:34:03.tar.xz (size 5472928) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -9462,7 +9434,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.8/pbench/archive/fs-version-001/master_40gb/TO-INDEX/uperf__2016-10-06_16:34:03.tar.xz (size 5472928) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -9485,17 +9457,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.8/pbench/archive/fs-version-001/master_40gb/TO-INDEX/uperf__2016-10-06_16:34:03.tar.xz (size 5472928) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.8/pbench/public_html/incoming/master_40gb/uperf__2016-10-06_16:34:03 /var/tmp/pbench-test-server/test-7.8/pbench/public_html/results/master_40gb/uperf__2016-10-06_16:34:03 run-1970-01-01T00:00:42-UTC: master_40gb/uperf__2016-10-06_16:34:03: success - elapsed time (secs): 0 - size (bytes): 5472928 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/gold/test-7.9.txt b/server/bin/gold/test-7.9.txt index f076dc1186..5fd31779a2 100644 --- a/server/bin/gold/test-7.9.txt +++ b/server/bin/gold/test-7.9.txt @@ -1,32 +1,4 @@ +++ Running pbench-unpack-tarballs -Template: pbench-unittests.v3.server-reports -Index: pbench-unittests.v3.server-reports.1970-01 1 -len(actions) = 1 -[ - { - "_id": "5ca1ab1e70015f100dedfab1ed0ff1ce", - "_index": "pbench-unittests.v3.server-reports.1970-01", - "_op_type": "create", - "_source": { - "@generated-by": { - "commit_id": "unit-test", - "group_id": 43, - "hostname": "example.com", - "pid": 42, - "user_id": 44, - "version": "" - }, - "@timestamp": "1970-01-01T00:00:42", - "chunk_id": 1, - "doctype": "status", - "name": "pbench-unpack-tarballs", - "text": "pbench-unpack-tarballs.run-1970-01-01T00:00:42-UTC(unit-test) - w/ 0 errors\nProcessed 1 result tar balls, 1 successfully, 0 warnings, 0 errors, and 0 duplicates\n\n", - "total_chunks": 1, - "total_size": 162 - }, - "_type": "pbench-server-reports" - } -] --- Finished pbench-unpack-tarballs (status=0) +++ Running pbench-index Template: pbench-unittests.v3.server-reports @@ -9231,11 +9203,11 @@ drwxrwxr-x - logs/pbench-audit-server -rw-rw-r-- 0 logs/pbench-audit-server/report.latest.txt drwxrwxr-x - logs/pbench-index drwxrwxr-x - logs/pbench-index-tool-data --rw-rw-r-- 6768 logs/pbench-index-tool-data/pbench-index-tool-data.log --rw-rw-r-- 3523 logs/pbench-index/pbench-index.log +-rw-rw-r-- 6780 logs/pbench-index-tool-data/pbench-index-tool-data.log +-rw-rw-r-- 3535 logs/pbench-index/pbench-index.log drwxrwxr-x - logs/pbench-unpack-tarballs -rw-rw-r-- 0 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.error --rw-rw-r-- 906 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log +-rw-rw-r-- 733 logs/pbench-unpack-tarballs/pbench-unpack-tarballs.log drwxrwxr-x - pbench-move-results-receive drwxrwxr-x - pbench-move-results-receive/fs-version-002 drwxrwxr-x - quarantine @@ -9294,7 +9266,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.9/pbench/archive/fs-version-001/dhcp31-144/TO-INDEX-TOOL/pbench-user-benchmark__2017-04-21_20:38:16.tar.xz (size 1137024) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- open tar ball @@ -9325,7 +9297,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.9/pbench/archive/fs-version-001/dhcp31-144/TO-INDEX-TOOL/pbench-user-benchmark__2017-04-21_20:38:16.tar.xz (size 1137024) 1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index-tool-data.pbench-index main -- pbench-index-tool-data.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index-tool-data.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index-tool-data/pbench-index-tool-data.log +++++ pbench-index/pbench-index.log 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: starting @@ -9333,7 +9305,7 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [start] 1970-01-01T00:00:42.000000 DEBUG pbench-index.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 10, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- update_templates [end] -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- start processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Starting /var/tmp/pbench-test-server/test-7.9/pbench/archive/fs-version-001/dhcp31-144/TO-INDEX/pbench-user-benchmark__2017-04-21_20:38:16.tar.xz (size 1137024) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- open tar ball @@ -9356,17 +9328,14 @@ drwxrwxr-x - tmp 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- Finished /var/tmp/pbench-test-server/test-7.9/pbench/archive/fs-version-001/dhcp31-144/TO-INDEX/pbench-user-benchmark__2017-04-21_20:38:16.tar.xz (size 1137024) 1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index main -- stopped processing list of tar balls 1970-01-01T00:00:42.000000 INFO pbench-index.pbench-index main -- pbench-index.run-1970-01-01T00:00:42-UTC: indexed 1 (skipped 0) results, 0 errors -1970-01-01T00:00:42.000000 DEBUG pbench-index.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +1970-01-01T00:00:42.000000 DEBUG pbench-index.pbench-index post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) ----- pbench-index/pbench-index.log +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.error ----- pbench-unpack-tarballs/pbench-unpack-tarballs.error +++++ pbench-unpack-tarballs/pbench-unpack-tarballs.log -run-1970-01-01T00:00:42-UTC ln -s /var/tmp/pbench-test-server/test-7.9/pbench/public_html/incoming/dhcp31-144/pbench-user-benchmark__2017-04-21_20:38:16 /var/tmp/pbench-test-server/test-7.9/pbench/public_html/results/dhcp31-144/pbench-user-benchmark__2017-04-21_20:38:16 run-1970-01-01T00:00:42-UTC: dhcp31-144/pbench-user-benchmark__2017-04-21_20:38:16: success - elapsed time (secs): 0 - size (bytes): 1137024 -run-1970-01-01T00:00:42-UTC: Processed 1 tar balls -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.indexer update_templates -- done templates (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, retries: 0) -1970-01-01T00:00:42.000000 DEBUG pbench-unpack-tarballs.report post_status -- posted status (start ts: 1970-01-01T00:00:42-UTC, end ts: 1970-01-01T00:00:42-UTC, duration: 0.00s, successes: 1, duplicates: 0, failures: 0, retries: 0) +run-1970-01-01T00:00:42-UTC(unit-test): Processed 1 result tar balls, 1 successfully, with 0 rechecks, 0 warnings, and 0 errors -- @cee:{"pbench": {"report": {"summary": {"end_ts": "1970-01-01T00:00:42-UTC", "errors": 0, "nrechecks": 0, "ntbs": 1, "ntotal": 1, "nwarn": 0, "prog": "pbench-unpack-tarballs", "start_ts": "1970-01-01T00:00:42-UTC"}}}} ----- pbench-unpack-tarballs/pbench-unpack-tarballs.log ---- pbench-local/logs ++++ pbench-satellite-local/logs diff --git a/server/bin/pbench-base.sh b/server/bin/pbench-base.sh index d1cf4ebb16..5f78ce9ad3 100755 --- a/server/bin/pbench-base.sh +++ b/server/bin/pbench-base.sh @@ -103,11 +103,8 @@ function log_finish { function log_exit { local _msg="${PROG}: ${1}" - if [[ -z "${3}" ]]; then - printf -- "%s\n" "${_msg}" >&4 - else - printf -- "%s\n" "${_msg}" | tee -a "${3}" >&4 - fi + printf -- "%s\n" "${_msg}" >&4 + logger -t ${PROG} -p daemon.err -- "${1}" log_finish if [[ -z "${2}" ]]; then exit 1 @@ -117,17 +114,11 @@ function log_exit { } function log_info { - if [[ -z "${2}" ]]; then - printf -- "%b\n" "${1}" - else - printf -- "%b\n" "${1}" | tee -a "${2}" - fi + printf -- "%b\n" "${1}" + logger -t ${PROG} -p daemon.info -- "${1}" } function log_error { - if [[ -z "${2}" ]]; then - printf -- "%b\n" "${1}" >&4 - else - printf -- "%b\n" "${1}" | tee -a "${2}" >&4 - fi + printf -- "%b\n" "${1}" >&4 + logger -t ${PROG} -p daemon.err -- "${1}" } diff --git a/server/bin/pbench-dispatch.sh b/server/bin/pbench-dispatch.sh index 250b09d6d3..caf7cd52f4 100755 --- a/server/bin/pbench-dispatch.sh +++ b/server/bin/pbench-dispatch.sh @@ -146,16 +146,11 @@ if [[ ${sts} != 0 ]]; then log_exit "Failed: \"mkdir -p ${tmp}\", status ${sts}" 4 fi -# Setup the report file. -status=${tmp}/status -> ${status} - -# Mark the beginning of execution. -log_info ${TS} - # File that will contain the list of all .md5 files to be processed. list=${tmp}/list.check +start_ts=$(timestamp) + # Check for results that are ready for processing: version 002 agents # upload the MD5 file as xxx.md5.check and they rename it to xxx.md5 # after they are done with MD5 checking so that's what we look for. @@ -201,7 +196,7 @@ while read tbmd5; do dest=${ARCHIVE}/${controller} if [[ -f ${dest}/${resultname}.tar.xz || -f ${dest}/${resultname}.tar.xz.md5 ]]; then - log_error "${TS}: Duplicate: ${tb} duplicate name" "${status}" + log_error "${TS}: Duplicate: ${tb} duplicate name" quarantine ${duplicates}/${controller} ${tbmd5} ${tb} (( ndups++ )) continue @@ -212,7 +207,7 @@ while read tbmd5; do sts=${?} popd > /dev/null 2>&4 if [[ ${sts} -ne 0 ]]; then - log_error "${TS}: Quarantined: ${tb} failed MD5 check" "${status}" + log_error "${TS}: Quarantined: ${tb} failed MD5 check" quarantine ${bad_md5}/${controller} ${tbmd5} ${tb} (( nquarantined++ )) continue @@ -223,14 +218,14 @@ while read tbmd5; do mkdir -p ${backup_dir}/${controller} sts=${?} if [[ ${sts} -ne 0 ]]; then - log_error "${TS}: failed to create backup directory ${backup_dir}/${controller}: code ${sts}" "${status}" + log_error "${TS}: failed to create backup directory ${backup_dir}/${controller}: code ${sts}" (( nberrs++ )) continue fi cp -a ${tbmd5} ${tb} ${backup_dir}/${controller}/ sts=${?} if [[ ${sts} -ne 0 ]]; then - log_error "${TS}: failed to backup ${tbmd5} and ${tb}: code ${sts}" "${status}" + log_error "${TS}: failed to backup ${tbmd5} and ${tb}: code ${sts}" (( nberrs++ )) continue fi @@ -254,7 +249,7 @@ while read tbmd5; do pbench-results-push ${tb} ${push_options} sts=${?} if [[ ${sts} -ne 0 ]]; then - log_info "${TS}: 'pbench-results-push ${tb} ${push_options}' failed, code ${sts}" "${status}" + log_error "${TS}: 'pbench-results-push ${tb} ${push_options}' failed, code ${sts}" fi fi @@ -262,7 +257,7 @@ while read tbmd5; do mk_dirs ${controller} sts=${?} if [[ ${sts} -ne 0 ]]; then - log_error "${TS}: Creation of ${controller} processing directories failed for ${tb}: code ${sts}" "${status}" + log_error "${TS}: Creation of ${controller} processing directories failed for ${tb}: code ${sts}" (( nerrs++ )) continue fi @@ -272,7 +267,7 @@ while read tbmd5; do cp -a ${tbmd5} ${dest}/ sts=${?} if [[ ${sts} -ne 0 ]]; then - log_error "${TS}: Error: \"cp -a ${tbmd5} ${dest}/\", status ${sts}" "${status}" + log_error "${TS}: Error: \"cp -a ${tbmd5} ${dest}/\", status ${sts}" (( nerrs++ )) continue fi @@ -285,11 +280,11 @@ while read tbmd5; do mv ${tb} ${dest}/ sts=${?} if [[ ${sts} -ne 0 ]]; then - log_error "${TS}: Error: \"mv ${tb} ${dest}/\", status ${sts}" "${status}" + log_error "${TS}: Error: \"mv ${tb} ${dest}/\", status ${sts}" rm ${dest}/${resultname}.tar.xz.md5 sts=${?} if [[ ${sts} -ne 0 ]]; then - log_error "${TS}: Warning: cleanup of move failure failed itself: \"rm ${dest}/${resultname}.tar.xz.md5\", status ${sts}" "${status}" + log_error "${TS}: Warning: cleanup of move failure failed itself: \"rm ${dest}/${resultname}.tar.xz.md5\", status ${sts}" fi (( nerrs++ )) continue @@ -302,7 +297,7 @@ while read tbmd5; do sts=${?} if [[ ${sts} -ne 0 ]]; then # log it but do not abort - log_error "${TS}: Error: \"restorecon ${dest}/${tbname} ${dest}/${tbname}.md5\", status ${sts}" "${status}" + log_error "${TS}: Error: \"restorecon ${dest}/${tbname} ${dest}/${tbname}.md5\", status ${sts}" fi # Now that we have successfully moved the tar ball and its .md5 to the @@ -310,7 +305,7 @@ while read tbmd5; do rm ${tbmd5} sts=${?} if [[ ${sts} -ne 0 ]]; then - log_error "${TS}: Warning: cleanup of successful copy operation failed: \"rm ${tbmd5}\", status ${sts}" "${status}" + log_error "${TS}: Warning: cleanup of successful copy operation failed: \"rm ${tbmd5}\", status ${sts}" fi # Create a link in each state dir - if any fail we don't delete them @@ -323,7 +318,7 @@ while read tbmd5; do if [[ ${sts} -eq 0 ]]; then (( totsuc++ )) else - log_error "${TS}: Cannot create ${dest}/${tbname} link to ${state}: code ${sts}" "${status}" + log_error "${TS}: Cannot create ${dest}/${tbname} link to ${state}: code ${sts}" (( toterr++ )) fi done @@ -338,22 +333,37 @@ while read tbmd5; do if [[ ${toterr} -gt 0 ]]; then # We have had some errors while processing this tar ball, so count # this as a partial success. - log_info "${TS}: ${controller}/${resultname}: success (partial)" "${status}" + log_info "${TS}: ${controller}/${resultname}: success (partial)" (( npartialsucc++ )) else - log_info "${TS}: ${controller}/${resultname}: success" "${status}" + log_info "${TS}: ${controller}/${resultname}: success" fi fi done < ${list} -summary_text="Processed ${ntotal} result tar balls, ${ntbs} successfully"\ -" (${npartialsucc} partial), with ${nquarantined} quarantined tar balls,"\ -" ${ndups} duplicately-named tar balls, and ${nerrs} errors." - -log_info "${PROG}.${TS}(${PBENCH_ENV}): ${summary_text}" "${status}" +end_ts=$(timestamp) + +if [[ ${ntotal} -gt 0 ]]; then + summary_text="Processed ${ntotal} result tar balls, ${ntbs} successfully" + summary_text+=" (${npartialsucc} partial), with ${nquarantined} quarantined" + summary_text+=" tar balls, ${ndups} duplicately-named tar balls, and" + summary_text+=" ${nerrs} errors." + printf -v summary_inner_json \ + "{\"%s\": \"%s\", \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": \"%s\", \"%s\": \"%s\"}" \ + "end_ts" "${end_ts}" \ + "errors" "${nerrs}" \ + "ndups" "${ndups}" \ + "npartialsucc" "${npartialsucc}" \ + "nquarantined" "${nquarantined}" \ + "ntbs" "${ntbs}" \ + "ntotal" "${ntotal}" \ + "prog" "${PROG}" \ + "start_ts" "${start_ts}" + printf -v summary_json "{\"pbench\": {\"report\": {\"summary\": %s}}}" "${summary_inner_json}" + + log_info "${TS}(${PBENCH_ENV}): ${summary_text} -- @cee:${summary_json}" +fi log_finish -pbench-report-status --name ${PROG} --pid ${$} --timestamp $(timestamp) --type status ${status} - exit ${nerrs} diff --git a/server/bin/pbench-index.py b/server/bin/pbench-index.py index a52c10ce8c..8493ae9d51 100755 --- a/server/bin/pbench-index.py +++ b/server/bin/pbench-index.py @@ -6,30 +6,38 @@ """ -import sys -import os import glob +import hashlib +import json +import lzma +import math +import os +import socket +import sys import tarfile import tempfile from argparse import ArgumentParser from configparser import Error as ConfigParserError +from configparser import NoOptionError, NoSectionError -from pbench import tstos, BadConfig, quarantine, rename_tb_link +import pbench +from pbench import BadConfig, quarantine, rename_tb_link, tstos from pbench.indexer import ( - IdxContext, - ConfigFileError, + VERSION, BadDate, - UnsupportedTarballFormat, BadMDLogFormat, - SosreportHostname, - PbenchTarBall, - es_index, + ConfigFileError, + IdxContext, JsonFileError, + PbenchTarBall, + PbenchTemplates, + SosreportHostname, TemplateError, - VERSION, + UnsupportedTarballFormat, + _op_type, + es_index, + get_es, ) -from pbench.report import Report - # Internal debugging flag. _DEBUG = 0 @@ -40,6 +48,236 @@ _filenotfounderror = getattr(__builtins__, "FileNotFoundError", IOError) +class Report(object): + """Encapsulation of server-side reporting information for recording who, + what, when, and how of a component's operation. + """ + + # We set the chunk size to 50 MB which is half the maximum payload size + # that Elasticsearch typically handles. + _CHUNK_SIZE = 50 * 1024 * 1024 + + def __init__( + self, + config, + name, + es=None, + pid=None, + group_id=None, + user_id=None, + hostname=None, + version=None, + templates=None, + ): + self.config = config + self.name = name + self.logger = pbench.get_pbench_logger(name, config) + + # We always create a base "tracking" document composed of parameters + # from the caller, and other environmental data. This document is used + # as the foundation for the first document posted to the target + # Elasticsearch instance with the `post_status()` method. All + # subsequent calls to the `post_status()` method will use that first + # document ID as their parent document ID. This allows us to have + # multiple status updates associated with the initial Report() caller. + if config._unittests: + _hostname = "example.com" + _pid = 42 + _group_id = 43 + _user_id = 44 + else: + _hostname = hostname if hostname else socket.gethostname() + _pid = pid if pid else os.getpid() + _group_id = group_id if group_id else os.getgid() + _user_id = user_id if user_id else os.getuid() + self.generated_by = dict( + [ + ("commit_id", self.config.COMMIT_ID), + ("group_id", _group_id), + ("hostname", _hostname), + ("pid", _pid), + ("user_id", _user_id), + ("version", version if version else ""), + ] + ) + # The "tracking_id" is the final MD5 hash of the first document + # indexed via the `post_status()` method. + self.tracking_id = None + try: + self.idx_prefix = config.get("Indexing", "index_prefix") + except (NoOptionError, NoSectionError): + # No index prefix so reporting will be performed via logging. + self.idx_prefix = None + self.es = None + else: + if es is None: + try: + self.es = get_es(config, self.logger) + except Exception: + self.logger.exception( + "Unexpected failure fetching" " Elasticsearch configuration" + ) + # If we don't have an Elasticsearch configuration just use + # None to indicate logging should be used instead. + self.es = None + else: + self.es = es + if templates is not None: + self.templates = templates + else: + self.templates = PbenchTemplates( + self.config.BINDIR, self.idx_prefix, self.logger + ) + + def init_report_template(self): + """Setup the Elasticsearch templates needed for properly indexing + report documents. This is only needed by non-'pbench-index' use cases. + """ + if self.es is None: + return + self.templates.update_templates(self.es, "server-reports") + + @staticmethod + def _make_json_payload(source): + """Given a source dictionary, return its ID, and a formatted JSON + payload. + """ + the_bytes = json.dumps(source, sort_keys=True).encode("utf-8") + source_id = hashlib.md5(the_bytes).hexdigest() + return source, source_id, the_bytes.decode("utf-8") + + def _gen_json_payload(self, base_source, file_to_index): + """Generate a series of JSON documents to be indexed, where the text + payload has a maximum size of 50 MB. + """ + total_size = os.path.getsize(file_to_index) + number_of_chunks = int(math.ceil(total_size / self._CHUNK_SIZE)) + chunk_id = 0 + with open(file_to_index, "r") as fp: + EOF = False + while not EOF: + chunk_id += 1 + text = fp.read(self._CHUNK_SIZE) + if not text: + EOF = True + else: + source = { + "chunk_id": chunk_id, + "total_chunks": number_of_chunks, + "total_size": total_size, + "text": text, + } + source.update(base_source) + yield self._make_json_payload(source) + + def _gen_no_json_payload(self, base_source): + """Just yield the base source document since we do not have a text + payload to add. + """ + yield self._make_json_payload(base_source) + + def post_status(self, timestamp, doctype, file_to_index=None): + """Post a status record, with an optional file payload to index along + with the base tracking document. + We return the tracking ID use for this report object. + """ + try: + if timestamp.startswith("run-"): + timestamp = timestamp[4:] + # Snip off the trailing "-" - assumes that does not + # contain a "-" + timestamp_noutc = timestamp.rsplit("-", 1)[0] + + base_source = { + "@timestamp": timestamp_noutc, + "@generated-by": self.generated_by, + "name": self.name, + "doctype": doctype, + } + if file_to_index: + payload_gen = self._gen_json_payload(base_source, file_to_index) + else: + payload_gen = self._gen_no_json_payload(base_source) + + if self.es is None: + # We don't have an Elasticsearch configuration, use syslog + # only. + + # Prepend the proper sequence to allow rsyslog to recognize + # JSON and process it. We only use the first chunk generated. + _, self.tracking_id, the_bytes = next(payload_gen) + payload = "@cee:{}".format(the_bytes) + if len(payload) > 4096: + # Compress the full message to a file. + fname = "report-status-payload.{}.{}.xz".format( + self.name, timestamp_noutc + ) + fpath = os.path.join(self.config.LOGSDIR, self.name, fname) + with lzma.open(fpath, mode="w", preset=9) as fp: + fp.write(the_bytes) + for _, _, the_bytes in payload_gen: + fp.write(the_bytes) + # Always log the first 4,096 bytes + self.logger.info("{}", payload[:4096]) + else: + # We have an Elasticsearch configuration. + + def _es_payload_gen(_payload_gen): + for source, source_id, _ in _payload_gen: + if self.tracking_id is None: + # First generated document becomes the tracking ID. + self.tracking_id = source_id + idx_name = self.templates.generate_index_name( + "server-reports", source + ) + action = { + "_op_type": _op_type, + "_index": idx_name, + "_type": "pbench-server-reports", + "_id": source_id, + "_source": source, + } + yield action + + es_res = es_index( + self.es, _es_payload_gen(payload_gen), sys.stderr, self.logger + ) + beg, end, successes, duplicates, failures, retries = es_res + if failures > 0: + log_action = self.logger.error + elif duplicates > 0 or retries > 0: + log_action = self.logger.warning + elif successes > 0: + # Don't log the normal success case + log_action = self.logger.debug + else: + # Warning when nothing happened + log_action = self.logger.warning + log_action( + "posted status (start ts: {}, end ts: {}, duration: {:.2f}s," + " successes: {:d}, duplicates: {:d}, failures: {:d}," + " retries: {:d})", + pbench.tstos(beg), + pbench.tstos(end), + end - beg, + successes, + duplicates, + failures, + retries, + ) + except Exception: + self.logger.exception( + "Failed to post status, name = {}," + " timestamp = {}, doctype = {}, file_to_index = {}", + self.name, + timestamp, + doctype, + file_to_index, + ) + raise + return self.tracking_id + + def _count_lines(fname): """Simple method to count the lines of a file.""" try: diff --git a/server/bin/pbench-pp-status b/server/bin/pbench-pp-status deleted file mode 100755 index 153285727a..0000000000 --- a/server/bin/pbench-pp-status +++ /dev/null @@ -1,59 +0,0 @@ -#!/usr/bin/env python3 - -# WIP use the following curl command to fetch report status to process: -# -# $ curl -X GET 'http://:/.server-reports.*/pbench-server-reports/_search?size=500' > ./response.json -# $ pbench-pp-status ./response.json | less -S - -import sys -import json -from operator import itemgetter - -with open(sys.argv[1], 'r') as fp: - doc = json.load(fp) -hits_d = None -query_ms = None -shards = None -timed_out = None -for key, val in doc.items(): - if key == 'hits': - hits_d = val - continue - if key == 'took': - query_ms = val - continue - if key == '_shards': - shards = val - continue - if key == 'timed_out': - timed_out = val - continue - print("{} = {}".format(key, val)) - -if shards['failed'] > 0: - shards_rpt = " ({:d} successful, {:d} failed)".format(shards['successful'], shards['failed']) -else: - shards_rpt = "" -if timed_out: - timed_out_rpt = " (timed out)" -else: - timed_out_rpt = "" - -total = None -hits_l = None -for key, val in hits_d.items(): - if key == 'total': - total = val - continue - if key == 'hits': - hits_l = val - continue - if key == 'max_score': - continue - print("{}".format(key)) - -print("Report Pull took {:d} ms{}, hitting {:d} total shards{}, matching {:d} total documents, returning {:d}".format(query_ms, timed_out_rpt, shards['total'], shards_rpt, total, len(hits_l))) - -for doc in sorted(hits_l, key=lambda k:"{} {} {:d}".format(k['_source']['@timestamp'], k['_source']['name'], k['_source']['chunk_id'])): - source = doc['_source'] - print("\n\n=======\n{} - {} ({:d} of {:d}) - text[{} of {:d} bytes]: \"{}\"".format(source['@timestamp'], source['name'], source['chunk_id'], source['total_chunks'], len(source['text']), source['total_size'], source['text'][:4096].strip())) diff --git a/server/bin/pbench-report-status b/server/bin/pbench-report-status deleted file mode 120000 index 09bdbcd420..0000000000 --- a/server/bin/pbench-report-status +++ /dev/null @@ -1 +0,0 @@ -pbench-trampoline \ No newline at end of file diff --git a/server/bin/pbench-report-status.py b/server/bin/pbench-report-status.py deleted file mode 100755 index f6a1cfd2d5..0000000000 --- a/server/bin/pbench-report-status.py +++ /dev/null @@ -1,89 +0,0 @@ -#!/usr/bin/env python3 -# -*- mode: python -*- - -import os -import sys -from argparse import ArgumentParser -from socket import gethostname - -from pbench import BadConfig, PbenchConfig -from pbench.report import Report - -if __name__ != "__main__": - sys.exit(1) - - -_prog = os.path.basename(sys.argv[0]) -if _prog.endswith(".py"): - _prog = _prog[:-3] - - -parser = ArgumentParser(_prog) -parser.add_argument("-C", "--config", dest="cfg_name", help="Specify config file") -parser.set_defaults(cfg_name=os.environ.get("_PBENCH_SERVER_CONFIG")) -parser.add_argument( - "-n", - "--name", - dest="name", - required=True, - help="Specify name of program reporting its status", -) -parser.add_argument( - "-t", - "--timestamp", - dest="timestamp", - required=True, - help="The timestamp that should be associated with the file to index," - " takes the form run---
T::-", -) -parser.add_argument("-p", "--pid", dest="pid", required=True, help="The caller's pid") -parser.add_argument( - "-g", - "--gid", - dest="group_id", - required=False, - help="The caller's group ID (optional)", -) -parser.add_argument( - "-u", - "--uid", - dest="user_id", - required=False, - help="The caller's user ID (optional)", -) -parser.add_argument( - "-T", - "--type", - dest="doctype", - required=True, - help="The type of report document to index, one of status|error", -) -parser.add_argument( - "file_to_index", nargs=1, help="The file containing the report to index" -) -parsed = parser.parse_args() - - -try: - config = PbenchConfig(parsed.cfg_name) -except BadConfig as e: - print("{}: {}".format(_prog, e), file=sys.stderr) - sys.exit(1) - - -hostname = gethostname() -pid = parsed.pid -group_id = parsed.group_id -user_id = parsed.user_id - -report = Report( - config, parsed.name, pid=pid, group_id=group_id, user_id=user_id, hostname=hostname -) -report.init_report_template() -try: - report.post_status(parsed.timestamp, parsed.doctype, parsed.file_to_index[0]) -except Exception: - status = 1 -else: - status = 0 -sys.exit(status) diff --git a/server/bin/pbench-satellite-cleanup.sh b/server/bin/pbench-satellite-cleanup.sh index a8c96f1080..0226819aa5 100755 --- a/server/bin/pbench-satellite-cleanup.sh +++ b/server/bin/pbench-satellite-cleanup.sh @@ -13,34 +13,28 @@ trap "rm -rf $tmp" EXIT log_init $PROG -log_info "$TS: $PROG starting" +start_ts=$(timestamp) tarballs=$(cd $ARCHIVE > /dev/null; find . -path '*/TO-DELETE/*.tar.xz' -printf '%P\n' | sort) hosts="$(for host in $tarballs ;do echo "${host%%/*}" ;done | sort -u )" -typeset -i ntb=0 -typeset -i nerrs=0 +typeset -i nhosts=0 +typeset -i ntbs=0 typeset -i ntberrs=0 typeset -i nmd5errs=0 typeset -i nstateerrs=0 typeset -i nincomingerrs=0 typeset -i nresultserrs=0 -mail_content=$tmp/mail.log -index_content=$tmp/index_mail_contents - -# Initialize mail content -> $mail_content -> $index_content - for host in $hosts; do + (( nhosts++ )) pushd $ARCHIVE/$host > /dev/null || log_exit "Failed to pushd to $ARCHIVE/$host" mkdir -p SATELLITE-DONE > /dev/null || log_exit "Failed to create $ARCHIVE/$host/SATELLITE-DONE" for tb in $tarballs; do if [ "$host" != "${tb%%/*}" ]; then continue fi - ntb=$ntb+1 + (( ntbs++ )) x=${tb##*/} # remove tar file if [ -e $x ]; then @@ -60,8 +54,8 @@ for host in $hosts; do # Note that if we do remove the tarball but fail to change the # state, this error will persist until the state is changed # manually. - log_error "$TS: Failed to remove $ARCHIVE/$host/$x, code: $rc" "${mail_content}" - ntberrs=$ntberrs+1 + log_error "$TS: Failed to remove $ARCHIVE/$host/$x, code: $rc" + (( ntberrs++ )) popd > /dev/null 2>&4 continue fi @@ -71,8 +65,8 @@ for host in $hosts; do rm $x.md5 rc=$? if [ $rc != 0 ]; then - log_error "$TS: Failed to remove $ARCHIVE/$host/$x.md5, code: $rc" "${mail_content}" - nmd5errs=$nmd5errs+1 + log_error "$TS: Failed to remove $ARCHIVE/$host/$x.md5, code: $rc" + (( nmd5errs++ )) fi fi # change the state to SATELLITE-DONE @@ -80,12 +74,12 @@ for host in $hosts; do mv -n TO-DELETE/$x SATELLITE-DONE/ rc=$? if [ $rc != 0 ]; then - log_error "$TS: Failed to move $ARCHIVE/$host/TO-DELETE/$x to SATELLITE-DONE, code: $rc" "${mail_content}" - nstateerrs=$nstateerrs+1 + log_error "$TS: Failed to move $ARCHIVE/$host/TO-DELETE/$x to SATELLITE-DONE, code: $rc" + (( nstateerrs++ )) else if [ -e TO-DELETE/$x ]; then - log_error "$TS: Failed to move $ARCHIVE/$host/TO-DELETE/$x: still exists after successful move to SATELLITE-DONE" "${mail_content}" - nstateerrs=$nstateerrs+1 + log_error "$TS: Failed to move $ARCHIVE/$host/TO-DELETE/$x: still exists after successful move to SATELLITE-DONE" + (( nstateerrs++ )) fi fi fi @@ -94,8 +88,8 @@ for host in $hosts; do rm -rf $INCOMING/$host/${x%%.tar.xz} rc=$? if [ $rc != 0 ]; then - log_error "$TS: Failed to remove the tarball from incoming directory: $INCOMING/$host/${x%%.tar.xz}, code: $rc" "${mail_content}" - nincomingerrs=$nincomingerrs+1 + log_error "$TS: Failed to remove the tarball from incoming directory: $INCOMING/$host/${x%%.tar.xz}, code: $rc" + (( nincomingerrs++ )) fi fi # remove the results @@ -104,31 +98,40 @@ for host in $hosts; do rm $sym_link rc=$? if [ $rc != 0 ]; then - log_error "$TS: Failed to remove results symlink: $sym_link, code: $rc" "${mail_content}" - nresultserrs=$nresultserrs+1 + log_error "$TS: Failed to remove results symlink: $sym_link, code: $rc" + (( nresultserrs++ )) fi fi done popd > /dev/null 2>&4 done -summary="Total $ntb tarballs cleaned up, with $ntberrs tarball removal errors, $nmd5errs md5 file \ -remove errors, $nstateerrs state change errors, $nincomingerrs incoming removal errors, and $nresultserrs \ -result removal errors errors." - -log_info "$TS: $PROG ends: $summary" +end_ts=$(timestamp) + +if [[ ${ntbs} -gt 0 ]]; then + summary_text="Total ${ntbs} tar balls cleaned up (for ${nhosts} hosts), with" + summary_text+=" ${ntberrs} tar ball removal errors, ${nmd5errs} md5 file" + summary_text+=" removal errors, ${nstateerrs} state change errors," + summary_text+=" ${nincomingerrs} incoming directory removal errors, and" + summary_text+=" ${nresultserrs} result directory removal errors." + printf -v summary_inner_json \ + "{\"%s\": \"%s\", \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": \"%s\", \"%s\": \"%s\"}" \ + "end_ts" "${end_ts}" \ + "errors" "$((nincomingerrs+nmd5errs+nresultserrs+nstateerrs+ntberrs))" \ + "nhosts" "${nhosts}" \ + "nincomingerrs" "${nincomingerrs}" \ + "nmd5errs" "${nmd5errs}" \ + "nresultserrs" "${nresultserrs}" \ + "nstateerrs" "${nstateerrs}" \ + "ntbs" "${ntbs}" \ + "ntberrs" "${ntberrs}" \ + "prog" "${PROG}" \ + "start_ts" "${start_ts}" + printf -v summary_json "{\"pbench\": {\"report\": {\"summary\": %s}}}" "${summary_inner_json}" + + log_info "${TS}(${PBENCH_ENV}): ${summary_text} -- @cee:${summary_json}" +fi log_finish -nerrs=$ntberrs+$nmd5errs+$nstateerrs+$nincomingerrs+$nresultserrs - -subj="$PROG.$TS($PBENCH_ENV) - w/ $nerrs total errors" -cat << EOF > $index_content -$subj -$summary - -EOF -cat $mail_content >> $index_content -pbench-report-status --name ${PROG} --pid ${$} --timestamp $(timestamp) --type status ${index_content} - exit 0 diff --git a/server/bin/pbench-sync-satellite.sh b/server/bin/pbench-sync-satellite.sh index 5f1b3d3bb0..dfa400c53a 100755 --- a/server/bin/pbench-sync-satellite.sh +++ b/server/bin/pbench-sync-satellite.sh @@ -59,24 +59,13 @@ function do_remote_sat_state_change { local status pbench-remote-satellite-state-change ${satellite_config} ${state_change_log} > ${logdir}/mv.log 2>&1 status=$? - if [[ $status != 0 ]] ;then - cat << EOF > $index_content -FAILED: $PROG: $remote_prefix: satellite state change failed twice. - -Unable to change the state directories of tarballs, in $remote_host, due to ssh failure. -EOF - pbench-report-status --name ${PROG} --pid ${$} --timestamp $(timestamp) --type error ${index_content} + if [[ $status != 0 ]]; then + log_error "${TS}: ${remote_prefix}: satellite state change failed twice, ssh failed to ${remote_host}" else rm ${state_change_log} status=$? if [[ $status != 0 ]]; then - cat << EOF > $index_content -$PROG ($remote_prefix): Failed to remove local "${state_change_log}" - -ssh to the $remote_host was sucessfull for changing the state of tarballs, -but failed to remove local "${state_change_log}". -EOF - pbench-report-status --name ${PROG} --pid ${$} --timestamp $(timestamp) --type error ${index_content} + log_error "${TS}: ${remote_prefix}: Failed to remove local '${state_change_log}'" else rm ${logdir}/mv.log fi @@ -84,14 +73,8 @@ EOF return $status } -# accumulate errors in a file for mailing at end -mail_content=$tmp/mail.log -> $mail_content -index_content=$tmp/index_mail_contents -> $index_content - let start_time=$(timestamp-seconds-since-epoch) -log_info "$TS: start - $(timestamp)" "${mail_content}" +start_ts=$(timestamp) # NOTE: the log file for tracking which tarballs have to be updated on # the remote satellite server is shared between runs, so it is not in @@ -101,19 +84,21 @@ state_change_log=$logdir_for_remote/change_state.log # check whether any previous ssh failure; if any try again here before # we try to process any new tar balls on the satellite server. If it # fails again then exit without going further. -if [ -s ${state_change_log} ] ;then - log_error "$PROG: completing previous satellite state changes ... (${state_change_log})" "${mail_content}" +if [ -s ${state_change_log} ]; then + log_error "${TS}: Completing previous satellite state changes ... (${state_change_log})" do_remote_sat_state_change status=$? if [[ $status != 0 ]]; then - log_exit "$PROG: unable to complete previous satellite state changes (${state_change_log})" + log_exit "${TS}: unable to complete previous satellite state changes (${state_change_log})" fi - log_error "$PROG: completed previous satellite state changes" "${mail_content}" + log_error "${TS}: completed previous satellite state changes" else # initialize state change log > ${state_change_log} fi +typeset -i nhosts=0 +typeset -i ntotal=0 typeset -i nprocessed=0 typeset -i nerrs=0 @@ -122,22 +107,18 @@ syncerr=${tmp}/syncerrors # Fetch all the tarballs from remote host's archive pbench-remote-sync-package-tarballs ${satellite_config} ${tmp}/satellite.${remote_prefix}.tar ${syncerr} rc=$? -if [[ $rc != 0 ]] ;then - log_exit "FAILED: $(cat ${syncerr})" 2 "${mail_content}" +if [[ $rc != 0 ]]; then + log_exit "${TS}: FAILED -- $(cat ${syncerr})" 2 fi if [ -s $tmp/satellite.$remote_prefix.tar ]; then - log_info "$TS: remote tarballs fetched, unpacking ... - $(timestamp)" "${mail_content}" - - # unpack the tarball into tmp directory + # Unpack the tarball into tmp directory, logging any errors reported. tar -xf $tmp/satellite.$remote_prefix.tar -C $unpack if [[ $? -ne 0 ]]; then cat $tmp/satellite.$remote_prefix.tar >&4 fi files=$(find $unpack -path '*.tar.xz' -printf '%P\n') hosts="$(for host in $files;do echo ${host%%/*};done | sort -u )" - - log_info "$TS: remote tarballs unpacked - $(timestamp)" "${mail_content}" else hosts="" fi @@ -146,6 +127,7 @@ dest=$(getconf.py pbench-receive-dir-prefix pbench-server)-002 let unpack_start_time=$(timestamp-seconds-since-epoch) for host in $hosts ;do + (( nhosts++ )) localdir=$dest/$remote_prefix::$host pushd $localdir > /dev/null 2>&1 @@ -154,15 +136,15 @@ for host in $hosts ;do mkdir -p $localdir rc=$? if [ $rc -ne 0 ]; then - nerrs=$nerrs+1 - log_error "$PROG: failed to create remote controller in archive, $localdir" "${mail_content}" + (( nerrs++ )) + log_error "${TS}: failed to create remote controller in archive, $localdir" continue else pushd $localdir > /dev/null rc=$? if [ $rc -ne 0 ]; then - nerrs=$nerrs+1 - log_error "$PROG: failed to pushd to remote controller in archive, $localdir" "${mail_content}" + (( nerrs++ )) + log_error "${TS}: failed to pushd to remote controller in archive, $localdir" continue fi fi @@ -176,18 +158,19 @@ for host in $hosts ;do # without a .md5 file present, then moving the .md5 files into place, and # recording success for updating the remote host. while read tbmd5; do + (( ntotal++ )) mv ${tbmd5%*.md5} ./ status=$? - if [[ $status != 0 ]] ;then - nerrs=$nerrs+1 + if [[ $status != 0 ]]; then + (( nerrs++ )) log_error "Failure moving tar ball ${tbmd5%*.md5} to $localdir" continue fi mv ${tbmd5} ./ status=$? - if [[ $status != 0 ]] ;then + if [[ $status != 0 ]]; then rm -f ${tbmd5%*.md5} - nerrs=$nerrs+1 + (( nerrs++ )) log_error "Failure moving tar ball MD5 file ${tbmd5} to $localdir" continue fi @@ -199,32 +182,40 @@ for host in $hosts ;do done # change the state of the tarballs on remote -if [ -s ${state_change_log} ] ;then - log_error "$PROG: completing satellite state changes ... (${state_change_log})" "${mail_content}" +if [ -s ${state_change_log} ]; then + log_error "$PROG: completing satellite state changes ... (${state_change_log})" do_remote_sat_state_change status=$? if [[ $status != 0 ]]; then - log_error "$PROG: unable to complete satellite state changes (${state_change_log})" "${mail_content}" + log_error "$PROG: unable to complete satellite state changes (${state_change_log})" else - log_error "$PROG: completed satellite state changes" "${mail_content}" + log_error "$PROG: completed satellite state changes" fi fi let end_time=$(timestamp-seconds-since-epoch) +end_ts=$(timestamp) let duration=end_time-start_time -log_info "$TS: end - $(timestamp)" "${mail_content}" -log_info "$TS: duration (secs): $duration" "${mail_content}" -log_info "$TS: Total $nprocessed files processed and $nerrs errors" "${mail_content}" +if [[ ${nhosts} -gt 0 ]]; then + summary_text="${remote_prefix} Processed ${nprocessed} files (from" + summary_text+=" ${nhosts} hosts, ${ntotal} total tar balls), with ${nerrs}" + summary_text+=" errors, in ${duration} seconds" + printf -v summary_inner_json \ + "{\"%s\": %d, \"%s\": \"%s\", \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": \"%s\", \"%s\": \"%s\", \"%s\": \"%s\"}" \ + "duration" "${duration}" \ + "end_ts" "${end_ts}" \ + "errors" "${nerrs}" \ + "nhosts" "${nhosts}" \ + "nprocessed" "${nprocessed}" \ + "ntotal" "${ntotal}" \ + "prog" "${PROG}" \ + "remote_prefix" "${remote_prefix}" \ + "start_ts" "${start_ts}" + printf -v summary_json "{\"pbench\": {\"report\": {\"summary\": %s}}}" "${summary_inner_json}" + + log_info "${TS}(${PBENCH_ENV}): ${summary_text} -- @cee:${summary_json}" +fi log_finish -subj="$PROG.$TS($PBENCH_ENV) - w/ $nerrs errors" -cat << EOF > $index_content -$subj -Remote $remote_prefix: processed $nprocessed files and $nerrs errors. - -EOF -cat $mail_content >> $index_content -pbench-report-status --name ${PROG} --pid ${$} --timestamp $(timestamp) --type status ${index_content} - exit 0 diff --git a/server/bin/pbench-unpack-tarballs.sh b/server/bin/pbench-unpack-tarballs.sh index 0df7cbcb9d..c826118340 100755 --- a/server/bin/pbench-unpack-tarballs.sh +++ b/server/bin/pbench-unpack-tarballs.sh @@ -88,13 +88,7 @@ mkdir -p ${tmp} || doexit "Failed to create ${tmp}" log_init ${PROG} -log_info "${TS}" - -# Accumulate errors and logs in files for reporting at the end. -mail_content=${tmp}/mail_content.log -> ${mail_content} -index_content=${tmp}/index_mail_contents -> ${index_content} +start_ts=$(timestamp) # The list of files we'll be operating on, when complete, sorted newest to # oldest by last modification time. @@ -135,7 +129,7 @@ function move_symlink() { mv ${ARCHIVE}/${hostname}/${linksrc}/${resultname}.tar.xz ${ARCHIVE}/${hostname}/${linkdest}/${resultname}.tar.xz local status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: Cannot move symlink ${ARCHIVE}/${hostname}/${resultname}.tar.xz from ${linksrc} to ${linkdest}: code ${status}" "${mail_content}" + log_error "${TS}: Cannot move symlink ${ARCHIVE}/${hostname}/${resultname}.tar.xz from ${linksrc} to ${linkdest}: code ${status}" fi return ${status} } @@ -145,15 +139,15 @@ function do_work() { local status=0 local max_seconds=${1} while read date size result; do - ntotal=${ntotal}+1 + (( ntotal++ )) resultname=$(basename ${result}) resultname=${resultname%.tar.xz} link=$(readlink -e ${result}) if [[ -z "${link}" ]]; then - log_error "${TS}: symlink target for ${result} does not exist" "${mail_content}" - nerrs=${nerrs}+1 + (( nerrs++ )) + log_error "${TS}: symlink target for ${result} does not exist" hostname=$(basename $(dirname $(dirname ${result}))) mkdir -p ${ARCHIVE}/${hostname}/${linkerr} move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for symlink" @@ -164,8 +158,8 @@ function do_work() { pbench-check-tb-age ${link} local status=${?} if [[ ${status} -gt 0 ]]; then - log_info "${TS}: ${result} is older than the configured maximum age (status = ${status})" "${mail_content}" - nwarn=${nwarn}+1 + (( nwarn++ )) + log_info "${TS}: ${result} is older than the configured maximum age (status = ${status})" hostname=$(basename $(dirname $(dirname ${result}))) mkdir -p ${ARCHIVE}/${hostname}/${linkerr} move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for symlink" @@ -180,16 +174,16 @@ function do_work() { mk_dirs ${hostname} status=${?} if [[ ${?} -ne 0 ]]; then - log_error "${TS}: Creation of ${hostname} processing directories failed for ${result}: code ${status}" "${mail_content}" - nerrs=${nerrs}+1 + (( nerrs++ )) + log_error "${TS}: Creation of ${hostname} processing directories failed for ${result}: code ${status}" if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi continue fi incoming=${INCOMING}/${hostname}/${resultname} if [[ -e ${incoming} ]]; then - log_error "${TS}: Incoming result, ${incoming}, already exists, skipping ${result}" "${mail_content}" - nerrs=${nerrs}+1 + (( nerrs++ )) + log_error "${TS}: Incoming result, ${incoming}, already exists, skipping ${result}" move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for already unpacked" if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi continue @@ -198,8 +192,8 @@ function do_work() { mkdir -p ${incoming}.unpack status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: 'mkdir ${incoming}.unpack' failed for ${result}: code ${status}" "${mail_content}" - nerrs=${nerrs}+1 + (( nerrs++ )) + log_error "${TS}: 'mkdir ${incoming}.unpack' failed for ${result}: code ${status}" popd > /dev/null 2>&1 if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi continue @@ -208,9 +202,9 @@ function do_work() { tar --extract --no-same-owner --touch --delay-directory-restore --file="${result}" --force-local --directory="${incoming}.unpack" status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: 'tar -xf ${result}' failed: code ${status}" "${mail_content}" + (( nerrs++ )) + log_error "${TS}: 'tar -xf ${result}' failed: code ${status}" rm -rf ${incoming}.unpack - nerrs=${nerrs}+1 move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for failed untar" if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi continue @@ -220,8 +214,8 @@ function do_work() { find ${incoming}.unpack/${resultname} -type d -print0 | xargs -0 chmod ugo+rx status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: 'chmod ugo+rx' of subdirs ${resultname} for ${result} failed: code ${status}" "${mail_content}" - nerrs=${nerrs}+1 + (( nerrs++ )) + log_error "${TS}: 'chmod ugo+rx' of subdirs ${resultname} for ${result} failed: code ${status}" rm -rf ${incoming}.unpack move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for failed find/chmod" if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi @@ -232,8 +226,8 @@ function do_work() { chmod -R ugo+r ${incoming}.unpack/${resultname} status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: 'chmod -R ugo+r ${resultname}' for ${result} failed: code ${status}" "${mail_content}" - nerrs=${nerrs}+1 + (( nerrs++ )) + log_error "${TS}: 'chmod -R ugo+r ${resultname}' for ${result} failed: code ${status}" rm -rf ${incoming}.unpack move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for failed chmod" if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi @@ -244,9 +238,9 @@ function do_work() { mv ${incoming}.unpack/${resultname} ${INCOMING}/${hostname}/ status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: '${result}' does not contain ${resultname} directory at the top level; skipping" "${mail_content}" + (( nerrs++ )) + log_error "${TS}: '${result}' does not contain ${resultname} directory at the top level; skipping" rm -rf ${incoming}.unpack - nerrs=${nerrs}+1 move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for failed mv" if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi continue @@ -254,9 +248,9 @@ function do_work() { rmdir ${incoming}.unpack status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: WARNING - '${result}' should only contain the ${resultname} directory at the top level, ignoring other content" "${mail_content}" + (( nwarn++ )) + log_error "${TS}: WARNING - '${result}' should only contain the ${resultname} directory at the top level, ignoring other content" rm -rf ${incoming}.unpack - nwarn=${nwarn}+1 fi # Version 002 agents use the metadata log to store a prefix. @@ -274,9 +268,9 @@ function do_work() { mkdir -p ${RESULTS}/${hostname}/${prefix} status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: mkdir -p ${RESULTS}/${hostname}/${prefix} for ${result} failed: code ${status}" "${mail_content}" + (( nerrs++ )) + log_error "${TS}: mkdir -p ${RESULTS}/${hostname}/${prefix} for ${result} failed: code ${status}" rm -rf ${incoming} - nerrs=${nerrs}+1 move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for failed mkdir results prefix" if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi continue @@ -286,8 +280,8 @@ function do_work() { ln -s ${incoming} ${RESULTS}/${hostname}/${prefix}${resultname} status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: ln -s ${incoming} ${RESULTS}/${hostname}/${prefix}${resultname} for ${result} failed: code ${status}" "${mail_content}" - nerrs=${nerrs}+1 + (( nerrs++ )) + log_error "${TS}: ln -s ${incoming} ${RESULTS}/${hostname}/${prefix}${resultname} for ${result} failed: code ${status}" rm -rf ${incoming} move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for failed ln results prefix" if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi @@ -299,10 +293,10 @@ function do_work() { mkdir -p ${USERS}/${user}/${hostname}/${prefix} status=${?} if [[ ${status} -ne 0 ]]; then - log_error "${TS}: mkdir -p ${USERS}/${user}/${hostname}/${prefix} for ${result} failed: code ${status}" "${mail_content}" + (( nerrs++ )) + log_error "${TS}: mkdir -p ${USERS}/${user}/${hostname}/${prefix} for ${result} failed: code ${status}" rm -rf ${incoming} rm -f ${RESULTS}/${hostname}/${prefix}${resultname} - nerrs=${nerrs}+1 move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for failed mkdir users prefix" if [[ ${SECONDS} -ge ${max_seconds} ]]; then break; fi continue @@ -311,10 +305,9 @@ function do_work() { log_info "ln -s ${incoming} ${USERS}/${user}/${hostname}/${prefix}${resultname}" ln -s ${incoming} ${USERS}/${user}/${hostname}/${prefix}${resultname} status=${?} - if [[ ${status} -ne 0 ]]; then - log_error "${TS}: code ${status}: ln -s ${incoming} ${USERS}/${user}/${hostname}/${prefix}${resultname}" "${mail_content}" - nerrs=${nerrs}+1 + (( nerrs++ )) + log_error "${TS}: code ${status}: ln -s ${incoming} ${USERS}/${user}/${hostname}/${prefix}${resultname}" rm -rf ${incoming} rm -f ${RESULTS}/${hostname}/${prefix}${resultname} move_symlink ${hostname} ${resultname} ${linksrc} ${linkerr} || doexit "Error handling failed for failed ln users prefix" @@ -326,7 +319,7 @@ function do_work() { move_symlink ${hostname} ${resultname} ${linksrc} ${linkdest} status=${?} if [[ ${status} -ne 0 ]]; then - nerrs=${nerrs}+1 + (( nerrs++ )) # Cleanup needed here but trap takes care of it. rm -rf ${incoming} rm -f ${RESULTS}/${hostname}/${prefix}${resultname} @@ -339,28 +332,25 @@ function do_work() { # Create a link in each state dir - if any fail, we should delete them all? No, that # would be racy. let toterr=0 - let totsuc=0 for state in ${linkdestlist}; do ln -sf ${ARCHIVE}/${hostname}/${resultname}.tar.xz ${ARCHIVE}/${hostname}/${state}/${resultname}.tar.xz status=${?} - if [[ ${status} -eq 0 ]]; then - let totsuc+=1 - else - log_error "${TS}: Cannot create ${ARCHIVE}/${hostname}/${resultname}.tar.xz link in state ${state}: code ${status}" "${mail_content}" - let toterr+=1 + if [[ ${status} -ne 0 ]]; then + log_error "${TS}: Cannot create ${ARCHIVE}/${hostname}/${resultname}.tar.xz link in state ${state}: code ${status}" + (( toterr++ )) fi done if [[ ${toterr} -gt 0 ]]; then # Count N link creations as one error since it is for handling of a # single tar ball. - let nerrs+=1 + (( nerrs++ )) fi let end_time=$(timestamp-seconds-since-epoch) let duration=end_time-start_time # log the success log_info "${TS}: ${hostname}/${resultname}: success - elapsed time (secs): ${duration} - size (bytes): ${size}" - ntb=${ntb}+1 + (( ntbs++ )) # The job currently default to running once a minute, but once unpack # tar balls starts running, we want to re-check for new tar balls that @@ -372,11 +362,11 @@ function do_work() { done } -typeset -i ntb=0 +typeset -i ntbs=0 typeset -i ntotal=0 typeset -i nerrs=0 -typeset -i ndups=0 typeset -i nwarn=0 +typeset -i nloops=0 while true; do gen_work_list @@ -385,19 +375,29 @@ while true; do break fi do_work ${max_seconds} < ${list} + (( nloops++ )) done -log_info "${TS}: Processed ${ntb} tar balls" +end_ts=$(timestamp) + +if [[ ${ntotal} -gt 0 ]]; then + (( nloops-- )) + summary_text="Processed ${ntotal} result tar balls, ${ntbs} successfully, with ${nloops} rechecks, ${nwarn} warnings, and ${nerrs} errors" + printf -v summary_inner_json \ + "{\"%s\": \"%s\", \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": %d, \"%s\": \"%s\", \"%s\": \"%s\"}" \ + "end_ts" "${end_ts}" \ + "errors" "${nerrs}" \ + "nrechecks" "${nloops}" \ + "ntbs" "${ntbs}" \ + "ntotal" "${ntotal}" \ + "nwarn" "${nwarn}" \ + "prog" "${PROG}" \ + "start_ts" "${start_ts}" + printf -v summary_json "{\"pbench\": {\"report\": {\"summary\": %s}}}" "${summary_inner_json}" + + log_info "${TS}(${PBENCH_ENV}): ${summary_text} -- @cee:${summary_json}" +fi log_finish -subj="${PROG}.${TS}(${PBENCH_ENV}) - w/ ${nerrs} errors" -cat << EOF > ${index_content} -${subj} -Processed ${ntotal} result tar balls, ${ntb} successfully, ${nwarn} warnings, ${nerrs} errors, and ${ndups} duplicates - -EOF -cat ${mail_content} >> ${index_content} -pbench-report-status --name ${PROG} --pid ${$} --timestamp $(timestamp) --type status ${index_content} - exit 0 diff --git a/server/bin/state/test-23.config/pbench-server.cfg b/server/bin/state/test-23.config/pbench-server.cfg deleted file mode 100644 index a533e0e31a..0000000000 --- a/server/bin/state/test-23.config/pbench-server.cfg +++ /dev/null @@ -1,51 +0,0 @@ -install-dir = %(unittest-dir)s/opt/pbench-server - -########################################################################### -## Deployment section -########################################################################### -[pbench-server] -pbench-top-dir = %(unittest-dir)s/pbench -pbench-local-dir = %(unittest-dir)s/pbench-local -pbench-backup-dir = %(pbench-local-dir)s/archive.backup -# Add role for sync'ing with satellites -roles = pbench-maintenance, pbench-results, pbench-sync-satellites -debug_unittest = True -commit_id = unit-test -environment = unit-test -admin-email = unit-test-user@example.com - -[Indexing] -server = elasticsearch.example.com:9280 -# No index_prefix so logger should be used in pbench-report-status - -[apache] -documentroot = %(unittest-dir)s/var-www-html - -[logging] -logger_type = file - -########################################################################### -# crontab roles - -# The definition of the crontab role for sync'ing satellite pbench servers. -[pbench-sync-satellites] -host = %(default-host)s -satellites = satellite-one -tasks = pbench-sync - -# Template values for this satellite -[satellite-one] -# NOTE WELL: this satellite host name, pbench-satellite.example.com, is the -# host name expected by the mock ssh command which triggers the behavior of -# running the actual commands instead of just echoing them. -satellite-host = pbench-satellite.example.com -satellite-prefix = ONE -satellite-lock = pbench-sync-satellite-%(satellite-prefix)s.lock -satellite-archive = %(unittest-dir)s/pbench-satellite/archive/fs-version-001 -satellite-opt = %(unittest-dir)s/opt/pbench-server-satellite - -########################################################################### -# The rest will come from the default config file. -[config] -path = %(install-dir)s/lib/config -files = pbench-server-default.cfg diff --git a/server/bin/state/test-23.setup b/server/bin/state/test-23.setup deleted file mode 120000 index 7d752b1825..0000000000 --- a/server/bin/state/test-23.setup +++ /dev/null @@ -1 +0,0 @@ -test-12.setup \ No newline at end of file diff --git a/server/bin/test-bin/curl b/server/bin/test-bin/curl deleted file mode 100755 index b3f3ea462e..0000000000 --- a/server/bin/test-bin/curl +++ /dev/null @@ -1,18 +0,0 @@ -#! /bin/bash - -echo "${0##*/} ${*}" | sed 's/pbench-report-status.[0-9][0-9]*/pbench-report-status.NNNN/' >> ${_testlog} - -file="" -while [ ! -z "${1}" ]; do - if [ "${1}" != "--data" ]; then - shift - continue - fi - shift - file=${1#@} -done -if [ ! -z "${file}" -a -e "${file}" ]; then - cat ${file} >> ${_testcurlpayload} -fi -printf "201" -exit 0 diff --git a/server/bin/test-bin/logger b/server/bin/test-bin/logger deleted file mode 100755 index 7d8d0eca5f..0000000000 --- a/server/bin/test-bin/logger +++ /dev/null @@ -1,6 +0,0 @@ -#! /bin/bash - -echo "${0##*/} $*" >> ${_testlog} -cat >> ${_testloggerpayload} -printf "\n" >> ${_testloggerpayload} -exit 0 diff --git a/server/bin/unittests b/server/bin/unittests index d537d641ee..0ba771dd27 100755 --- a/server/bin/unittests +++ b/server/bin/unittests @@ -24,6 +24,8 @@ if [[ ! -d $_testbase ]]; then exit 1 fi +alias logger="logger --no-act --stderr" + function remove_path { # PATH ($2) => /bin:/opt/a dir/bin:/sbin WORK=:$2: @@ -212,18 +214,6 @@ function _dump_logs { _normalize_output $_testlog >> $_testout 2>&1 echo "--- $(basename $_testlog) file contents" >> $_testout fi - - if [ -s $_testcurlpayload ]; then - echo "+++ $(basename $_testcurlpayload) file contents" >> $_testout - _normalize_output $_testcurlpayload >> $_testout 2>&1 - echo "--- $(basename $_testcurlpayload) file contents" >> $_testout - fi - - if [ -s $_testloggerpayload ]; then - echo "+++ $(basename $_testloggerpayload) file contents" >> $_testout - _normalize_output $_testloggerpayload >> $_testout 2>&1 - echo "--- $(basename $_testloggerpayload) file contents" >> $_testout - fi } function _verify_output { @@ -486,8 +476,6 @@ function _run_test { export _testactout=$_testroot/actoutput.txt export _testactlog=$_testroot/test-activation-execution.log export _testlog=$_testroot/test-execution.log - export _testcurlpayload=$_testroot/test-curl-payload.log - export _testloggerpayload=$_testroot/test-logger-payload.log export _testdir=$_testroot/pbench export _testdir_local=$_testroot/pbench-local export _testdir_sat=$_testroot/pbench-satellite @@ -533,8 +521,6 @@ function _reset_state { rm -f $_testactout rm -f $_testlog rm -f $_testactlog - rm -f $_testcurlpayload - rm -f $_testloggerpayload rm -rf $_testdir if [[ -d $_testdir ]]; then echo "ERROR: unable to remove pbench hierarchy" >&2 @@ -705,9 +691,6 @@ declare -A cmds=( # Actually exercises satellite code [test-22]="_run pbench-satellite-cleanup" - # Uses pbench-dispatch to test report status via syslog - [test-23]="_run pbench-dispatch" - # Tests that the behavior of the find command works as we expect to # ensure that buckets for pbench-unpack-tarballs work properly. [test-25]="_run test-find-behavior" diff --git a/server/lib/pbench/report.py b/server/lib/pbench/report.py deleted file mode 100644 index cb6386c8df..0000000000 --- a/server/lib/pbench/report.py +++ /dev/null @@ -1,246 +0,0 @@ -"""Reporting module for pbench server. -""" - -import hashlib -import json -import lzma -import math -import os -import socket -import sys -from configparser import Error as NoSectionError -from configparser import NoOptionError - -import pbench -from pbench.indexer import PbenchTemplates, _op_type, es_index, get_es - - -class Report(object): - """Encapsulation of server-side reporting information for recording who, - what, when, and how of a component's operation. - """ - - # We set the chunk size to 50 MB which is half the maximum payload size - # that Elasticsearch typically handles. - _CHUNK_SIZE = 50 * 1024 * 1024 - - def __init__( - self, - config, - name, - es=None, - pid=None, - group_id=None, - user_id=None, - hostname=None, - version=None, - templates=None, - ): - self.config = config - self.name = name - self.logger = pbench.get_pbench_logger(name, config) - - # We always create a base "tracking" document composed of parameters - # from the caller, and other environmental data. This document is used - # as the foundation for the first document posted to the target - # Elasticsearch instance with the `post_status()` method. All - # subsequent calls to the `post_status()` method will use that first - # document ID as their parent document ID. This allows us to have - # multiple status updates associated with the initial Report() caller. - if config._unittests: - _hostname = "example.com" - _pid = 42 - _group_id = 43 - _user_id = 44 - else: - _hostname = hostname if hostname else socket.gethostname() - _pid = pid if pid else os.getpid() - _group_id = group_id if group_id else os.getgid() - _user_id = user_id if user_id else os.getuid() - self.generated_by = dict( - [ - ("commit_id", self.config.COMMIT_ID), - ("group_id", _group_id), - ("hostname", _hostname), - ("pid", _pid), - ("user_id", _user_id), - ("version", version if version else ""), - ] - ) - # The "tracking_id" is the final MD5 hash of the first document - # indexed via the `post_status()` method. - self.tracking_id = None - try: - self.idx_prefix = config.get("Indexing", "index_prefix") - except (NoOptionError, NoSectionError): - # No index prefix so reporting will be performed via logging. - self.idx_prefix = None - self.es = None - else: - if es is None: - try: - self.es = get_es(config, self.logger) - except Exception: - self.logger.exception( - "Unexpected failure fetching" " Elasticsearch configuration" - ) - # If we don't have an Elasticsearch configuration just use - # None to indicate logging should be used instead. - self.es = None - else: - self.es = es - if templates is not None: - self.templates = templates - else: - self.templates = PbenchTemplates( - self.config.BINDIR, self.idx_prefix, self.logger - ) - - def init_report_template(self): - """Setup the Elasticsearch templates needed for properly indexing - report documents. This is only needed by non-'pbench-index' use cases. - """ - if self.es is None: - return - self.templates.update_templates(self.es, "server-reports") - - @staticmethod - def _make_json_payload(source): - """Given a source dictionary, return its ID, and a formatted JSON - payload. - """ - the_bytes = json.dumps(source, sort_keys=True).encode("utf-8") - source_id = hashlib.md5(the_bytes).hexdigest() - return source, source_id, the_bytes.decode("utf-8") - - def _gen_json_payload(self, base_source, file_to_index): - """Generate a series of JSON documents to be indexed, where the text - payload has a maximum size of 50 MB. - """ - total_size = os.path.getsize(file_to_index) - number_of_chunks = int(math.ceil(total_size / self._CHUNK_SIZE)) - chunk_id = 0 - with open(file_to_index, "r") as fp: - EOF = False - while not EOF: - chunk_id += 1 - text = fp.read(self._CHUNK_SIZE) - if not text: - EOF = True - else: - source = { - "chunk_id": chunk_id, - "total_chunks": number_of_chunks, - "total_size": total_size, - "text": text, - } - source.update(base_source) - yield self._make_json_payload(source) - - def _gen_no_json_payload(self, base_source): - """Just yield the base source document since we do not have a text - payload to add. - """ - yield self._make_json_payload(base_source) - - def post_status(self, timestamp, doctype, file_to_index=None): - """Post a status record, with an optional file payload to index along - with the base tracking document. - - We return the tracking ID use for this report object. - """ - try: - if timestamp.startswith("run-"): - timestamp = timestamp[4:] - # Snip off the trailing "-" - assumes that does not - # contain a "-" - timestamp_noutc = timestamp.rsplit("-", 1)[0] - - base_source = { - "@timestamp": timestamp_noutc, - "@generated-by": self.generated_by, - "name": self.name, - "doctype": doctype, - } - if file_to_index: - payload_gen = self._gen_json_payload(base_source, file_to_index) - else: - payload_gen = self._gen_no_json_payload(base_source) - - if self.es is None: - # We don't have an Elasticsearch configuration, use syslog - # only. - - # Prepend the proper sequence to allow rsyslog to recognize - # JSON and process it. We only use the first chunk generated. - _, self.tracking_id, the_bytes = next(payload_gen) - payload = "@cee:{}".format(the_bytes) - if len(payload) > 4096: - # Compress the full message to a file. - fname = "report-status-payload.{}.{}.xz".format( - self.name, timestamp_noutc - ) - fpath = os.path.join(self.config.LOGSDIR, self.name, fname) - with lzma.open(fpath, mode="w", preset=9) as fp: - fp.write(the_bytes) - for _, _, the_bytes in payload_gen: - fp.write(the_bytes) - # Always log the first 4,096 bytes - self.logger.info("{}", payload[:4096]) - else: - # We have an Elasticsearch configuration. - - def _es_payload_gen(_payload_gen): - for source, source_id, _ in _payload_gen: - if self.tracking_id is None: - # First generated document becomes the tracking ID. - self.tracking_id = source_id - idx_name = self.templates.generate_index_name( - "server-reports", source - ) - action = { - "_op_type": _op_type, - "_index": idx_name, - "_type": "pbench-server-reports", - "_id": source_id, - "_source": source, - } - yield action - - es_res = es_index( - self.es, _es_payload_gen(payload_gen), sys.stderr, self.logger - ) - beg, end, successes, duplicates, failures, retries = es_res - if failures > 0: - log_action = self.logger.error - elif duplicates > 0 or retries > 0: - log_action = self.logger.warning - elif successes > 0: - # Don't log the normal success case - log_action = self.logger.debug - else: - # Warning when nothing happened - log_action = self.logger.warning - log_action( - "posted status (start ts: {}, end ts: {}, duration: {:.2f}s," - " successes: {:d}, duplicates: {:d}, failures: {:d}," - " retries: {:d})", - pbench.tstos(beg), - pbench.tstos(end), - end - beg, - successes, - duplicates, - failures, - retries, - ) - except Exception: - self.logger.exception( - "Failed to post status, name = {}," - " timestamp = {}, doctype = {}, file_to_index = {}", - self.name, - timestamp, - doctype, - file_to_index, - ) - raise - return self.tracking_id diff --git a/server/rpm/pbench-server.spec.j2 b/server/rpm/pbench-server.spec.j2 index 0391346473..5ce26fecff 100644 --- a/server/rpm/pbench-server.spec.j2 +++ b/server/rpm/pbench-server.spec.j2 @@ -61,7 +61,7 @@ cp -a ./web-server/* %{buildroot}/%{installdir}/%{static} # for the npm install below mv %{buildroot}/%{installdir}/%{static}/package.json %{buildroot}/%{installdir} -%py3_shebang_fix %{buildroot}/%{installdir}/bin %{buildroot}/%{installdir}/bin/pbench-pp-status +%py3_shebang_fix %{buildroot}/%{installdir}/bin %post # Install python dependencies @@ -126,7 +126,6 @@ fi /%{installdir}/lib/pbench/__init__.py /%{installdir}/lib/pbench/indexer.py /%{installdir}/lib/pbench/mock.py -/%{installdir}/lib/pbench/report.py /%{installdir}/bin/pbench-base.sh @@ -145,13 +144,10 @@ fi /%{installdir}/bin/pbench-dispatch.sh /%{installdir}/bin/pbench-index /%{installdir}/bin/pbench-index.py -/%{installdir}/bin/pbench-pp-status /%{installdir}/bin/pbench-reindex /%{installdir}/bin/pbench-reindex.py /%{installdir}/bin/pbench-remote-satellite-state-change /%{installdir}/bin/pbench-remote-sync-package-tarballs -/%{installdir}/bin/pbench-report-status -/%{installdir}/bin/pbench-report-status.py /%{installdir}/bin/pbench-satellite-cleanup /%{installdir}/bin/pbench-satellite-cleanup.sh /%{installdir}/bin/pbench-satellite-state-change