Refactoring log format by removing metadata addition in log statements. #4220
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: AmarnatReddy The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
AmarnatReddy
force-pushed
the
log_rewap
branch
from
918ba99
to
31059ab
Compare
There was a problem hiding this comment.
@AmarnatReddy thanks for taking the initiative on fixing the logging.
if I understand correctly, we decided to remove this entire file.
|
Thanks @AmarnatReddy for the explanation. This is good approach and the modification effort is minimal. Let's evaluate parallel. if it is working, we should be able to get this merged. |
AmarnatReddy
force-pushed
the
log_rewap
branch
2 times, most recently
from
975974b
to
1e56ec3
Compare
| ) | ||
| console_handler = logging.StreamHandler() |
There was a problem hiding this comment.
By default logging to console is enabled. Do we need this?
There was a problem hiding this comment.
HI Satya
With out this parallel logs of the suites were not logging to console
tests/parallel/test_parallel.py
Outdated
| rc = test_mod.run( | ||
| ceph_cluster=args["ceph_cluster"], | ||
| ceph_nodes=args["ceph_nodes"], | ||
| config=args["config"], | ||
| test_data=args["test_data"], | ||
| ceph_cluster_dict=args["ceph_cluster_dict"], | ||
| clients=args["clients"], |
There was a problem hiding this comment.
Can we ensure we run.py and test_parallel.py use the same signature when calling test module run method?
| self.info = self._logger.info | ||
| self.debug = self._logger.debug | ||
| self.warning = self._logger.warning | ||
| self.error = self._logger.error |
There was a problem hiding this comment.
We have a method error and an attribute error likewize self._logger.error is a method. This is a bit confusing.
Since we are adding messages as part of error method, I believe we shouldn't have this attribute.
There was a problem hiding this comment.
We have attribute self._log_errors = [] which we are using to log all the errors collected. This list is used in run.py in Exception block to display the errors
Signed-off-by: Amarnath K <[email protected]>
AmarnatReddy
force-pushed
the
log_rewap
branch
from
1e56ec3
to
344c9e5
Compare
Description
Refactoring log format by removing metadata addition in log statements.
Parallel tests : http://magna002.ceph.redhat.com/cephci-jenkins/cephci-run-EDH5VP/
as part of suite i have 2 tests running in parallel. I did not make any changes in test files
test1: http://magna002.ceph.redhat.com/cephci-jenkins/cephci-run-EDH5VP/Increase%20and%20Decrease%20of%20MDS.log
test2:http://magna002.ceph.redhat.com/cephci-jenkins/cephci-run-EDH5VP/nfs-ganesha_with_cephfs.log
suite file :
name: Parallel run
desc: FS tier-0 parallelly.
module: test_parallel.py
parallel:
- test:
name: nfs-ganesha_with_cephfs
module: cephfs_tier1_ops.py
desc: Configure nfs-ganesha on nfs server,do mount on any client and do IOs
polarion-id: CEPH-83574439
abort-on-fail: false
- test:
name: Increase and Decrease of MDS
module: mds_inc_dec.py
desc: Deploy mds using cephadm and increase & decrease number of mds.
polarion-id: CEPH-83574286
abort-on-fail: false
Running test: Parallel run
Test logfile location: http://magna002.ceph.redhat.com/cephci-jenkins/cephci-run-EDH5VP/Parallel_run_0.log
2024-11-21 14:50:40,287 (utility.utils) - utils:1138 - email_results - [INFO] - Results have been emailed to ['[email protected]']
2024-11-21 14:50:40,287 (root) - run:1007 - run - [INFO] -
Generating sosreports for all the nodes due to failures in testcase
2024-11-21 14:50:40,287 (root) - run:1011 - run - [INFO] - Installing Ceph-common on ceph nodes to gather Sos report
2024-11-21 14:50:40,289 (ceph.ceph) - ceph:1570 - long_running - [INFO] - Execute yum install -y ceph-common --nogpgcheck on 10.0.65.58
2024-11-21 14:50:42,856 (ceph.ceph) - ceph:1600 - long_running - [INFO] - Execution of yum install -y ceph-common --nogpgcheck on 10.0.65.58 took 2.56767 seconds.
2024-11-21 14:50:42,858 (ceph.ceph) - ceph:1570 - long_running - [INFO] - Execute mkdir -p /etc/ceph on 10.0.65.58
2024-11-21 14:50:43,902 (ceph.ceph) - ceph:1600 - long_running - [INFO] - Execution of mkdir -p /etc/ceph on 10.0.65.58 took 1.04412 seconds.
2024-11-21 14:50:43,904 (ceph.ceph) - ceph:1570 - long_running - [INFO] - Execute cephadm shell -- ceph auth get client.admin on 10.0.65.58
2024-11-21 14:50:45,382 (ceph.ceph) - ceph:1600 - long_running - [INFO] - Execution of cephadm shell -- ceph auth get client.admin on 10.0.65.58 took 1.477583 seconds.
2024-11-21 14:50:45,448 (paramiko.transport.sftp) - sftp:169 - _log - [INFO] - [chan 5] Opened sftp connection (server version 3)
2024-11-21 14:50:45,451 (ceph.ceph) - ceph:1570 - long_running - [INFO] - Execute cephadm shell -- ceph config generate-minimal-conf on 10.0.65.58
2024-11-21 14:50:47,800 (ceph.ceph) - ceph:1600 - long_running - [INFO] - Execution of cephadm shell -- ceph config generate-minimal-conf on 10.0.65.58 took 2.348358 seconds.
2024-11-21 14:50:47,825 (paramiko.transport.sftp) - sftp:169 - _log - [INFO] - [chan 7] Opened sftp connection (server version 3)
2024-11-21 14:50:47,831 (ceph.ceph) - ceph:1570 - long_running - [INFO] - Execute yum install -y ceph-common --nogpgcheck on 10.0.65.49
Single test : http://magna002.ceph.redhat.com/cephci-jenkins/cephci-run-XRBX1X/
Please include Automation development guidelines. Source of Test case - New Feature/Regression Test/Close loop of customer BZs
click to expand checklist