YARN-11579. Fix 'Physical Mem Used' and 'Physical VCores Used' are not displaying data. #6123

slfan1989 · 2023-09-27T16:22:52Z

Description of PR

JIRA: YARN-11579. Fix 'Physical Mem Used' and 'Physical VCores Used' are not displaying data.

Before the fix
After the fix

How was this patch tested?

Add Junit Test.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

…t displaying data.

hadoop-yetus · 2023-09-27T20:36:00Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 12s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 28s		Maven dependency ordering for branch
+1 💚	mvninstall	32m 47s		trunk passed
+1 💚	compile	2m 40s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	compile	2m 24s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 23s		trunk passed
+1 💚	mvnsite	1m 42s		trunk passed
+1 💚	javadoc	1m 35s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 20s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 56s		trunk passed
+1 💚	shadedclient	33m 4s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 33s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 17s		the patch passed
+1 💚	compile	2m 26s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javac	2m 26s		the patch passed
+1 💚	compile	2m 14s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	2m 14s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 11s		the patch passed
+1 💚	mvnsite	1m 23s		the patch passed
+1 💚	javadoc	1m 15s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 8s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 56s		the patch passed
+1 💚	shadedclient	33m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	101m 16s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	unit	0m 33s		hadoop-yarn-server-router in the patch passed.
+1 💚	asflicense	0m 42s		The patch does not generate ASF License warnings.
		251m 38s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6123/1/artifact/out/Dockerfile
GITHUB PR	#6123
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 442428018cfe 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1c11943`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6123/1/testReport/
Max. process+thread count	976 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6123/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2023-09-28T18:41:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 38s		Maven dependency ordering for branch
+1 💚	mvninstall	32m 12s		trunk passed
+1 💚	compile	2m 37s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	compile	2m 21s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 24s		trunk passed
+1 💚	mvnsite	1m 39s		trunk passed
+1 💚	javadoc	1m 37s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 26s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	3m 2s		trunk passed
+1 💚	shadedclient	34m 8s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 19s		the patch passed
+1 💚	compile	2m 32s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javac	2m 32s		the patch passed
+1 💚	compile	2m 24s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	2m 24s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 13s		the patch passed
+1 💚	mvnsite	1m 22s		the patch passed
+1 💚	javadoc	1m 12s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 6s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	3m 3s		the patch passed
+1 💚	shadedclient	35m 21s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	77m 39s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌	unit	0m 29s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt	hadoop-yarn-server-router in the patch failed.
+0 🆗	asflicense	0m 29s		ASF License check generated no output?
		229m 22s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesConfigurationMutation
	hadoop.yarn.server.resourcemanager.webapp.fairscheduler.TestRMWebServicesFairSchedulerCustomResourceTypes
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6123/2/artifact/out/Dockerfile
GITHUB PR	#6123
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 9ff118c58e58 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1e76fe4`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6123/2/testReport/
Max. process+thread count	995 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6123/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2023-09-29T03:42:57Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 59s		Maven dependency ordering for branch
+1 💚	mvninstall	30m 52s		trunk passed
+1 💚	compile	2m 50s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	compile	2m 25s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 32s		trunk passed
+1 💚	mvnsite	1m 42s		trunk passed
+1 💚	javadoc	1m 37s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 26s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 56s		trunk passed
+1 💚	shadedclient	33m 9s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 41s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 16s		the patch passed
+1 💚	compile	2m 23s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javac	2m 23s		the patch passed
+1 💚	compile	2m 16s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	2m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 12s		the patch passed
+1 💚	mvnsite	1m 22s		the patch passed
+1 💚	javadoc	1m 15s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 10s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 59s		the patch passed
+1 💚	shadedclient	33m 41s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	100m 54s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	unit	0m 31s		hadoop-yarn-server-router in the patch passed.
+1 💚	asflicense	0m 41s		The patch does not generate ASF License warnings.
		249m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6123/3/artifact/out/Dockerfile
GITHUB PR	#6123
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 66a00fda243e 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0ff1ee3`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6123/3/testReport/
Max. process+thread count	954 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6123/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2023-10-02T13:51:03Z

@goiri Can you help review this PR? Thank you very much!

slfan1989 · 2023-10-05T06:21:31Z

@goiri Thank you very much for your help in reviewing the code! I will continue to complete other PRs.

…t displaying data. (apache#6123) Contributed by Shilun Fan. Reviewed-by: Inigo Goiri <[email protected]> Signed-off-by: Shilun Fan <[email protected]>

YARN-11579. Fix 'Physical Mem Used' and 'Physical VCores Used' are no…

1c11943

…t displaying data.

github-actions bot added YARN trunk labels Sep 27, 2023

YARN-11579. Fix CheckStyle.

1e76fe4

Merge branch 'apache:trunk' into YARN-11579

0ff1ee3

goiri approved these changes Oct 3, 2023

View reviewed changes

slfan1989 merged commit f3a27f2 into apache:trunk Oct 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YARN-11579. Fix 'Physical Mem Used' and 'Physical VCores Used' are not displaying data. #6123

YARN-11579. Fix 'Physical Mem Used' and 'Physical VCores Used' are not displaying data. #6123

slfan1989 commented Sep 27, 2023

hadoop-yetus commented Sep 27, 2023

hadoop-yetus commented Sep 28, 2023

hadoop-yetus commented Sep 29, 2023

slfan1989 commented Oct 2, 2023

slfan1989 commented Oct 5, 2023

YARN-11579. Fix 'Physical Mem Used' and 'Physical VCores Used' are not displaying data. #6123

YARN-11579. Fix 'Physical Mem Used' and 'Physical VCores Used' are not displaying data. #6123

Conversation

slfan1989 commented Sep 27, 2023

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Sep 27, 2023

hadoop-yetus commented Sep 28, 2023

hadoop-yetus commented Sep 29, 2023

slfan1989 commented Oct 2, 2023

slfan1989 commented Oct 5, 2023