HADOOP-18924. upgrade to grpc 1.53.0 due to CVEs #6161

pjfanning · 2023-10-09T10:53:00Z

Description of PR

https://issues.apache.org/jira/browse/HADOOP-18924

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2023-11-26T01:59:27Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 24s		Maven dependency ordering for branch
+1 💚	mvninstall	30m 47s		trunk passed
+1 💚	compile	16m 17s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	compile	14m 40s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	mvnsite	17m 42s		trunk passed
+1 💚	javadoc	8m 24s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	7m 38s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	shadedclient	47m 28s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 44s		Maven dependency ordering for patch
+1 💚	mvninstall	28m 54s		the patch passed
+1 💚	compile	15m 38s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javac	15m 38s		the patch passed
+1 💚	compile	14m 43s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	14m 43s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	13m 29s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	8m 16s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	7m 38s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	shadedclient	48m 46s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	730m 43s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 29s		The patch does not generate ASF License warnings.
		1001m 3s

Reason	Tests
Failed junit tests	hadoop.fs.http.client.TestHttpFSWithHttpFSFileSystem
	hadoop.hdfs.server.datanode.TestDirectoryScanner

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6161/7/artifact/out/Dockerfile
GITHUB PR	#6161
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux a7b8f1233087 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1b9e42c`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6161/7/testReport/
Max. process+thread count	3178 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6161/7/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

pjfanning · 2023-11-26T11:13:02Z

@steveloughran @ayushtkn @slfan1989 is this a change we could consider merging? I think the test failures are random intermittent failures.

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml

ayushtkn

LGTM

hadoop-yetus · 2023-11-28T10:26:05Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 5s		Maven dependency ordering for branch
+1 💚	mvninstall	30m 48s		trunk passed
+1 💚	compile	16m 17s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	compile	14m 49s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	mvnsite	18m 33s		trunk passed
+1 💚	javadoc	8m 27s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	7m 34s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	shadedclient	47m 14s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 45s		Maven dependency ordering for patch
+1 💚	mvninstall	28m 45s		the patch passed
+1 💚	compile	15m 27s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javac	15m 27s		the patch passed
+1 💚	compile	14m 41s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	14m 41s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	13m 23s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	javadoc	8m 17s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	7m 32s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	shadedclient	48m 27s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	731m 24s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 31s		The patch does not generate ASF License warnings.
		1002m 22s

Reason	Tests
Failed junit tests	hadoop.fs.http.server.TestHttpFSAccessControlled
	hadoop.fs.http.client.TestHttpFSFileSystemLocalFileSystem

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6161/8/artifact/out/Dockerfile
GITHUB PR	#6161
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux 031498c007af 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `abb0013`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6161/8/testReport/
Max. process+thread count	3677 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6161/8/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

ayushtkn · 2023-12-01T04:22:54Z

Failed tests passes locally

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.http.server.TestHttpFSAccessControlled
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.561 s - in org.apache.hadoop.fs.http.server.TestHttpFSAccessControlled
[INFO] Running org.apache.hadoop.fs.http.client.TestHttpFSFileSystemLocalFileSystem
[WARNING] Tests run: 100, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 13.127 s - in org.apache.hadoop.fs.http.client.TestHttpFSFileSystemLocalFileSystem
[INFO] 
[INFO] Results:
[INFO] 
[WARNING] Tests run: 101, Failures: 0, Errors: 0, Skipped: 6

…ibuted by PJ Fanning. Signed-off-by: Ayush Saxena <[email protected]> Update LICENSE-binary

… by PJ Fanning.. (#6313)

…ibuted by PJ Fanning. Signed-off-by: Ayush Saxena <[email protected]>

…ibuted… (apache#439) * HADOOP-18924. Upgrade to grpc 1.53.0 due to CVEs (apache#6161). Contributed by PJ Fanning. Signed-off-by: Ayush Saxena <[email protected]> * fix conflicts * exclude conflicting animal-sniffer-annotations version * empty commit for ACLOVERRIDE --------- Signed-off-by: Ayush Saxena <[email protected]> Co-authored-by: PJ Fanning <[email protected]> Co-authored-by: Joseph DellAringa <[email protected]>

github-actions bot added YARN build trunk labels Oct 9, 2023

This comment was marked as outdated.

Sign in to view

pjfanning force-pushed the HADOOP-18924-grpc branch from dc04c25 to 8126ead Compare October 24, 2023 22:56

This comment was marked as outdated.

Sign in to view

pjfanning force-pushed the HADOOP-18924-grpc branch from 299f66b to 7e3d26c Compare November 3, 2023 00:19

This comment was marked as outdated.

Sign in to view

pjfanning force-pushed the HADOOP-18924-grpc branch from 7e3d26c to f64224c Compare November 4, 2023 10:39

This comment was marked as outdated.

Sign in to view

pjfanning added 2 commits November 25, 2023 10:12

HADOOP-18924. upgrade to grpc 1.53.0 due to CVEs

358856c

Update pom.xml

1b9e42c

pjfanning force-pushed the HADOOP-18924-grpc branch from f64224c to 1b9e42c Compare November 25, 2023 09:17

ayushtkn reviewed Nov 27, 2023

View reviewed changes

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml Show resolved Hide resolved

Update LICENSE-binary

abb0013

ayushtkn approved these changes Nov 28, 2023

View reviewed changes

ayushtkn merged commit 3cb3dfa into apache:trunk Dec 1, 2023
1 of 4 checks passed

pjfanning deleted the HADOOP-18924-grpc branch December 1, 2023 08:57

pjfanning added a commit to pjfanning/hadoop that referenced this pull request Dec 1, 2023

HADOOP-18924. Upgrade to grpc 1.53.0 due to CVEs (apache#6161). Contr…

a9a756e

…ibuted by PJ Fanning. Signed-off-by: Ayush Saxena <[email protected]> Update LICENSE-binary

pjfanning added a commit to pjfanning/hadoop that referenced this pull request Dec 5, 2023

HADOOP-18924. Upgrade to grpc 1.53.0 due to CVEs (apache#6161). Contr…

587d6af

…ibuted by PJ Fanning. Signed-off-by: Ayush Saxena <[email protected]> Update LICENSE-binary

ayushtkn pushed a commit that referenced this pull request Dec 5, 2023

HADOOP-18924. Upgrade to grpc 1.53.0 due to CVEs (#6161). Contributed…

fad316a

… by PJ Fanning.. (#6313)

jiajunmao pushed a commit to jiajunmao/hadoop-MLEC that referenced this pull request Feb 6, 2024

HADOOP-18924. Upgrade to grpc 1.53.0 due to CVEs (apache#6161). Contr…

51adf80

…ibuted by PJ Fanning. Signed-off-by: Ayush Saxena <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-18924. upgrade to grpc 1.53.0 due to CVEs #6161

HADOOP-18924. upgrade to grpc 1.53.0 due to CVEs #6161

pjfanning commented Oct 9, 2023

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

hadoop-yetus commented Nov 26, 2023

pjfanning commented Nov 26, 2023

ayushtkn left a comment

hadoop-yetus commented Nov 28, 2023

ayushtkn commented Dec 1, 2023

HADOOP-18924. upgrade to grpc 1.53.0 due to CVEs #6161

HADOOP-18924. upgrade to grpc 1.53.0 due to CVEs #6161

Conversation

pjfanning commented Oct 9, 2023

Description of PR

How was this patch tested?

For code changes:

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

hadoop-yetus commented Nov 26, 2023

pjfanning commented Nov 26, 2023

ayushtkn left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Nov 28, 2023

ayushtkn commented Dec 1, 2023