TEZ-4543: Throw a special exception to DagClient when there is no current DAG #338

abstractdog · 2024-02-27T08:31:14Z

This is an AM + client-side change to make DagClient realize much faster when its asking for the status of a DAG which is not present.

…rent DAG

tez-yetus · 2024-02-27T09:45:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	26m 57s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ master Compile Tests _
+0 🆗	mvndep	6m 3s	Maven dependency ordering for branch
+1 💚	mvninstall	13m 34s	master passed
+1 💚	compile	1m 20s	master passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	compile	1m 13s	master passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+1 💚	checkstyle	1m 27s	master passed
+1 💚	javadoc	1m 13s	master passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	javadoc	1m 9s	master passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+0 🆗	spotbugs	2m 2s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	4m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 51s	the patch passed
+1 💚	compile	1m 2s	the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	javac	1m 2s	the patch passed
+1 💚	compile	0m 51s	the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+1 💚	javac	0m 51s	the patch passed
+1 💚	checkstyle	0m 13s	tez-api: The patch generated 0 new + 73 unchanged - 1 fixed = 73 total (was 74)
+1 💚	checkstyle	0m 20s	The patch passed checkstyle in tez-dag
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 35s	the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	javadoc	0m 35s	the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+1 💚	findbugs	2m 24s	the patch passed
		_ Other Tests _
+1 💚	unit	2m 14s	tez-api in the patch passed.
+1 💚	unit	5m 13s	tez-dag in the patch passed.
+1 💚	asflicense	0m 21s	The patch does not generate ASF License warnings.
		73m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-338/1/artifact/out/Dockerfile
GITHUB PR	#338
JIRA Issue	TEZ-4543
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux 11c00730d383 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / `f8c2e11`
Default Java	Private Build-1.8.0_392-8u392-ga-1~22.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~22.04-b08
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-338/1/testReport/
Max. process+thread count	486 (vs. ulimit of 5500)
modules	C: tez-api tez-dag U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-338/1/console
versions	git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ayushtkn

minor comments, rest looks good

ayushtkn · 2024-02-27T11:24:48Z

tez-api/src/main/java/org/apache/tez/dag/api/NoCurrentDAGException.java

+ *  Fatal exception: thrown by the AM if there is no DAG running when
+ *  when a DAG's status is queried. This is different from {@link org.apache.tez.dag.api.DAGNotRunningException}


nit
two times when, once in first line & then in second

:) fixing it

ayushtkn · 2024-02-27T11:24:55Z

tez-api/src/main/java/org/apache/tez/dag/api/client/DAGClientImpl.java

@@ -408,6 +409,8 @@ private DAGStatus getDAGStatusViaAM(@Nullable Set<StatusGetOpts> statusOptions,
    } catch (ApplicationNotFoundException e) {
      LOG.info("DAG is no longer running - application not found by YARN", e);
      dagCompleted = true;
+    } catch (NoCurrentDAGException e) {
+      return dagLost(e);


the exception e isn't used in the method dagLost, we can remove that param., moreover I think we can explore having a info log with the exception before returning

abstractdog · 2024-02-27T12:06:41Z

minor comments, rest looks good

thanks, fixed in ba93717

tez-yetus · 2024-02-27T12:48:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ master Compile Tests _
+0 🆗	mvndep	6m 1s	Maven dependency ordering for branch
+1 💚	mvninstall	11m 7s	master passed
+1 💚	compile	1m 18s	master passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	compile	1m 9s	master passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+1 💚	checkstyle	1m 21s	master passed
+1 💚	javadoc	1m 10s	master passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	javadoc	0m 57s	master passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+0 🆗	spotbugs	1m 19s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	2m 55s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 48s	the patch passed
+1 💚	compile	0m 57s	the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	javac	0m 57s	the patch passed
+1 💚	compile	0m 46s	the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+1 💚	javac	0m 46s	the patch passed
+1 💚	checkstyle	0m 13s	tez-api: The patch generated 0 new + 73 unchanged - 1 fixed = 73 total (was 74)
+1 💚	checkstyle	0m 20s	The patch passed checkstyle in tez-dag
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 34s	the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	javadoc	0m 34s	the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+1 💚	findbugs	2m 15s	the patch passed
		_ Other Tests _
+1 💚	unit	2m 13s	tez-api in the patch passed.
+1 💚	unit	4m 59s	tez-dag in the patch passed.
+1 💚	asflicense	0m 24s	The patch does not generate ASF License warnings.
		42m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-338/3/artifact/out/Dockerfile
GITHUB PR	#338
JIRA Issue	TEZ-4543
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux 1de697df0d9f 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / `f8c2e11`
Default Java	Private Build-1.8.0_392-8u392-ga-1~22.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~22.04-b08
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-338/3/testReport/
Max. process+thread count	473 (vs. ulimit of 5500)
modules	C: tez-api tez-dag U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-338/3/console
versions	git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ayushtkn

LGTM

TEZ-4543: Throw a special exception to DagClient when there is no cur…

3c13443

…rent DAG

abstractdog force-pushed the TEZ-4543 branch from 48423ae to 3c13443 Compare February 27, 2024 08:35

abstractdog requested a review from ayushtkn February 27, 2024 08:36

This comment was marked as outdated.

Sign in to view

ayushtkn reviewed Feb 27, 2024

View reviewed changes

addressing PR comments

ba93717

abstractdog requested a review from ayushtkn February 27, 2024 12:22

ayushtkn approved these changes Feb 27, 2024

View reviewed changes

ayushtkn merged commit 34bb628 into apache:master Feb 27, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TEZ-4543: Throw a special exception to DagClient when there is no current DAG #338

TEZ-4543: Throw a special exception to DagClient when there is no current DAG #338

abstractdog commented Feb 27, 2024 •

edited

Loading

This comment was marked as outdated.

tez-yetus commented Feb 27, 2024

ayushtkn left a comment

ayushtkn Feb 27, 2024

abstractdog Feb 27, 2024

ayushtkn Feb 27, 2024

abstractdog Feb 27, 2024

abstractdog commented Feb 27, 2024

tez-yetus commented Feb 27, 2024

ayushtkn left a comment

		* Fatal exception: thrown by the AM if there is no DAG running when
		* when a DAG's status is queried. This is different from {@link org.apache.tez.dag.api.DAGNotRunningException}

TEZ-4543: Throw a special exception to DagClient when there is no current DAG #338

TEZ-4543: Throw a special exception to DagClient when there is no current DAG #338

Conversation

abstractdog commented Feb 27, 2024 • edited Loading

This comment was marked as outdated.

tez-yetus commented Feb 27, 2024

ayushtkn left a comment

Choose a reason for hiding this comment

ayushtkn Feb 27, 2024

Choose a reason for hiding this comment

abstractdog Feb 27, 2024

Choose a reason for hiding this comment

ayushtkn Feb 27, 2024

Choose a reason for hiding this comment

abstractdog Feb 27, 2024

Choose a reason for hiding this comment

abstractdog commented Feb 27, 2024

tez-yetus commented Feb 27, 2024

ayushtkn left a comment

Choose a reason for hiding this comment

abstractdog commented Feb 27, 2024 •

edited

Loading