Nonblocking Collectives #456

manjugv · 2020-10-26T15:00:49Z

No description provided.

content/shmem_collective_test.tex

content/nb_collectives_intro.tex

naveen-rn · 2023-01-26T13:40:31Z

content/shmem_alltoall_nb.tex

+    Like data exchange semantics, the entry and completion
+    criteria of blocking and nonblocking alltoall is similar. 
+
+    {\bf Entry criteria}: Before any \ac{PE} calls a \FUNC{shmem\_alltoall\_nb} routine,


This entry criteria is confusing - if we have the same entry criteria as blocking operations - then the users are supposed to make sure the src/dst buffers are available at the point of entry - meaning the users are expected to sync with themselves before launching the nb collective operation.

AFAIU - the src/dst needs to be available when everyone in the team reaches the state - which can be determined during runtime - without the need for explicit sync before launching the nb collective operation.

Please clarify the intended semantics.

Naveen, my intent is to keep it consistent with the semantics of blocking collectives. I wanted to improve the description. I don’t see how the current wording is violating that. If the wording doesn’t reflect that, then we can fix it.

content/shmem_barrier_all_nb.tex

content/nb_collectives_intro.tex

manjugv · 2023-05-26T18:45:47Z

Feedback from Jim (May Spec Meeting): Should we consider shmem_local_complete ?

davidozog · 2023-09-29T17:59:37Z

content/nb_collectives_intro.tex

+\item Collective Types: The nonblocking variants supported include barrier, alltoall,
+and broadcast collectives. Other collective operations such as
+reductions, collect, barrier, alltoalls, and sync will not have nonblocking variants.
+


Should sync_all be supported? My hunch is it seems possibly useful...

content/shmem_alltoall_nb.tex

content/nb_collectives_intro.tex

content/shmem_alltoall_nb.tex

content/shmem_barrier_all_nb.tex

content/shmem_broadcast_nb.tex

jdinan · 2024-02-27T16:43:45Z

Feedback from Feb. 27, 2024 discussion:

Request handle should be passed by reference to wait/test to allow implementation to reset the request handle's value to SHMEM_REQUEST_INVALID.
We discussed a counter proposal to make "nbi" collectives on the team that can be completed in bulk via shmem_team_sync or other similar operation. This would mirror the contexts behavior for nonblocking RMA/AMO.
Discussion about how to manage internal psync space used by the collectives

Wait/test are local operations which prevents freeing symmetric memory in the wait/test. If a pool allocator is used, the pool will become inconsistent across PEs. This can be addressed through periodic synchronization when initiating a collective to "garbage collect" pSync operations.

yfguo · 2024-02-27T16:49:52Z

content/shmem_collective_wait.tex

+threads but on different request objects. It is the responsibility of the
+\openshmem user to ensure that proper synchronization is used to prevent race
+conditions or deadlock. Specifically, the \FUNC{shmem\_req\_wait} operation should
+be called after the collective operation to ensure that the request object is


If the requests are always deallocated in wait or test call, and if request handles are not reused, it will not run into the scenario of "premature deallocation".

davidozog · 2024-05-30T19:19:41Z

content/shmem_collective_test.tex

+\apidescription{
+    A call to \FUNC{shmem\_req\_test} returns immediately. If the
+    operation identified by the request is completed, it returns
+    zero, and the request object is deallocated and cannot be reused.


Could request objects have value SHMEM_REQUEST_INVALID after they are deallocated? In which case this routine and shmem_req_wait would take an argument of type shmem_req_h *.

@davidozog Just updated. Please let me know if it's missing something.

jdinan · 2024-06-27T18:13:57Z

Notes from today's discussion -- 06-27-2024 -- NBI Collectives pSync Management.pptx

Co-authored-by: James Dinan <[email protected]>

Add SHMEM_REQ_INVALID library constant Address WG Feedback Clarify collective start/execution Remove barrier_all_nb Address Feedback Minor update to nb_collective_intro.tex Clarify completion of broadcast_nb

bcmIntc · 2024-08-23T17:17:12Z

content/nb_collectives_intro.tex

+\item Completion semantics: \openshmem programs can learn the status of the collective operations
+using the \FUNC{shmem\_req\_test} routine. The operation is completed after
+a call to \FUNC{shmem\_req\_test} or a call to \FUNC{shmem\_req\_wait}.
+


Similar to what @kwaters4 mentioned. Add something like "Completion of the operation can be observed through one or more calls to \FUNC{shmem_req_test} or a single call to \FUNC{shmem_req_wait}."

bcmIntc · 2024-08-23T17:26:59Z

content/shmem_alltoall_nb.tex

@@ -0,0 +1,123 @@
+\apisummary{
+  Exchanges a fixed amount of contiguous data blocks between all pairs
+  of \acp{PE} participating in the collective routine.


Since we mention later that "All PEs in the provided team must participate in the collective.", can we just say that here, instead of keeping it in the abstract?
i.e. "Exchanges a fixed amount of contiguous data blocks between all PEs on the specified team."

jdinan · 2024-08-23T18:14:46Z

content/shmem_broadcast_nb.tex

-    \item The \source{} data object may be safely reused.
+    \item The \dest{} data object is updated.
+    \item If the local \ac{PE} is \VAR{PE\_root}, the data has been copied
+          out of the \source{} data object.


Move this change to a separate PR and ask section committee to add it. @davidozog

jdinan · 2024-08-23T19:26:00Z

This proposal was brought forward for a vote on August 23, 2024 and the ballot did not pass. The proposal will need to be re-read before it can be voted on again.

bcmIntc · 2024-08-23T19:55:55Z

content/shmem_collective_wait.tex

@@ -0,0 +1,38 @@
+\apisummary{
+    The routine waits until a operation identified by a request


nit: "an operation" instead of "a operation"

wfaderhold21 · 2024-08-27T17:48:49Z

Capturing feedback from specification meeting:

General feedback including from HPE and DOD: not confident regarding the proposal and want to delay out of abundance of caution (get it right once rather than continually fix later). Would like to see some performance numbers with a working reference implementation on InfiniBand networks. Would also like to see some driving use cases.
Intel Feedback: Need examples in proposal and they feel all proposals should have reference implementations.
No feedback from AMD or ANL...

manjugv force-pushed the topic/nb_collectives branch from 2683ab3 to a77c345 Compare January 21, 2022 19:10

manjugv changed the title ~~Nonblocking Collectives (Draft)~~ Nonblocking Collectives Jan 21, 2022

manjugv added the Feedback Requested label Jan 21, 2022

manjugv self-assigned this Jan 21, 2022

jdinan reviewed Jan 28, 2022

View reviewed changes

content/shmem_collective_test.tex Outdated Show resolved Hide resolved

jdinan reviewed Jan 28, 2022

View reviewed changes

content/nb_collectives_intro.tex Outdated Show resolved Hide resolved

naveen-rn reviewed Jan 26, 2023

View reviewed changes

manjugv added the PendingReading label Mar 16, 2023

manjugv commented May 26, 2023

View reviewed changes

content/shmem_barrier_all_nb.tex Outdated Show resolved Hide resolved

manjugv commented May 26, 2023

View reviewed changes

content/nb_collectives_intro.tex Outdated Show resolved Hide resolved

manjugv commented May 26, 2023

View reviewed changes

content/nb_collectives_intro.tex Outdated Show resolved Hide resolved

davidozog reviewed Sep 29, 2023

View reviewed changes

content/shmem_alltoall_nb.tex Outdated Show resolved Hide resolved

jdinan reviewed Sep 29, 2023

View reviewed changes

content/nb_collectives_intro.tex Outdated Show resolved Hide resolved

jdinan reviewed Sep 29, 2023

View reviewed changes

content/shmem_alltoall_nb.tex Outdated Show resolved Hide resolved

jdinan reviewed Sep 29, 2023

View reviewed changes

content/shmem_barrier_all_nb.tex Outdated Show resolved Hide resolved

jdinan reviewed Sep 29, 2023

View reviewed changes

content/shmem_broadcast_nb.tex Outdated Show resolved Hide resolved

jdinan reviewed Sep 29, 2023

View reviewed changes

content/shmem_broadcast_nb.tex Outdated Show resolved Hide resolved

jdinan mentioned this pull request Sep 29, 2023

Clarify progress of nonblocking operations #500

Open

manjugv added HadReading PendingBallot SpecialBallot and removed PendingReading labels Sep 29, 2023

naveen-rn added this to the OpenSHMEM 1.6 milestone Oct 19, 2023

wfaderhold21 mentioned this pull request Feb 14, 2024

Add Clarification of Progress Model #507

Closed

yfguo reviewed Feb 27, 2024

View reviewed changes

davidozog reviewed May 30, 2024

View reviewed changes

manjugv and others added 15 commits August 22, 2024 15:25

Add missing 32 and 64 bit interfaces; minor edits

a972738

Update content/nb_collectives_intro.tex

db8528c

Co-authored-by: James Dinan <[email protected]>

Update content/shmem_alltoall_nb.tex

558e0d3

Co-authored-by: James Dinan <[email protected]>

Update content/shmem_barrier_all_nb.tex

ef3ecf1

Co-authored-by: James Dinan <[email protected]>

Update content/shmem_broadcast_nb.tex

db79fcb

Co-authored-by: James Dinan <[email protected]>

Update content/shmem_broadcast_nb.tex

7d74314

Co-authored-by: James Dinan <[email protected]>

Remove 32 and 64 bit nb variants

2355bf8

Minor edits: Remove active set mention

b15c727

Make shmem_req_test/wait functions neutral

030a019

Update content/nb_collectives_intro.tex

82e5d54

Update content/shmem_alltoall_nb.tex

fb0b482

Update content/shmem_broadcast_nb.tex

930b39c

Update content/shmem_barrier_all_nb.tex

96f0b97

Update content/shmem_collective_test.tex

ff96cf5

Update content/shmem_collective_wait.tex

ef49e8f

wfaderhold21 force-pushed the topic/nb_collectives branch from 80d70be to e40b263 Compare August 22, 2024 22:28

Minor update to nb_collectives_intro

e40b263

Add SHMEM_REQ_INVALID library constant Address WG Feedback Clarify collective start/execution Remove barrier_all_nb Address Feedback Minor update to nb_collective_intro.tex Clarify completion of broadcast_nb

bcmIntc reviewed Aug 23, 2024

View reviewed changes

jdinan reviewed Aug 23, 2024

View reviewed changes

Inclusion of text from PR openshmem-org#507

02a539a

jdinan removed PendingBallot SpecialBallot HadReading Feedback Requested labels Aug 23, 2024

jdinan modified the milestones: OpenSHMEM 1.6, OpenSHMEM 1.7 Aug 23, 2024

bcmIntc reviewed Aug 23, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nonblocking Collectives #456

Nonblocking Collectives #456

manjugv commented Oct 26, 2020

naveen-rn Jan 26, 2023

manjugv Mar 31, 2023

manjugv commented May 26, 2023

davidozog Sep 29, 2023 •

edited

Loading

jdinan commented Feb 27, 2024

yfguo Feb 27, 2024

davidozog May 30, 2024

wfaderhold21 Jun 12, 2024

jdinan commented Jun 27, 2024

bcmIntc Aug 23, 2024

bcmIntc Aug 23, 2024

jdinan Aug 23, 2024

jdinan commented Aug 23, 2024

bcmIntc Aug 23, 2024

wfaderhold21 commented Aug 27, 2024

		@@ -0,0 +1,38 @@
		\apisummary{
		The routine waits until a operation identified by a request

Nonblocking Collectives #456

Are you sure you want to change the base?

Nonblocking Collectives #456

Conversation

manjugv commented Oct 26, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

manjugv commented May 26, 2023

davidozog Sep 29, 2023 • edited Loading

Choose a reason for hiding this comment

jdinan commented Feb 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdinan commented Jun 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdinan commented Aug 23, 2024

Choose a reason for hiding this comment

wfaderhold21 commented Aug 27, 2024

davidozog Sep 29, 2023 •

edited

Loading