Skip to content

Commit

Permalink
updating nccl interface documantation
Browse files Browse the repository at this point in the history
  • Loading branch information
caiomcbr committed Jul 26, 2024
1 parent a920a6c commit 9019f8e
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 0 deletions.
19 changes: 19 additions & 0 deletions apps/nccl/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,22 @@ The table below lists all NCCL APIs (v2.21). We may cover more APIs in the futur
| ncclRecv | X |
| ncclRedOpCreatePreMulSum | X |
| ncclRedOpDestroy | X |

### Executor Support

The executor is a versatile tool designed to specify how mscclpp executes algorithms. Currently, only the allReduce operation allows for algorithm customization. The following environment variables can be managed:

- ALLREDUCEPKT_IP_JSON_FILE: Specifies the path to the JSON file that defines the algorithm for small-sized, in-place operations.
- ALLREDUCEPKT_OP_JSON_FILE: Specifies the path to the JSON file that defines the algorithm for small-sized, out-of-place operations.
- ALLREDUCE_IP_JSON_FILE: Specifies the path to the JSON file that defines the algorithm for larger-sized, in-place operations.
- ALLREDUCE_OP_JSON_FILE: Specifies the path to the JSON file that defines the algorithm for larger-sized, out-of-place operations.
- ALLREDUCE_SMALL_MSG_BOUNDARY: Defines the size threshold at which the algorithm will switch between fallback code and the customized algorithm for small messages.
- ALLREDUCE_LARGE_MSG_BOUNDARY: Defines the size threshold at which the algorithm will switch between the customized algorithm for small messages and that for larger messages.

| <center>Decision Flowchart for Message Size-Based Algorithm Execution |
|-------------------------------|
| <img src="../.././docs/figs/size_boundary_diagram.png" alt="MSCCL++ Abstractions" style="width: 800px;"/> |

This is an example of executing the interface with the executor:
``` bash
mpirun -np 8 -x ALLREDUCEPKT_IP_JSON_FILE=/root/azure-mscclpp/nccl/test/execution-files/allreducepacket.json -x ALLREDUCE_IP_JSON_FILE=/root/azure-mscclpp/nccl/test/execution-files/allreducesm.json -x ALLREDUCE_SMALL_MSG_BOUNDARY=16K -x ALLREDUCE_LARGE_MSG_BOUNDARY=1M ./apps/nccl/test/nccl_api_test
Binary file added docs/figs/size_boundary_diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 9019f8e

Please sign in to comment.