Add distributed Scatter/Gather #2113

Merged

bvanessen merged 31 commits into

LBANN:developfrom

szaman19:nvhsmem-scatter-gather

Feb 1, 2023

Commits on Feb 1, 2023

Initial commit for dist-graph-conv scatter and gather layer
szaman19
committed
Added function definitions for dc scatter and gather
szaman19
committed
- Added dimension guards for distconv-enabled scatter and gather
szaman19
committed
Defined NVShmemScatter and NVShmemGather operators on distconv::tensor namespace
szaman19
committed
Updated to dc side scatter gather operator with correct nvshmem func for getting
szaman19
committed
Adding dc scatter-gather impl files
szaman19
committed
Added vector_addressing headers and .cu files instead of calling them utils
szaman19
committed
Fix distconv CMakeLists for NVSHMEM scatter-gather kernels
szaman19
committed
scatter layer updated to fix typo
szaman19
committed
Typo fixes to distconv layer
szaman19
committed
Fixed all include issues and defined all functions to have a minimal compiling build
szaman19
committed
Added nvshmem kernel code
szaman19
committed
Added the CI test for distconv gather
szaman19
committed
Adding base implementation of distributed scatter/gather.
szaman19
committed
Adding benchmarking code for distconv scatter gather
szaman19
committed
Added a distributed GCN implementation for benchmarking and testing
szaman19
committed
Fix output dimension of distconv adapter to allow output matrix dimension to be different from input
szaman19
committed
- Fixed NVSHMEM scatter implementation
szaman19
committed
- Removed debug statements
szaman19
committed
Fixed gather layer buffer copy
szaman19
committed
Adding fix for non-zero elements in output layer
szaman19
committed
Added better include guards
szaman19
committed
Bebugging new error found on dist scatter
szaman19
committed
Adding the updated code for debugging session
szaman19
committed
Fixed Scatter troubles
szaman19
committed
Removed unnecessary quiet operation inside scatter kernel
szaman19
committed
Removed copy kernel in favor os MemcpyAsync because we are reasonably sure the participant buffers are contiguous.
szaman19
committed
Clean up following Tom's suggestions
szaman19
committed
Updated unit tests
szaman19
committed
CI tests passing
szaman19
committed
Update CMakeList to add an NVSHMEM check following Tom's suggestions
szaman19
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add distributed Scatter/Gather #2113

Add distributed Scatter/Gather #2113
bvanessen merged 31 commits into
LBANN:developfrom
szaman19:nvhsmem-scatter-gather

Commits on Feb 1, 2023

Initial commit for dist-graph-conv scatter and gather layer

Added function definitions for dc scatter and gather

- Added dimension guards for distconv-enabled scatter and gather

Defined NVShmemScatter and NVShmemGather operators on distconv::tensor namespace

Updated to dc side scatter gather operator with correct nvshmem func for getting

Adding dc scatter-gather impl files

Added vector_addressing headers and .cu files instead of calling them utils

Fix distconv CMakeLists for NVSHMEM scatter-gather kernels

scatter layer updated to fix typo

Typo fixes to distconv layer

Fixed all include issues and defined all functions to have a minimal compiling build

Added nvshmem kernel code

Added the CI test for distconv gather

Adding base implementation of distributed scatter/gather.

Adding benchmarking code for distconv scatter gather

Added a distributed GCN implementation for benchmarking and testing

Fix output dimension of distconv adapter to allow output matrix dimension to be different from input

- Fixed NVSHMEM scatter implementation

- Removed debug statements

Fixed gather layer buffer copy

Adding fix for non-zero elements in output layer

Added better include guards

Bebugging new error found on dist scatter

Adding the updated code for debugging session

Fixed Scatter troubles

Removed unnecessary quiet operation inside scatter kernel

Removed copy kernel in favor os MemcpyAsync because we are reasonably sure the participant buffers are contiguous.

Clean up following Tom's suggestions

Updated unit tests

CI tests passing

Update CMakeList to add an NVSHMEM check following Tom's suggestions