add sharded checkpoint loading for AutoTP path to reduce the peak mem…#6

delock merged 1 commit into

delock:gma/cpu_supportdelock/DeepSpeedSYCLSupport:gma/cpu_supportfrom

sywangyi:autoTP_reduce_peaksywangyi/DeepSpeed:autoTP_reduce_peakCopy head branch name to clipboard