From 60712cc3f1efa6b7309221b113bb122cc0d75ff5 Mon Sep 17 00:00:00 2001 From: Molly Smith Date: Thu, 23 Feb 2023 19:40:01 +0000 Subject: [PATCH 1/2] Tutorial table compatible web formwat again? --- .../automatic-tensor-parallelism.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/_tutorials/automatic-tensor-parallelism.md b/docs/_tutorials/automatic-tensor-parallelism.md index e480fbf42ac8..89bddb7b9686 100644 --- a/docs/_tutorials/automatic-tensor-parallelism.md +++ b/docs/_tutorials/automatic-tensor-parallelism.md @@ -88,18 +88,18 @@ deepspeed --num_gpus DeepSpeedExamples/inference/huggingface/text-gen The following results were collected using V100 SXM2 32GB GPUs. ### Max New Tokens = 50 -| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU | -|----------|--------------------------|----------------|------------------------| -| No TP | 23.94 GB | 64 | 18.84 TFlops | -| 2 GPU TP | 12.23 GB | 320 | 27.17 TFlops | -| 4 GPU TP | 6.36 GB | 664 | 27.63 TFlops | +| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU | +| ---------- | -------------------------- | ---------------- | ------------------------ | +| No TP | 23.94 GB | 64 | 18.84 TFlops | +| 2 GPU TP | 12.23 GB | 320 | 27.17 TFlops | +| 4 GPU TP | 6.36 GB | 664 | 27.63 TFlops | ### Max New Tokens = 1024 -| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU | -|----------|--------------------------|----------------|------------------------| -| No TP | 23.94 GB | 2 | 1.65 TFlops | -| 2 GPU TP | 12.23 GB | 20 | 4.61 TFlops | -| 4 GPU TP | 6.36 GB | 56 | 4.90 TFlops | +| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU | +| ---------- | -------------------------- | ---------------- | ------------------------ | +| No TP | 23.94 GB | 2 | 1.65 TFlops | +| 2 GPU TP | 12.23 GB | 20 | 4.61 TFlops | +| 4 GPU TP | 6.36 GB | 56 | 4.90 TFlops | # Supported Models From 3729a9f6701c1e24c0ab3dade28c52783bc483f9 Mon Sep 17 00:00:00 2001 From: Molly Smith Date: Thu, 23 Feb 2023 19:43:07 +0000 Subject: [PATCH 2/2] update web latest news --- docs/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/index.md b/docs/index.md index 996c2fdfe925..79c5ad21f058 100755 --- a/docs/index.md +++ b/docs/index.md @@ -7,6 +7,7 @@ title: "Latest News" --- DeepSpeed trained the world's most powerful language models ([MT-530B](https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/), [BLOOM](https://huggingface.co/blog/bloom-megatron-deepspeed)); [learn how](https://www.deepspeed.ai/tutorials/large-models-w-deepspeed/). +* [2023/02] [Automatic Tensor Parallelism: Enables tensor parallelism by default without providing an injection policy](https://www.deepspeed.ai/tutorials/automatic-tensor-parallelism/) * [2022/12] [DeepSpeed Data Efficiency: A composable library that makes better use of data, increases training efficiency, and improves model quality](https://www.deepspeed.ai/2022/12/11/data-efficiency.html) * [2022/11] [Stable Diffusion Image Generation under 1 second w. DeepSpeed MII](https://github.com/microsoft/DeepSpeed-MII/tree/main/examples/benchmark/txt2img) * [2022/10] [DeepSpeed-MII: instant speedup on 24,000+ open-source DL models with up to 40x cheaper inference](https://www.deepspeed.ai/2022/10/10/mii.html)