PyPI · #3857 most downloaded on PyPI

deepspeed

0.19.71.1M downloads/modeepspeedai/DeepSpeed ↗New releases in your daily digest, changelog included.

DeepSpeed library

Last release 18 days ago
16 Sep 2026
Ships on a steady schedule
a new release about every 3 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
1 version withdrawn
withdrawn after publishing
6 years old
132 releases · first in 2020

Release timeline

132 releases since 2020

2020202120222023202420252026

20202026

One column per quarter.

Releases

0.19.716 Sept 2026Deprecated
Deprecate unused DeepSpeed features by @sfc-gh-truwase in #8490
What's Changed

Add configurable dtype for ZeRO checkpoint export by @gaoxiaomo in #8318

Raise the per-tensor norms to norm_type when combining them by @vineethsaivs in #8313

Fix the seq-first Ulysses all2all output layout by @vineethsaivs in #8317

Count each module object once when aggregating flops profiler totals by @ebarkhordar in #8320

Fallback for unsupported Hybrid Engine policies by @LiRunGuo in #8265

[tiled mlp] reshape instead of view by @stas00 in #8348

Add pin_empty helper for empty scratch destinations by @sfc-gh-truwase in #8332

Fix Triton NFS detection crash when df wraps long device names by @atirna in #8259

[AutoTP] Replace tp_shard process-wide globals with per-model AutoTPMeta by @delock in #8241

feat(cpu-adam): add ARM SVE update kernel by @xylian86 in #8365

Add macOS (MPS) CI workflow and a torch floor check for the MPS accelerator by @PKUWZP in #8335

Add an opt-in DeepEP transport for the AutoEP expert all-to-all by @yh0903 in #8213

remove the dead non-Triton attention path from TritonSelfAttention by @Anai-Guo in #8349

perf(rollout): profile prefill and decode forwards by @nathon-lee in #8350

DeepCompile: stabilize ZeRO-3 parameter guards by @tohtana in #8328

fix: Only bind device id when needed, Fixes #8248 by @pengdurice in #8269

Make the WarmupCosineLR ratio flags reach the config by @vineethsaivs in #8268

Forward free_data through partition() instead of hardcoding True by @vineethsaivs in #8305

Recognize Qwen3.5's RMSNorm variants in AutoTP module loading by @promptsmith1990 in #8306

Forward barrier device_ids to communication backends by @tohtana in #8312

Fix flops profiler counts for transposed convolutions by @vineethsaivs in #8323

Broadcast elementwise flops from the trailing dimension by @vineethsaivs in #8324

Replace cuda_graph assertion with explicit ValueError; fix custom_op type annotations by @chakshu-dhannawat in #8336

Keep the elasticity batch overrides out of the caller's config dict by @alanhuangyoo in #8329

Stop the curriculum schedule starting below min_difficulty by @vineethsaivs in #8334

Add an opt-in fused weighted restore for AutoEP by @yh0903 in #8326

Add nccl_version to the source-checkout torch_info fallback by @alanhuangyoo in #8383

docs: add test discipline rules to agent guidelines by @delock in #8372

Preallocate the static KV cache with config.head_dim by @ebarkhordar in #8389

Skip fp16-config tests on accelerators without fp16 support by @delock in #8398

Do not pin DDP device_ids for CPU reference models by @delock in #8399

Use device names, not rank ids, for device placement in test helpers by @delock in #8397

[Ulysses] Carry the KV head count per DistributedAttention by @alanhuangyoo in #8316

[Workflow] Raise modal CI timeouts to absorb slower sandbox provisioning by @delock in #8403

Preserve AutoEP score correction bias buffers by @poorpaper in #8369

op_builder: use C++20 for nvcc on CUDA 13+ by @bheu in #8422

Reshape instead of view in TiledFusedLogitsLoss by @alanhuangyoo in #8362

Fix AutoTP + deep compile collectives silently drop when AC is on by @pengdurice in #8355

fix(lr_schedules): make --lr_range_test_staircase an opt-in flag by @Anai-Guo in #8337

Split the modal CI budget into acquisition and test phases by @delock in #8404

[Workflow] run modal GPU workflows only from the merge queue by @delock in #8412

Read rope_theta from rope_parameters in the Llama injection policy by @alanhuangyoo in #8341

Stop the debug name maps from pinning the model they snapshot by @alanhuangyoo in #8356

Muon runs no Newton-Schulz at ZeRO stage 0, the default: run it by @alanhuangyoo in #8442

Filter --include against the real slots, not against itself by @vineethsaivs in #8239

Add an opt-in ZeRO-1/2 gradient norm fast path by @yh0903 in #8331

[AutoEP]Fix optimizer and replaced MOE parameter mismatch by @pengdurice in #8377

Muon is silently disabled under ZeRO-3 when the model is built with zero.Init by @alanhuangyoo in #8438

Avoid collective token preparation for AutoEP DeepEP by @yh0903 in #8423

[MPS] Update C++ Standard in CPUAdamBuilder by @jinyouzhi in #8466

Split DeepCompile ZeRO-3 memory scheduler by @tohtana in #8233

Describe universal checkpoint shards as affine maps by @Achyuthan-S in #8385

Fix comms logger KeyError when log_name is omitted by @jinyouzhi in #8267

Fix universal checkpoint resume across AutoTP sizes by @0z5a in #8474

feat(rollout): add continuous batching generation prototype by @nathon-lee in #8368

Fix ZeRO parameter alignment for grouped_mm by @fwerkor in #8277

Carry the affine scale on the replicated map, not the split by @Achyuthan-S in #8477

[muon] Reconcile the momentum dtype when a checkpoint is restored by @alanhuangyoo in #8433

Deprecate unused DeepSpeed features by @sfc-gh-truwase in #8490

[muon] Keep the momentum out of steps the loss scaler discards by @alanhuangyoo in #8435

Remove triton compatibility check for fp_quantizer by @hwchen2017 in #8492

Muon silently discards the param groups it is given by @alanhuangyoo in #8440

[Phase 2] Add NEON SIMD path for CPU Adam on AArch64 by @PKUWZP in #8453

Fix sequence overlap backward gradient permutation by @taking-lying-flat in #8342

Validate positive inference and HybridEngine max output tokens by @taking-lying-flat in #8343

Offload a saved view when it is the last value holding its storage by @pengdurice in #8388

Honor --include/--exclude in the SLURM launcher by @vineethsaivs in #8304

Make wait() idempotent on AllGatherHandle and NoGatherHandle by @pengdurice in #8487

Deprecate sparse attention by @hwchen2017 in #8493

feat(rollout): add continuous batching profiling by @nathon-lee in #8494

Probe the device module for train_cifar's fork_rng device entries by @delock in #8407

Gate the offload-state memory deltas on allocator-backed stats by @delock in #8409

Read rope_theta from rope_parameters across Inference V2 by @alanhuangyoo in #8345

Partition AutoEP expert parameters per layer under ZeRO-3 by @tohtana in #8424

Give Muon's momentum the dtype of the gradient it is combined with by @alanhuangyoo in #8483

Update version.txt after 0.19.6 release by @loadams in #8333

Add DCO sign-off to release commits by @loadams in #8527

Emit affine maps from AutoTP layers by @Achyuthan-S in #8519

Cache the Modal sandbox install chain in image layers by @delock in #8536

[bugfix] _DimZeroAllToAll silently fails to send grads under torch.compile by @stas00 in #8491

New Contributors

@atirna made their first contribution in #8259

@Anai-Guo made their first contribution in #8349

@promptsmith1990 made their first contribution in #8306

@chakshu-dhannawat made their first contribution in #8336

@poorpaper made their first contribution in #8369

@bheu made their first contribution in #8422

@0z5a made their first contribution in #8474

@taking-lying-flat made their first contribution in #8342

Full Changelog: v0.19.6...v0.19.7

0.19.627 Aug 2026

Fix checkpoint rank selection for Ulysses sequence parallelism by @sfc-gh-thonguyen in #8226

What's Changed

Fix checkpoint rank selection for Ulysses sequence parallelism by @sfc-gh-thonguyen in #8226
Return a copy from OnebitLamb.get_lamb_coeffs by @vineethsaivs in #8227
Update version.txt after 0.19.5 release by @loadams in #8243
Implement the documented per-param-group lists in OneCycle by @vineethsaivs in #8201
Keep the elastic batch size within max_train_batch_size by @vineethsaivs in #8237
[DeepCompile] Fix KeyError on frozen parameters in ZeRO-3 by @roycho96 in #8214
[DeepCompile] Fix staticmethod handling on Python 3.9 by @baremetaldevx86 in #8240
Upgrade pip in Python install-smoke jobs by @tohtana in #8247
(1/2) Implementing Compiler Pass for AutoTP by @therealnaveenkamal in #8204
Fix DeepCompile last use for unconsumed waits by @tohtana in #8254
Add configurable sum gradient reduction by @zupengwang in #8232
Add native (DeepNVMe) host-memory pinning backend for accelerators by @sfc-gh-truwase in #8211
Add compile.offload_activation_pin_memory for DeepCompile activation offload by @sfc-gh-truwase in #8258
Fix int32 overflow in Triton grouped-GEMM expert offset by @PKUWZP in #8261
NCCL and other backend PG timeout 30min -> 10min by @stas00 in #8253
Fix ZeRO-3 crash in AutoTP universal-checkpoint metadata by @PKUWZP in #8270
Add CUDA graph support for HybridEngine generation by @PKUWZP in #8271
Fix docstring Args entries that name a parameter the function does not take by @darkdi in #8223
Keep parameter dtype through ZeRO-3 weight quantization by @adityasingh2400 in #8215
[AutoTP] Complete uneven sharding and universal checkpoint support by @jinyouzhi in #8185
Route FPDT and checkpoint writer pins through accelerator pin_memory by @sfc-gh-truwase in #8257
Add fused triton kernel for swiglu by @hwchen2017 in #8244
Make pass contracts cover the passes DeepCompile actually schedules; Renaming files by @pengdurice in #8251
Non-reentrant activation checkpoint CPU offload by @sfc-gh-truwase in #8282
Enable activation offloading by @pengdurice in #8255
[typo] Add raise for RuntimeError by @jinyouzhi in #8281
Fix ZeRO-1/2 with zero-sized parameters by @fwerkor in #8280
docs: note async cpu_checkpointing perf and expandable_segments by @sfc-gh-truwase in #8287
Fix ZeroDivisionError in compute_elastic_config return_microbatch on non-0.2 elasticity by @ebarkhordar in #8286
Fix repeated FlopsProfiler metric accumulation by @baremetaldevx86 in #8246
fix: stop DeepSpeedConfig writing max_grad_norm back into the caller's config dict by @ebarkhordar in #8289
Enable DeepSpeed support on Apple Silicon (MPS) with ZeRO Stage 1-3 by @PKUWZP in #8293
fix(engine): resolve inference workspace attribute lookup by @nathon-lee in #8288
[AutoTP] Convert HF embedding_rowwise tp_plan entries to SKIP specs by @ebarkhordar in #8294
Fix zero reduce bucket size validation by @fwerkor in #8266
Register native pinned host memory with CUDA for GPU DMA by @sfc-gh-truwase in #8283
Drop the documented grad_hooks ZeRO option, which does not exist by @vineethsaivs in #8242
Skip zero-sized parameters in HP fragment mapping by @fwerkor in #8298
perf(rollout): add HybridEngine rollout profiling by @nathon-lee in #8295
Route isend/irecv to nonblocking backend methods and stage them as async on MPS by @PKUWZP in #8303
Fix device mismatch in test_gate_up_partition_covers_the_whole_weight by @alanhuangyoo in #8308
Import print_dist in auto_tp by @alanhuangyoo in #8311
Reject invalid ZenFlow ratio and update interval boundaries by @tandede in #8274
Avoid redundant copies in MPS P2P staging by @FU-max-boop in #8314
Fix AutoTP metadata updates for unsharded modules by @zupengwang in #8299
[Apple Silicon Support Phase 1] Add Metal FusedAdam kernel and CPU Adam build for Apple Silicon by @PKUWZP in #8300
Enable optimized Adam backend for MuonWithAuxAdam optimizer by @jinyouzhi in #8278
feat(opsd): share prompt prefill across rollout samples by @nathon-lee in #8296
stop allocating per-element temporaries in the overflow check by @stas00 in #8325
Fix shared loss gradient accumulation by @tohtana in #8245
Isolate DeepCompile list-schedule test ops by @tohtana in #8319
Fix ZeRO-3 synchronization during OPSD rollout by @LiRunGuo in #8264
[AutoTP] Fix training lm_head routing by @gaoxiaomo in #8302
Route ZeRO/SuperOffload pin sites through accelerator pin_memory by @sfc-gh-truwase in #8256
Honor adam_w_mode in the CPU multi_tensor_adam binding by @PKUWZP in #8307

New Contributors

@sfc-gh-thonguyen made their first contribution in #8226
@baremetaldevx86 made their first contribution in #8240
@darkdi made their first contribution in #8223
@fwerkor made their first contribution in #8280
@alanhuangyoo made their first contribution in #8308
@tandede made their first contribution in #8274
@FU-max-boop made their first contribution in #8314
@LiRunGuo made their first contribution in #8264
@gaoxiaomo made their first contribution in #8302

Full Changelog: v0.19.5...v0.19.6

Open source →

0.19.46 Aug 2026

Validate warmup_type in WarmupCosineLR like WarmupLR by @sohumt123 in https://github.com/deepspeedai/DeepSpeed/pull/8151

What's Changed

Validate warmup_type in WarmupCosineLR like WarmupLR by @sohumt123 in https://github.com/deepspeedai/DeepSpeed/pull/8151
Run pull request code only inside a Modal Sandbox by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8170
[AutoTP] Allow ZeRO stage 3 inference with tensor parallelism by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8167
Fix autotuning get_val_by_key to search all nested subdicts by @ebarkhordar in https://github.com/deepspeedai/DeepSpeed/pull/8177
Enable support for Tutel when k != 1 for shared moe by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8174
NVMe write warning by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8164
Guard LRRangeTest and OneCycle schedulers against zero step sizes by @ebarkhordar in https://github.com/deepspeedai/DeepSpeed/pull/8166
Fix WarmupLR collapsing multi-group base LRs to group 0's by @ebarkhordar in https://github.com/deepspeedai/DeepSpeed/pull/8171
Stabilize fork-sensitive nightly tests and AutoSP coverage by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8172
Make the OneCycle stair counts actually change the schedule by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8179
[AutoTP] Enable HF colwise_gather_output to support lm_head replace by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8146
Scope DeepCompile compiler state to graph and engine lifecycles by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8159
Clarify merge commit exemption for signoff check in AGENTS.md and CLAUDE.md by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8194
[AutoTP] Preserve universal checkpoint metadata for HuggingFace tp_plan by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8187
Derive AutoEP rank splits from the per-expert count exchange by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8190
Warn when zero.Init silently falls back to a single rank (#8084) by @akshansh47 in https://github.com/deepspeedai/DeepSpeed/pull/8089
Release GIL in blocking AIO bindings by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8196
Enable ZeRO-3 linear wrapper for existing models by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8189
Skip CUDA transformer forward tests when TransformerBuilder is incompatible by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8200
Clamp capacity to num_tokens in MoE gating drop branches (complete #5353) by @ebarkhordar in https://github.com/deepspeedai/DeepSpeed/pull/8155
Preserve tensor learning rates across scheduler updates by @n33levo in https://github.com/deepspeedai/DeepSpeed/pull/8202
[AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8168
Add managed_gradient_accumulation for ZeRO stage 0/1 by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8184
MoE dispatch: fix silent Tutel + TP corruption, speed up native and Tutel paths by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8195
Add keyword argument support to activation checkpointing by @zupengwang in https://github.com/deepspeedai/DeepSpeed/pull/8182
Remove redundant DCO workflow by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8188
Preserve tensor learning rates in OneCycle by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8205
Add Triton grouped-GEMM for MoE experts on Ampere/Ada by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8180
Wiring up offload_opt_states by @pengdurice in https://github.com/deepspeedai/DeepSpeed/pull/8186
Fix DeepCompile ZeRO-3 gathered parameter ownership by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8157
Consolidate LR scheduler tensor-LR tests into a contract suite by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8206
Fix MLU_Accelerator conformance to the DeepSpeedAccelerator ABC by @ebarkhordar in https://github.com/deepspeedai/DeepSpeed/pull/8208
Reduce redundant work in AutoEP token routing by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8209
Make PipelineModule.set_checkpoint_interval actually change the interval by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8178
Share DeepNVMe pinned-tensor manager and route swap buffers through I/O handles by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8212
Stop trim_mean sorting the caller's list in place by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8199
Unmanaged gradient accumulation: ZeRO stage 2 support by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8203
Fix ZeRO-3 all_reduce param fetch stride for padded parameters by @ebarkhordar in https://github.com/deepspeedai/DeepSpeed/pull/8158
Unmanaged gradient accumulation: ZeRO stage 3 support by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8217
Update version.txt pre-0.19.4 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/8218

New Contributors

@sohumt123 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8151
@ebarkhordar made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8177
@yh0903 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8190
@akshansh47 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8089
@n33levo made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8202
@zupengwang made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8182
@pengdurice made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8186

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.19.3...v0.19.4

Open source →

Additional notes

What's Changed

Validate warmup_type in WarmupCosineLR like WarmupLR by @sohumt123 in #8151
Run pull request code only inside a Modal Sandbox by @tohtana in #8170
[AutoTP] Allow ZeRO stage 3 inference with tensor parallelism by @delock in #8167
Fix autotuning get_val_by_key to search all nested subdicts by @ebarkhordar in #8177
Enable support for Tutel when k != 1 for shared moe by @jinyouzhi in #8174
NVMe write warning by @sfc-gh-truwase in #8164
Guard LRRangeTest and OneCycle schedulers against zero step sizes by @ebarkhordar in #8166
Fix WarmupLR collapsing multi-group base LRs to group 0's by @ebarkhordar in #8171
Stabilize fork-sensitive nightly tests and AutoSP coverage by @tohtana in #8172
Make the OneCycle stair counts actually change the schedule by @vineethsaivs in #8179
[AutoTP] Enable HF colwise_gather_output to support lm_head replace by @jinyouzhi in #8146
Scope DeepCompile compiler state to graph and engine lifecycles by @tohtana in #8159
Clarify merge commit exemption for signoff check in AGENTS.md and CLAUDE.md by @jinyouzhi in #8194
[AutoTP] Preserve universal checkpoint metadata for HuggingFace tp_plan by @jinyouzhi in #8187
Derive AutoEP rank splits from the per-expert count exchange by @yh0903 in #8190
Warn when zero.Init silently falls back to a single rank (#8084) by @akshansh47 in #8089
Release GIL in blocking AIO bindings by @tohtana in #8196
Enable ZeRO-3 linear wrapper for existing models by @tohtana in #8189
Skip CUDA transformer forward tests when TransformerBuilder is incompatible by @sfc-gh-truwase in #8200
Clamp capacity to num_tokens in MoE gating drop branches (complete #5353) by @ebarkhordar in #8155
Preserve tensor learning rates across scheduler updates by @n33levo in #8202
[AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP by @delock in #8168
Add managed_gradient_accumulation for ZeRO stage 0/1 by @sfc-gh-truwase in #8184
MoE dispatch: fix silent Tutel + TP corruption, speed up native and Tutel paths by @jinyouzhi in #8195
Add keyword argument support to activation checkpointing by @zupengwang in #8182
Remove redundant DCO workflow by @tohtana in #8188
Preserve tensor learning rates in OneCycle by @tohtana in #8205
Add Triton grouped-GEMM for MoE experts on Ampere/Ada by @hwchen2017 in #8180
Wiring up offload_opt_states by @pengdurice in #8186
Fix DeepCompile ZeRO-3 gathered parameter ownership by @tohtana in #8157
Consolidate LR scheduler tensor-LR tests into a contract suite by @delock in #8206
Fix MLU_Accelerator conformance to the DeepSpeedAccelerator ABC by @ebarkhordar in #8208
Reduce redundant work in AutoEP token routing by @hwchen2017 in #8209
Make PipelineModule.set_checkpoint_interval actually change the interval by @vineethsaivs in #8178
Share DeepNVMe pinned-tensor manager and route swap buffers through I/O handles by @sfc-gh-truwase in #8212
Stop trim_mean sorting the caller's list in place by @vineethsaivs in #8199
Unmanaged gradient accumulation: ZeRO stage 2 support by @sfc-gh-truwase in #8203
Fix ZeRO-3 all_reduce param fetch stride for padded parameters by @ebarkhordar in #8158
Unmanaged gradient accumulation: ZeRO stage 3 support by @sfc-gh-truwase in #8217
Update version.txt pre-0.19.4 release by @loadams in #8218

New Contributors

@sohumt123 made their first contribution in #8151
@ebarkhordar made their first contribution in #8177
@yh0903 made their first contribution in #8190
@akshansh47 made their first contribution in #8089
@n33levo made their first contribution in #8202
@zupengwang made their first contribution in #8182
@pengdurice made their first contribution in #8186

Full Changelog: v0.19.3...v0.19.4

0.19.323 Jul 2026Deprecated

Feat: zero3 deprecate elastic checkpoint by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8099

What's Changed

Update version post release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/8070
Validate fp16 dynamic loss scaling parameters are positive by @aryanputta in https://github.com/deepspeedai/DeepSpeed/pull/8050
Add merge queue DCO workflow by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8086
Stop obsolete CI jobs on workflow cancellation by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8034
Fix ZeRO-3: Use per-param dtype for output buffers in _allgather_params_coalesced by @albertvillanova in https://github.com/deepspeedai/DeepSpeed/pull/8073
fix: close file descriptor in deepspeed_io_handle_t::wait() to prevent fd leak by @MarkCLChang in https://github.com/deepspeedai/DeepSpeed/pull/8075
Add type hints to top-level public API functions by @arunshar in https://github.com/deepspeedai/DeepSpeed/pull/8079
Default gradient_clipping to 1.0 by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8068
activation_checkpointing: default num_layers to None so configure() assert fires by @Kymi808 in https://github.com/deepspeedai/DeepSpeed/pull/8041
Add configurable engine log level by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8067
[CI] diff driven test selection by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/8077
Enable bf16 check_grad_overflow by default (matching fp16) by @yongzhe-wang in https://github.com/deepspeedai/DeepSpeed/pull/8035
Support AutoEP with ZeRO-3 zero.Init source modules by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8060
Update publication list in README.md by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/8095
fix: AutoTP partition_config uses full hierarchical module path by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8088
Reject Muon optimizer with reduce_scatter in ZeRO-1/2 by @whycoming in https://github.com/deepspeedai/DeepSpeed/pull/8090
Fix DeepCompile profile metadata backfill by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8094
Add numerical-correctness test for Muon under ZeRO-1/2 by @whycoming in https://github.com/deepspeedai/DeepSpeed/pull/8091
Don't swallow KeyboardInterrupt/SystemExit in NPUOpBuilder by @ajinkyajawale14499 in https://github.com/deepspeedai/DeepSpeed/pull/8096
Avoid CUDA context initialization during op compatibility checks at import by @Achyuthan-S in https://github.com/deepspeedai/DeepSpeed/pull/8078
[DeepCompile] fix gather params in dynamo skipped frames for ZeRO3 by @XAheli in https://github.com/deepspeedai/DeepSpeed/pull/8059
Feat: zero3 deprecate elastic checkpoint by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8099
Fix eigenvalue parsing for compression-only quantize configs by @sowndappan5 in https://github.com/deepspeedai/DeepSpeed/pull/8057
fix: log eigenvalue monitor values by @he-yufeng in https://github.com/deepspeedai/DeepSpeed/pull/8049
Fix DeepCompile profiling memory cleanup by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8106
Add optional torchembed RoPE backend to apply_rotary_pos_emb by @py-ai-dev in https://github.com/deepspeedai/DeepSpeed/pull/8052
Fix minor comment/docstring typos in runtime and inference modules by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8046
feat(zenflow): run the overlapped CPU optimizer in a native process by @Antlera in https://github.com/deepspeedai/DeepSpeed/pull/8058
fix: add buffer-length check in shm.cpp by @orbisai0security in https://github.com/deepspeedai/DeepSpeed/pull/8082
Fix FlopsProfiler crash when dp_world_size is None under sequence parallelism by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8122
Fix Python 3.9 import-time TypeError in AutoEP ep_router by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8119
Add input validation for save_checkpoint and warmup LR schedulers by @jahnavi-yelamanchi in https://github.com/deepspeedai/DeepSpeed/pull/8126
Fix ZeRO-3: use per-param dtype for quantize-scale buffer in _allgather_params_coalesced by @roycho96 in https://github.com/deepspeedai/DeepSpeed/pull/8128
Fix ZeRO-3 autocast gather with mixed parameter dtypes by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8113
doc: Remove suggestion to build extensions in parallel by @Flamefire in https://github.com/deepspeedai/DeepSpeed/pull/7899
feat(types): add type hints to public comm API functions by @NovusEdge in https://github.com/deepspeedai/DeepSpeed/pull/8118
Add Hybrid Engine rollout in DeepSpeed to support On-Policy Distillation (OPSD) Trainer by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/8027
Add AutoEP + AutoTP parallel folding by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8064
Make DCO workflow Probot compatible by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8110
ZeRO 1/2: wait on all IPG-bucket producer streams in average_tensor (#8061) by @arunshar in https://github.com/deepspeedai/DeepSpeed/pull/8080
Fix CommsLogger.stop_profiling_comms not disabling global profiling by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8137
Guard WarmupCosineLR against total_num_steps == warmup_num_steps (ZeroDivisionError) by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8142
Add EXAONE 4.5 model support for Inference V2 by @Bias92 in https://github.com/deepspeedai/DeepSpeed/pull/8121
fix(autotp): when using autotp, ignore the consistency of certain data within the tp_group. by @1787648106 in https://github.com/deepspeedai/DeepSpeed/pull/8125
Reduce blockDim in fake_quantize_kernel for improved SM occupancy by @flutist in https://github.com/deepspeedai/DeepSpeed/pull/8115
Fix DeepSpeedInferenceConfig crash on bool moe backward-compat value by @winklemad in https://github.com/deepspeedai/DeepSpeed/pull/8145
fix(exceptions): narrow broad Exception clauses to specific types by @sowndappan5 in https://github.com/deepspeedai/DeepSpeed/pull/8136
fix(muon): support ZeRO-1/2 reduce scatter by @Micuks in https://github.com/deepspeedai/DeepSpeed/pull/8141
[DeepCompile] Add lightweight pass contracts for optimization passes by @jahnavi-yelamanchi in https://github.com/deepspeedai/DeepSpeed/pull/8139
Fix DeepCompile fallback handling for scalar outputs by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8149
Use synthetic CIFAR data in AlexNet tests by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8143
Update CI badge by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8150
Update Zhipeng Wang affiliation to Google in COMMITTERS.md by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/8153
Fix ZeRO-3 hooks for attribute-delegating modules by @Functionhx in https://github.com/deepspeedai/DeepSpeed/pull/8144
Fix repeated gradient scaling across pipeline stages by @Functionhx in https://github.com/deepspeedai/DeepSpeed/pull/8154
Z3: Support for activation ckpt with frozen params by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8148
Adopt PTF Code of Conduct by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8165

New Contributors

@aryanputta made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8050
@albertvillanova made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8073
@MarkCLChang made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8075
@arunshar made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8079
@Kymi808 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8041
@yongzhe-wang made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8035
@whycoming made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8090
@ajinkyajawale14499 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8096
@Achyuthan-S made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8078
@XAheli made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8059
@he-yufeng made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8049
@py-ai-dev made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8052
@vineethsaivs made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8122
@jahnavi-yelamanchi made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8126
@NovusEdge made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8118
@1787648106 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8125
@winklemad made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8145
@Micuks made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8141
@Functionhx made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8144

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.19.2...v0.19.3

Open source →

Additional notes

What's Changed

Update version post release by @loadams in #8070
Validate fp16 dynamic loss scaling parameters are positive by @aryanputta in #8050
Add merge queue DCO workflow by @tohtana in #8086
Stop obsolete CI jobs on workflow cancellation by @tohtana in #8034
Fix ZeRO-3: Use per-param dtype for output buffers in _allgather_params_coalesced by @albertvillanova in #8073
fix: close file descriptor in deepspeed_io_handle_t::wait() to prevent fd leak by @MarkCLChang in #8075
Add type hints to top-level public API functions by @arunshar in #8079
Default gradient_clipping to 1.0 by @sfc-gh-truwase in #8068
activation_checkpointing: default num_layers to None so configure() assert fires by @Kymi808 in #8041
Add configurable engine log level by @sfc-gh-truwase in #8067
[CI] diff driven test selection by @stas00 in #8077
Enable bf16 check_grad_overflow by default (matching fp16) by @yongzhe-wang in #8035
Support AutoEP with ZeRO-3 zero.Init source modules by @tohtana in #8060
Update publication list in README.md by @PKUWZP in #8095
fix: AutoTP partition_config uses full hierarchical module path by @delock in #8088
Reject Muon optimizer with reduce_scatter in ZeRO-1/2 by @whycoming in #8090
Fix DeepCompile profile metadata backfill by @tohtana in #8094
Add numerical-correctness test for Muon under ZeRO-1/2 by @whycoming in #8091
Don't swallow KeyboardInterrupt/SystemExit in NPUOpBuilder by @ajinkyajawale14499 in #8096
Avoid CUDA context initialization during op compatibility checks at import by @Achyuthan-S in #8078
[DeepCompile] fix gather params in dynamo skipped frames for ZeRO3 by @XAheli in #8059
Feat: zero3 deprecate elastic checkpoint by @nathon-lee in #8099
Fix eigenvalue parsing for compression-only quantize configs by @sowndappan5 in #8057
fix: log eigenvalue monitor values by @he-yufeng in #8049
Fix DeepCompile profiling memory cleanup by @tohtana in #8106
Add optional torchembed RoPE backend to apply_rotary_pos_emb by @py-ai-dev in #8052
Fix minor comment/docstring typos in runtime and inference modules by @nathon-lee in #8046
feat(zenflow): run the overlapped CPU optimizer in a native process by @Antlera in #8058
fix: add buffer-length check in shm.cpp by @orbisai0security in #8082
Fix FlopsProfiler crash when dp_world_size is None under sequence parallelism by @vineethsaivs in #8122
Fix Python 3.9 import-time TypeError in AutoEP ep_router by @vineethsaivs in #8119
Add input validation for save_checkpoint and warmup LR schedulers by @jahnavi-yelamanchi in #8126
Fix ZeRO-3: use per-param dtype for quantize-scale buffer in _allgather_params_coalesced by @roycho96 in #8128
Fix ZeRO-3 autocast gather with mixed parameter dtypes by @tohtana in #8113
doc: Remove suggestion to build extensions in parallel by @Flamefire in #7899
feat(types): add type hints to public comm API functions by @NovusEdge in #8118
Add Hybrid Engine rollout in DeepSpeed to support On-Policy Distillation (OPSD) Trainer by @PKUWZP in #8027
Add AutoEP + AutoTP parallel folding by @tohtana in #8064
Make DCO workflow Probot compatible by @tohtana in #8110
ZeRO 1/2: wait on all IPG-bucket producer streams in average_tensor (#8061) by @arunshar in #8080
Fix CommsLogger.stop_profiling_comms not disabling global profiling by @vineethsaivs in #8137
Guard WarmupCosineLR against total_num_steps == warmup_num_steps (ZeroDivisionError) by @vineethsaivs in #8142
Add EXAONE 4.5 model support for Inference V2 by @Bias92 in #8121
fix(autotp): when using autotp, ignore the consistency of certain data within the tp_group. by @1787648106 in #8125
Reduce blockDim in fake_quantize_kernel for improved SM occupancy by @flutist in #8115
Fix DeepSpeedInferenceConfig crash on bool moe backward-compat value by @winklemad in #8145
fix(exceptions): narrow broad Exception clauses to specific types by @sowndappan5 in #8136
fix(muon): support ZeRO-1/2 reduce scatter by @Micuks in #8141
[DeepCompile] Add lightweight pass contracts for optimization passes by @jahnavi-yelamanchi in #8139
Fix DeepCompile fallback handling for scalar outputs by @tohtana in #8149
Use synthetic CIFAR data in AlexNet tests by @tohtana in #8143
Update CI badge by @tohtana in #8150
Update Zhipeng Wang affiliation to Google in COMMITTERS.md by @PKUWZP in #8153
Fix ZeRO-3 hooks for attribute-delegating modules by @Functionhx in #8144
Fix repeated gradient scaling across pipeline stages by @Functionhx in #8154
Z3: Support for activation ckpt with frozen params by @sfc-gh-truwase in #8148
Adopt PTF Code of Conduct by @sfc-gh-truwase in #8165

New Contributors

@aryanputta made their first contribution in #8050
@albertvillanova made their first contribution in #8073
@MarkCLChang made their first contribution in #8075
@arunshar made their first contribution in #8079
@Kymi808 made their first contribution in #8041
@yongzhe-wang made their first contribution in #8035
@whycoming made their first contribution in #8090
@ajinkyajawale14499 made their first contribution in #8096
@Achyuthan-S made their first contribution in #8078
@XAheli made their first contribution in #8059
@he-yufeng made their first contribution in #8049
@py-ai-dev made their first contribution in #8052
@vineethsaivs made their first contribution in #8122
@jahnavi-yelamanchi made their first contribution in #8126
@NovusEdge made their first contribution in #8118
@1787648106 made their first contribution in #8125
@winklemad made their first contribution in #8145
@Micuks made their first contribution in #8141
@Functionhx made their first contribution in #8144

Full Changelog: v0.19.2...v0.19.3

0.19.216 Jun 2026

fix(fp16): filter requires_grad in FP16 optimizer flat buffer init by @avicooper1 in https://github.com/deepspeedai/DeepSpeed/pull/8029

What's Changed

fix(fp16): filter requires_grad in FP16 optimizer flat buffer init by @avicooper1 in https://github.com/deepspeedai/DeepSpeed/pull/8029
Run AutoSP compile tests sequentially by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8020
Fix PR-target workflow concurrency groups by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8017
Fix full CI test isolation for ZeRO chmod and NVMe quantization tests by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8008
Keep required CI checks visible for ignored paths by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8019
Bump version by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8030
Add engine.coalesce_grad_reduction() for ZeRO 1/2/3 multi-backward by @roycho96 in https://github.com/deepspeedai/DeepSpeed/pull/7992
feat(zero): enable torch.func transforms on engine for ZeRO 0/1/2 by @roycho96 in https://github.com/deepspeedai/DeepSpeed/pull/8026
Simplify module_inject.transpose by @xbcReal in https://github.com/deepspeedai/DeepSpeed/pull/8028
Fix DeepCompile all-gather scheduler candidate selection by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8033
Version fix to unblock pypi by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8039
Bump version after 0.19.1 release by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8040
Fix DeepCompile ZeRO-3 release parameter lifetime by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8032
Fix ZenFlow ZeRO-3 selective optimizer crash with parameter offload on nvme by @Antlera in https://github.com/deepspeedai/DeepSpeed/pull/8042
Add test coverage for Muon muon_lr/adam_lr overrides by @sowndappan5 in https://github.com/deepspeedai/DeepSpeed/pull/8047
Avoid HF Hub access in CPU unit test setup by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8053
Fix DeepCompile ZeRO-1 grad target lifetime by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8036
Normalize ZeRO-3 DeepCompile grad dtype before reduction by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8038
Remove AutoSP assertion against Transformers version by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8044
fix(transformer): use correct stride in Transpose_Kernel shared memory indexing to eliminate bank conflicts by @flutist in https://github.com/deepspeedai/DeepSpeed/pull/8055
zero3: invalidate coordinator trace on hook re-registration by @roycho96 in https://github.com/deepspeedai/DeepSpeed/pull/8043
Consistent fp32 grads flow by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8056
Add AutoEP by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7938
Fix: ZenFlow Adam integration for updated PyTorch backward flow (#7759) by @Antlera in https://github.com/deepspeedai/DeepSpeed/pull/7771
Pass expected grad dtype to register_z3_param in ZeRO-3 release test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8063
Add Biren SUPA accelerator support by @frozenleaves in https://github.com/deepspeedai/DeepSpeed/pull/8054
Mixed-precision: per-policy param/buffer dtype cast (preserve fp32 buffers) by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8066

New Contributors

@avicooper1 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8029
@sowndappan5 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8047
@flutist made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8055
@frozenleaves made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8054

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.19.1...v0.19.2

Open source →

Additional notes

What's Changed

fix(fp16): filter requires_grad in FP16 optimizer flat buffer init by @avicooper1 in #8029
Run AutoSP compile tests sequentially by @tohtana in #8020
Fix PR-target workflow concurrency groups by @tohtana in #8017
Fix full CI test isolation for ZeRO chmod and NVMe quantization tests by @tohtana in #8008
Keep required CI checks visible for ignored paths by @tohtana in #8019
Bump version by @sfc-gh-truwase in #8030
Add engine.coalesce_grad_reduction() for ZeRO 1/2/3 multi-backward by @roycho96 in #7992
feat(zero): enable torch.func transforms on engine for ZeRO 0/1/2 by @roycho96 in #8026
Simplify module_inject.transpose by @xbcReal in #8028
Fix DeepCompile all-gather scheduler candidate selection by @tohtana in #8033
Version fix to unblock pypi by @sfc-gh-truwase in #8039
Bump version after 0.19.1 release by @tohtana in #8040
Fix DeepCompile ZeRO-3 release parameter lifetime by @tohtana in #8032
Fix ZenFlow ZeRO-3 selective optimizer crash with parameter offload on nvme by @Antlera in #8042
Add test coverage for Muon muon_lr/adam_lr overrides by @sowndappan5 in #8047
Avoid HF Hub access in CPU unit test setup by @tohtana in #8053
Fix DeepCompile ZeRO-1 grad target lifetime by @tohtana in #8036
Normalize ZeRO-3 DeepCompile grad dtype before reduction by @tohtana in #8038
Remove AutoSP assertion against Transformers version by @tohtana in #8044
fix(transformer): use correct stride in Transpose_Kernel shared memory indexing to eliminate bank conflicts by @flutist in #8055
zero3: invalidate coordinator trace on hook re-registration by @roycho96 in #8043
Consistent fp32 grads flow by @sfc-gh-truwase in #8056
Add AutoEP by @tohtana in #7938
Fix: ZenFlow Adam integration for updated PyTorch backward flow (#7759) by @Antlera in #7771
Pass expected grad dtype to register_z3_param in ZeRO-3 release test by @tohtana in #8063
Add Biren SUPA accelerator support by @frozenleaves in #8054
Mixed-precision: per-policy param/buffer dtype cast (preserve fp32 buffers) by @sfc-gh-truwase in #8066

New Contributors

@avicooper1 made their first contribution in #8029
@sowndappan5 made their first contribution in #8047
@flutist made their first contribution in #8055
@frozenleaves made their first contribution in #8054

Full Changelog: v0.19.1...v0.19.2

0.18.930 Mar 2026

Respect $TRITON_HOME by @Flamefire in https://github.com/deepspeedai/DeepSpeed/pull/7907

What's Changed

Respect $TRITON_HOME by @Flamefire in https://github.com/deepspeedai/DeepSpeed/pull/7907
Add Feature Universal Checkpoint for AutoTP by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/7908
fix: remove unnecessary shell=True in ROCm GPU architecture detection by @instantraaamen in https://github.com/deepspeedai/DeepSpeed/pull/7915
Don't detect local GPU if $DS_IGNORE_CUDA_DETECTION is set by @Flamefire in https://github.com/deepspeedai/DeepSpeed/pull/7896
Add HuggingFace tp_plan support for AutoTP by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7901
fix: handle non-existent path in is_nfs_path for Triton autotune cache by @Krishnachaitanyakc in https://github.com/deepspeedai/DeepSpeed/pull/7921
Fix backward compatibility of torch.amp.custom_fwd for PyTorch < 2.4 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7920
Extending Muon Optimizer Support for ZeRO Stage 3 by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/7919
Add news item for ASPLOS 2026 Best Paper Award by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/7923
fix(superoffload) preserve multi-group updates with shared cpu buffers (#7905) by @xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/7906
AGENTS.md: Add pre-commit command to existing CI requirements line by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7930
Update README with latest news from DeepSpeed by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/7931
Merging AutoSP into DeepSpeed by @neeldani in https://github.com/deepspeedai/DeepSpeed/pull/7860
Add fallback to full test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7933
Remove Microsoft Corporation copyright from AGENTS.md and CLAUDE.md by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/7932
Update version.txt for latest incoming release 0.18.9 by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7935

New Contributors

@instantraaamen made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7915
@Krishnachaitanyakc made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7921
@neeldani made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7860

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.18.8...v0.18.9

Open source →

Additional notes

What's Changed

Respect $TRITON_HOME by @Flamefire in #7907
Add Feature Universal Checkpoint for AutoTP by @nathon-lee in #7908
fix: remove unnecessary shell=True in ROCm GPU architecture detection by @instantraaamen in #7915
Don't detect local GPU if $DS_IGNORE_CUDA_DETECTION is set by @Flamefire in #7896
Add HuggingFace tp_plan support for AutoTP by @delock in #7901
fix: handle non-existent path in is_nfs_path for Triton autotune cache by @Krishnachaitanyakc in #7921
Fix backward compatibility of torch.amp.custom_fwd for PyTorch < 2.4 by @tohtana in #7920
Extending Muon Optimizer Support for ZeRO Stage 3 by @PKUWZP in #7919
Add news item for ASPLOS 2026 Best Paper Award by @PKUWZP in #7923
fix(superoffload) preserve multi-group updates with shared cpu buffers (#7905) by @xylian86 in #7906
AGENTS.md: Add pre-commit command to existing CI requirements line by @delock in #7930
Update README with latest news from DeepSpeed by @PKUWZP in #7931
Merging AutoSP into DeepSpeed by @neeldani in #7860
Add fallback to full test by @tohtana in #7933
Remove Microsoft Corporation copyright from AGENTS.md and CLAUDE.md by @PKUWZP in #7932
Update version.txt for latest incoming release 0.18.9 by @loadams in #7935

New Contributors

@instantraaamen made their first contribution in #7915
@Krishnachaitanyakc made their first contribution in #7921
@neeldani made their first contribution in #7860

Full Changelog: v0.18.8...v0.18.9

0.18.75 Mar 2026

Update version post release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7850

What's Changed

Update version post release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7850
Z1/2 init: flatten params on device by @ksugama in https://github.com/deepspeedai/DeepSpeed/pull/7828
Enable shm_comm support for arm by @phalani-paladugu in https://github.com/deepspeedai/DeepSpeed/pull/7800
Add news entry for DeepSpeed updates by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/7854
Add EXAONE 4.0 model support for Inference V2 by @Bias92 in https://github.com/deepspeedai/DeepSpeed/pull/7853
Fix ROCm BF16 conversion intrinsics in inference v2 (#7843) by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7846
Fix compilation of Evoformer by @Flamefire in https://github.com/deepspeedai/DeepSpeed/pull/7862
Throw error when parameter is modified in GatheredParameters by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7832
Fix Zero-3 static scale assertion in fp16 test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7866
Schedule nightly full test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7870
Fix broken links and add AutoTP Training tutorial to sidebar nav by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7874
fix: replace 35 bare except clauses with except Exception by @haosenwang1018 in https://github.com/deepspeedai/DeepSpeed/pull/7873
perf: use deque for FIFO queues in sequence parallel, superoffload, and compile by @giulio-leone in https://github.com/deepspeedai/DeepSpeed/pull/7880
Fix: only add parameter with grads to parameter group by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7869
Fix no-grad grad-fn lookup in ZeRO hook counting on PyTorch 2.3 (#7830) by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7841
Fix import deepspeed crash on PyTorch v2.3 + Python 3.12 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7875
XPU use stock pytorch instead of Intel Extension for PyTorch by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7877
Remove amp() from abstract accelerator by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7879
Add document section explaining autocast nesting by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7883
Fix hook count performance regression from v0.18.5 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7886

New Contributors

@ksugama made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7828
@phalani-paladugu made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7800
@Bias92 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7853
@haosenwang1018 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7873
@giulio-leone made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7880

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.18.6...v0.18.7

Open source →

Additional notes

What's Changed

Update version post release by @loadams in #7850
Z1/2 init: flatten params on device by @ksugama in #7828
Enable shm_comm support for arm by @phalani-paladugu in #7800
Add news entry for DeepSpeed updates by @PKUWZP in #7854
Add EXAONE 4.0 model support for Inference V2 by @Bias92 in #7853
Fix ROCm BF16 conversion intrinsics in inference v2 (#7843) by @tohtana in #7846
Fix compilation of Evoformer by @Flamefire in #7862
Throw error when parameter is modified in GatheredParameters by @tohtana in #7832
Fix Zero-3 static scale assertion in fp16 test by @tohtana in #7866
Schedule nightly full test by @tohtana in #7870
Fix broken links and add AutoTP Training tutorial to sidebar nav by @tohtana in #7874
fix: replace 35 bare except clauses with except Exception by @haosenwang1018 in #7873
perf: use deque for FIFO queues in sequence parallel, superoffload, and compile by @giulio-leone in #7880
Fix: only add parameter with grads to parameter group by @delock in #7869
Fix no-grad grad-fn lookup in ZeRO hook counting on PyTorch 2.3 (#7830) by @tohtana in #7841
Fix import deepspeed crash on PyTorch v2.3 + Python 3.12 by @tohtana in #7875
XPU use stock pytorch instead of Intel Extension for PyTorch by @delock in #7877
Remove amp() from abstract accelerator by @delock in #7879
Add document section explaining autocast nesting by @tohtana in #7883
Fix hook count performance regression from v0.18.5 by @tohtana in #7886

New Contributors

@ksugama made their first contribution in #7828
@phalani-paladugu made their first contribution in #7800
@Bias92 made their first contribution in #7853
@haosenwang1018 made their first contribution in #7873
@giulio-leone made their first contribution in #7880

Full Changelog: v0.18.6...v0.18.7

0.18.530 Jan 2026

Update version.txt after 0.18.4 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7765

What's Changed

Update version.txt after 0.18.4 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7765
Various fixes to run on mps by @jeffra in https://github.com/deepspeedai/DeepSpeed/pull/7767
Udpate workflow trigger by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7768
fix: delete using namespace std. by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/7766
fix: update Megatron-DeepSpeed tutorial to match current repo structure by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/7761
Add timeout to test workflows by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7774
Remove cron/PR triggers for outdated V100 tests by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7777
[Docs] Fix docs/_pages/config-json.md format by @ooooo-create in https://github.com/deepspeedai/DeepSpeed/pull/7779
Update CLA to refer to DCO by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7778
Fix multiprocessing testcase by @k-artem in https://github.com/deepspeedai/DeepSpeed/pull/7743
fix: skip compressed allreduce for empty tensors by @T1mn in https://github.com/deepspeedai/DeepSpeed/pull/7769
docs: update README.md by @eltociear in https://github.com/deepspeedai/DeepSpeed/pull/7781
Fix gradient checkpointing with use_reentrant=True / PyTorch-style backward / ZeRO-3 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7780
Fix Ulysses PEFT test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7784
Fix Evoformer compilation by @sdvillal in https://github.com/deepspeedai/DeepSpeed/pull/7760
fix checkpointing/loading of z0+bf16 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7786
Add sequential allgather optimization for ZeRO-3 by @aeeeeeep in https://github.com/deepspeedai/DeepSpeed/pull/7661
Fix AutoTP test numerical tolerance with rtol by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7794
Fix backward for pipeline engine by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7787
Skip empty parameters in gradient reduction by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7789
Fix issue with BF16 optimizer selection by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7788
Fix BF16_Optimizer being used without ZeRO by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7790
Add full test suite workflow by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7795
Fix Muon optimizer module path by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7802
Fix ping-pong buffer index reset and removing redundant stream sync by @undersilence in https://github.com/deepspeedai/DeepSpeed/pull/7805
Fix ZeRO stage to choose BF16 optimizer in test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7803
Run Evoformer tests sequentially by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7810
Improve engine's cleanup by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7813
Ignore evoformer test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7815
Fix typos in accelerator setup guide by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/7818
Raise clear error on in-place GatheredParameters edits without modifier_rank by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7817
[Bugfix] Resolve Rank index out of range during BWD when sp_size < world_size in Ulysses by @Flink-ddd in https://github.com/deepspeedai/DeepSpeed/pull/7809
Update PyTorch to v2.9 for modal tests by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7816

New Contributors

@ooooo-create made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7779
@T1mn made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7769
@sdvillal made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7760
@undersilence made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7805

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.18.4...v0.18.5

Open source →

0.18.47 Jan 2026

Update version by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7719

What's Changed

Update version by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7719
Disable deterministic option in compile tests by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7720
Fix SuperOffloadOptimizer_Stage3 crash due to missing param_names parameter by @ImaGoodFella in https://github.com/deepspeedai/DeepSpeed/pull/7715
[AMD][ROCm] Improve support of AMD by @k-artem in https://github.com/deepspeedai/DeepSpeed/pull/7448
fix typo by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7722
Skip none in backward hook by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7725
[Engine] Only scale gradients if scale_wrt_gas is True by @kashif in https://github.com/deepspeedai/DeepSpeed/pull/7724
Fix testcases that depends on triton by @k-artem in https://github.com/deepspeedai/DeepSpeed/pull/7731
Fix rare hang in DeepSpeed Async I/O wait by releasing the Python GIL by @xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/7727
Fix #7733: Replace torch.sqrt with math.sqrt in scale_lr for sqrt method by @Rakshit-gen in https://github.com/deepspeedai/DeepSpeed/pull/7735
replace moe checkpoint dp_world_size with seq_dp_world_size by @wukong1992 in https://github.com/deepspeedai/DeepSpeed/pull/7732
[BUG] Fix UlyssesSPAttentionHF.register_with_transformers() crash with PEFT models by @Rakshit-gen in https://github.com/deepspeedai/DeepSpeed/pull/7737
Add core api update blog by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7738
Fix Nebula checkpoint engine commit() API mismatch by @Rakshit-gen in https://github.com/deepspeedai/DeepSpeed/pull/7740
Fix DecoupledCheckpointEngine deadlock and improve reliability by @Rakshit-gen in https://github.com/deepspeedai/DeepSpeed/pull/7742
Fix OnebitLamb NaN propagation with empty parameters by @Rakshit-gen in https://github.com/deepspeedai/DeepSpeed/pull/7736
fix: remove premature MPI environment variable check in OpenMPIRunner by @leejianwoo-collab in https://github.com/deepspeedai/DeepSpeed/pull/7751
Enable python 3.11 and 3.12 tests by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7007
Add CI workflow to run tests on AWS by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7753
Add fallback to BF16 support check by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7754
Fix DeepCompile for PyTorch 2.8/2.9 compatibility by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7755
Removed amp testcases by @k-artem in https://github.com/deepspeedai/DeepSpeed/pull/7745
fix: avoid IndexError in BF16_Optimizer.destroy() when using DummyOptim by @leejianwoo-collab in https://github.com/deepspeedai/DeepSpeed/pull/7763

New Contributors

@ImaGoodFella made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7715
@k-artem made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7448
@kashif made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7724
@Rakshit-gen made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7735
@leejianwoo-collab made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7751

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.18.3...v0.18.4

Open source →

0.18.07 Oct 2025

Update version.txt post 0.17.6 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7572

What's Changed

Update version.txt post 0.17.6 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7572
DeepCompile ZeRO-3: robust allgather for uneven shards; fix profiling… by @juyterman1000 in https://github.com/deepspeedai/DeepSpeed/pull/7489
logging: Also set log level of logger handlers by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7576
Deepcompile: Fix bugs when applying deepcompile to VLA-like models by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7569
Broadcast fp16 overflow in Z1 by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7580
Deepcompile: Make size of activation to free configurable by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7582
SuperOffload Release by @xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/7559
Include init file for superoffload folder by @nguyen599 in https://github.com/deepspeedai/DeepSpeed/pull/7591
disables ZeRO checkpoint loading path when stage=0 by @therealnaveenkamal in https://github.com/deepspeedai/DeepSpeed/pull/7586
Simplify leaf module hook by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7592
Fix the universal checkpoint issue for stage3 when there are multiple subgroups. by @zhengchenyu in https://github.com/deepspeedai/DeepSpeed/pull/7585
Change current_device() to current_device_name() by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7600
Fixed the problem of loading universal checkpoint error in multi-machine mode. by @zhengchenyu in https://github.com/deepspeedai/DeepSpeed/pull/7601
DeepCompile: Specify tensor aliasing in C++ op schema by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7597
DeepCompile: Fuse allgather and downcast by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7588
Add blog for SuperOffload by @xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/7594
Add venv to .gitignore by @zhengchenyu in https://github.com/deepspeedai/DeepSpeed/pull/7605
Handle the case of DeepCompile's enabled but not activated by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7603
DeepCompile: Fix IPG bucket clearing by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7610
Minor fix in the SuperOffload blog by @xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/7612
Fixed the issue that universal checkpoint cannot be loaded for stage3 when world size expansion. by @zhengchenyu in https://github.com/deepspeedai/DeepSpeed/pull/7599
Fixed save_checkpoint race when consolidating NVMe offloaded tensors by @H1manshu21 in https://github.com/deepspeedai/DeepSpeed/pull/7613
[wall_clock_breakdown] always log stats when enabled by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7617
DeepCompile: Use min_cut_rematerialization for partitioning joint graphs by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7609
Show mismatching values when DeepCompile test fails by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7618
Improve leaf module interface (enable via config, relax matching criteria, add document, etc.) by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7604
add print_dist util by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7621
Super offload blog Chinese version by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7620
Enable grad scaler for ZeRO-0 + torch.autocast path by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7619
Blog of zenflow binding study by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7614
Clarify document of leaf module config by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7623
[TiledMLP] moe support by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7622
Update email address by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7624

New Contributors

@juyterman1000 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7489
@nguyen599 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7591
@zhengchenyu made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7585
@H1manshu21 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7613

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.17.6...v0.18.0

Open source →

0.17.619 Sept 2025

Update version.txt after 0.17.5 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7502

What's Changed

Update version.txt after 0.17.5 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7502
Support DeepSpeed offload and reload states with ZeRO1 and ZeRO2 by @LYMDLUT in https://github.com/deepspeedai/DeepSpeed/pull/7421
CI funding shout out to modal.com by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7503
Fix assert when 'pp_int' object has no attribute 'custom_print_str' by @aeeeeeep in https://github.com/deepspeedai/DeepSpeed/pull/7507
Update TSC Committers by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/7517
Enabling Muon Optimizer in DeepSpeed by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/7509
Enable non-ZeRO mode by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7515
Update README with ZenFlow release blog featured by PyTorch. by @Antlera in https://github.com/deepspeedai/DeepSpeed/pull/7520
Add riscv64 cpu support in deepspeed_shm_comm op by @heyujiao99 in https://github.com/deepspeedai/DeepSpeed/pull/7519
ZeRO3: Improve mismatch detection by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7525
fix typo s/1014 /1024 by @digger-yu in https://github.com/deepspeedai/DeepSpeed/pull/7528
undo the revert by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7536
[logging] less startup noise by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7526
[doc] fixing moe tutorial by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7538
docs typo: lrrt.md, reference to cycle_min_lr should be cycle_max_lr by @jakehemmerle in https://github.com/deepspeedai/DeepSpeed/pull/7530
fixed DeepSpeedCPULion with ZeRO-Offload bug by @qibin0506 in https://github.com/deepspeedai/DeepSpeed/pull/7531
Fix scaling and allgather with torch.autocast by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7534
Fix zenflow_torch_adam.py by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7544
Relax restrictions of torch.autocast integration by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7543
Autotune ZenFlow affinity by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7506
fix get_cuda_compile_flag by @mingjielu in https://github.com/deepspeedai/DeepSpeed/pull/7521
avoid setting device_id to init_process_group by @kaixuanliu in https://github.com/deepspeedai/DeepSpeed/pull/7542
Improve error message and reduce validation in autocast test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7547
Revert "Add index to HPU devices (#7497)" by @deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7545
[ALST tutorial] support bs>1 by @sfc-gh-sbekman in https://github.com/deepspeedai/DeepSpeed/pull/7550
[MoE] Fix misuse of num_experts as expert parallel group size (ep_size) by @Flakes342 in https://github.com/deepspeedai/DeepSpeed/pull/7551
Limit random seed range in tests by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7553
Fix gradient buffer access for DeepCompile Z1/2 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7548
Move modal tests to tests/v1 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7557
Add dependency for deepcompile test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7558
deepcompile: Create dummy inputs using empty_strided by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7564
deepcompile: Record graph order using OrderedDict by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7563
deepcompile: Create a full list of no-copy ops by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7562
fix npu device_id AttributeError issue by @we1sper in https://github.com/deepspeedai/DeepSpeed/pull/7560
Make Muon optimizer easier to enable by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7555
scripts: Check .is_cuda only in non-C++ files by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7561
[bugfix] fix partition context unpatch by @hjh0119 in https://github.com/deepspeedai/DeepSpeed/pull/7566

New Contributors

@LYMDLUT made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7421
@aeeeeeep made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7507
@heyujiao99 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7519
@jakehemmerle made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7530
@qibin0506 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7531
@mingjielu made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7521
@kaixuanliu made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7542
@sfc-gh-sbekman made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7550
@Flakes342 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7551
@we1sper made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7560
@hjh0119 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7566

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.17.5...v0.17.6

Open source →

0.17.520 Aug 2025

Update version.txt after v0.17.4 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7460

What's Changed

Update version.txt after v0.17.4 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7460
Update README.md by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/7465
Add getter APIs for TP/PP/DP ranks in DeepSpeedEngine by @WoosungMyung in https://github.com/deepspeedai/DeepSpeed/pull/7427
fix issues raised by Coverity scans by @NirSonnenschein in https://github.com/deepspeedai/DeepSpeed/pull/7431
Fix all-gather duplicate params and wrong dtype by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7462
fix #7188 by @lpnpcs in https://github.com/deepspeedai/DeepSpeed/pull/7371
add --bind_cores_to_rank to zero offload tutorial by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7474
Add blog for ZenFlow by @Antlera in https://github.com/deepspeedai/DeepSpeed/pull/7463
Fix cpu CI by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7481
fix deepspeed --venv_script by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7469
Modal CI by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7289
[UlyssesSPDataLoaderAdapter] fix iterator reset by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7472
[TiledFusedLogitsLoss] support inference by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7477
Fix pre-compile on cpu-only machines by @AlongWY in https://github.com/deepspeedai/DeepSpeed/pull/7168
Enable forked PRs by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7486
fix xpu device_id AttributeError issue by @yao-matrix in https://github.com/deepspeedai/DeepSpeed/pull/7488
Add Zenflow code for Stage 1 & 2 by @Antlera in https://github.com/deepspeedai/DeepSpeed/pull/7391
Fix invalid f-strings by @cyyever in https://github.com/deepspeedai/DeepSpeed/pull/7457
Fix DeepCompile for PyTorch v2.8 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7496
Reduce performance impact of compiler.enable decorator by @deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7498
Add index to HPU devices by @deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7497

New Contributors

@WoosungMyung made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7427
@eternalNight made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7462
@lpnpcs made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7371
@Antlera made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7463
@AlongWY made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7168
@yao-matrix made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7488
@cyyever made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7457

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.17.4...v0.17.5

Open source →

0.17.27 Jul 2025

Update version after 0.17.1 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7345

What's Changed

Update version after 0.17.1 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7345
s/UlyssesPlus/Arctic Long Sequence Training (ALST)/ by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7348
Don't break set_start_method by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7349
Fix error of <glog/logging.h> by @Freed-Wu in https://github.com/deepspeedai/DeepSpeed/pull/7351
Improve padding util for compile by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7355
Fix 404s by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7363
Fix tutorial title by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7365
Restore real inputs for recompilation by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7356
Fix(scheduler): WarmupLR inherits optimizer lr when not specified by @Flink-ddd in https://github.com/deepspeedai/DeepSpeed/pull/7360
sequence parallel default dtype by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7364
Enable torch.autocast with ZeRO by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/6993
add Arctic Long Sequence Training paper reference by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7372
Flops profiler support for F.interpolate by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7353
Relax tolerances for FP8 unit test only for ROCm + FP16 by @rraminen in https://github.com/deepspeedai/DeepSpeed/pull/7373
Update latest news with DeepNVMe by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7375
Fix release of IPG buffer by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7376
fix wandb.log() call by removing sync kwarg by @ned2 in https://github.com/deepspeedai/DeepSpeed/pull/7383
Fix dtype mismatch in TestParamPartitioningSkipInit by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7377
Add support for ws=1 scenario by @NirSonnenschein in https://github.com/deepspeedai/DeepSpeed/pull/7379
fix(inference): Add missing dtype attribute to ParameterBase setter by @Flink-ddd in https://github.com/deepspeedai/DeepSpeed/pull/7378
add blog link by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7385
fix broken url by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7390
add support for CUDAtk12.9 by @loscrossos in https://github.com/deepspeedai/DeepSpeed/pull/7394
Fix unbound local error for return_val by @HollowMan6 in https://github.com/deepspeedai/DeepSpeed/pull/7395
Fix ZeRO stage 1 and add stage 2 support with DeepCompile by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7366
Improve coverage of DeepCompile by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7386
Added device detection to communication logging by @alexk101 in https://github.com/deepspeedai/DeepSpeed/pull/7398
fix: Add csrc/compile to include paths for DeepCompile builder by @HollowMan6 in https://github.com/deepspeedai/DeepSpeed/pull/7401
fix: DeepCompile for torch 2.8 by @HollowMan6 in https://github.com/deepspeedai/DeepSpeed/pull/7402
fix(comm): Expose GradBucket in deepspeed.comm API by @Flink-ddd in https://github.com/deepspeedai/DeepSpeed/pull/7400
fix: fix FileNotFoundError for build_win.bat by @gjj2828 in https://github.com/deepspeedai/DeepSpeed/pull/7399
fix: engine initializes optimizer attributes at the beginning by @HollowMan6 in https://github.com/deepspeedai/DeepSpeed/pull/7410

New Contributors

@Freed-Wu made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7351
@Flink-ddd made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7360
@ned2 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7383
@alexk101 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7398
@gjj2828 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7399

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.17.1...v0.17.2

Open source →

0.16.527 Mar 2025

Update version.txt after 0.16.4 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7063

What's Changed

Update version.txt after 0.16.4 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7063
fix an outdated doc wrt CUDA_VISIBLE_DEVICES by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7058
Tecorigin sdaa accelerator by @siqi654321 in https://github.com/deepspeedai/DeepSpeed/pull/6903
Handle special case of libuv for Windows by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7064
Bug Fix for offload_states API by @U-rara in https://github.com/deepspeedai/DeepSpeed/pull/7050
Update README with info on newest accelerator by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7065
Fix TOCTOU issues, switch to fstat by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7067
config torch to avoid graph breaks caused by logger by @ShellyNR in https://github.com/deepspeedai/DeepSpeed/pull/6999
Fix meta load tensor imcompatible issue by @Yejing-Lai in https://github.com/deepspeedai/DeepSpeed/pull/7073
Replace calls to python setup.py sdist with python -m build --sdist by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7069
Revert "Handle special case of libuv for Windows (#7064)" by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7076
Add DeepseekV3 AutoTP. by @Yejing-Lai in https://github.com/deepspeedai/DeepSpeed/pull/7045
Improve inference tutorial docs by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7083
Pin transformers version on tests that use latest. by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7085
Update README.md with ICS '23 MoE paper link by @siddharth9820 in https://github.com/deepspeedai/DeepSpeed/pull/7087
Update parallelism for nv-torch-latest/nightly tests due to more GPUs/runner by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7086
Remove workflows for very old torch versions by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7090
Use new dlpack api; Formatting fixes by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7101
Avoid graph breaks by disabling sourceless calls in instrument_w_nvtx by @deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7081
Avoid graph breaks in torch.compile caused by inner classes in the backward hooks by @deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7062
Only run pre-commit on the changes by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/7106
Avoid graph break due to unsupported frozenset by @deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7105
Fix fused_qkv print model ValueError by @Yejing-Lai in https://github.com/deepspeedai/DeepSpeed/pull/7109
Update references to new X/Twitter handle by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7110
Update gaudi2 nightly,ci to latest 1.20.0 build by @raza-sikander in https://github.com/deepspeedai/DeepSpeed/pull/7093
fix keep_module_on_host by @inkcherry in https://github.com/deepspeedai/DeepSpeed/pull/7112
Add sequential pytest mark to TestNVMeCheckpointing to resolve pytest forked hangs by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7131
Training multiple models by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7018
Update CONTRIBUTING.md to reflect changes from CLA to DCO by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7135
Avoid missing attr error by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7133
Add conditional expression by @A-transformer in https://github.com/deepspeedai/DeepSpeed/pull/7119
Unpin transformers version for most workflows by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7139
Conditionally quote env vars by @saurabhkoshatwar in https://github.com/deepspeedai/DeepSpeed/pull/7071
Correct the BACKWARD_PREFETCH_SUBMIT mismatch by @A-transformer in https://github.com/deepspeedai/DeepSpeed/pull/7120
Enhance Gaudi2 CI/Nightly Coverage with Model Parallelism and Linear Tests by @raza-sikander in https://github.com/deepspeedai/DeepSpeed/pull/7146
Update container version that runs on A6000 tests. by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7153
hf tp+zero training doc. by @inkcherry in https://github.com/deepspeedai/DeepSpeed/pull/7151
Avoid graph break by removing redundant requires_grad attr change by @deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7158
Add destroy to tests to free memory by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7160
[NFC] Typo fix in SP layer. by @c8ef in https://github.com/deepspeedai/DeepSpeed/pull/7152
Link AutoTP blog in the front page by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/7167
fix seq_parallel_communication_data_type constant. by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7175
Fix typos in GDS blog by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7177
Variable batch size and LR scheduler by @bm-synth in https://github.com/deepspeedai/DeepSpeed/pull/7104

New Contributors

@siqi654321 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/6903
@A-transformer made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7119
@saurabhkoshatwar made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7071
@c8ef made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7152

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.16.4...v0.16.5

Open source →

0.16.420 Feb 2025

Update version.txt after 0.16.3 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/6965

What's Changed

Update version.txt after 0.16.3 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/6965
Precisely track nvme optimizer offload by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/6963
Update build_win.bat script to exclue GDS op as it lacks Windows support. by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/6971
Add CUDA 12.8 support and comment on CUDA 12.7 by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/6975
Update cpu torch latest to use torch 2.6 by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/6977
generalize deepspeed linear and implement it for non cuda systems by @oelayan7 in https://github.com/deepspeedai/DeepSpeed/pull/6932
Update recommended Windows whl building versions by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/6983
Title: Fix setup_env_ranks to Properly Set Environment Variables Instead of Raising Error by @fabiosanger in https://github.com/deepspeedai/DeepSpeed/pull/6979
Specify torchvision in nv-ds-chat workflow (prevents errors with torch 2.6) by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/6982
Remove assumption that padding only occurs on last rank by @xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/6974
Use ds-specific module id to avoid conflicts by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/6847
Update A6000 workflows to use newer docker container - 24.09 vs 24.03 by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/6967
Allow NVIDIA Blackwell by @fabiendupont in https://github.com/deepspeedai/DeepSpeed/pull/6991
Update GH org references by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/6998
[XPU] max1100 workflow update for docker and softwares by @Liangliang-Ma in https://github.com/deepspeedai/DeepSpeed/pull/7003
autotp training(fix dco) by @inkcherry in https://github.com/deepspeedai/DeepSpeed/pull/7004
import triton files when triton is supported and installed by @oelayan7 in https://github.com/deepspeedai/DeepSpeed/pull/6989
Update A6000 tests transformers version by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7016
Fix ds-chat CI regression by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7015
[Ulysses tutorial] typos by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7024
fix hostname -I for macOS #6497 by @fitzjalen in https://github.com/deepspeedai/DeepSpeed/pull/6990
Update workflows to cuda 12.4 by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7000
[ROCm] Enable fp_quantizer on ROCm by @rraminen in https://github.com/deepspeedai/DeepSpeed/pull/7027
add gds chinese blog by @GuanhuaWang in https://github.com/deepspeedai/DeepSpeed/pull/7034
Add chinese blog for deepspeed windows, and fix format by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/7035
AIO on ROCM by @jomayeri in https://github.com/deepspeedai/DeepSpeed/pull/7023
Control trace cache warnings by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7039
Update CUDA compute capability to support Blackwell by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/7047
Update setup.py handling of ROCm cupy by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7051
nv-ds-chat breaks with latest transformers by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7052
Rename aio_thread_count to intra_op_parallelism by @tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7056
add autoTP training zero2 tests by @inkcherry in https://github.com/deepspeedai/DeepSpeed/pull/7049
Fix, bf16 optimizer remove dup loop by @wukong1992 in https://github.com/deepspeedai/DeepSpeed/pull/7054

New Contributors

@fabiosanger made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/6979
@fabiendupont made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/6991
@fitzjalen made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/6990
@wukong1992 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7054

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.16.3...v0.16.4

Open source →

0.16.321 Jan 2025

Update version.txt after 0.16.2 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6893

What's Changed

Update version.txt after 0.16.2 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6893
Allow to compile collective for PT>2.3 by @NirSonnenschein in https://github.com/microsoft/DeepSpeed/pull/6899
Zero2: avoid graph breaks in torch.compile by using param_idx by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/6803
hpu_accelerator: use torch.use_deterministic_algorithms by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/6897
Fix error caused by all_reduce call in domino by @hwchen2017 in https://github.com/microsoft/DeepSpeed/pull/6880
Update Gaudi2 jobs to latest 1.19 build by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6905
Change compile for pipeline module torch.compile by @NirSonnenschein in https://github.com/microsoft/DeepSpeed/pull/6478
Stage3: Use new torch grad accumulation hooks API by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/6773
Cleanup ops/transformer/inference tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/6830
Fix checkpointable_layers Logic by @Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/6881
[BUG FIX]:fix get torch.version.cuda error when cuda is None in rocm by @hj-wei in https://github.com/microsoft/DeepSpeed/pull/6909
Add fp8_gemm fallback for non-triton systems by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/6916
Reduce the device bubble introduced by heavy loop synchronization in coalesced fetch/release(z3_leaf_module) by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/6694
Cleanup ops/transformer/inference tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/6925
Check transformers version in BLOOM for inference v1 by @lekurile in https://github.com/microsoft/DeepSpeed/pull/6766
inference: remove unused _validate_args function by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5505
Use torch.log1p by @kit1980 in https://github.com/microsoft/DeepSpeed/pull/6930
Update python version classifiers by @loadams in https://github.com/microsoft/DeepSpeed/pull/6933
Fix building on Windows with presence of Triton by @woct0rdho in https://github.com/microsoft/DeepSpeed/pull/6749
Fix windows blog examples by @loadams in https://github.com/microsoft/DeepSpeed/pull/6934
Add deepseek autotp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/6937
Add position_ids arg to OPTEmbedding forward function by @lekurile in https://github.com/microsoft/DeepSpeed/pull/6939
Add information on security expectations with this software by @loadams in https://github.com/microsoft/DeepSpeed/pull/6941
Support pure meta model lm_head tp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/6812
Remove op compilation flags due to perf issue by @NirSonnenschein in https://github.com/microsoft/DeepSpeed/pull/6944
Pin nv-a6000 workflow by @loadams in https://github.com/microsoft/DeepSpeed/pull/6938
[inf] Add config var to enable keeping module on host by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/6846
warn to warning by @qgallouedec in https://github.com/microsoft/DeepSpeed/pull/6952
Add extra_repr to Linear classes for debugging purpose by @Xia-Weiwen in https://github.com/microsoft/DeepSpeed/pull/6954
Update import for torchvision.transformers by @loadams in https://github.com/microsoft/DeepSpeed/pull/6958
Remove Duplicate Declaration of pandas in Dockerfile by @Zerohertz in https://github.com/microsoft/DeepSpeed/pull/6959
Add the missing view operations from sequence parallel(async). by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/6750
Update torch.norm to torch.linalg.norm and torch.linalg.vector_norm by @loadams in https://github.com/microsoft/DeepSpeed/pull/6931
Using explicit GPU upcast for ZeRO-Offload by @xylian86 in https://github.com/microsoft/DeepSpeed/pull/6962

New Contributors

@hj-wei made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6909
@kit1980 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6930
@woct0rdho made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6749
@Xia-Weiwen made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6954
@Zerohertz made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6959

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.16.2...v0.16.3

Open source →

0.16.218 Dec 2024

Update pre-commit version by @loadams in https://github.com/microsoft/DeepSpeed/pull/6821

What's Changed

Update pre-commit version by @loadams in https://github.com/microsoft/DeepSpeed/pull/6821
Update version.txt after 0.16.1 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6826
Pin HPU tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/6831
Flops profiler support einops.einsum by @lvhoaa in https://github.com/microsoft/DeepSpeed/pull/6755
Pin pytest-subtests version for accelerate tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/6842
Inference UTs check for trition support from accelerator by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6782
Unpin pytest-subtests now that 0.14.1 is released by @loadams in https://github.com/microsoft/DeepSpeed/pull/6844
Merge LoCo with Zero++ by @XingyuXie in https://github.com/microsoft/DeepSpeed/pull/6730
Fix type error in ZeROOrderedDict by @oraluben in https://github.com/microsoft/DeepSpeed/pull/6794
Fix uneven head sequence parallelism bug (#6774) by @Eugene29 in https://github.com/microsoft/DeepSpeed/pull/6797
Fix nv-torch-nightly test by pinning transformers by @loadams in https://github.com/microsoft/DeepSpeed/pull/6849
Remove broken links to non-active site by @kaiksi-bb in https://github.com/microsoft/DeepSpeed/pull/6854
Avoid poisoning process with CUDA calls as soon as importing by @HollowMan6 in https://github.com/microsoft/DeepSpeed/pull/6810
Fix xpu tests workflow failure by changing pip index url by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/6864
Domino updates by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/6861
add domino navigation by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/6866
Update TSC by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6867
Remove warnings from autodoc and sphinx by @loadams in https://github.com/microsoft/DeepSpeed/pull/6788
Update real_accelerator.py by @keiwoo in https://github.com/microsoft/DeepSpeed/pull/6845
Fix assertion for offloading states by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6855
Remove pin from transformers version and fix Processing/Threading issues in tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/6822
Add MLP/lm_head tp grain size setting. by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/6828
Fix --enable_each_rank_log when used with PDSH multi-node runner by @akeshet in https://github.com/microsoft/DeepSpeed/pull/6863
Update transformers ops unit tests to use requried_torch_version by @loadams in https://github.com/microsoft/DeepSpeed/pull/6884
Don't error out when cpu accelerator doesn't have torch (as default for whl building) by @loadams in https://github.com/microsoft/DeepSpeed/pull/6886
Add arctic model support by adding w2 to all_reduce by @pi314ever in https://github.com/microsoft/DeepSpeed/pull/6856
Update code owners by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6890

New Contributors

@lvhoaa made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6755
@XingyuXie made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6730
@Eugene29 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6797
@kaiksi-bb made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6854
@HollowMan6 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6810
@keiwoo made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6845
@akeshet made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6863
@pi314ever made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6856

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.16.1...v0.16.2

Open source →

0.16.025 Nov 2024

Update version.txt after 0.15.4 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6731

What's Changed

Update version.txt after 0.15.4 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6731
Update GH hosted workflows to 24.04 by @loadams in https://github.com/microsoft/DeepSpeed/pull/6717
Add COMMITTER file by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6741
Update AMD apex version by @loadams in https://github.com/microsoft/DeepSpeed/pull/6739
Fix Type Name Inconsistency & Typo in cpu_adam by @xylian86 in https://github.com/microsoft/DeepSpeed/pull/6732
Add Domino code by @zhangsmallshark in https://github.com/microsoft/DeepSpeed/pull/6733
Add data type check for bf16 by @hwchen2017 in https://github.com/microsoft/DeepSpeed/pull/6742
add zero3 module_granularity_threshold to zero optimization. by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/6649
AIO File Offsets by @jomayeri in https://github.com/microsoft/DeepSpeed/pull/6641
Update path for BingBertSquad from DeepSpeedExamples by @loadams in https://github.com/microsoft/DeepSpeed/pull/6746
Sanitize inputs to eval() by @loadams in https://github.com/microsoft/DeepSpeed/pull/6745
Adding the governance doc by @minjiazhang in https://github.com/microsoft/DeepSpeed/pull/6748
Add no_sync context manager by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6675
Gaudi2 Nightly job for daily check by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6753
Disable failing python tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/6758
A faster and more memory-efficient implementation of zero_to_fp32 by @xu-song in https://github.com/microsoft/DeepSpeed/pull/6658
Pin transformers version to work around latest torch requirements by @loadams in https://github.com/microsoft/DeepSpeed/pull/6759
make xpu ops compatible with oneapi 2025.0 by @baodii in https://github.com/microsoft/DeepSpeed/pull/6760
Add explicit parameters for torch.load by @loadams in https://github.com/microsoft/DeepSpeed/pull/6751
Fix setup.py bash cmd generation to correctly extract git info by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/6762
Use json_schema_extra instead of extra keyword in Field by @qgallouedec in https://github.com/microsoft/DeepSpeed/pull/6764
Fix potential memory issues when use deepspeed Z3 by @wenbinc-Bin in https://github.com/microsoft/DeepSpeed/pull/6726
Removes unnecessary cloning by @swigls in https://github.com/microsoft/DeepSpeed/pull/6761
Enable torch compile on _allgather_params by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/6769
Unpin with latest transformers fixes by @loadams in https://github.com/microsoft/DeepSpeed/pull/6763
docs: fix HF links by @imba-tjd in https://github.com/microsoft/DeepSpeed/pull/6780
Fix Doc Error: ZeRO Stage 2 gradient partitioning by @yewentao256 in https://github.com/microsoft/DeepSpeed/pull/6775
Cleanup code docs warnings by @loadams in https://github.com/microsoft/DeepSpeed/pull/6783
Domino Blog by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/6776
Update version.txt before release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6784
Revert release workflow by @loadams in https://github.com/microsoft/DeepSpeed/pull/6785

New Contributors

@zhangsmallshark made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6733
@hwchen2017 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6742
@minjiazhang made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6748
@qgallouedec made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6764
@wenbinc-Bin made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6726
@swigls made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6761
@imba-tjd made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6780
@yewentao256 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6775

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.15.4...v0.16.0

Open source →

0.15.48 Nov 2024

Update version.txt after 0.15.3 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6652

What's Changed

Update version.txt after 0.15.3 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6652
Fix expert grad scaling problem with ZeRO optimizer by @wyooyw in https://github.com/microsoft/DeepSpeed/pull/6546
Add attribute check for language_model when replace last linear module by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/6650
fix init_device_mesh for torch 2.4 by @Lzhang-hub in https://github.com/microsoft/DeepSpeed/pull/6614
Fix dynamo issue by @oraluben in https://github.com/microsoft/DeepSpeed/pull/6527
sequence parallel for uneven heads by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/6392
Add fallback for is_compiling by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6663
Update profiler registration check by @loadams in https://github.com/microsoft/DeepSpeed/pull/6668
Add support for H100/sm_90 arch compilation by @loadams in https://github.com/microsoft/DeepSpeed/pull/6669
Update Gaudi2 docker image by @loadams in https://github.com/microsoft/DeepSpeed/pull/6677
Update gaudi2 docker version to latest release (1.18) by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6648
Update base docker image for A6000 GPU tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/6681
Remove packages that no longer need to be updated in the latest container by @loadams in https://github.com/microsoft/DeepSpeed/pull/6682
Fix training of pipeline based peft's lora model by @xuanhua in https://github.com/microsoft/DeepSpeed/pull/5477
Update checkout action to latest version by @loadams in https://github.com/microsoft/DeepSpeed/pull/5021
Add attribute check to support git-base autotp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/6688
fix memcpy issue on backward for zero-infinity by @xylian86 in https://github.com/microsoft/DeepSpeed/pull/6670
Free memory in universal checkpointing tests by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6693
Explictly set device when reusing dist env by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6696
Update URL in README Pipeline Status for Huawei Ascend NPU by @xuedinge233 in https://github.com/microsoft/DeepSpeed/pull/6706
Pin transformers to 4.45.2 in nv-ds-chat workflow by @loadams in https://github.com/microsoft/DeepSpeed/pull/6710
[Bug Fix] Support threads_per_head < 64 for wavefront size of 64 by @jagadish-amd in https://github.com/microsoft/DeepSpeed/pull/6622
Use one param coordinator for both train/inference scenarios by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6662
Update yapf version by @loadams in https://github.com/microsoft/DeepSpeed/pull/6721
Update flake8 version by @loadams in https://github.com/microsoft/DeepSpeed/pull/6722
Switch what versions of python are supported by @loadams in https://github.com/microsoft/DeepSpeed/pull/5676

New Contributors

@wyooyw made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6546
@xuanhua made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5477

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.15.3...v0.15.4

Open source →

0.15.29 Oct 2024

Update version.txt after 0.15.1 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6493

What's Changed

Update version.txt after 0.15.1 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6493
HPU: add required ENV vars to acccelerator init by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/6495
Op_builder->is_compatible quite warning by @terry-for-github in https://github.com/microsoft/DeepSpeed/pull/6093
fix pipeline eval_batch micro_batches argument for schedule by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/6484
Fix the broken url link by @rogerxfeng8 in https://github.com/microsoft/DeepSpeed/pull/6500
fix environment variable export bug for MultiNodeRunner by @TideDra in https://github.com/microsoft/DeepSpeed/pull/5878
Revert "BF16 optimizer: Clear lp grads after updating hp grads in hook" by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/6508
wrap include cuda_bf16.h with ifdef BF16_AVAILABLE by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/6520
Avoid security issues of subprocess shell by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6498
Add conditional on torch version for scaled_dot_product_attention by @loadams in https://github.com/microsoft/DeepSpeed/pull/6517
Added Intel Gaudi to Accelerator Setup Guide by @ShifaAbu in https://github.com/microsoft/DeepSpeed/pull/6543
Skip failing newly added tests in accelerate by @loadams in https://github.com/microsoft/DeepSpeed/pull/6574
Use msgpack for p2p comm by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6547
DeepNVMe perf tuning by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6560
[Accelerator] Cambricon MLU support by @Andy666G in https://github.com/microsoft/DeepSpeed/pull/6472
Fix gradient accumulation for Z2+offload by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6550
fix errors when setting zero3 leaf modules with torch.compile by @NirSonnenschein in https://github.com/microsoft/DeepSpeed/pull/6564
[XPU] Support DeepNVMe new code structure by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/6532
Add APIs to offload states of model, optimizer, and engine by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6011
add bfloat16 to inference support dtypes by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/6528
[COMPILE] workflow for deepspeed + torch.compile by @YizhouZ in https://github.com/microsoft/DeepSpeed/pull/6570
Fixes on the accelerate side mean we do not need to skip this test by @loadams in https://github.com/microsoft/DeepSpeed/pull/6583
Fix torch include in op_builder/mlu/fused_adam.py and update no-torch workflow triggers by @loadams in https://github.com/microsoft/DeepSpeed/pull/6584
[ROCm] Fix subprocess error by @jagadish-amd in https://github.com/microsoft/DeepSpeed/pull/6587
Cleanup CODEOWNERS file to be valid by @loadams in https://github.com/microsoft/DeepSpeed/pull/6603
Add SSF Best practices badge by @loadams in https://github.com/microsoft/DeepSpeed/pull/6604
Move V100 workflows from cuda 11.1/11.7 to 12.1 by @loadams in https://github.com/microsoft/DeepSpeed/pull/6607
Fix SD workflow by @loadams in https://github.com/microsoft/DeepSpeed/pull/6609
Pin accelerate to fix CI failures/issues by @loadams in https://github.com/microsoft/DeepSpeed/pull/6610
Add llama3.2 vision autotp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/6577
Improve DS logging control by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6602
Fix device selection using CUDA_VISIBLE_DEVICES by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6530
Handle when backend is also in compile_kwargs by @oraluben in https://github.com/microsoft/DeepSpeed/pull/6502
Rearrange inference OPS and stop using builder.load by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/5490
Unpin accelerate tests, update lightning with node16 removal. by @loadams in https://github.com/microsoft/DeepSpeed/pull/6611
Enabled Qwen2-MoE Tensor Parallelism (TP) inference by @gyou2021 in https://github.com/microsoft/DeepSpeed/pull/6551

New Contributors

@TideDra made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5878
@ShifaAbu made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6543
@jagadish-amd made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6587
@gyou2021 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6551

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.15.1...v0.15.2

Open source →

0.15.15 Sept 2024

Update version.txt after 0.15.0 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6403

What's Changed

Update version.txt after 0.15.0 release by @loadams in https://github.com/microsoft/DeepSpeed/pull/6403
Fix Type Mismatch by @jomayeri in https://github.com/microsoft/DeepSpeed/pull/6410
Fix redundant seq data parallel grp argument in Z3/MiCS by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/5352
add Huawei Ascend NPU setup guide by @xuedinge233 in https://github.com/microsoft/DeepSpeed/pull/6445
Add documentation for launcher without SSH by @dogacancolak-kensho in https://github.com/microsoft/DeepSpeed/pull/6455
Dtype support check for accelerator in UTs by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6360
Store/Load CIFAR from local/offline by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6390
Add the accelerator setup guide link in Getting Started page by @rogerxfeng8 in https://github.com/microsoft/DeepSpeed/pull/6452
Allow triton==3.0.x for fp_quantizer by @siddartha-RE in https://github.com/microsoft/DeepSpeed/pull/6447
Change GDS to 1 AIO thread by @jomayeri in https://github.com/microsoft/DeepSpeed/pull/6459
[CCL] fix condition issue in ccl.py by @YizhouZ in https://github.com/microsoft/DeepSpeed/pull/6443
Avoid gds build errors on ROCm by @rraminen in https://github.com/microsoft/DeepSpeed/pull/6456
TestLowCpuMemUsage UT get device by device_name by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6397
Add workflow to build DS without torch to better test before releases by @loadams in https://github.com/microsoft/DeepSpeed/pull/6450
Fix patch for parameter partitioning in zero.Init() by @tohtana in https://github.com/microsoft/DeepSpeed/pull/6388
Add default value to "checkpoint_folder" in "load_state_dict" of bf16_optimizer by @ljcc0930 in https://github.com/microsoft/DeepSpeed/pull/6446
DeepNVMe tutorial by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6449
bf16_optimizer: fixes to different grad acc dtype by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/6485
print warning if actual triton cache dir is on NFS, not just for default by @jrandall in https://github.com/microsoft/DeepSpeed/pull/6487
DS_BUILD_OPS should build only compatible ops by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6489
Safe usage of popen by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/6490
Handle an edge case where CUDA_HOME is not defined on ROCm systems by @amorehead in https://github.com/microsoft/DeepSpeed/pull/6488

New Contributors

@xuedinge233 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6445
@siddartha-RE made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6447
@ljcc0930 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6446
@jrandall made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6487
@amorehead made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6488

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.15.0...v0.15.1

Open source →

0.14.515 Aug 2024

Update version.txt after 0.14.4 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5694

What's Changed

Update version.txt after 0.14.4 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5694
Fixed Windows inference build. by @costin-eseanu in https://github.com/microsoft/DeepSpeed/pull/5609
Fix memory leak from _hp_mapping by @chiragjn in https://github.com/microsoft/DeepSpeed/pull/5643
Bug fix for the "Link bit16 and fp32 parameters in partition" by @U-rara in https://github.com/microsoft/DeepSpeed/pull/5681
[CPU] add fp16 support to shm inference_all_reduce by @delock in https://github.com/microsoft/DeepSpeed/pull/5669
Universal checkpoint for zero stage 3 by @xylian86 in https://github.com/microsoft/DeepSpeed/pull/5475
inference unit test injectionPolicy split world_size to multiple tests by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/5687
ENV var added for recaching in INF Unit tests by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/5688
Disable nvtx decorator to avoid graph break by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5697
Add an argument to enable the injection of missing state during the conversion of universal checkpoints by @xylian86 in https://github.com/microsoft/DeepSpeed/pull/5608
Change source of CPUAdam for xpu accelerator by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/5703
Add additional paths to trigger xpu tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/5707
Update XPU docker version by @loadams in https://github.com/microsoft/DeepSpeed/pull/5712
update xpu fusedadam opbuilder for pytorch 2.3 by @baodii in https://github.com/microsoft/DeepSpeed/pull/5702
DeepSpeed Universal Checkpointing: Blog and Tutorial by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/5711
UCP Chinese Blog by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/5713
Fix tutorial links by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/5714
Update node16 check on self-hosted runners and remove python 3.6 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5756
fix the missing argument in test and typo by @xylian86 in https://github.com/microsoft/DeepSpeed/pull/5730
[INF] Enable torch compile for inference by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/5612
Update checkout action for nv-human-eval workflow by @loadams in https://github.com/microsoft/DeepSpeed/pull/5757
Add Windows scripts (deepspeed, ds_report). by @costin-eseanu in https://github.com/microsoft/DeepSpeed/pull/5699
Unit Test: Add error handling for rate limit exceeded in model list by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/5715
Fix memory leak for pipelined optimizer swapper by @mauryaavinash95 in https://github.com/microsoft/DeepSpeed/pull/5700
Remove duplicated variable by @xu-song in https://github.com/microsoft/DeepSpeed/pull/5727
Fix phi3 mini 128k load error by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/5765
[CPU] Allow deepspeed.comm.inference_all_reduce in torch.compile graph by @delock in https://github.com/microsoft/DeepSpeed/pull/5604
Added wrappers for hpu tensors based on dtype by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/5771
[bugfix] promote state in bf16_optimizer by @billishyahao in https://github.com/microsoft/DeepSpeed/pull/5767
Launcher mode with SSH bypass by @dogacancolak-kensho in https://github.com/microsoft/DeepSpeed/pull/5728
Update the list of supported models in the Chinese README of fastgen by @beep-bebop in https://github.com/microsoft/DeepSpeed/pull/5773
Add support for Microsoft Phi-3 model to DeepSpeed-FastGen by @adk9 in https://github.com/microsoft/DeepSpeed/pull/5559
Misplaced global variable warned by @anferico in https://github.com/microsoft/DeepSpeed/pull/5725
Fixes for latest Huggingface_hub changes on modelId -> id by @loadams in https://github.com/microsoft/DeepSpeed/pull/5789
reduce all-to-all communication volume when both expert and non-expert are tensor-parallel by @taozhiwei in https://github.com/microsoft/DeepSpeed/pull/5626
Update Ubuntu version for running python tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/5783
fix: quantization with DeepSpeed HE by @Atry in https://github.com/microsoft/DeepSpeed/pull/5624
[INF] Add Qwen2RMSNorm to loaded layers in auto_tp by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/5786
Add chatglm2 & chatglm3 autotp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/5540
Add new autotp supported model in doc by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/5785
Fix accuracy error of NPUFusedAdam by @penn513 in https://github.com/microsoft/DeepSpeed/pull/5777
Update torch version in cpu-torch-latest and nv-torch-latest-v100 tests to 2.4 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5797
move is_checkpointable call reducing torch.compile Graph breaks by @NirSonnenschein in https://github.com/microsoft/DeepSpeed/pull/5759
Unpin transformers version by @loadams in https://github.com/microsoft/DeepSpeed/pull/5650
Update other workflows to run on Ubuntu 22.04 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5798
[XPU]Use host time to replace xpu time when IPEX version slower than 2.5. by @ys950902 in https://github.com/microsoft/DeepSpeed/pull/5796
Update MII tests to pull correct torchvision by @loadams in https://github.com/microsoft/DeepSpeed/pull/5800
Add fp8-fused gemm kernel by @sfc-gh-reyazda in https://github.com/microsoft/DeepSpeed/pull/5764
Add doc of compressed backend in Onebit optimizers by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/5782
fix: handle exception when loading cache file in test_inference.py by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/5802
Pin transformers version for MII tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/5807
Fix op_builder for CUDA 12.5 by @keshavkowshik in https://github.com/microsoft/DeepSpeed/pull/5806
Find ROCm on Fedora by @trixirt in https://github.com/microsoft/DeepSpeed/pull/5705
Fix CPU Adam JIT compilation by @lekurile in https://github.com/microsoft/DeepSpeed/pull/5780
GDS AIO Blog by @jomayeri in https://github.com/microsoft/DeepSpeed/pull/5817
[ROCm] Get rocm version from /opt/rocm/.info/version by @rraminen in https://github.com/microsoft/DeepSpeed/pull/5815
sequence parallel with communication overlap by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/5691
Update to ROCm6 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5491
Add fp16 support of Qwen1.5MoE models (A2.7B) to DeepSpeed-FastGen by @ZonePG in https://github.com/microsoft/DeepSpeed/pull/5403
Use accelerator to replace cuda in setup and runner by @Andy666G in https://github.com/microsoft/DeepSpeed/pull/5769
Link GDS blog to site by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/5820
Non-reentrant checkpointing hook fix by @ic-synth in https://github.com/microsoft/DeepSpeed/pull/5781
Fix NV references by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/5821
Fix docs building guide by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/5825
Update clang-format version from 16 to 18. by @loadams in https://github.com/microsoft/DeepSpeed/pull/5839
Add Japanese translation of DeepNVMe blog by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5845
Fix the bug of deepspeed sequence parallel working with batch size larger than 1 by @YJHMITWEB in https://github.com/microsoft/DeepSpeed/pull/5823
Upgrade HPU image to v1.16.2. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5610
OptimizedLinear updates by @jeffra in https://github.com/microsoft/DeepSpeed/pull/5791
Log operator warnings only in verbose mode by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/5917
Use torch.nan_to_num replace numpy wrapper one by @jinyouzhi in https://github.com/microsoft/DeepSpeed/pull/5877
[Zero2] Reduce the unnecessary all-reduce when tensor size is 0. by @ys950902 in https://github.com/microsoft/DeepSpeed/pull/5868
Update container version for Gaudi2 CI by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/5937
Fix missing ds_id bug by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/5824
Update LR scheduler configuration by @xiyang-aads-lilly in https://github.com/microsoft/DeepSpeed/pull/5846
HPUAccelerator: remove support in set_visible_devices_envs by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5929
Z3: optimizations for grad norm calculation and gradient clipping by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5504
Update xpu-max1100.yml with new config and add some tests by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/5668
Add accelerator setup guides by @delock in https://github.com/microsoft/DeepSpeed/pull/5827
Allow accelerator to instantiate the device by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5255

New Contributors

@U-rara made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5681
@xylian86 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5475
@mauryaavinash95 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5700
@billishyahao made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5767
@dogacancolak-kensho made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5728
@beep-bebop made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5773
@anferico made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5725
@Atry made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5624
@sfc-gh-reyazda made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5764
@keshavkowshik made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5806
@trixirt made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5705
@Andy666G made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5769
@ic-synth made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5781
@xiyang-aads-lilly made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5846

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.14.4...v0.14.5

Open source →

0.14.312 Jun 2024

Update version.txt after 0.14.2 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5458

What's Changed

Update version.txt after 0.14.2 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5458
Add getter and setter methods for compile_backend across accelerators. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5299
Fix torch.compile error for PyTorch v2.3 by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5463
Revert "stage3: efficient compute of scaled_global_grad_norm (#5256)" by @lekurile in https://github.com/microsoft/DeepSpeed/pull/5461
Update ds-chat CI workflow paths to include zero stage 1-3 files by @lekurile in https://github.com/microsoft/DeepSpeed/pull/5462
Update with ops not supported on Windows by @loadams in https://github.com/microsoft/DeepSpeed/pull/5468
fix: swapping order of parameters in create_dir_symlink method. by @alvieirajr in https://github.com/microsoft/DeepSpeed/pull/5465
Un-pin torch version in nv-torch-latest back to latest and skip test_compile_zero tests on v100 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5459
re-introduce: stage3: efficient compute of scaled_global_grad_norm by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5493
Fix crash when creating Torch tensor on NPU with device=get_accelerator().current_device() by @harygo2 in https://github.com/microsoft/DeepSpeed/pull/5464
Fix compile wrapper by @BacharL in https://github.com/microsoft/DeepSpeed/pull/5455
enable phi3_mini autotp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/5501
Fused adam for HPU by @BacharL in https://github.com/microsoft/DeepSpeed/pull/5500
[manifest] update mainfest to add hpp file in csrc. by @ys950902 in https://github.com/microsoft/DeepSpeed/pull/5522
enable phi2 autotp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/5436
Switch pynvml to nvidia-ml-py by @loadams in https://github.com/microsoft/DeepSpeed/pull/5529
Switch from double quotes to match single quotes by @loadams in https://github.com/microsoft/DeepSpeed/pull/5530
[manifest] update mainfest to add hpp file in deepspeed. by @ys950902 in https://github.com/microsoft/DeepSpeed/pull/5533
New integration - CometMonitor by @alexkuzmik in https://github.com/microsoft/DeepSpeed/pull/5466
Improve _configure_optimizer() final optimizer log by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5528
Enhance testing: Skip fused_optimizer tests if not supported. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5159
Skip the UT cases that use unimplemented op builders. by @foin6 in https://github.com/microsoft/DeepSpeed/pull/5372
rocblas -> hipblas changes for ROCm by @rraminen in https://github.com/microsoft/DeepSpeed/pull/5401
Rocm warp size fix by @rraminen in https://github.com/microsoft/DeepSpeed/pull/5402
CPUAdam fp16 and bf16 support by @BacharL in https://github.com/microsoft/DeepSpeed/pull/5409
Optimize zero3 fetch params using all_reduce by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/5420
Fix the TypeError for XPU Accelerator by @shiyang-weng in https://github.com/microsoft/DeepSpeed/pull/5531
Fix RuntimeError for moe on XPU: tensors found at least two devices by @shiyang-weng in https://github.com/microsoft/DeepSpeed/pull/5519
Remove synchronize calls from allgather params by @BacharL in https://github.com/microsoft/DeepSpeed/pull/5516
Avoid overwrite of compiled module wrapper attributes by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/5549
Small typos in functions set_none_gradients_to_zero by @TravelLeraLone in https://github.com/microsoft/DeepSpeed/pull/5557
Adapt doc for #4405 by @oraluben in https://github.com/microsoft/DeepSpeed/pull/5552
Update to HF_HOME from TRANSFORMERS_CACHE by @loadams in https://github.com/microsoft/DeepSpeed/pull/4816
[INF] DSAttention allow input_mask to have false as value by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/5546
Add throughput timer configuration by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/5363
Add Ulysses DistributedAttention compatibility by @Kwen-Chen in https://github.com/microsoft/DeepSpeed/pull/5525
Add hybrid_engine.py as path to trigger the DS-Chat GH workflow by @lekurile in https://github.com/microsoft/DeepSpeed/pull/5562
Update HPU docker version by @loadams in https://github.com/microsoft/DeepSpeed/pull/5566
Rename files in fp_quantize op from quantize.* to fp_quantize.* by @loadams in https://github.com/microsoft/DeepSpeed/pull/5577
[MiCS] Remove the handle print on DeepSpeed side by @ys950902 in https://github.com/microsoft/DeepSpeed/pull/5574
Update to fix sidebar over text by @loadams in https://github.com/microsoft/DeepSpeed/pull/5567
DeepSpeedCheckpoint: support custom final ln idx by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5506
Update minor CUDA version compatibility by @adk9 in https://github.com/microsoft/DeepSpeed/pull/5591
Add slide deck for meetup in Japan by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5598
Fixed the Windows build. by @costin-eseanu in https://github.com/microsoft/DeepSpeed/pull/5596
estimate_zero2_model_states_mem_needs: fixing memory estiamtion by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5099
Fix cuda hardcode for inference woq by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/5565
fix sequence parallel(Ulysses) grad scale for zero0 by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/5555
Add Compressedbackend for Onebit optimizers by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/5473
Updated hpu-gaudi2 tests content. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5622
Pin transformers version for MII tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/5629
WA for Torch-compile-Z3-act-apt accuracy issue from the Pytorch repo by @NirSonnenschein in https://github.com/microsoft/DeepSpeed/pull/5590
stage_1_and_2: optimize clip calculation to use clamp by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5632
Fix overlap communication of ZeRO stage 1 and 2 by @penn513 in https://github.com/microsoft/DeepSpeed/pull/5606
fixes in _partition_param_sec function by @mmhab in https://github.com/microsoft/DeepSpeed/pull/5613
assumption of torch.initial_seed function accepting seed arg in DeepSpeedAccelerator abstract class is incorrect by @polisettyvarma in https://github.com/microsoft/DeepSpeed/pull/5569
pipe/_exec_backward_pass: fix immediate grad update by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5605
Monitor was always enabled causing performance degradation by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/5633

New Contributors

@alvieirajr made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5465
@harygo2 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5464
@alexkuzmik made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5466
@foin6 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5372
@shiyang-weng made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5531
@TravelLeraLone made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5557
@oraluben made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5552
@Kwen-Chen made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5525
@adk9 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5591
@costin-eseanu made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5596
@NirSonnenschein made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5590
@penn513 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5606

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.14.2...v0.14.3

Open source →

0.14.223 Apr 2024

Update version.txt after 0.14.1 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5413

What's Changed

Update version.txt after 0.14.1 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5413
Remove dtype(fp16) condition check for residual_add unit test by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/5329
[XPU] Use non_daemonic_proc by default on XPU device by @ys950902 in https://github.com/microsoft/DeepSpeed/pull/5412
Fix a convergence issues in TP topology caused by incorrect grad_norm. by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/5411
Update 'create-pr' action in release workflow to latest by @loadams in https://github.com/microsoft/DeepSpeed/pull/5415
Update engine.py to avoid torch warning by @etiennebonnafoux in https://github.com/microsoft/DeepSpeed/pull/5408
Update _sidebar.scss by @fasterinnerlooper in https://github.com/microsoft/DeepSpeed/pull/5293
Add more tests into XPU CI by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/5427
[CPU] Support SHM based inference_all_reduce in TorchBackend by @delock in https://github.com/microsoft/DeepSpeed/pull/5391
Add required paths to trigger AMD tests on PRs by @loadams in https://github.com/microsoft/DeepSpeed/pull/5406
Bug fix in split_index method by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5292
Parallel map step for DistributedDataAnalyzer map-reduce by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5291
Selective dequantization by @RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/5375
Fix sorting of shard optimizer states files for universal checkpoint by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5395
add device config env for the accelerator by @shiyuan680 in https://github.com/microsoft/DeepSpeed/pull/5396
64bit indexing fused adam by @garrett4wade in https://github.com/microsoft/DeepSpeed/pull/5187
Improve parallel process of universal checkpoint conversion by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5343
set the default to use set_to_none for clearing gradients in BF16 optimizer. by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/5434
OptimizedLinear implementation by @jeffra in https://github.com/microsoft/DeepSpeed/pull/5355
Update README.md by @Jhonso7393 in https://github.com/microsoft/DeepSpeed/pull/5453
Update PyTest torch version to match PyTorch latest official (2.3.0) by @loadams in https://github.com/microsoft/DeepSpeed/pull/5454

New Contributors

@etiennebonnafoux made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5408
@fasterinnerlooper made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5293
@shiyuan680 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5396
@garrett4wade made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5187
@Jhonso7393 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5453

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.14.1...v0.14.2

Open source →

0.14.115 Apr 2024

Update version.txt after 0.14.0 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5238

What's Changed

Update version.txt after 0.14.0 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5238
Fp6 blog chinese by @xiaoxiawu-microsoft in https://github.com/microsoft/DeepSpeed/pull/5239
Add contributed HW support into README by @delock in https://github.com/microsoft/DeepSpeed/pull/5240
Set tp world size to 1 in ckpt load, if MPU is not provided by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/5243
Make op builder detection adapt to accelerator change by @delock in https://github.com/microsoft/DeepSpeed/pull/5206
Replace HIP_PLATFORM_HCC with HIP_PLATFORM_AMD by @rraminen in https://github.com/microsoft/DeepSpeed/pull/5264
Add CI for Habana Labs HPU/Gaudi2 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5244
Fix attention mask handling in the Hybrid Engine Bloom flow by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/5101
Skip 1Bit Compression and sparsegrad tests for HPU. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5270
Enabled LMCorrectness inference tests on HPU. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5271
Added HPU backend support for torch.compile tests. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5269
Average only valid part of the ipg buffer. by @BacharL in https://github.com/microsoft/DeepSpeed/pull/5268
Add HPU accelerator support in unit tests. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5162
Fix loading a universal checkpoint by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5263
Add Habana Gaudi2 CI badge to the README by @loadams in https://github.com/microsoft/DeepSpeed/pull/5286
Add intel gaudi to contributed HW in README by @BacharL in https://github.com/microsoft/DeepSpeed/pull/5300
Fixed Accelerate Link by @wkaisertexas in https://github.com/microsoft/DeepSpeed/pull/5314
Enable mixtral 8x7b autotp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/5257
support bf16_optimizer moe expert parallel training and moe EP grad_scale/grad_norm fix by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/5259
fix comms dtype by @mayank31398 in https://github.com/microsoft/DeepSpeed/pull/5297
Modified regular expression by @igeni in https://github.com/microsoft/DeepSpeed/pull/5306
Docs typos fix and grammar suggestions by @Gr0g0 in https://github.com/microsoft/DeepSpeed/pull/5322
Added Gaudi2 CI tests. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5275
Improve universal checkpoint by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5289
Increase coverage for HPU by @loadams in https://github.com/microsoft/DeepSpeed/pull/5324
Add NFS path check for default deepspeed triton cache directory by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/5323
Correct typo in checking on bf16 unit test support by @loadams in https://github.com/microsoft/DeepSpeed/pull/5317
Make NFS warning print only once by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/5345
resolve KeyError: 'PDSH_SSH_ARGS_APPEND' by @Lzhang-hub in https://github.com/microsoft/DeepSpeed/pull/5318
BF16 optimizer: Clear lp grads after updating hp grads in hook by @YangQun1 in https://github.com/microsoft/DeepSpeed/pull/5328
Fix sort of zero checkpoint files by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5342
Add distributed_port for deepspeed.initialize by @LZHgrla in https://github.com/microsoft/DeepSpeed/pull/5260
[fix] fix typo s/simultanenously /simultaneously by @digger-yu in https://github.com/microsoft/DeepSpeed/pull/5359
Update container version for Gaudi2 CI by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/5360
compute global norm on device by @BacharL in https://github.com/microsoft/DeepSpeed/pull/5125
logger update with torch master changes by @rogerxfeng8 in https://github.com/microsoft/DeepSpeed/pull/5346
Ensure capacity does not exceed number of tokens by @jeffra in https://github.com/microsoft/DeepSpeed/pull/5353
Update workflows that use cu116 to cu117 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5361
FP [6,8,12] quantizer op by @jeffra in https://github.com/microsoft/DeepSpeed/pull/5336
CPU SHM based inference_all_reduce improve by @delock in https://github.com/microsoft/DeepSpeed/pull/5320
Auto convert moe param groups by @jeffra in https://github.com/microsoft/DeepSpeed/pull/5354
Support MoE for pipeline models by @mosheisland in https://github.com/microsoft/DeepSpeed/pull/5338
Update pytest and transformers with fixes for pytest>= 8.0.0 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5164
Increase CI coverage for Gaudi2 accelerator. by @vshekhawat-hlab in https://github.com/microsoft/DeepSpeed/pull/5358
Add CI for Intel XPU/Max1100 by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/5376
Update path name on xpu-max1100.yml, add badge in README by @loadams in https://github.com/microsoft/DeepSpeed/pull/5386
Update checkout action on workflows on ubuntu 20.04 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5387
Cleanup required_torch_version code and references. by @loadams in https://github.com/microsoft/DeepSpeed/pull/5370
Update README.md for intel XPU support by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/5389
Optimize the fp-dequantizer to get high memory-BW utilization by @RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/5373
Removal of cuda hardcoded string with get_device function by @raza-sikander in https://github.com/microsoft/DeepSpeed/pull/5351
Add custom reshaping for universal checkpoint by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5390
fix pagable h2d memcpy by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/5301
stage3: efficient compute of scaled_global_grad_norm by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5256
Fix the FP6 kernels compilation problem on non-Ampere GPUs. by @JamesTheZ in https://github.com/microsoft/DeepSpeed/pull/5333

New Contributors

@vshekhawat-hlab made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5270
@wkaisertexas made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5314
@igeni made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5306
@Gr0g0 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5322
@Lzhang-hub made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5318
@YangQun1 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5328
@raza-sikander made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5360
@rogerxfeng8 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5346
@JamesTheZ made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5333

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.14.0...v0.14.1

Open source →

0.13.323 Feb 2024Deprecated

Update deprecated HuggingFace function by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5144

What's Changed

Update version.txt after 0.13.2 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5119
Stop tracking backward chain of broadcast (ZeRO3) by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5113
[NPU]ZeRO-Infinity feature compatibility by @misstek in https://github.com/microsoft/DeepSpeed/pull/5077
BF16 optimizer: Improve device utilization by immediate grad update by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/4975
removed if condition in if collate_fn is None by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5107
disable compile tests for torch<2.1 by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5121
Update inference test model names by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5127
Fix issue with zero-sized file after merging file on curriculum map_reduce by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5106
Update return codes in PyTest to properly error out if tests fail by @loadams in https://github.com/microsoft/DeepSpeed/pull/5122
add missing methods to MPS_Accelerator by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5134
Solve tensor vs numpy dtype conflicts in data efficiency map-reduce. by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5108
Fix broadcast deadlock for incomplete batches in data sample for data analysis by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5117
Avoid zero-sized microbatches for incomplete minibatches when doing curriculum learning by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5118
remove mandatory index key from output of metric_function in DataAnalysis map operation by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5112
tensorboard logging: avoid item() outside gas to improve performance by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5135
Check overflow on device without host synchronization for each tensor by @BacharL in https://github.com/microsoft/DeepSpeed/pull/5115
Update nv-inference torch version by @loadams in https://github.com/microsoft/DeepSpeed/pull/5128
Method run_map_reduce to fix errors when running run_map followed by run_reduce by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5131
Added missing isinstance check in PR 5112 by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5142
Fix UserWarning: The torch.cuda.*DtypeTensor constructors are no long… by @ShukantPal in https://github.com/microsoft/DeepSpeed/pull/5018
TestEmptyParameterGroup: replace fusedAdam with torch.optim.AdamW by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5139
Update deprecated HuggingFace function by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5144
Pin to PyTest 8.0.0 by @loadams in https://github.com/microsoft/DeepSpeed/pull/5163
get_grad_norm_direct: fix a case of empty norm group by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5148
Distributed in-memory map-reduce for data analyzer by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5129
DeepSpeedZeroOptimizer_Stage3: remove cuda specific optimizer by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5138
MOE: Fix save checkpoint when TP > 1 by @mosheisland in https://github.com/microsoft/DeepSpeed/pull/5157
Fix gradient clipping by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5150
Use ninja to speed up build by @jinzhen-lin in https://github.com/microsoft/DeepSpeed/pull/5088
Update flops profiler to handle attn and matmul by @KimmiShi in https://github.com/microsoft/DeepSpeed/pull/4724
Fix allreduce for BF16 and ZeRO0 by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5170
Write multiple items to output file at once, in distributed data analyzer. by @bm-synth in https://github.com/microsoft/DeepSpeed/pull/5169
Fix typos in blogs/ by @jinyouzhi in https://github.com/microsoft/DeepSpeed/pull/5172
Inference V2 Human Eval by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4804
Reduce ds_id name length by @jomayeri in https://github.com/microsoft/DeepSpeed/pull/5176
Switch cpu-inference workflow from --extra-index-url to --index-url by @loadams in https://github.com/microsoft/DeepSpeed/pull/5182

New Contributors

@bm-synth made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5107
@KimmiShi made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4724

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.13.2...v0.13.3

Open source →

0.13.212 Feb 2024

Update version.txt after 0.13.1 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5002

What's Changed

Update version.txt after 0.13.1 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5002
Support exclude_frozen_parameters for save_16bit_model by @LZHgrla in https://github.com/microsoft/DeepSpeed/pull/4999
Allow nightly tests dispatch by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5014
Enable hpz based on secondary tensor presence by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/4906
Enable workflow dispatch on all workflows by @loadams in https://github.com/microsoft/DeepSpeed/pull/5016
[minor] improve code quality and readablilty by @ByronHsu in https://github.com/microsoft/DeepSpeed/pull/5011
Update falcon fused type order by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/5007
Fix error report of DSElasticAgent._set_master_addr_port() by @RobinDong in https://github.com/microsoft/DeepSpeed/pull/4985
DS #4993 #662 : autotune single node hostfile bugfix by @oushu1zhangxiangxuan1 in https://github.com/microsoft/DeepSpeed/pull/4996
[minor] Improve logging for multiprocesses by @ByronHsu in https://github.com/microsoft/DeepSpeed/pull/5004
deepspeed/launcher: add launcher_helper as each rank's start portal by @YizhouZ in https://github.com/microsoft/DeepSpeed/pull/4699
Graph capture support on HPU accelerators by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/5013
launcher/launcher_helper.py: fix PMI name and add EnvironmentError by @YizhouZ in https://github.com/microsoft/DeepSpeed/pull/5025
Remove MI100 badge from landing page by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5036
Remove coverage reports from workflows and fix for inference CI by @loadams in https://github.com/microsoft/DeepSpeed/pull/5028
Remove Megatron-DeepSpeed CI workflow by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5038
Fix P40 CI failures by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5037
Fix for nightly torch CI by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5039
Fix nv-accelerate and nv-torch-latest-v100. by @loadams in https://github.com/microsoft/DeepSpeed/pull/5035
update inference pages to point to FastGen by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5029
launcher_helper: enable fds passing by @YizhouZ in https://github.com/microsoft/DeepSpeed/pull/5042
Fix nv-torch-latest-cpu CI by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5045
[NPU] Add NPU to support hybrid engine by @CurryRice233 in https://github.com/microsoft/DeepSpeed/pull/4831
MoE type hints by @ringohoffman in https://github.com/microsoft/DeepSpeed/pull/5043
[doc] update inference related docs from mp_size to tensor_parallel for TP by @yundai424 in https://github.com/microsoft/DeepSpeed/pull/5048
Fix broken model names in inference CI by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5053
[NPU] Change log level to debug by @CurryRice233 in https://github.com/microsoft/DeepSpeed/pull/5051
Delay reduce-scatter for ZeRO3 leaf modules by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5008
Optimize grad_norm calculations by reducing device/host dependency by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4974
load linear layer weight with given dtype by @polisettyvarma in https://github.com/microsoft/DeepSpeed/pull/4044
Update import for changes to latest diffusers by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5065
adding hccl to init_distributed function description by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5034
[Zero++ qgZ] Fall back to reduce_scatter if tensor.numel() % (2 * global_world_size) != 0 by @ByronHsu in https://github.com/microsoft/DeepSpeed/pull/5056
Make batch size documentation clearer by @segyges in https://github.com/microsoft/DeepSpeed/pull/5072
[doc/1-line change] default stage3_param_persistence_threshold is wrong in the doc by @ByronHsu in https://github.com/microsoft/DeepSpeed/pull/5073
Further refactor deepspeed.moe.utils + deepspeed.moe.layer type hints by @ringohoffman in https://github.com/microsoft/DeepSpeed/pull/5060
Fix verification for ZeRO3 leaf module by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5074
Stop tracking backward chain of broadcast in initialization by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5075
Update torch version for nv-torch-latest-cpu by @loadams in https://github.com/microsoft/DeepSpeed/pull/5086
Add backwards compatibility w/ older versions of diffusers (<0.25.0) by @lekurile in https://github.com/microsoft/DeepSpeed/pull/5083
Enable torch.compile with ZeRO (Experimental) by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4878
Update nv-accelerate to latest torch by @loadams in https://github.com/microsoft/DeepSpeed/pull/5040
HPU Accelerator: fix supported_dtypes API by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/5094
[NPU] replace 'cuda' with get_accelerator().device_name() by @minchao-sun in https://github.com/microsoft/DeepSpeed/pull/5095
optimize clip_grad_norm_ function by @mmhab in https://github.com/microsoft/DeepSpeed/pull/4915
[xs] fix ZEROPP convergence test by @yundai424 in https://github.com/microsoft/DeepSpeed/pull/5061
Switch hasattr check from compile to compiler by @loadams in https://github.com/microsoft/DeepSpeed/pull/5096
Split is_synchronized_device api to multiple apis by @BacharL in https://github.com/microsoft/DeepSpeed/pull/5026
47% FastGen speedup for low workload - refactor allocator by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/5090
Support exclude_frozen_parameters for zero_to_fp32.py script by @andstor in https://github.com/microsoft/DeepSpeed/pull/4979
Fix alignment of optimizer states when loading by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5105
Skip Triton import for AMD by @lekurile in https://github.com/microsoft/DeepSpeed/pull/5110
Add HIP conversion file outputs to .gitignore by @lekurile in https://github.com/microsoft/DeepSpeed/pull/5111
Remove optimizer step on initialization by @tohtana in https://github.com/microsoft/DeepSpeed/pull/5104

New Contributors

@ByronHsu made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5011
@RobinDong made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4985
@oushu1zhangxiangxuan1 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4996
@yundai424 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5048
@segyges made their first contribution in https://github.com/microsoft/DeepSpeed/pull/5072
@andstor made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4979

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.13.1...v0.13.2

Open source →

0.13.019 Jan 2024

DeepSpeed-FastGen: Introducting Mixtral, Phi-2, and Falcon support with major performance and feature enhancements.

New Features

DeepSpeed-FastGen: Introducting Mixtral, Phi-2, and Falcon support with major performance and feature enhancements.

What's Changed

Update version.txt after 0.12.6 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4850
doc corrections by @goodship1 in https://github.com/microsoft/DeepSpeed/pull/4861
Fix exception handling in get_all_ranks_from_group() function by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/4862
deepspeed engine: fp16 support validation on init by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4843
Remove hooks on gradient accumulation on engine/optimizer destroy by @chiragjn in https://github.com/microsoft/DeepSpeed/pull/4858
optimize grad_norm calculation in stage3.py by @mmhab in https://github.com/microsoft/DeepSpeed/pull/4436
Fix f-string messages by @li-plus in https://github.com/microsoft/DeepSpeed/pull/4865
[NPU] Fix npu offload bug by @CurryRice233 in https://github.com/microsoft/DeepSpeed/pull/4883
Partition parameters: Minor refactoring of use_secondary_tensor condition by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/4868
Pipeline: Add support to eval micro bs configuration by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4859
zero_to_fp32.py: Handle a case where shape doesn't have numel attr by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4842
Add support of Microsoft Phi-2 model to DeepSpeed-FastGen by @arashb in https://github.com/microsoft/DeepSpeed/pull/4812
Support cpu tensors without direct device invocation by @abhilash1910 in https://github.com/microsoft/DeepSpeed/pull/3842
add sharded loading for safetensors in AutoTP by @sywangyi in https://github.com/microsoft/DeepSpeed/pull/4854
[XPU] XPU accelerator support for Intel GPU device by @delock in https://github.com/microsoft/DeepSpeed/pull/4547
enable starcode((kv_head=1)) autotp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4896
Release overlap_comm & contiguous_gradients restrictions for ZeRO 1 by @li-plus in https://github.com/microsoft/DeepSpeed/pull/4887
[NPU]Add ZeRO-Infinity feature for NPU by @misstek in https://github.com/microsoft/DeepSpeed/pull/4809
fix num_kv_heads sharding in uneven autoTP for Falcon-40b by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4712
Nvme offload checkpoint by @eisene in https://github.com/microsoft/DeepSpeed/pull/4707
Add WarmupCosineLR to Read the Docs by @dwyatte in https://github.com/microsoft/DeepSpeed/pull/4916
Add Habana Labs HPU accelerator support by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/4912
Unit tests for MiCS by @zarzen in https://github.com/microsoft/DeepSpeed/pull/4792
Fix SD workflow to work with latest diffusers version by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4918
[Fix] Fix cpu inference UT failure by @delock in https://github.com/microsoft/DeepSpeed/pull/4430
Add paths to run SD tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/4919
Change PR/schedule triggers for CPU-inference by @loadams in https://github.com/microsoft/DeepSpeed/pull/4924
fix falcon-40b accuracy issue by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4895
Refactor the positional emebdding config code by @arashb in https://github.com/microsoft/DeepSpeed/pull/4920
Pin to triton 2.1.0 to fix issues with nv-inference by @loadams in https://github.com/microsoft/DeepSpeed/pull/4929
Add support of Qwen models (7b, 14b, 72b) to DeepSpeed-FastGen by @ZonePG in https://github.com/microsoft/DeepSpeed/pull/4913
DeepSpeedZeroOptimizer: refactor bit16 flattening to support more accelerators by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4833
Fix confusing width in simd_load by @yzhblind in https://github.com/microsoft/DeepSpeed/pull/4714
Specify permissions for secrets.GITHUB_TOKEN by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4927
Enable quantizer op on ROCm by @rraminen in https://github.com/microsoft/DeepSpeed/pull/4114
autoTP for Qwen by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/4902
Allow specifying mii branch for nv-a6000 workflow by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4936
Only run MII CI for inference changes by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4939
InfV2 - remove generation config requirement by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4938
Cache HF model list for inference tests by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4940
Fix docs inconsistency on default value for ignore_unused_parameters by @loadams in https://github.com/microsoft/DeepSpeed/pull/4949
Fix bug in CI model caching by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4951
fix uneven issue & add balance autotp by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4697
Optimize preprocess for ragged batching by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4942
Fix bug where ZeRO2 never uses the reduce method. by @CurryRice233 in https://github.com/microsoft/DeepSpeed/pull/4946
[docs] Add new autotp supported model in tutorial by @delock in https://github.com/microsoft/DeepSpeed/pull/4960
Add missing op_builder.hpu component for HPU accelerator by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4963
Stage_1_and_2.py: fix assert for reduce_scatter configurations combinations by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4964
[MiCS]Add the path to support sequence_data_parallel on MiCS by @ys950902 in https://github.com/microsoft/DeepSpeed/pull/4926
Update the DeepSpeed Phi-2 impl. to work with the HF latest changes by @arashb in https://github.com/microsoft/DeepSpeed/pull/4950
Prevent infinite recursion when DS_ACCELERATOR is set to cuda by @ShukantPal in https://github.com/microsoft/DeepSpeed/pull/4962
Fixes for training models with bf16 + freshly initialized optimizer via load_module_only by @haileyschoelkopf in https://github.com/microsoft/DeepSpeed/pull/4141
params partition for skip_init by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/4722
Enhance query APIs for text generation by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4965
Add API to set a module as a leaf node when recursively setting Z3 hooks by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4966
Fix T5 and mistral model meta data error by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4958
FastGen Jan 2024 blog by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4980

New Contributors

@chiragjn made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4858
@li-plus made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4865
@misstek made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4809
@dwyatte made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4916
@ZonePG made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4913
@yzhblind made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4714
@ShukantPal made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4962
@haileyschoelkopf made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4141

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.12.6...v0.13.0

Open source →

0.12.516 Dec 2023

Fix DS Stable Diffusion for latest diffusers version by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4770

What's Changed

Fix DS Stable Diffusion for latest diffusers version by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4770
Resolve any '..' in the file paths using os.path.abspath() by @rraminen in https://github.com/microsoft/DeepSpeed/pull/4709
Update dockerfile with updated versions by @loadams in https://github.com/microsoft/DeepSpeed/pull/4780
Run workflows when they are edited by @loadams in https://github.com/microsoft/DeepSpeed/pull/4779
BF16_Optimizer: add support for bf16 grad acc by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4713
fix autoTP issue for mpt (trust_remote_code=True) by @sywangyi in https://github.com/microsoft/DeepSpeed/pull/4787
Fix Hybrid Engine metrics printing by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4789
[BUG] partition_balanced return wrong result. by @zjjMaiMai in https://github.com/microsoft/DeepSpeed/pull/4312
improve the way to determine whether a variable is None by @RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4782
[NPU] Add HcclBackend for 1-bit adam, 1-bit lamb, 0/1 adam by @RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4733
Fix for stage3 when setting different communication data type by @BacharL in https://github.com/microsoft/DeepSpeed/pull/4540
Add support of Falcon models (7b, 40b, 180b) to DeepSpeed-FastGen by @arashb in https://github.com/microsoft/DeepSpeed/pull/4790
Switch paths-ignore to single quotes, update paths-ignore on nv-pre-compile-ops by @loadams in https://github.com/microsoft/DeepSpeed/pull/4805
fix for tests using torch<2.1 by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4818
Universal Checkpoint for Sequence Parallelism by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/4752
Accelerate CI fix by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4819
fix [BUG] 'DeepSpeedGPTInference' object has no attribute 'dtype' for… by @jxysoft in https://github.com/microsoft/DeepSpeed/pull/4814
Update broken link in docs by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4822
Update imports from Transformers by @loadams in https://github.com/microsoft/DeepSpeed/pull/4817
Minor updates to CI workflows by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4823
fix falcon model load from_config meta_data error by @baodii in https://github.com/microsoft/DeepSpeed/pull/4783
mv DeepSpeedEngine param_names dict init post _configure_distributed_model by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4803
Refactor launcher user arg parsing by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4824
Fix 4649 by @Alienfeel in https://github.com/microsoft/DeepSpeed/pull/4650

New Contributors

@zjjMaiMai made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4312
@jxysoft made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4814
@baodii made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4783
@Alienfeel made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4650

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.12.4...v0.12.5

Open source →

0.12.41 Dec 2023

Update version.txt after 0.12.3 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4673

What's Changed

Update version.txt after 0.12.3 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4673
[MII] catch error wrt HF version and Mistral by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4634
[NPU] Add NPU support for unit test by @RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4569
[op-builder] use unique exceptions for cuda issues by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4653
Add stable diffusion unit test by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/2496
[CANN] Support cpu offload optimizer for Ascend NPU by @hipudding in https://github.com/microsoft/DeepSpeed/pull/4568
Inference Checkpoints in V2 by @cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4664
KV Cache Improved Flexibility by @cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4668
Fix for when prompt contains an odd num of apostrophes by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/4660
universal-ckp: support megatron-deepspeed llama model by @mosheisland in https://github.com/microsoft/DeepSpeed/pull/4666
Add new MII unit tests by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4693
[Bug fix] WarmupCosineLR issues by @sbwww in https://github.com/microsoft/DeepSpeed/pull/4688
infV2 fix for OPT size variants by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4694
Add get and set APIs for the ZeRO-3 partitioned parameters by @yiliu30 in https://github.com/microsoft/DeepSpeed/pull/4681
Remove unneeded dict reinit (fix for #4565) by @eisene in https://github.com/microsoft/DeepSpeed/pull/4702
Update flops profiler to recurse by @loadams in https://github.com/microsoft/DeepSpeed/pull/4374
Communication Optimization for Large-Scale Training by @RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/4695
[docs] Intel inference blog by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4734
use all_gather_into_tensor instead of all_gather by @taozhiwei in https://github.com/microsoft/DeepSpeed/pull/4705
Install deepspeed-kernels only on Linux by @aphedges in https://github.com/microsoft/DeepSpeed/pull/4739
Add nv-sd badge to README by @loadams in https://github.com/microsoft/DeepSpeed/pull/4747
Re-organize .gitignore file to be parsed properly by @aphedges in https://github.com/microsoft/DeepSpeed/pull/4740
fix mics run with offload++ by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4749
Fix logger formatting for partitioning flags by @OAfzal in https://github.com/microsoft/DeepSpeed/pull/4728
fix: to solve #4726 by @RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4727
Add safetensors support by @jihnenglin in https://github.com/microsoft/DeepSpeed/pull/4659

New Contributors

@RUAN-ZX made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4569
@oelayan7 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4660
@sbwww made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4688
@yiliu30 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4681
@eisene made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4702
@taozhiwei made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4705
@OAfzal made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4728
@jihnenglin made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4659

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.12.3...v0.12.4

Open source →

0.12.313 Nov 2023

Stable Diffusion now supported with latest Torch, diffusers, and Triton versions.

New Bug Fixes

Stable Diffusion now supported with latest Torch, diffusers, and Triton versions.

What's Changed

Update version.txt after 0.12.2 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4617
Fix figure in FlexGen blog by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4624
Fix figure of llama2 13B in DS-FlexGen blog by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4625
Fix config format by @xu-song in https://github.com/microsoft/DeepSpeed/pull/4594
Guanhua/partial offload rebase v2 (#590) by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4636
offload++ blog (#623) by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4637
Update README in offloadpp blog by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4641
[docs] update news items by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4640
DeepSpeed-FastGen Chinese Blog by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/4642
Fix issues with torch cpu builds by @loadams in https://github.com/microsoft/DeepSpeed/pull/4639
Isolate src code and testing for DeepSpeed-FastGen by @cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4610
Add Japanese blog for DeepSpeed-FastGen by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4651
Fix for MII unit tests by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4652
Enhance the robustness of module_state_dict by @LZHgrla in https://github.com/microsoft/DeepSpeed/pull/4587
Enable ZeRO3 allgather for multiple dtypes by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4647
add option to disable pipeline partitioning by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4322
Added HIP_PLATFORM_AMD=1 for non JIT build by @rraminen in https://github.com/microsoft/DeepSpeed/pull/4585
Fix rope_theta arg for diffusers_attention by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4656
tl.dot(a,b, trans_b=True) is not supported by triton2.0+ , updating this api by @bmedishe in https://github.com/microsoft/DeepSpeed/pull/4541
Update ds-chat workflow to work w/ deepspeed-chat install by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4598
Diffusers attention script update triton2.1 by @bmedishe in https://github.com/microsoft/DeepSpeed/pull/4573
Fix the openfold training. by @cctry in https://github.com/microsoft/DeepSpeed/pull/4657
Universal ckp fixes by @mosheisland in https://github.com/microsoft/DeepSpeed/pull/4588
Update .gitignore [Adding comments , Improved documentation] by @Nadav23AnT in https://github.com/microsoft/DeepSpeed/pull/4631
Update lr_schedules.py by @CoinCheung in https://github.com/microsoft/DeepSpeed/pull/4563
Fix UNET and VAE implementations for new diffusers version by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4663
fix num_kv_heads sharding in autoTP for the new in-repo Falcon-40B by @dc3671 in https://github.com/microsoft/DeepSpeed/pull/4654

New Contributors

@xu-song made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4594
@LZHgrla made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4587
@mosheisland made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4588
@Nadav23AnT made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4631
@CoinCheung made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4563

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.12.2...v0.12.3

Open source →

0.11.23 Nov 2023

Update version.txt after 0.11.1 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4484

What's Changed

Update version.txt after 0.11.1 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4484
Update DS_BUILD_* references. by @loadams in https://github.com/microsoft/DeepSpeed/pull/4485
Introduce pydantic_v1 compatibility module for pydantic>=2.0.0 support by @ringohoffman in https://github.com/microsoft/DeepSpeed/pull/4407
Enable control over timeout with environment variable by @BramVanroy in https://github.com/microsoft/DeepSpeed/pull/4405
Update ROCm verison by @loadams in https://github.com/microsoft/DeepSpeed/pull/4486
adding 8bit dequantization kernel for asym fine-grained block quantization in zero-inference by @stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4450
Fix scale factor on flops profiler by @loadams in https://github.com/microsoft/DeepSpeed/pull/4500
add DeepSpeed4Science white paper by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4502
[CCLBackend] update API by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/4378
Ulysses: add col-ai evaluation by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/4517
Ulysses: Update README.md by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/4518
add available memory check to accelerators by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4508
clear redundant parameters in zero3 bwd hook by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/4520
Add NPU FusedAdam support by @CurryRice233 in https://github.com/microsoft/DeepSpeed/pull/4343
fix error type issue in deepspeed/comm/ccl.py by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/4521
Fixed deepspeed.comm.monitored_barrier call by @Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/4496
[Bug fix] Add rope_theta for llama config by @cupertank in https://github.com/microsoft/DeepSpeed/pull/4480
[ROCm] Add rocblas header by @rraminen in https://github.com/microsoft/DeepSpeed/pull/4538
[docs] ZeRO infinity slides and blog by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4542
Switch from HIP_PLATFORM_HCC to HIP_PLATFORM_AMD by @loadams in https://github.com/microsoft/DeepSpeed/pull/4539
Turn off I_MPI_PIN for impi launcher by @delock in https://github.com/microsoft/DeepSpeed/pull/4531
[docs] paper updates by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4543
ROCm 6.0 prep changes by @loadams in https://github.com/microsoft/DeepSpeed/pull/4537
Fix RTD builds by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4558
pipe engine _aggregate_total_loss: more efficient loss concatenation by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4327
Add missing rocblas include by @loadams in https://github.com/microsoft/DeepSpeed/pull/4557
Enable universal checkpoint for zero stage 1 by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/4516
[AutoTP] Make AutoTP work when num_heads not divisible by number of workers by @delock in https://github.com/microsoft/DeepSpeed/pull/4011
Fix the sequence-parallelism for the dense model architecture by @RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/4530
engine.py - save_checkpoint: only rank-0 should create the save dir by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4536
Remove PP Grad Tail Check by @Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/2538
Added HIP_PLATFORM_AMD=1 by @rraminen in https://github.com/microsoft/DeepSpeed/pull/4570
fix multiple definition while building evoformer by @fecet in https://github.com/microsoft/DeepSpeed/pull/4556
Don't check overflow for bf16 data type by @hablb in https://github.com/microsoft/DeepSpeed/pull/4512
Public update by @yaozhewei in https://github.com/microsoft/DeepSpeed/pull/4583
[docs] paper updates by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4584
Disable CPU inference on PRs by @loadams in https://github.com/microsoft/DeepSpeed/pull/4590

New Contributors

@ringohoffman made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4407
@BramVanroy made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4405
@cupertank made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4480

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.11.1...v0.11.2

Open source →