PyPI · #127 most downloaded on PyPI

sglang

0.5.20315M downloads/mosgl-project/sglang ↗New releases in your daily digest, changelog included.

SGLang is a fast serving framework for large language models and vision language models.

Last release today
18 Sep 2026
Ships fairly regularly
a new release about every 2 weeks
Some releases are documented
notes for 26 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
3 years old
157 releases · first in 2024

Release timeline

157 releases since 2024

202420252026

Releases

0.5.71 Jan 2026Security
[model-gateway] Fixed WASM Security Vulnerability - Execution Timeout by @slin1237 in https://github.com/sgl-project/sglang/pull/14588
Highlights

New Model Support:

Day 0 Support for Mimo-V2-Flash: #15207, https://lmsys.org/blog/2025-12-16-mimo-v2-flash/

Day 0 Support for Nemotron-Nano-v3: https://lmsys.org/blog/2025-12-15-run-nvidia-nemotron-3-nano/

Day 0 Support for LLaDA 2.0: https://lmsys.org/blog/2025-12-19-diffusion-llm/

[SGLang-Diffusion] Day 0 Support for Qwen-Image-Edit-2509, Qwen-Image-Edit-2511, Qwen-Image-2512 and Qwen-Image-Layered

EAGLE 3 speculative decoding draft models for popular models: https://lmsys.org/blog/2025-12-23-spec-bundle-phase-1/

Model Gateway v0.3.0 Release: https://docs.sglang.io/advanced_features/sgl_model_gateway.html

Scalable pipeline parallelism with dynamic chunking support for ultra-long contexts (PP Refactor Roadmap #11857）

Encoder Disaggregation for Multi-modal models (Roadmap #15118)

SGLang-Diffusion:

Set --dit-layerwise-offload true to reduce peak VRAM usage by up to 30GB, and improve performance by up to 58% for all models

Significantly reduce the latency of Qwen-Image-Edit, making it one-of-the-fastest among all open-source solutions. More improvements are on the way

Add support for AMD/4090/5090, along with additional attention choices (sage-attn, sage-attn3), more parallelism options (TP) and enhancements to HTTP API (Google vertex supported)

Cache-dit integration to improve performance by up to 165%

What's Changed

Refactor custom allreduce logics by @iforgetmyname in https://github.com/sgl-project/sglang/pull/13710

[Doc] Update DeepSeek-V3.2 document by @Fridge003 in https://github.com/sgl-project/sglang/pull/14321

Feature/support distilled vae generic by @baonudesifeizhai in https://github.com/sgl-project/sglang/pull/14195

[Performance] Optimize NSA Indexer K/S Buffer Access with Fused Triton Kernels by @Johnsonms in https://github.com/sgl-project/sglang/pull/13812

Update CODEOWNERS for multimodal by @mickqian in https://github.com/sgl-project/sglang/pull/14329

[bug fix] use npu phy id in container env by @jinke446 in https://github.com/sgl-project/sglang/pull/14266

[model-gateway] multimodality initialization by @slin1237 in https://github.com/sgl-project/sglang/pull/13350

[Doc] Fix DeepSeek V32 Doc by @Fridge003 in https://github.com/sgl-project/sglang/pull/14336

sync attention, deepseek doc by @b8zhong in https://github.com/sgl-project/sglang/pull/14335

[PD] Support decode pp for PD disaggregation by @ShangmingCai in https://github.com/sgl-project/sglang/pull/14265

[model-gateway] add image processor and transformer structure by @slin1237 in https://github.com/sgl-project/sglang/pull/14344

[CPU] Support chunk_gated_delta_rule kernel for Qwen3-Next by @Valentine233 in https://github.com/sgl-project/sglang/pull/12441

[bugfix] Fix prefill tbo disabled when --deepep-mode=auto by @yuhyao in https://github.com/sgl-project/sglang/pull/14333

[CI] update estimated elapsed time of some unittests by @ch-wan in https://github.com/sgl-project/sglang/pull/14347

[NPU] bug fix: w_vc need contiguous for NPU batch_matmul_transpose ops by @ZhengdQin in https://github.com/sgl-project/sglang/pull/13980

[bugfix] NpuFuseEPMoE miss initialization parameters by @chenxu140 in https://github.com/sgl-project/sglang/pull/14295

[Ascend] fix AscendAttnMaskBuilder bug to support float16 models by @MichelleWu351 in https://github.com/sgl-project/sglang/pull/14271

Tiny adjust CI testcases by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14362

[NPU][Doc] updated installation guide for Ascend NPU by @VDV1985 in https://github.com/sgl-project/sglang/pull/13585

Feature/add vae path to cli doc#14004 by @baonudesifeizhai in https://github.com/sgl-project/sglang/pull/14355

[CPU] add fused_qkvzba_split_reshape_cat kernel for Qwen3-next by @blzheng in https://github.com/sgl-project/sglang/pull/12330

Single Batch Overlap for MoE Models by @Sulfur6 in https://github.com/sgl-project/sglang/pull/9660

Move custom_ops under layers; move _custom_ops.py → custom_all_reduce_ops.py by @merrymercy in https://github.com/sgl-project/sglang/pull/14326

[model-gateway] add llava model image processor and tests by @slin1237 in https://github.com/sgl-project/sglang/pull/14371

ci: Migrate AMD workflows to new MI325 runners; temporarily disabled failed CI's to be added back by @sunxxuns in https://github.com/sgl-project/sglang/pull/14226

[Tiny]Small fixes in deepseek v32 doc by @Fridge003 in https://github.com/sgl-project/sglang/pull/14372

Fix validation to detect missing model files before loading by @alisonshao in https://github.com/sgl-project/sglang/pull/14253

[model-gateway] add qwen2_vl model image processor and tests by @slin1237 in https://github.com/sgl-project/sglang/pull/14374

[model-gateway] add qwen2.5_vl model image processor by @slin1237 in https://github.com/sgl-project/sglang/pull/14375

Revert "Revert "enable csgmv automatically on cuda"" by @b8zhong in https://github.com/sgl-project/sglang/pull/14277

[model-gateway] use worker crate in openai router by @slin1237 in https://github.com/sgl-project/sglang/pull/14330

[model-gateway] add qwen3_vl model image processor by @slin1237 in https://github.com/sgl-project/sglang/pull/14377

Fix sgl-router silently parse selector wrongly causing OME fail to discover pods by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14359

[sgl-kernel][Feat][B200][1/N]Support MXFP8 Grouped GEMM in Blackwell by @HydraQYH in https://github.com/sgl-project/sglang/pull/13731

[CPU] document updates by @ZailiWang in https://github.com/sgl-project/sglang/pull/14272

Support PP x PD decode with nixl backend by @bluecoffee8 in https://github.com/sgl-project/sglang/pull/14392

[VLM] Introduce Cache for positional embedding ids for Qwen-VL family by @yuan-luo in https://github.com/sgl-project/sglang/pull/14292

use faster covnersion from float8_e4m3fn to bfloat16 by @mingfeima in https://github.com/sgl-project/sglang/pull/12316

[model-gateway][doc] Add STDIO Explicitly to Example in README by @xuwenyihust in https://github.com/sgl-project/sglang/pull/14393

[CPU] add support for mamba causal conv1d for qwen3-next by @mingfeima in https://github.com/sgl-project/sglang/pull/12309

[model-gateway] add phi3 vision image processor by @slin1237 in https://github.com/sgl-project/sglang/pull/14381

[model-gateway] introduce provider in openai router by @slin1237 in https://github.com/sgl-project/sglang/pull/14394

[AMD] fix the regression issue for DeepseekV3 on MI300 by @yctseng0211 in https://github.com/sgl-project/sglang/pull/14383

[NPU][1/N] NPU basic functions refactor and new modelslim quant type by @iforgetmyname in https://github.com/sgl-project/sglang/pull/13359

[CPU] Optimize small oc GEMM for Qwen3-next on CPU by @jianan-gu in https://github.com/sgl-project/sglang/pull/12446

Try to fix B200 DeepEP error by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14399

[1/2] Add rope kernel in sgl-kernel by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/14334

[bug fix] fix ima with get_mla_kv_buffer_kernel overflow by @XucSh in https://github.com/sgl-project/sglang/pull/14224

Add Mistral Large 3 support. by @dcampora in https://github.com/sgl-project/sglang/pull/14213

[diffusion] fix gen video doc by @yeahdongcn in https://github.com/sgl-project/sglang/pull/14409

Add 'NPU' to the runtime exception message in get_device by @rauletorresc in https://github.com/sgl-project/sglang/pull/14225

Add mooncake transfer_engine_bench into maunal test by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14429

[model-gateway] add phi4 vision image processor by @slin1237 in https://github.com/sgl-project/sglang/pull/14430

diffusion: Add Configurable Generator Device and Seed Support via API by @niehen6174 in https://github.com/sgl-project/sglang/pull/14366

[model-gateway] introduce request ctx for oai router by @slin1237 in https://github.com/sgl-project/sglang/pull/14434

[NPU]add nightly-test-npu by @cherryblo in https://github.com/sgl-project/sglang/pull/14143

[model-gateway] add llama4 vision image processor by @slin1237 in https://github.com/sgl-project/sglang/pull/14438

[model-gateway] extract conversation out of oai router by @slin1237 in https://github.com/sgl-project/sglang/pull/14440

[DeepseekV3.2][NSA][Indexer] Fix PAGED top-k transform for NSA indexer chunked execution on H200 by @YAMY1234 in https://github.com/sgl-project/sglang/pull/14325

[model-gateway] move oai header util to router header util by @slin1237 in https://github.com/sgl-project/sglang/pull/14441

[FIX] trtllm-moe-fp4-renorm for Qwen series models by @samuellees in https://github.com/sgl-project/sglang/pull/14350

add doc for quantized kv cache by @b8zhong in https://github.com/sgl-project/sglang/pull/14348

fix: Correct environment variable syntax in docker-compose configuration by @yankay in https://github.com/sgl-project/sglang/pull/8287

[model-gateway] move all responses api event from oai to proto by @slin1237 in https://github.com/sgl-project/sglang/pull/14446

[model-gateway] add mistral 3 image processor by @slin1237 in https://github.com/sgl-project/sglang/pull/14445

[model-gateway] grpc to leverage event type by @slin1237 in https://github.com/sgl-project/sglang/pull/14450

ministral3 by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/14251

[Bug] fix not desired disable fused share experts caused by rocm logic by @ocss884 in https://github.com/sgl-project/sglang/pull/14432

Rename secrets.WHL_TOKEN -> secrets.GH_PAT_FOR_WHL_RELEASE by @sglang-bot in https://github.com/sgl-project/sglang/pull/14421

further optimze model load by @zyksir in https://github.com/sgl-project/sglang/pull/13836

Add CI permissions for user 'yushengsu-thu' by @alisonshao in https://github.com/sgl-project/sglang/pull/14468

[ez] Fix typing by @yinghai in https://github.com/sgl-project/sglang/pull/14473

Add AMD stage support to /rerun-stage command and fix related bugs by @alisonshao in https://github.com/sgl-project/sglang/pull/14463

Add YAMY1234 to CI Permission by @Fridge003 in https://github.com/sgl-project/sglang/pull/14475

clean up gemlite usage by @zminglei in https://github.com/sgl-project/sglang/pull/14444

[diffusion] chore: further improve model searching logic by @mickqian in https://github.com/sgl-project/sglang/pull/14484

fix bug about pin memory by @zyksir in https://github.com/sgl-project/sglang/pull/14472

[diffusion] cli: add argument --adjust-frames, --override-protected-fields by @gmixiaojin in https://github.com/sgl-project/sglang/pull/13996

dockerfile: add lightweight runtime stage and refactors by @ishandhanani in https://github.com/sgl-project/sglang/pull/13861

diffusion: Fix CLIP text encoder attention mask and causal masking bugFix clip attention by @niehen6174 in https://github.com/sgl-project/sglang/pull/14364

Enable RadixCache for Mamba2 models by @roikoren755 in https://github.com/sgl-project/sglang/pull/13584

[Diffusion] Fix profiler trace missing Python stack in diffusion pipeline by @BBuf in https://github.com/sgl-project/sglang/pull/14499

support GLM-V vision model dp by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/14097

[misc] add model arch and type to server info and use it for harmony by @slin1237 in https://github.com/sgl-project/sglang/pull/14456

Add Mistral Large 3 Eagle Support by @elvischenv in https://github.com/sgl-project/sglang/pull/14466

Add Mistral Large 3 to nightly CI tests by @alisonshao in https://github.com/sgl-project/sglang/pull/14459

[diffusion] chore: set allowing overriding protected fields of sampling params as default behavior by @mickqian in https://github.com/sgl-project/sglang/pull/14471

[model-gateway] move conversation to first class routing by @slin1237 in https://github.com/sgl-project/sglang/pull/14506

[Spec] Mamba2 support in target models by @roikoren755 in https://github.com/sgl-project/sglang/pull/13434

[diffusion] Support cache-dit by @Brain97 in https://github.com/sgl-project/sglang/pull/14234

Add fused FP8 KV cache write kernel for TRTLLM MHA backend by @harvenstar in https://github.com/sgl-project/sglang/pull/14093

[model-gateway] Add WASM support for middleware by @tonyluj in https://github.com/sgl-project/sglang/pull/12471

[model-gateway] reorganized conversation handler by @slin1237 in https://github.com/sgl-project/sglang/pull/14507

tiny remove deprecated endpoint call by @b8zhong in https://github.com/sgl-project/sglang/pull/13607

[model-gateway] fix server info comment by @slin1237 in https://github.com/sgl-project/sglang/pull/14508

Add Mistral Large 3 basic test to PR CI by @alisonshao in https://github.com/sgl-project/sglang/pull/14460

Fix removing worker will make it healthy forever in prometheus metrics by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14420

[model-gateway] Make Tokenizer Builder Aware of Env Vars Like HF_ENDPOINT by @xuwenyihust in https://github.com/sgl-project/sglang/pull/14405

[model-gateway] change sgl-router to sgl-model-gateway by @slin1237 in https://github.com/sgl-project/sglang/pull/14312

[model-gateway] fix left over sgl-router names to sgl-model-gateway by @slin1237 in https://github.com/sgl-project/sglang/pull/14512

[model-gateway] fix logs in smg workflow by @slin1237 in https://github.com/sgl-project/sglang/pull/14513

[model-gateway] fix left over sgl-router names in wasm by @slin1237 in https://github.com/sgl-project/sglang/pull/14514

[model-gateway] fix code owner for wasm by @slin1237 in https://github.com/sgl-project/sglang/pull/14516

chore: bump sgl-kernel version to 0.3.18.post3 by @sglang-bot in https://github.com/sgl-project/sglang/pull/14427

Tiny use trtllm_mha as default when possible by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14291

[Docs] Add /rerun-stage command to contribution guide by @alisonshao in https://github.com/sgl-project/sglang/pull/14521

Fix safetensors validation to catch corruption after download by @alisonshao in https://github.com/sgl-project/sglang/pull/14465

[CODEOWNER] update codeowner for qwen3-next related by @hanming-lu in https://github.com/sgl-project/sglang/pull/14522

fix rmsnorm -> layernorm in qwen3 omni by @vincentzed in https://github.com/sgl-project/sglang/pull/11791

[diffusion] chore: temporarily upgrade diffusers to make Z-image compatible with Cache-DiT by @mickqian in https://github.com/sgl-project/sglang/pull/14530

[bug] fix notebook to include new keys from model_info by @slin1237 in https://github.com/sgl-project/sglang/pull/14528

Revise DP Multi-Modal Encoder Document by @yhyang201 in https://github.com/sgl-project/sglang/pull/14290

[CPU] add mamba fla kernels for Qwen3-next by @blzheng in https://github.com/sgl-project/sglang/pull/12324

Revert "tiny remove deprecated endpoint call" by @Fridge003 in https://github.com/sgl-project/sglang/pull/14533

support mtp with deepseek r1 nvfp4 model by @rainj-me in https://github.com/sgl-project/sglang/pull/13115

[diffusion] refactor: simplify SamplingParams override logic by @mickqian in https://github.com/sgl-project/sglang/pull/14539

[Diffusion] Add QKV fusion optimization for Flux models by @BBuf in https://github.com/sgl-project/sglang/pull/14505

[model-gateway][tracing]: implement request tracing using OpenTelemetry with trace context propagation (HTTP) by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/13897

diffusion: fix LoRA dtype handling and weight attribute access for z-image model by @niehen6174 in https://github.com/sgl-project/sglang/pull/14543

fix "GrammarMatcher has terminated after accepting the stop token, but is trying to find the next token mask" when both reasoning and spec are enabled by @gongwei-130 in https://github.com/sgl-project/sglang/pull/14464

[1/n] Fix hanging during DeepGemm Warmup by @Fridge003 in https://github.com/sgl-project/sglang/pull/14493

[Bug fix] Add /model_info endpoint to mini_lb by @alisonshao in https://github.com/sgl-project/sglang/pull/14535

[Qwen3-next] remove heuristics and add radix cache kl test by @hanming-lu in https://github.com/sgl-project/sglang/pull/14520

[Misc]Register and refactor some environs for dpsk-fp4 and DeepEp by @Fridge003 in https://github.com/sgl-project/sglang/pull/14538

chore: bump sgl-kernel version to 0.3.18.post3 by @sglang-bot in https://github.com/sgl-project/sglang/pull/14518

Update CI_PERMISSIONS.json by @harrisonlimh in https://github.com/sgl-project/sglang/pull/14552

Update DeepSeek V3 docs to use B200 by @leejnau in https://github.com/sgl-project/sglang/pull/14447

[Doc] Add short explanation on page size by @b8zhong in https://github.com/sgl-project/sglang/pull/14557

[docs] Add missing word in argument description by @almaslof in https://github.com/sgl-project/sglang/pull/14205

support piecewise cuda graph for Olmo models by @zminglei in https://github.com/sgl-project/sglang/pull/14476

Enhance prefill PP node robustness by @qhsc in https://github.com/sgl-project/sglang/pull/14494

DOC update nemo-skills in docs by @gwarmstrong in https://github.com/sgl-project/sglang/pull/14555

remove unecessary dual stream token threshold from the rest of models (qwen moe, kimi linear, etc.) by @b8zhong in https://github.com/sgl-project/sglang/pull/14337

feat(ci): add framework target to release-docker workflows by @ishandhanani in https://github.com/sgl-project/sglang/pull/14559

Fix attention backend logic for Qwen3-Next on SM100 by @Chen-0210 in https://github.com/sgl-project/sglang/pull/14560

[FLA] Add explicit kernel arguments to kda.py for Kimi Linear support by @alisonshao in https://github.com/sgl-project/sglang/pull/14561

Add CUDA kernel size analysis tool for sgl-kernel optimization by @BBuf in https://github.com/sgl-project/sglang/pull/14544

[DLLM] Add threshold based parallel decoding support by @btw616 in https://github.com/sgl-project/sglang/pull/14412

Add unit-test-backend-8-gpu-b200 to rerun-stage command by @alisonshao in https://github.com/sgl-project/sglang/pull/14569

[apply][2/2] Fused qk_norm_rope for Qwen3-MoE by @yuan-luo in https://github.com/sgl-project/sglang/pull/13998

Add Expert Parallelism (EP) support for kimi-k2-thinking by @BBuf in https://github.com/sgl-project/sglang/pull/13725

Tiny remove wrong import from python.sglang by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14577

Add small model test for spec v2 + dp + trtllm_mla by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14576

[diffusion] cli: profiling utilities support by @AichenF in https://github.com/sgl-project/sglang/pull/14185

[NPU]LoRA: Adding Torch Native backend by @vlserov in https://github.com/sgl-project/sglang/pull/14132

[BugFix] fix prefixcache performance and accuracy on ascend by @khalil2ji3mp6 in https://github.com/sgl-project/sglang/pull/13573

Fix FP8 KV Triton type issue and add regression test by @harvenstar in https://github.com/sgl-project/sglang/pull/14553

Rename TensorRT Model Optimizer to Model Optimizer by @Edwardf0t1 in https://github.com/sgl-project/sglang/pull/14455

[CI] Tiny speed up VLM CI by @b8zhong in https://github.com/sgl-project/sglang/pull/14517

[Minor] Temporarily skipping deepep large mtp test by @Fridge003 in https://github.com/sgl-project/sglang/pull/14586

[model-gateway] extra accumulator and tool handler in oai router by @slin1237 in https://github.com/sgl-project/sglang/pull/14587

[model-gateway] Fixed WASM Security Vulnerability - Execution Timeout by @slin1237 in https://github.com/sgl-project/sglang/pull/14588

[model-gateway] reorganize metrics, logging, and otel to its own module by @slin1237 in https://github.com/sgl-project/sglang/pull/14590

Refactor tuning block wise kernel and opt Qwen/Qwen3-VL-32B-Instruct-FP8 by @BBuf in https://github.com/sgl-project/sglang/pull/14141

[CI]Unblock and split spec v2+dp test by @Fridge003 in https://github.com/sgl-project/sglang/pull/14551

[Tool Call] Fix DeepSeekV32Detector skipping functions with no params in streaming mode by @momaek in https://github.com/sgl-project/sglang/pull/14573

[feat] use cachebuffer to store mm feature to speedup hash by @liusy58 in https://github.com/sgl-project/sglang/pull/14386

[CI] Fix unit-test-backend-8-gpu-b200 running on every /rerun-stage by @alisonshao in https://github.com/sgl-project/sglang/pull/14591

[model-gateway] fix WASM memory limit per module by @slin1237 in https://github.com/sgl-project/sglang/pull/14600

Tiny fix missing policy decision recording by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14605

Super tiny remove unneeded policy flag by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14608

[model-gateway] refactor otel to be more efficient by @slin1237 in https://github.com/sgl-project/sglang/pull/14604

Super tiny remove unused select_worker_pair by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14609

[model-gateway] fix WASM unbounded request/response body read vuln by @slin1237 in https://github.com/sgl-project/sglang/pull/14612

[2/2] Add rope kernel in sgl-kernel by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/14452

[DLLM] Add initial cuda graph support by @btw616 in https://github.com/sgl-project/sglang/pull/14203

Super tiny fix unused code in router by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14618

[Glm46v] Bug fix for accuracy drop and unable to launch server by @byjiang1996 in https://github.com/sgl-project/sglang/pull/14585

Fix amd rope definition by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/14556

modify the sgl-kernel to be compatible with transformers 5.x. by @yhyang201 in https://github.com/sgl-project/sglang/pull/14625

[Reasoning + Structured Output] make reasoning compatible with structured output by @Muqi1029 in https://github.com/sgl-project/sglang/pull/12551

[Feat] add support for LoRA layers in transformer_2 within LoRAPipeline by @Prozac614 in https://github.com/sgl-project/sglang/pull/14606

chore: bump sgl-kernel version to 0.3.19 by @sglang-bot in https://github.com/sgl-project/sglang/pull/14632

[cpu] Implement all gather/reduce for arm64 cpu by @cyb70289 in https://github.com/sgl-project/sglang/pull/12527

[diffusion] chore: further refine output resolution adjustment logic by @mickqian in https://github.com/sgl-project/sglang/pull/14558

Fix dp-aware incompatible with service-discovery by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14629

update transformers package version to 5.0.0rc0 by @yhyang201 in https://github.com/sgl-project/sglang/pull/14356

chore: bump sgl-kernel version to 0.3.19 by @sglang-bot in https://github.com/sgl-project/sglang/pull/14649

chore: bump SGLang version to 0.5.6.post1 by @sglang-bot in https://github.com/sgl-project/sglang/pull/14651

[AMD] change fused rms quant interface for aiter upgrade by @yctseng0211 in https://github.com/sgl-project/sglang/pull/14497

[model-gateway] reducing cpu overhead in various of places by @slin1237 in https://github.com/sgl-project/sglang/pull/14658

[model-gateway] reduce cpu overhead in grpc router by @slin1237 in https://github.com/sgl-project/sglang/pull/14663

[model-gateway] fix WASM arbitrary file read security vol by @slin1237 in https://github.com/sgl-project/sglang/pull/14664

vlm: Use fa3 as the default backend for qwen3 vl by @mickqian in https://github.com/sgl-project/sglang/pull/14634

[model-gateway] Optimize memory usage in HTTP router by @slin1237 in https://github.com/sgl-project/sglang/pull/14667

fix: use .get() when accessing strict mem-check env variable by @yhyang201 in https://github.com/sgl-project/sglang/pull/14657

improve default glm mtp setting by @b8zhong in https://github.com/sgl-project/sglang/pull/14457

Fix cache-aware router should pick min load instead of min tenant size by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14650

Bump up diffusers to latest official release version by @byjiang1996 in https://github.com/sgl-project/sglang/pull/14670

[model-gateway] add OTEL integration to grpc router by @slin1237 in https://github.com/sgl-project/sglang/pull/14671

[CI] Increase max-parallel to 15 for high priority PRs by @alisonshao in https://github.com/sgl-project/sglang/pull/14675

[HiCache] fix condition check when use decode offload by @ssssnow in https://github.com/sgl-project/sglang/pull/14489

[RadixTree] Optimize the Time Complexity of Node Retrieval Operation from O(n*m) to O(n) by @CLFutureX in https://github.com/sgl-project/sglang/pull/13334

Tiny support printing requests in bench_serving for observability by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14652

Aiter fp8 kv cache by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/13147

[SMG]feat: implement TokenGuardBody for managing token return by @jimmy-evo in https://github.com/sgl-project/sglang/pull/14653

[NPU] chore: bump basic software version to 8.3.rc2 by @iforgetmyname in https://github.com/sgl-project/sglang/pull/14614

[CI] Unblock gb200 cutedsl test by @Fridge003 in https://github.com/sgl-project/sglang/pull/14469

Add ffmpeg into sglang docker - required by transformers multimodal V… by @byjiang1996 in https://github.com/sgl-project/sglang/pull/14679

[Bugfix] Fix KeyError for Mistral-Large-3 rope_scaling config by @alisonshao in https://github.com/sgl-project/sglang/pull/14627

Tiny support sgl-router http response status code metrics by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14689

[CI] Migrate Eagle 1-GPU tests to test/registered/ by @alisonshao in https://github.com/sgl-project/sglang/pull/14529

Revert "[Bug] fix not desired disable fused share experts caused by r… by @zhyncs in https://github.com/sgl-project/sglang/pull/14676

Add per-request decode tp size by @merrymercy in https://github.com/sgl-project/sglang/pull/14678

[ci][smg] fix docker release ci and add it to pr test by @slin1237 in https://github.com/sgl-project/sglang/pull/14683

Tiny extract select_worker_min_load by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14648

Fix dp-aware incompatible with completions and chat completions APIs by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14647

[CI] Fix Llama 3.1 8B FP4 CI by @b8zhong in https://github.com/sgl-project/sglang/pull/14699

fix: make override DeepseekV2Model work by @zhyncs in https://github.com/sgl-project/sglang/pull/14707

chore: add code owners for deepseek_v2.py by @zhyncs in https://github.com/sgl-project/sglang/pull/14714

[CI] Move mistral large 3 basic to nightly by @alisonshao in https://github.com/sgl-project/sglang/pull/14622

fix the deepep 8 gpu unit test by @rainj-me in https://github.com/sgl-project/sglang/pull/14601

Add fuse_marlin_moe test to ci and add new ep test by @BBuf in https://github.com/sgl-project/sglang/pull/14686

[Bugfix] Fix environ error in scheduler_runtime_checker_mixin.py by @llfl in https://github.com/sgl-project/sglang/pull/14461

[Feat] Add received_time in serving_base by @zhanghaotong in https://github.com/sgl-project/sglang/pull/13432

fix: prevent HugginqFace access when SGLANG_USE_MODELSCOPE is enabled by @yrk111222 in https://github.com/sgl-project/sglang/pull/12039

[Test] Skip STANDALONE speculative decoding tests for different hidden sizes by @alisonshao in https://github.com/sgl-project/sglang/pull/14733

[diffusion] support batch compare by @Brain97 in https://github.com/sgl-project/sglang/pull/14738

Revert "[Feat] Add received_time in serving_base" by @merrymercy in https://github.com/sgl-project/sglang/pull/14743

[Model] Add PaddleOCR-VL Model Support by @yudian0504 in https://github.com/sgl-project/sglang/pull/12953

fix rope parameter initialization error caused by transformers v5.0 update by @yhyang201 in https://github.com/sgl-project/sglang/pull/14745

[model-gateway] optimize core modules by @slin1237 in https://github.com/sgl-project/sglang/pull/14751

[SMG] perf: optimize tokenizer for reduced CPU and memory overhead by @slin1237 in https://github.com/sgl-project/sglang/pull/14752

Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend by @b8zhong in https://github.com/sgl-project/sglang/pull/14379

fix: checking if tokenizer is in cache before downloading from HF by @dougyster in https://github.com/sgl-project/sglang/pull/14698

fix: making rate limit a warning instead of error by @dougyster in https://github.com/sgl-project/sglang/pull/14753

move multi-item scoring functions in tokenizer manager into a separate file by @merrymercy in https://github.com/sgl-project/sglang/pull/14740

Improve CI by trying a warmup before unit tests by @merrymercy in https://github.com/sgl-project/sglang/pull/14669

[Perf] Optimize radix tree for cache-aware load balancin by @slin1237 in https://github.com/sgl-project/sglang/pull/14758

[Feature] Add LoRA support for embedding layers by @yushengsu-thu in https://github.com/sgl-project/sglang/pull/14177

[model-gateway] release gateway 0.2.4 by @slin1237 in https://github.com/sgl-project/sglang/pull/14763

[ci]: Enable the new hf API by @MingxuZh in https://github.com/sgl-project/sglang/pull/14687

Re-add the API serving timing metrics. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14744

fix: adding rate limit warning at verify token permission stage by @dougyster in https://github.com/sgl-project/sglang/pull/14756

Disable 8-gpu-b200 runner in PR tests by @alisonshao in https://github.com/sgl-project/sglang/pull/14768

[fix] Fix issues for in-flight weight updates by @ShawnY112358 in https://github.com/sgl-project/sglang/pull/14064

[Auto Sync] Update data_parallel_controller.py, detokenizer... (20251209) by @merrymercy in https://github.com/sgl-project/sglang/pull/14759

fix: race condition between validation and download locks by @alisonshao in https://github.com/sgl-project/sglang/pull/14761

Fix VLM accuracy thresholds for nightly tests by @alisonshao in https://github.com/sgl-project/sglang/pull/14777

fix server args bug by @TomerBN-Nvidia in https://github.com/sgl-project/sglang/pull/14725

handling incomplete rope_scaling confign ci after transformers upgrade by @yhyang201 in https://github.com/sgl-project/sglang/pull/14784

fix b200 ci by @b8zhong in https://github.com/sgl-project/sglang/pull/14786

[RL] support weight reload for low-bit rollout by @AniZpZ in https://github.com/sgl-project/sglang/pull/9650

fix: add missing logic for SGLANG_USE_MODELSCOPE variable by @yrk111222 in https://github.com/sgl-project/sglang/pull/14794

fix b200 fa4 ci by @b8zhong in https://github.com/sgl-project/sglang/pull/14788

[diffusion] profile: early exit when enough steps are captured to red… by @mickqian in https://github.com/sgl-project/sglang/pull/14803

[GLM-4.6V] Support Pipeline Parallelism for GLM-4.6V & GLM-4.1V by @yuan-luo in https://github.com/sgl-project/sglang/pull/14720

[CI] Add LoRA support to diffusion server configuration and test cases by @Prozac614 in https://github.com/sgl-project/sglang/pull/14697

Revert "fix: checking if tokenizer is in cache before downloading from HF" by @yhyang201 in https://github.com/sgl-project/sglang/pull/14808

[Difussion] Refactor diffusion fuse qkv and support qwen-image by @BBuf in https://github.com/sgl-project/sglang/pull/14793

[Router-GO] implement a Go SGLang Router - OpenAI Compatible API Server by @whybeyoung in https://github.com/sgl-project/sglang/pull/14770

[model-gateway] Dynamically Populate Tool Call Parser Choices by @xuwenyihust in https://github.com/sgl-project/sglang/pull/14807

Support HTTP response status code prometheus metrics by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14710

Fix router keep nonzero metrics after worker is deleted by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14819

Tiny fix incorrect worker removal command by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14822

[NPU] bug fix for mtp and w4a8 by @liupeng374 in https://github.com/sgl-project/sglang/pull/14806

[CI] fix UT success check in test_eagle_infer_beta_dp_attention.py by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14831

Fix CI registry scan to only check test/registered directory by @alisonshao in https://github.com/sgl-project/sglang/pull/14812

[model-gateway] add anthropic message api spec by @slin1237 in https://github.com/sgl-project/sglang/pull/14834

Fix tiny typo in multimodal_gen/README.md by @wplf in https://github.com/sgl-project/sglang/pull/14830

Tiny support customizing Prometheus duration buckets by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14716

Tiny support engine response http status statistics in router by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14712

[CI] Reduce stage-b auto-partition from 4 to 2 by @alisonshao in https://github.com/sgl-project/sglang/pull/14769

Apply back moe_sum_reduce for fused_marlin_moe by @ispobock in https://github.com/sgl-project/sglang/pull/14829

[diffusion] parallel: pad tokens for video models under sp by @mickqian in https://github.com/sgl-project/sglang/pull/14833

[diffusion] CI: use unified sampling_params for CI by @mickqian in https://github.com/sgl-project/sglang/pull/14045

[Auto Sync] Update tool_chat_template_deepseekv31.jinja (20251210) by @zhyncs in https://github.com/sgl-project/sglang/pull/14837

Revert transformers to 4.57.1 by @yhyang201 in https://github.com/sgl-project/sglang/pull/14801

[model-gateway] Fix incompatible metric comparison in PowerOfTwo policy by @ppraneth in https://github.com/sgl-project/sglang/pull/14823

[bugfix] qwen25-VL support lora by @SYChen123 in https://github.com/sgl-project/sglang/pull/14638

fix lora target all + csgmv backend by @b8zhong in https://github.com/sgl-project/sglang/pull/14796

[model-gateway] adds default implementations to RouterTrait in mod.rs by @slin1237 in https://github.com/sgl-project/sglang/pull/14841

[AMD] Add model to AMD nightly test by @michael-amd in https://github.com/sgl-project/sglang/pull/14442

Treat unittest SkipTest exception as pass instead of as failure by @byjiang1996 in https://github.com/sgl-project/sglang/pull/14847

[model-gateway] code clean up on oai router by @slin1237 in https://github.com/sgl-project/sglang/pull/14850

[model-gateway] fix import order in oai conversation by @slin1237 in https://github.com/sgl-project/sglang/pull/14851

fix fp8 gemm nightly CI by @b8zhong in https://github.com/sgl-project/sglang/pull/14844

fix: restrict cache validation behaviors to CI only by @alisonshao in https://github.com/sgl-project/sglang/pull/14849

Fix CUDA version handling in ci_install_deepep.sh by @merrymercy in https://github.com/sgl-project/sglang/pull/14854

Fix TestGLM41VPPAccuracy test flakiness by @byjiang1996 in https://github.com/sgl-project/sglang/pull/14848

Minor code style fix for dllm by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14836

Enable TP for Mamba-based models by @roikoren755 in https://github.com/sgl-project/sglang/pull/14811

[CI] Temp disable gb200 test by @Fridge003 in https://github.com/sgl-project/sglang/pull/14865

Refactor Marlin MoeRunner by @trangdough in https://github.com/sgl-project/sglang/pull/14554

[6/n] Fix num_token_non_padded computation in prefill by @yuchengz816-bot in https://github.com/sgl-project/sglang/pull/14313

Remove myself to test CI gate issue by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/14871

fix: creating blobs only once for publish trace retries by @dougyster in https://github.com/sgl-project/sglang/pull/14845

Move and update MindSpore docs, make it appear on the online documentation by @wangtiance in https://github.com/sgl-project/sglang/pull/14861

fix nightly vlm ci : restore original eval for requests without regex by @yhyang201 in https://github.com/sgl-project/sglang/pull/14875

Only count limitations for previous runs that reaches the test stages by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/14856

[CI][BUG] fix ib setup for disaggregation hicache test by @luketong777 in https://github.com/sgl-project/sglang/pull/14877

[Fix] Remove unused import from test_disaggregation_hicache.py by @ShangmingCai in https://github.com/sgl-project/sglang/pull/14880

fix: adding temporary bypass for nightly tests by @dougyster in https://github.com/sgl-project/sglang/pull/14876

Avoid deleting entire cache for missing shards (#14754 follow-up) by @alisonshao in https://github.com/sgl-project/sglang/pull/14853

Tiny add more error info for bench_serving by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14827

Tiny support range ratio in GSP in bench serving by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14828

[diffusion] feat: enable torch compile to eliminate GPU bubble by @AichenF in https://github.com/sgl-project/sglang/pull/13641

[NPU]dsv3.2 cp for npu by @liupeng374 in https://github.com/sgl-project/sglang/pull/14541

[diffusion] feat: support sageattn & sageattn3 backend by @mickqian in https://github.com/sgl-project/sglang/pull/14878

[Ascend]Support of piecewise graph compilation for prefill on NPU by @Vladimir221 in https://github.com/sgl-project/sglang/pull/12287

Introduce server_fixtures in sglang.test by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14899

[diffusion] UX: suppress excessive loggers by @mickqian in https://github.com/sgl-project/sglang/pull/14900

Tiny refactor cleanup WorkflowContext.get_or_err by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14890

Tiny clean router load report logic by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14889

[model-gateway] code clean up on oai router in responses by @slin1237 in https://github.com/sgl-project/sglang/pull/14852

[model-gateway] fix annotation error and code formating by @slin1237 in https://github.com/sgl-project/sglang/pull/14910

[model-gateway] fix imports and delete unused code by @slin1237 in https://github.com/sgl-project/sglang/pull/14911

[docs] Fix kernel name by @almaslof in https://github.com/sgl-project/sglang/pull/14887

[SMG][DS32][fix] support dsv32, add role developer by @jimmy-evo in https://github.com/sgl-project/sglang/pull/14307

Update CI_PERMISSIONS.json by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/14917

[FIX][DS32]openai protocol: support openai message role: developer by @jimmy-evo in https://github.com/sgl-project/sglang/pull/14304

[loader] enable private loader by @yinghai in https://github.com/sgl-project/sglang/pull/14620

chore: bump SGLang version to 0.5.6.post2 by @sglang-bot in https://github.com/sgl-project/sglang/pull/14858

extend timeout for b200 test by @b8zhong in https://github.com/sgl-project/sglang/pull/14925

ci: adding more nightly tests to bot bump workflows by @dougyster in https://github.com/sgl-project/sglang/pull/14928

update mistral detector by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/14921

support non-disturbing remote-instance-weight-loader by @amysaq2023 in https://github.com/sgl-project/sglang/pull/13125

[refactor] Update reasoning parameter to require_reasoning by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/14922

[CPU] layernorm & fused add-layernorm kernels by @ZailiWang in https://github.com/sgl-project/sglang/pull/14074

Add retry logic for scheduled CI tests by @alisonshao in https://github.com/sgl-project/sglang/pull/14771

[CI] Add Mistral Large 3 Eagle nightly performance test by @alisonshao in https://github.com/sgl-project/sglang/pull/14525

fix: handle Jinja2 template errors as client errors in OpenAIServingChat by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/14748

Fix black formatting in ci_utils.py by @alisonshao in https://github.com/sgl-project/sglang/pull/14932

[bugfix] fix TBO crashes when attn_tp_size > 1 by @yuhyao in https://github.com/sgl-project/sglang/pull/13730

fix: making the publish trace error check broader by @dougyster in https://github.com/sgl-project/sglang/pull/14931

[CI]add nightly CI for glm4v_moe arch model by @zminglei in https://github.com/sgl-project/sglang/pull/14927

Check KV4 compatibility with attention backends and add KV4 support to the attention_backend doc by @JackChuang in https://github.com/sgl-project/sglang/pull/14467

Re-org eagle unit tests by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14909

Super tiny remove sgl_router_active_workers by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14891

remove dpsk3.2 sys prompt by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/14923

[DLLM] Add documentation for diffusion LLMs by @ClawSeven in https://github.com/sgl-project/sglang/pull/14358

[RL] refactor flash rl weight reload in sglang by @AniZpZ in https://github.com/sgl-project/sglang/pull/14870

[PP] Refactor PP to async mode by @XucSh in https://github.com/sgl-project/sglang/pull/11852

[Fix] Enable applying different LoRA adapters to different transformers in multi-transformer pipelines by @Prozac614 in https://github.com/sgl-project/sglang/pull/14839

[model-gateway] optimize worker selection by @ppraneth in https://github.com/sgl-project/sglang/pull/14894

Fix negative duration panic in token bucket wait time calculation by @xiaguan in https://github.com/sgl-project/sglang/pull/14941

Tiny add router e2e duration histogram by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14892

Tiny add e2e http request arrival metric by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14893

Super tiny remove non-updated sgl_router_worker_load by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14888

Super tiny move error.rs by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14944

direct register custom op for mm_fp4 by @b8zhong in https://github.com/sgl-project/sglang/pull/13699

fix: trtllm mha attention auto-selection on sm120 by @b8zhong in https://github.com/sgl-project/sglang/pull/14842

Super tiny refactor error.rs logic by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14949

[NPU] optimization for dsv3.2 by @ZhengdQin in https://github.com/sgl-project/sglang/pull/14572

[NVIDIA] Enable TRTLLM BF16 MoE on Blackwell GPUs by @samuellees in https://github.com/sgl-project/sglang/pull/13798

[Fix] suppress remote weight loading engine w/o mooncake installed by @ZailiWang in https://github.com/sgl-project/sglang/pull/14937

enable flashinfer-jit-cache in image build and ci install to speed up model launch by @gongwei-130 in https://github.com/sgl-project/sglang/pull/14959

[diffusion] chore: minor code cleanups and improve logging by @mickqian in https://github.com/sgl-project/sglang/pull/14916

[Diffusion] upgrade cache-dit for better compatiblity by @DefTruth in https://github.com/sgl-project/sglang/pull/14534

[1/N] Update doc of Pipeline Parallelism by @ShangmingCai in https://github.com/sgl-project/sglang/pull/14985

[PD] Add decode PP event loop for PD disaggregation by @bluecoffee8 in https://github.com/sgl-project/sglang/pull/14945

[Diffusion] Tiny fix Docker Hub link in installation documentation by @BBuf in https://github.com/sgl-project/sglang/pull/14987

Update CODEOWNERS for multimodal_gen by @mickqian in https://github.com/sgl-project/sglang/pull/14995

[model-gateway] refactor: extract workflow engine to src/workflow module by @slin1237 in https://github.com/sgl-project/sglang/pull/14996

[model-gateway] feat: add DAG parallel execution support and workflow optimization by @slin1237 in https://github.com/sgl-project/sglang/pull/14999

[model-gateway] fix: handle workflow deadlock and optimize cycle detection by @slin1237 in https://github.com/sgl-project/sglang/pull/15000

[model-gateway] refactor: workflow engine cleanup and minor optimization by @slin1237 in https://github.com/sgl-project/sglang/pull/15001

Fix CI by reverting incorrect metric check logic by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/15004

Super tiny extract route_typed_request_once by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14951

Revert several PRs by @zhyncs in https://github.com/sgl-project/sglang/pull/14958

Add KV4-capable backend flashmla and update server args by @JackChuang in https://github.com/sgl-project/sglang/pull/14989

Refactor of http and engine entrypoints to allow custom override by @merrymercy in https://github.com/sgl-project/sglang/pull/14869

Update ci permission by @merrymercy in https://github.com/sgl-project/sglang/pull/15014

[model-gateway] refactor: unify worker management into modular workflow structure by @slin1237 in https://github.com/sgl-project/sglang/pull/15010

Tune triton fused moe for the case of glm-4.6-fp8 b200 tp4 by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/15020

Feature/Fix multi lora scheduler blocking issue and evict LoRA None lastly by @ConnorLi96 in https://github.com/sgl-project/sglang/pull/14795

[registry] Add a strict mode to model registration by @yinghai in https://github.com/sgl-project/sglang/pull/14933

Super tiny remove unused argument by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14966

Super tiny fix confusing slash_command_handler hint by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14976

Super tiny add gsp-fast-prepare by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14992

Tiny extract SchedulerWatchdog by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15021

Add soft watchdogs to debug soft hangs by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15023

Clean up server args and engine startup processes by @merrymercy in https://github.com/sgl-project/sglang/pull/15015

tiny update: use rope kernel in sgl-kernel for amd by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/14955

Tiny remove the duplicate function in spec v2 by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14957

Fix regression caused by fa3 block_table by @wenscarl in https://github.com/sgl-project/sglang/pull/15009

Add a special label for b200 CI runner that can run kernel tests by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/15033

[CI]Add gb200 runner back by @Fridge003 in https://github.com/sgl-project/sglang/pull/15024

Fix decode OOM caused by retraction by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14939

Super tiny remove unused log_request by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15035

Add code field and unify error responses for router by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15028

Tiny unify grpc existing error responses into new format by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15030

Tiny change http router response format to unify by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15031

Provide more fine grained error reason for reqwest error by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15032

Add error code in prometheus metrics and add X-SMG-Error-Code header by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15036

Add sgl_router_attempt_http_responses_total for single attempt information by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15037

call check_quantized_moe_compatibility after initialize by @chunyuan-w in https://github.com/sgl-project/sglang/pull/13876

Mistral Large 3 NVFP4 support by @dcampora in https://github.com/sgl-project/sglang/pull/14485

[diffusion] fix: use NDRotaryEmbedding in flux_2 by @mickqian in https://github.com/sgl-project/sglang/pull/15034

[Fix] Disable trtllm moe backend for draft model for a qucik fix by @samuellees in https://github.com/sgl-project/sglang/pull/15002

feat: Improve LoRA compatibility by adding unified format detection and diffusers-based normalization by @MikukuOvO in https://github.com/sgl-project/sglang/pull/14659

feature: adding nightly wheel workflow and indexer by @dougyster in https://github.com/sgl-project/sglang/pull/14924

Fix GLM-4.6 tool calls don't support streaming output for arguments i… by @cynial in https://github.com/sgl-project/sglang/pull/13989

[PP Prefill][NIXL] Fix PP mode transfer completion tracking to wait for all ranks by @YAMY1234 in https://github.com/sgl-project/sglang/pull/15027

[NPU] perf update with kvcache nz & w4a8 quant by @liupeng374 in https://github.com/sgl-project/sglang/pull/14423

Clean up GDN Init by @hebiao064 in https://github.com/sgl-project/sglang/pull/14855

[VLM] Support VLM ViT Piecewise CUDA Graph by @yuan-luo in https://github.com/sgl-project/sglang/pull/14422

Fix load metric not updated when using guard by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15059

Fix double decrease load by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15060

[Diffusion] Add multimodal gen profiling doc by @BBuf in https://github.com/sgl-project/sglang/pull/15069

Fix IMA with flashinfer + spec + topk & Add radix attention test cases for eagle by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13740

[diffusion] doc: update profiling.md with output location details by @mickqian in https://github.com/sgl-project/sglang/pull/15072

Tiny adjust CI run suite by @hnyls2002 in https://github.com/sgl-project/sglang/pull/15074

Fix spec info's filter when reqs are finished right after prefill by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14742

[model-gateway] Simplify error response creation by @slin1237 in https://github.com/sgl-project/sglang/pull/15079

[bug] fix grpc secheduler launcher breaking change by @slin1237 in https://github.com/sgl-project/sglang/pull/15080

feature: ci failure monitor improvements by @dougyster in https://github.com/sgl-project/sglang/pull/15055

fix: adding schedule for nightly wheel by @dougyster in https://github.com/sgl-project/sglang/pull/15054

fix flaky image access in ci by switching to raw content url by @yhyang201 in https://github.com/sgl-project/sglang/pull/14940

[scheduler] enhance scheduler in dp_attention mixed case with spec by @liupeng374 in https://github.com/sgl-project/sglang/pull/14201

add transformers version validation for glm-4.6v moe models by @yhyang201 in https://github.com/sgl-project/sglang/pull/14998

[model-gateway] Avoid MCP Server Initialization Issue by @xuwenyihust in https://github.com/sgl-project/sglang/pull/15065

Add nightly accuracy test for DeepSeek V3.2 by @Fridge003 in https://github.com/sgl-project/sglang/pull/14935

fix: dpskv32 chat history processing, default drop_thinking to true by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/15064

[model-gateway] Refactor worker steps and add update workflow by @slin1237 in https://github.com/sgl-project/sglang/pull/15085

Add sglang:decode_sum_seq_lens metric by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15066

[Doc][TPU]add sglang-jax tpu docs by @JamesBrianD in https://github.com/sgl-project/sglang/pull/15056

[Fix] avoid stream sync in _compute_mrope_positions by @narutolhy in https://github.com/sgl-project/sglang/pull/14956

Support prefill max requests limitation by @fzyzcjy in https://github.com/sgl-project/sglang/pull/14993

[model-gateway] Remove unused TokenizerMetrics to reduce CPU overhead by @slin1237 in https://github.com/sgl-project/sglang/pull/15087

[Fix] Environment variable SGL_* is deprecated by @miter6 in https://github.com/sgl-project/sglang/pull/14943

[model-gateway] Fix metric emission gaps and name mismatch by @slin1237 in https://github.com/sgl-project/sglang/pull/15093

[model-gateway] Add circuit breaker and discovery watcher metrics by @slin1237 in https://github.com/sgl-project/sglang/pull/15094

[model-gateway] optimize metric labels to avoid unnecessary allocations by @slin1237 in https://github.com/sgl-project/sglang/pull/15095

Fix issue not reported when load decrement is incorrect by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15061

Avoid confusing zero value metric when worker is removed by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15096

[NPU][CI] change de trigger of release image workflow by @monkeyLoveding in https://github.com/sgl-project/sglang/pull/14969

[ci] Move dpsk-r1-fp4 b200 test to stage b by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/15084

[scheduler] remove scheduler allgather for best throughout by @liupeng374 in https://github.com/sgl-project/sglang/pull/14294

[NPU] bug fix for multi stream by @liupeng374 in https://github.com/sgl-project/sglang/pull/15048

Introduce native kv cache move by @hnyls2002 in https://github.com/sgl-project/sglang/pull/15108

[diffusion] feat support resolution check for video model by @Brain97 in https://github.com/sgl-project/sglang/pull/14881

[Diffusion] Tiny fix _templated_ring_attention bug by @BBuf in https://github.com/sgl-project/sglang/pull/15053

[Diffusion] feat: Add support for additional sampling parameters in video generation API by @BBuf in https://github.com/sgl-project/sglang/pull/15062

diffusion: support webui by @wplf in https://github.com/sgl-project/sglang/pull/14961

[kernel][moe] add moe topk fast by @thenumberouscode in https://github.com/sgl-project/sglang/pull/13969

feat: support EPD disaggregation by @gty111 in https://github.com/sgl-project/sglang/pull/12263

[CI] Add disaggregation decode PP test by @ShangmingCai in https://github.com/sgl-project/sglang/pull/15114

[Diffusion] Refactor fuse qkv with QKVParallelLinear linear by @BBuf in https://github.com/sgl-project/sglang/pull/15090

[model-gateway] Add new SMG metrics architecture with 6 layers by @slin1237 in https://github.com/sgl-project/sglang/pull/15106

Fix tensor mismatch error in sepc + topk > 1 + page_size > 1 by @ZeldaHuang in https://github.com/sgl-project/sglang/pull/14874

[model-gateway] Implement Layer 1 HTTP metrics instrumentation by @slin1237 in https://github.com/sgl-project/sglang/pull/15121

feat(metrics): implement Layer 2 router metrics (smg_router_*) by @slin1237 in https://github.com/sgl-project/sglang/pull/15124

Fix Mamba2-based models' default attention backend by @roikoren755 in https://github.com/sgl-project/sglang/pull/15117

Add NanoV3 reasoning parser support by @danielafrimi in https://github.com/sgl-project/sglang/pull/15113

[hotfix]: Add missing args for 3FS bench_client.py by @hzh0425 in https://github.com/sgl-project/sglang/pull/14791

feature: ci failure monitor slack bot by @dougyster in https://github.com/sgl-project/sglang/pull/15110

[model-gateway] add streaming metrics (TTFT, TPOT, tokens, duration) for gRPC router by @slin1237 in https://github.com/sgl-project/sglang/pull/15125

[refactor] Move trtllm_fp8_kv_kernel to triton_ops directory by @harvenstar in https://github.com/sgl-project/sglang/pull/15044

feat(gateway): Add server-side TLS support by @Ratish1 in https://github.com/sgl-project/sglang/pull/15052

[model-gateway] Parallelize metrics requests by @ppraneth in https://github.com/sgl-project/sglang/pull/14953

Super tiny cleanup circuit breaker code by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15098

Fix circuit breaker wrong metrics by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15099

[NSA] Fix NSA backend assertion error when running DeepSeek-V3.2 PP with radix-cache by @YAMY1234 in https://github.com/sgl-project/sglang/pull/15086

[Diffusion] Fix default resolution 720p width from 1080 to 1280 by @BBuf in https://github.com/sgl-project/sglang/pull/15058

fix: adjusting frequency for ci failure monitor by @dougyster in https://github.com/sgl-project/sglang/pull/15134

Add cyb70289 to CI permissions by @cyb70289 in https://github.com/sgl-project/sglang/pull/14938

Fix cache aware wrong routing caused by incorrect load tracking by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15101

Fix H200 CI by commenting out Warmup Weights and JIT Compilation by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/15139

docs: update usage by @zhyncs in https://github.com/sgl-project/sglang/pull/15142

[Qwen3-next] support mamba radix cache for overlap scheduler by @hanming-lu in https://github.com/sgl-project/sglang/pull/14792

Enable TRT Allreduce Fusion by default for compatible models by @b8zhong in https://github.com/sgl-project/sglang/pull/14764

[VLM] Support chunked vit attention by @yuan-luo in https://github.com/sgl-project/sglang/pull/14907

[model-gateway] Add Layer 3 worker metrics (smg_worker_*) by @slin1237 in https://github.com/sgl-project/sglang/pull/15130

[model-gateway] upgrade axum and axum server by @slin1237 in https://github.com/sgl-project/sglang/pull/15146

[model-gateway] Add streaming metrics for harmony gRPC router by @slin1237 in https://github.com/sgl-project/sglang/pull/15147

ci: adding errors to Github summary by @dougyster in https://github.com/sgl-project/sglang/pull/14778

Fix import warnings by @merrymercy in https://github.com/sgl-project/sglang/pull/15144

fix: move ci-bot by @dougyster in https://github.com/sgl-project/sglang/pull/15154

[model-gateway] add mcp and discovery metrics by @slin1237 in https://github.com/sgl-project/sglang/pull/15156

Tiny improve summary text in bench_one_batch_server.py by @hnyls2002 in https://github.com/sgl-project/sglang/pull/15158

fix CompressedTensorsW8A8Int8 min_capability by @mmdbhs in https://github.com/sgl-project/sglang/pull/13914

[diffusion]: support mutli image input and qwen-image-edit-2509 by @yhyang201 in https://github.com/sgl-project/sglang/pull/15005

feature: PR wheel by @dougyster in https://github.com/sgl-project/sglang/pull/15170

[CPU] Add Gemma3RMSNorm kernel in sgl-kernel and add ut by @blzheng in https://github.com/sgl-project/sglang/pull/9324

fix: adding date and fixing release name issue by @dougyster in https://github.com/sgl-project/sglang/pull/15174

Fused two elementwise kernels for k_nope and k_pe concat by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/14862

Fix num running requests (load) wrong cleared for ongoing requests by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15116

[Feature] Fuse mrope all in 1 kernel by @DarkSharpness in https://github.com/sgl-project/sglang/pull/14906

chore: change npu pr-test a2 runner by @Goalina in https://github.com/sgl-project/sglang/pull/15152

[Diffusion] Cache dit support parallel by @BBuf in https://github.com/sgl-project/sglang/pull/15163

[diffusion] fix: Fixed pytorch non-writable array warning by @RuixiangMa in https://github.com/sgl-project/sglang/pull/15017

[diffusion] fix: fix video model sp when resolution is not specified by @mickqian in https://github.com/sgl-project/sglang/pull/15047

[model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels by @slin1237 in https://github.com/sgl-project/sglang/pull/15160

Add missing assertion in NemotronH path by @roikoren755 in https://github.com/sgl-project/sglang/pull/15193

[Diffusion] Fix AttributeError in _build_parallelism_config when acce… by @BBuf in https://github.com/sgl-project/sglang/pull/15196

[diffusion] chore: minor code cleanups by @mickqian in https://github.com/sgl-project/sglang/pull/15190

[Diffusion] Zimage support pack qkv by @BBuf in https://github.com/sgl-project/sglang/pull/15191

fix(attention): Prevent trtllm_mha auto-selection with eagle3 speculative decoding by @Ratish1 in https://github.com/sgl-project/sglang/pull/15127

[Diffusion] Z-Image FFN pack gate and up proj by @BBuf in https://github.com/sgl-project/sglang/pull/15201

[NPU][eagle3] support qwen eagle3 on NPU by @Liwansi in https://github.com/sgl-project/sglang/pull/14820

Add cache for flashinfer installation by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/15153

feature: create docker image from pr branch by @dougyster in https://github.com/sgl-project/sglang/pull/15185

chore: update CI_PERMISSIONS by @zhyncs in https://github.com/sgl-project/sglang/pull/15212

[Feature] npu support enable_torch_compile for torchair backend by @XDaoHong in https://github.com/sgl-project/sglang/pull/13410

Add EPD disaggregation doc by @gty111 in https://github.com/sgl-project/sglang/pull/15224

[Bugfix][Tool Call] Add null system prompt to support tool system prompt by @Muqi1029 in https://github.com/sgl-project/sglang/pull/15092

[AMD CI] Temporarily disable 2 gpu accuracy test. by @saienduri in https://github.com/sgl-project/sglang/pull/15204

[BugFix] Fix CPU inference failure by @cyb70289 in https://github.com/sgl-project/sglang/pull/15231

[AMD CI] Fix typo. by @saienduri in https://github.com/sgl-project/sglang/pull/15229

[Feature] Add AIME25 dataset support for SGLang simple_eval by @yurekami in https://github.com/sgl-project/sglang/pull/14990

Remove duplicate bs=1 in nightly benchmark by @Fridge003 in https://github.com/sgl-project/sglang/pull/15162

[Diffusion] fix pack qkv opt break tensor parallel by @BBuf in https://github.com/sgl-project/sglang/pull/15225

[Qwen3-next] Add PD disaggregation support for mamba with extra_buffer by @ShangmingCai in https://github.com/sgl-project/sglang/pull/15180

[bugfix][quark] Fixed an issue where per_token could not be properly recognized when the token count was 1. by @haoyangli-amd in https://github.com/sgl-project/sglang/pull/14415

Adding tool calling and reasoning parser support for Intern-S1 by @KennyYao2001 in https://github.com/sgl-project/sglang/pull/14866

fix: removing latest-sglang=1 by @dougyster in https://github.com/sgl-project/sglang/pull/15220

Increase timeout for TestDeepseekV3MTP for potential DeepGEMM cold start by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/15239

[CPU] Add 4D input support for ROPE in sgl-kernel by @blzheng in https://github.com/sgl-project/sglang/pull/9337

Support piecewise cuda graph for fused marlin moe by @ispobock in https://github.com/sgl-project/sglang/pull/15100

Enhance runtime memory check in CI by @hnyls2002 in https://github.com/sgl-project/sglang/pull/15192

[Diffusion] Use current_platform.device_type to replace hard-coded cuda device by @yeahdongcn in https://github.com/sgl-project/sglang/pull/15232

[diffusion] doc: update profiling.md by @mickqian in https://github.com/sgl-project/sglang/pull/15270

[CI] Improve flaky 4 GPU test success rate by @ShangmingCai in https://github.com/sgl-project/sglang/pull/15234

Fix accuracy issue when using a16w16 mla_decode_fwd by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/14936

[AMD] Support fused_rms_mxfp4_quant in the prefill stage for DeepSeek-R1-MXFP4 by @yichiche in https://github.com/sgl-project/sglang/pull/14975

[misc] Upgrade cutedsl to 4.3.1 by @Fridge003 in https://github.com/sgl-project/sglang/pull/14857

Fix lint by @Fridge003 in https://github.com/sgl-project/sglang/pull/15281

Remove incorrect BlockRemoved event emission during node splits by @nealvaidya in https://github.com/sgl-project/sglang/pull/14934

support non disturbing remote instance weight loader v2 by @amysaq2023 in https://github.com/sgl-project/sglang/pull/14997

[sgl-kernel] Update flashmla to include fp8 sparse_mla optimizations by @hlu1 in https://github.com/sgl-project/sglang/pull/15242

Fix lora doc by @Fridge003 in https://github.com/sgl-project/sglang/pull/15282

Fix test_pp_single_node.py estimated time from 800s to 500s by @alisonshao in https://github.com/sgl-project/sglang/pull/15291

fix: skipping TestEPDDisaggregationOneEncoder test by @dougyster in https://github.com/sgl-project/sglang/pull/15292

Revert "[misc] Upgrade cutedsl to 4.3.1 (#14857)" by @zhyncs in https://github.com/sgl-project/sglang/pull/15293

[NVIDIA] Fixes for NVFP4 all-gather with spec decoding by @trevor-m in https://github.com/sgl-project/sglang/pull/15280

[NPU] fix for NPU memory settings logic by @iforgetmyname in https://github.com/sgl-project/sglang/pull/15258

fix: moving decorator to header by @dougyster in https://github.com/sgl-project/sglang/pull/15297

Minor style fixes to the scheduler.py by @merrymercy in https://github.com/sgl-project/sglang/pull/15218

[Test] Update LoRA eviction policy tests to match current behavior by @alisonshao in https://github.com/sgl-project/sglang/pull/15283

[BugFix] fix gptq_marlin_gemm has no parameter called b_bias by @ehuaa in https://github.com/sgl-project/sglang/pull/13571

fix(function_call): fallback to decode when batch decode options differ by @luqitao in https://github.com/sgl-project/sglang/pull/15155

Add Ollama-compatible API endpoints + Smart Router by @alisonshao in https://github.com/sgl-project/sglang/pull/14376

[DeepSeekV3.2] Add pure TP+MTP test by @ashtonchew in https://github.com/sgl-project/sglang/pull/15088

[Perf] Enable Flashinfer autotune by default by @elvischenv in https://github.com/sgl-project/sglang/pull/14357

Update FP4 GEMM Benchmark by @b8zhong in https://github.com/sgl-project/sglang/pull/14449

Revert "direct register custom op for mm_fp4 (#13699)" by @b8zhong in https://github.com/sgl-project/sglang/pull/15284

diffusion: Add sampling parameters and model info endpoint to OpenAI API by @niehen6174 in https://github.com/sgl-project/sglang/pull/15071

[PP] Add pp support for Qwen3-VL by @XucSh in https://github.com/sgl-project/sglang/pull/12333

[Hotfix] Fix required enable_mamba_track argument for Flashinfer autotune path by @elvischenv in https://github.com/sgl-project/sglang/pull/15314

Fix gpu-fault when running mtp in eager mode by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/15233

[bug fix][pp] fix qwen3 model load by @XucSh in https://github.com/sgl-project/sglang/pull/15223

Fix the accuracy issue when running mxfp4 dsv3 model and enable ep by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/15304

fix qwenvl compressed tensors quantization weight loader by @LHXuuu in https://github.com/sgl-project/sglang/pull/11914

[Piecewise CUDA Graph] Support INT8 by @b8zhong in https://github.com/sgl-project/sglang/pull/14918

[bug fix][pp] fix weight load for qwen2.5-vl by @XucSh in https://github.com/sgl-project/sglang/pull/15138

[Diffusion] Add flux2 tp2 test in ci to avoid break diffusion tensor parallel by @BBuf in https://github.com/sgl-project/sglang/pull/15237

[Diffusion] Enhance trace export with gzip and integrity check by @BBuf in https://github.com/sgl-project/sglang/pull/15326

Add cuda_graph_forward_passes_total and num_retracted_reqs_total by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15189

Add realtime token counter metrics by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15198

Tiny dump native stacktraces in watchdog by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15222

Super tiny rename failure_count for consistency by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15186

[PP] Minor code cleanup for Pipeline Parallelism by @ShangmingCai in https://github.com/sgl-project/sglang/pull/15329

tiny unify environ usage by @hnyls2002 in https://github.com/sgl-project/sglang/pull/15335

[model-gateway] reduce cpu overhead by @slin1237 in https://github.com/sgl-project/sglang/pull/15316

[model-gateway] optimize worker registry and reduce lock contention in grpc client fetch by @slin1237 in https://github.com/sgl-project/sglang/pull/15336

[DeepSeek-V32]Update nightly performance benchmark by @Fridge003 in https://github.com/sgl-project/sglang/pull/15308

Fix dp run error with fp8-kv enable in high concurrency test by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/15241

fix: prevent points regex from matching checkpoints/endpoints by @xvlincaigou in https://github.com/sgl-project/sglang/pull/15120

Fix condition check for require_gathered_buffer by @ch-wan in https://github.com/sgl-project/sglang/pull/15328

Reserve more memory for DeepSeekOCR model and adjust server start timeout for DeepGEMM to reduce flakiness by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/15277

[CI] Migrate LoRA tests to test/registered/lora/ by @alisonshao in https://github.com/sgl-project/sglang/pull/15176

Add request-level timestamp for when prefill finishes by @scottjlee in https://github.com/sgl-project/sglang/pull/14860

[Deepseek V3.2] Support Overlap Spec + NSA by @b8zhong in https://github.com/sgl-project/sglang/pull/15307

[VLM] Support cos sin cache for Qwen3-VL & GLM-4.1V by @yuan-luo in https://github.com/sgl-project/sglang/pull/15205

Feature/trtllm mha workspace size configurable #15089 by @baonudesifeizhai in https://github.com/sgl-project/sglang/pull/15131

feat: DeepSeek-V3.2 Streaming tool call output by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/15278

Add doc for qwen3 next by @yizhang2077 in https://github.com/sgl-project/sglang/pull/15337

fix: adjust time for test_epd_disaggregation.py by @dougyster in https://github.com/sgl-project/sglang/pull/15354

Mistral Large 3 NVFP4 TRTLLM MoE support by @elvischenv in https://github.com/sgl-project/sglang/pull/15049

unified management of environment variables for vlm cuda ipc transport by @yhyang201 in https://github.com/sgl-project/sglang/pull/14501

Split test_piecewise_cuda_graph.py to optimize CI resource usage by @alisonshao in https://github.com/sgl-project/sglang/pull/15290

Fix issue: ENABLE_BELOW_SM90 cannot be enabled on aarch64 CPU by @MarcoDWei in https://github.com/sgl-project/sglang/pull/12967

[PP] Fix dynamic chunking strategy for PP by @ShangmingCai in https://github.com/sgl-project/sglang/pull/15372

[model-gateway] Replace PolicyRegistry RwLock with DashMap for lock-free policy lookups by @slin1237 in https://github.com/sgl-project/sglang/pull/15361

Monkey patch deepseek-ocr's v_head_dim by @hnyls2002 in https://github.com/sgl-project/sglang/pull/15384

Fix gpt-oss yarn with truncate argument by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14270

Fix warp illegal instruction in kimi k2 thinking PCG by @BBuf in https://github.com/sgl-project/sglang/pull/15306

[bug fix][pp] fix inconsistent latency between tp by @XucSh in https://github.com/sgl-project/sglang/pull/15379

[sgl-kernel][1/2] Fused qk_norm_rope for GLM4.6 by @Kevin-XiongC in https://github.com/sgl-project/sglang/pull/15141

Clean up init function of the scheduler and event loop for PD by @merrymercy in https://github.com/sgl-project/sglang/pull/15298

[perf]optimize w4afp8 kernel on deepseek-v3-0324 by @Bruce-x-1997 in https://github.com/sgl-project/sglang/pull/12921

[Diffusion] Fix sglang generate --perf-dump-path to include per-denoising-step timings by @BBuf in https://github.com/sgl-project/sglang/pull/15397

Tiny fix unknown route in prometheus metrics by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15404

Support GPU execution time breakdown by forward mode metrics by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15396

Tiny extract ModelRunnerOutput by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15400

Super tiny add moe_ep_rank to prometheus labels by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15407

Support EPLB balancedness prometheus metric without GPU->CPU synchronize by @fzyzcjy in https://github.com/sgl-project/sglang/pull/15401

[PP] Add dynamic chunking PP test by @ShangmingCai in https://github.com/sgl-project/sglang/pull/15395

[Tiny]Add warning for deepgemm on Blackwell by @Fridge003 in https://github.com/sgl-project/sglang/pull/15352

Update benchmarks to use HF token from environment. by @FrankD412 in https://github.com/sgl-project/sglang/pull/15421

[Performance] optimize NSA backend metadata computation for multi-step speculative decoding by @Johnsonms in https://github.com/sgl-project/sglang/pull/14781

[AMD] Clear pre-built AITER kernels and warmup to prevent segfaults and test timeouts by @sunxxuns in https://github.com/sgl-project/sglang/pull/15318

multimodal: precompute hash for MultimodalDataItem by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/14354

tiny fix lint on main by @b8zhong in https://github.com/sgl-project/sglang/pull/15424

feat(dsv32): better error handling for DeepSeek-v3.2 encoder by @jimmy-evo in https://github.com/sgl-project/sglang/pull/14353

Support using different attention backend for draft decoding. by @pyc96 in https://github.com/sgl-project/sglang/pull/14843

[DLLM] Add CI for diffusion LLMs by @ClawSeven in https://github.com/sgl-project/sglang/pull/14723

chore: update CI_PERMISSIONS by @zhyncs in https://github.com/sgl-project/sglang/pull/15431

[Deepseek V3.2] Fix Deepseek MTP in V1 mode by @b8zhong in https://github.com/sgl-project/sglang/pull/15429

[DLLM] Fix dLLM regression by @ClawSeven in https://github.com/sgl-project/sglang/pull/15371

[diffusion] profiling: add bench_serving.py and VBench by @mickqian in https://github.com/sgl-project/sglang/pull/15410

[Feature] Xiaomi MiMo-V2-Flash day0 support by @acelyc111 in https://github.com/sgl-project/sglang/pull/15207

fix mindspore import warning by @b8zhong in https://github.com/sgl-project/sglang/pull/15287

Update readme by @merrymercy in https://github.com/sgl-project/sglang/pull/15425

Add customized sampler registration by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/15423

[Fix]: Refactor _build_req_from_sampling to use shallow_asdict by @cocoshe in https://github.com/sgl-project/sglang/pull/13782

[amd] Add deterministic all-reduce kernel for AMD (ROCm) by @sunxxuns in https://github.com/sgl-project/sglang/pull/15340

[AMD] Enable all diffusion models and fix encoder loading on MI325 by @zyzshishui in https://github.com/sgl-project/sglang/pull/13760

[sgl-kernel] chore: update deepgemm version by @FlamingoPg in https://github.com/sgl-project/sglang/pull/13402

fix: unreachable error check in retraction by @alphabetc1 in https://github.com/sgl-project/sglang/pull/15433

[AMD] Fix and add accuracy-test-2-gpu-amd back by @yctseng0211 in https://github.com/sgl-project/sglang/pull/15415

[AMD] add unit-test-backend-8-gpu-amd back by @yctseng0211 in https://github.com/sgl-project/sglang/pull/15253

Support FP8 MLA prefill and 128k context. by @weireweire in https://github.com/sgl-project/sglang/pull/14395

[Auto Sync] Update scheduler_runtime_checker_mixin.py (20251219) by @merrymercy in https://github.com/sgl-project/sglang/pull/15437

[diffusion]Support url image input by @IPostYellow in https://github.com/sgl-project/sglang/pull/15262

diffusion: support qwen-image-edit-2511 by @yhyang201 in https://github.com/sgl-project/sglang/pull/15458

Fix: Support multiple input images to SGLang Diffusion when using generate mode by @suyedu in https://github.com/sgl-project/sglang/pull/15394

[Diffusion] kernel: timestep embedding kernel implementation by @66RING in https://github.com/sgl-project/sglang/pull/12995

[diffusion] Add Sage Attention 3 Support for sm 120 (RTX5090) by @ryang-max in https://github.com/sgl-project/sglang/pull/15382

[diffusion] fix: fix wrong validation on 2k resolution by @mickqian in https://github.com/sgl-project/sglang/pull/15478

Add MiDasheng Model Support by @Jacki1223 in https://github.com/sgl-project/sglang/pull/15219

fix: update model name after weights update by @alphabetc1 in https://github.com/sgl-project/sglang/pull/15416

[Diffusion] Add diffusion attention backends doc by @BBuf in https://github.com/sgl-project/sglang/pull/15408

[NPU]Fix for ipc handle with npu by @hustmf in https://github.com/sgl-project/sglang/pull/14138

[NPU] bugfix for chunkedprefill by @Hexq0210 in https://github.com/sgl-project/sglang/pull/15166

Tiny fix mimo model conflicts with main by @hnyls2002 in https://github.com/sgl-project/sglang/pull/15483

Enhance protection rules of code owners by @hnyls2002 in https://github.com/sgl-project/sglang/pull/15406

Vertex generate pathway in server by @yashikagandhi-google in https://github.com/sgl-project/sglang/pull/15348

EP Support for Piecewise Cuda Graph by @Oasis-Git in https://github.com/sgl-project/sglang/pull/14164

fixed trtllm nvfp4 backend for moe by @khushgx in https://github.com/sgl-project/sglang/pull/15022

[model-gateway] fix graceful shutdown for TLS/Non-TLS server by @slin1237 in https://github.com/sgl-project/sglang/pull/15491

[model-gateway] refactor: extract common graceful shutdown code before TLS branch by @slin1237 in https://github.com/sgl-project/sglang/pull/15494

[model-gateway] Improve logging in data_connector module by @slin1237 in https://github.com/sgl-project/sglang/pull/15495

[model-gateway] Improve logging in policies module by @slin1237 in https://github.com/sgl-project/sglang/pull/15496

[AMD] Add TP=8 models to nightly test and make TP=2 test stable by @michael-amd in https://github.com/sgl-project/sglang/pull/15296

[DSv32] Move deep_gemm.get_paged_mqa_logits_metadata to init time as metadata by @qianlihuang in https://github.com/sgl-project/sglang/pull/15040

[model-gateway] Improve logging across core modules by @slin1237 in https://github.com/sgl-project/sglang/pull/15497

[model-gateway] Optimize workflow engine with pre-computed dependency graph by @slin1237 in https://github.com/sgl-project/sglang/pull/15503

[model-gateway] Run workflow event subscribers concurrently by @slin1237 in https://github.com/sgl-project/sglang/pull/15504

[model-gateway] simplify workflow engine backoff and reduce duplicate reads by @slin1237 in https://github.com/sgl-project/sglang/pull/15505

[router] bugfix: cache_aware in grpc inbalance forward by @llfl in https://github.com/sgl-project/sglang/pull/15473

Clean hidden_states_before_norm by @hnyls2002 in https://github.com/sgl-project/sglang/pull/15485

[ci] remove rust benchmark in unit test ci by @slin1237 in https://github.com/sgl-project/sglang/pull/15510

[model-gateway] Implement RAII load guard with response body attachment by @slin1237 in https://github.com/sgl-project/sglang/pull/15507

[diffusion] refactor: deprecate WorkloadType by @mickqian in https://github.com/sgl-project/sglang/pull/15267

[GLM-4.7] GLM-4.7 Tool Parser and Doc Update by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/15333

tiny fix sampling seed for completion api by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/15498

[AMD] remove the redundant projection by @yctseng0211 in https://github.com/sgl-project/sglang/pull/15178

[AMD] Support fast_topk kernels in sgl-kernel by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/15172

[NPU] [BUGFIX] Fix NPU inference (torch_npu._npu_reshape_and_cache() crash) by @OrangeRedeng in https://github.com/sgl-project/sglang/pull/15484

Optimize MiMo-V2-Flash by flashinfer fused allreduce by @yuan-luo in https://github.com/sgl-project/sglang/pull/15464

vlm: Refactor engine vlm params and support precessor output as input by @minleminzui in https://github.com/sgl-project/sglang/pull/14091

[VLM] Support ViT Piecewise CUD

Note truncated.

0.5.63 Dec 2025Deprecated

remove deprecated tile_tokens_dim by @b8zhong in https://github.com/sgl-project/sglang/pull/13186

Highlights

Support for DeepSeek V3.2/V3.2 Speciale #14249
Blockwise diffusion language model support #12588
Support for new diffusion models (Flux2 #14000, Z-image #14067)
Introduce JIT Kernels #13453
Upgrade to Torch 2.9 #12969
Kimi-K2-Thinking model enhancement #12882
Memory management/Overlap spec compatibility #12224 #12839
More performance optimization: DeepSeek-v3-fp4/GLM-4.6/Kimi-K2/DeepSeek-V3.2...
CI/CD Enhancement

What's Changed

[router][grpc] Add more mcp test cases to responses api by @CatherineSue in https://github.com/sgl-project/sglang/pull/12749
[Intel]Add 'intel_xpu' attention backend for llama4 by @gaopengff in https://github.com/sgl-project/sglang/pull/11051
[Intel XPU]Update pytorch xpu to 2.9 by @gaopengff in https://github.com/sgl-project/sglang/pull/12363
[Docs] fix dead links in multiple documentation pages by @mattheliu in https://github.com/sgl-project/sglang/pull/12764
[mem pool] bugfix: wrong position for self.device in Mamba by @stmatengss in https://github.com/sgl-project/sglang/pull/12684
[Fix]HTTP Stream raise exception by @jimmy-evo in https://github.com/sgl-project/sglang/pull/11904
[CPU] Fix TP padding case with weight block size by @jianan-gu in https://github.com/sgl-project/sglang/pull/8243
[docs] Remove redundant --disable-radix-cache option from by @rchalamala in https://github.com/sgl-project/sglang/pull/12717
Pin uvloop to 0.21.0 by @yeahdongcn in https://github.com/sgl-project/sglang/pull/12279
[fix] Only enable flashinfer all reduce fusion by default for single-node servers by @leejnau in https://github.com/sgl-project/sglang/pull/12724
chore: update CODEOWNERS by @zhyncs in https://github.com/sgl-project/sglang/pull/12795
Fix hang in deepgemm compilation with symmetric memory enabled by @nvcastet in https://github.com/sgl-project/sglang/pull/12715
Add bot-bump-kernel-version-to-sglang workflow by @alisonshao in https://github.com/sgl-project/sglang/pull/12794
ignore the deepgemm check when the model weight with nvfp4 and moe ba… by @rainj-me in https://github.com/sgl-project/sglang/pull/12782
[AMD] Update wave-lang to 3.8.2 by @xintin in https://github.com/sgl-project/sglang/pull/12576
[DeepSeek-V3.2][NSA] Enable MHA Pathway for Short Sequence Prefill on B200 (SM100) by @YAMY1234 in https://github.com/sgl-project/sglang/pull/12788
[hotfix]: Resolve ModuleNotFoundError in PD deployment for is_in_ci() by @hzh0425 in https://github.com/sgl-project/sglang/pull/12772
[HotFix]: Add missing SGLANG_EPLB_HEATMAP_COLLECTION_INTERVAL env var by @hzh0425 in https://github.com/sgl-project/sglang/pull/12776
Add PP support for dots_vlm by @gty111 in https://github.com/sgl-project/sglang/pull/12763
fixes hardcoded "cuda" device references in unit tests to use a dynamic device selection by @kalyank007 in https://github.com/sgl-project/sglang/pull/12761
fix multimodal gen issues by @yhyang201 in https://github.com/sgl-project/sglang/pull/12765
[Test] Add DeepSeekV3.2 NSA Indexer Test Suite by @Johnsonms in https://github.com/sgl-project/sglang/pull/12520
[Bugfix] Fix illegal memory access by @elvischenv in https://github.com/sgl-project/sglang/pull/12758
[MoE] Add Comprehensive MoE Integration Tests by @Jonahcb in https://github.com/sgl-project/sglang/pull/12090
[Deepseek V3.2] Only skip Indexer logits computation when is_extend_without_speculative by @hlu1 in https://github.com/sgl-project/sglang/pull/12816
Fix missing dp_max_padding argument in set_dp_buffer_len by @Chen-0210 in https://github.com/sgl-project/sglang/pull/12812
optm(checkpoint-engine): disable multi-thread loading when update weights by @BraveY in https://github.com/sgl-project/sglang/pull/12374
Fix piecewise cuda graph ci test by @ispobock in https://github.com/sgl-project/sglang/pull/12836
update multimodal_gen readme by @mickqian in https://github.com/sgl-project/sglang/pull/12825
[router] Support structured model output for openai and grpc router by @key4ng in https://github.com/sgl-project/sglang/pull/12431
Fix data parallel controller launch for num nodes > 2 by @merrymercy in https://github.com/sgl-project/sglang/pull/12822
remove the fa4 page_size hardcode to 128 restriction on mla model arch by @rainj-me in https://github.com/sgl-project/sglang/pull/12801
sglang diffusion announcement by @wisclmy0611 in https://github.com/sgl-project/sglang/pull/12856
add back flashinfer jit cache to dev docker by @b8zhong in https://github.com/sgl-project/sglang/pull/12851
[router][grpc] Refactor: Add builders for chat and responses by @CatherineSue in https://github.com/sgl-project/sglang/pull/12852
[router][grpc] Move all error logs to their call sites by @CatherineSue in https://github.com/sgl-project/sglang/pull/12859
[router] Switch MCP tests from DeepWiki to self-hosted Brave search server by @key4ng in https://github.com/sgl-project/sglang/pull/12849
Add nightly performance test for GPT-OSS 4GPU models by @alisonshao in https://github.com/sgl-project/sglang/pull/12805
[sgl-kernel][Deepseek V3.2] Add row_starts to topk kernel by @hlu1 in https://github.com/sgl-project/sglang/pull/12582
[CI] Fix huggingface access for test_flash_attention_4.py by @Fridge003 in https://github.com/sgl-project/sglang/pull/12846
[Auto Sync] Update activation.py, logits_processor.py, rota... (20251107) by @merrymercy in https://github.com/sgl-project/sglang/pull/12853
[Docs][DeepseekV3.2] Update deepseekv3.2 docs for mha short seq prefill by @YAMY1234 in https://github.com/sgl-project/sglang/pull/12868
Support capturing aux_hidden_states for minimax m2. by @pyc96 in https://github.com/sgl-project/sglang/pull/12798
[CI] Tiny adjust CI esitmation time by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12886
[DP-Attn] Clarify MLP sync / idle batch preparation logic by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12843
Fix sending all requests to the first rank in DP attention by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12832
Apply moe_reduce_sum kernel for fused_marlin_moe by @ispobock in https://github.com/sgl-project/sglang/pull/12888
use fast stream instead of torch.cuda.current_stream in llama 4 shared experts overlap by @b8zhong in https://github.com/sgl-project/sglang/pull/12811
[Fix] Fix trtllm-mla backend when chunked prefix cache is disabled by @Fridge003 in https://github.com/sgl-project/sglang/pull/12361
Refs/heads/add nightly test multi gpu configs by @alisonshao in https://github.com/sgl-project/sglang/pull/12870
chore: bump sgl-kernel version to 0.3.16.post6 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12889
Update CODEOWNERS by @ispobock in https://github.com/sgl-project/sglang/pull/12897
Tiny simplify can_run_dp_cuda_graph gather logic by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12891
Fix spec decoding acc length for dpsk-r1-fp4 tp8 by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/12896
Revert "Fix spec decoding acc length for dpsk-r1-fp4 tp8" by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/12900
Add Deepseek models into nightly tests by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12865
Fix empty server args in marlin moe test by @ispobock in https://github.com/sgl-project/sglang/pull/12904
Fix duplicate nightly test name by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12905
Add HF cleanup logic in ci_install_dependency.sh by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12895
fallback to triton mm_persistent kernel when deepGemm fail by @zminglei in https://github.com/sgl-project/sglang/pull/12911
Add kimi k2 thinking to ci by @ispobock in https://github.com/sgl-project/sglang/pull/12907
Fix Deepseek nightly tests by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12906
Add Jet-Nemotron by @futrime in https://github.com/sgl-project/sglang/pull/12448
[CI] increase ut buckets & adjust estimation time. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12919
[PD] feat: refactor custom mem pool and add barex pd support by @stmatengss in https://github.com/sgl-project/sglang/pull/12332
[CI] Fix matrix.part in pr-test. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12920
Adjust server launch time in ci by @ispobock in https://github.com/sgl-project/sglang/pull/12917
feat: basic support for server-level multimodal cache by @mickqian in https://github.com/sgl-project/sglang/pull/10775
Refactor / Unify event loop across PD-Disagg, Overlap, DP-Attn cases by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12839
[lint] tiny fix unimported packages. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12927
ci: try to fix gpg error during kernel build by @ishandhanani in https://github.com/sgl-project/sglang/pull/12928
Support piecewise cuda graph for MLA by @ispobock in https://github.com/sgl-project/sglang/pull/11812
diffusion: skip full CI suite for multimodal_gen changes by @mickqian in https://github.com/sgl-project/sglang/pull/12940
Minor code cleanup / improvement for PREBUILT_EXTEND mode by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12948
Bugfix: LMCache Connector with Sglang by @MMuzzammil1 in https://github.com/sgl-project/sglang/pull/12946
[Docs] Add docs for Qwen3-VL image and video support by @adarshxs in https://github.com/sgl-project/sglang/pull/12554
[Refactor] rename set_index_k_and_scale_buffer to set_index_k_scale_b… by @edwingao28 in https://github.com/sgl-project/sglang/pull/12956
Refactor KTransformers heterogeneous compute with unified GPU-quantization backend by @Atream in https://github.com/sgl-project/sglang/pull/12834
diffusion: fix detected file changes rule in CI by @mickqian in https://github.com/sgl-project/sglang/pull/12943
clean redundant code in previous PR by @Atream in https://github.com/sgl-project/sglang/pull/12957
Fix the run-time error when calling fused_rms_mxfp4_quant that change return output number by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/12803
diffusion: fix wan-2.2-TI2V and support sp by @mickqian in https://github.com/sgl-project/sglang/pull/12926
[Refactor / Style] Unify all event loops (except for PP) by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12959
chore: bump sgl-kernel version to 0.3.17 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12931
chore: include a minimum image for vlms when warming-up by @mickqian in https://github.com/sgl-project/sglang/pull/9528
[PP] put pp assert in model runner by @XucSh in https://github.com/sgl-project/sglang/pull/12934
Fix errors of page head kernels in sgl-kernel for ROCm by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/12604
[Fix] Add validation for served model name to reserve : for LoRA adapter syntax by @neelabhsinha in https://github.com/sgl-project/sglang/pull/12912
Support hidden_dim % 4 == 0 in per_token_quant_fp8 by @BBuf in https://github.com/sgl-project/sglang/pull/12883
[RadixTree] Reduce Syscalls, Optimize Collection Filtering and Align with cpp by @CLFutureX in https://github.com/sgl-project/sglang/pull/12239
[Auto Sync] Update batch_invariant_ops.py (20251109) by @merrymercy in https://github.com/sgl-project/sglang/pull/12916
[router] bucket policy by @syy-hw in https://github.com/sgl-project/sglang/pull/11719
fix missing output_token_logprobs when using ngram speculative decoding by @a4zhangfei in https://github.com/sgl-project/sglang/pull/10702
feat(metrics): add scheduler and hiradix cache metrics (#10218) by @ShawnKung in https://github.com/sgl-project/sglang/pull/10225
diffusion: reduce effort of supporting new model by @mickqian in https://github.com/sgl-project/sglang/pull/12982
vlm: fix tiny multimodal cache bug by @yhyang201 in https://github.com/sgl-project/sglang/pull/12984
chore: bump sgl-kernel version to 0.3.17 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12966
[1 / 2] register weak_ref_tensor in sgl-kernel by @BBuf in https://github.com/sgl-project/sglang/pull/12999
Support piecewise cuda graph for deepseek v3 by @ispobock in https://github.com/sgl-project/sglang/pull/12996
minor: fix notebook bug with new model_info fields added for warmup by @mickqian in https://github.com/sgl-project/sglang/pull/13005
Super tiny fix typo by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13001
Add process_prefill_chunk back to fix PP event loop by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13009
[misc][ci] Add run-ci after auto-labeler by @CatherineSue in https://github.com/sgl-project/sglang/pull/13013
Unify memory management across (overlap, non-overlap) x (page>=1) x (spec, non-spec, spec v2) x (retract, finished) by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12224
Enhance retract test (page cases, long output cases) by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12781
[AMD CI] Remove SRT docker build. by @saienduri in https://github.com/sgl-project/sglang/pull/11850
[CI] Limit the CI trigger frequency of low-privilege actors by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13010
Resolve HF download issue and download models before CI run starts by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12952
Add pre-suffle weight for new aiter MoE support. by @sogalin in https://github.com/sgl-project/sglang/pull/12908
chore: bump SGLang version to 0.5.5.post1 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13000
[router][ci] Fix maturin build by @key4ng in https://github.com/sgl-project/sglang/pull/13012
Simplify the BatchMultimodalOutput in io_struct.py by @merrymercy in https://github.com/sgl-project/sglang/pull/12993
[router][ci] Quick Improvement to make CI more stable by @key4ng in https://github.com/sgl-project/sglang/pull/12869
[9/n] decouple quantization impl from vllm dependency - adjust ci by @AniZpZ in https://github.com/sgl-project/sglang/pull/12753
[router] add postgres databases data connector by @lengrongfu in https://github.com/sgl-project/sglang/pull/12218
[AMD CI] Update docker release workflows docker file name. by @saienduri in https://github.com/sgl-project/sglang/pull/13028
fix tuning_fused_moe_triton_sep tool per_channel_quant bug by @BBuf in https://github.com/sgl-project/sglang/pull/13027
fix(ci): workflow id in permission rate limit by @cicirori in https://github.com/sgl-project/sglang/pull/13035
[PieceWise CUDA Graph] Support awq/gptq model in piecewise cudagraph by @BBuf in https://github.com/sgl-project/sglang/pull/12518
Re-enable Flashinfer TRTLLM GEN MHA and Add Unit Test by @samuellees in https://github.com/sgl-project/sglang/pull/12885
[AMD CI] Update CI Version Logic. by @saienduri in https://github.com/sgl-project/sglang/pull/13029
[diffusion] doc: add support_new_models by @mickqian in https://github.com/sgl-project/sglang/pull/13043
Sglang Tracing: optimize trace_event_batch() by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/13036
[CI] Auto format code by @BBuf in https://github.com/sgl-project/sglang/pull/13053
disable overlap schedule if mamba radix cache open by @yizhang2077 in https://github.com/sgl-project/sglang/pull/13057
[AMD] Add PD test for AMD CI by @michael-amd in https://github.com/sgl-project/sglang/pull/11938
Remove duplicate import by @LHXuuu in https://github.com/sgl-project/sglang/pull/12980
[Bug] TypeError: maybe_executor_submit() by @Johnsonms in https://github.com/sgl-project/sglang/pull/13050
[Fix] Add TPOT back to bench_serving by @elvischenv in https://github.com/sgl-project/sglang/pull/12976
[bug][rocm]fix qr when variable inp by @haoyangli-amd in https://github.com/sgl-project/sglang/pull/11609
[ROCM] Optimized deepseek-r1 model with rmsnorm + fp8 quant fusion by @yctseng0211 in https://github.com/sgl-project/sglang/pull/12689
Update rope dtype config by @ispobock in https://github.com/sgl-project/sglang/pull/13037
Tiny simplify evcition metrics collector by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12983
[RadixTree] Reduce Stack Push/Pop Overhead for Leaf Nodes, Improve radix_tree Leaf Collection Performance by @CLFutureX in https://github.com/sgl-project/sglang/pull/12199
fix: display served_model_name in /v1/models by @Sunhaihua1 in https://github.com/sgl-project/sglang/pull/13063
refine stdout logging codes by @cicirori in https://github.com/sgl-project/sglang/pull/13015
Support file:// scheme in load_video by @netanel-haber in https://github.com/sgl-project/sglang/pull/13076
[BugFix] Fix prefill memory leak in PD + GDN by @ZeldaHuang in https://github.com/sgl-project/sglang/pull/12994
[Bug] Login shell error: bash: /root/.cargo/env: No such file or directory by @Johnsonms in https://github.com/sgl-project/sglang/pull/12941
Fix cached tokens usage bug by @FrankMinions in https://github.com/sgl-project/sglang/pull/12814
Revert "[AMD] Add PD test for AMD CI (#11938)" by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13088
[Test] Handle streaming chunks with null content in case of stream end. by @vshekhawat-hlab in https://github.com/sgl-project/sglang/pull/10862
[Fix] Update text_chunks in bench_serving chat completions by @ZeldaHuang in https://github.com/sgl-project/sglang/pull/13041
Fix CPP Radix Cache and add test to CI by @cctry in https://github.com/sgl-project/sglang/pull/11645
[AMD] Apply AITER_MXFP4_MOE_SF=1 only to gfx950 in aiter build by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/13092
Export runner labels via env var by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13018
Fix spec decoding acc length for dpsk-r1-fp4 tp8 (2nd attempt) by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/12915
[Fix] Fix nan error for large scale ep by @Fridge003 in https://github.com/sgl-project/sglang/pull/12866
[AMD CI] Update nightly docker build CI config. by @saienduri in https://github.com/sgl-project/sglang/pull/13090
overlap shared + routed expert computation in kimi linear by @b8zhong in https://github.com/sgl-project/sglang/pull/12660
Revert "fix: display served_model_name in /v1/models" by @CatherineSue in https://github.com/sgl-project/sglang/pull/13093
[Deepseek V3.2] Fix accuracy bug in the Indexer by @hlu1 in https://github.com/sgl-project/sglang/pull/12583
[Router] use call_id instead of id for matching function calls in Responses API for Harmony by @zhaowenzi in https://github.com/sgl-project/sglang/pull/13056
Upgrade to ROCm 7.0 image by @yctseng0211 in https://github.com/sgl-project/sglang/pull/13105
Improve overlap scheduling for better TTFT by @vipwangerxiao in https://github.com/sgl-project/sglang/pull/11856
At least tell the user that ngram verify is greedy! by @MayDomine in https://github.com/sgl-project/sglang/pull/13039
diffusion: remove unused workflows folder by @mickqian in https://github.com/sgl-project/sglang/pull/13114
Don't fuse wk+weight_proj for nextn by @trevor-m in https://github.com/sgl-project/sglang/pull/12863
[diffusion] log: improve logging while multiprocessing by @mickqian in https://github.com/sgl-project/sglang/pull/12997
Fix gpt oss 4gpu b200 trace links by @alisonshao in https://github.com/sgl-project/sglang/pull/12872
diffusion: refactor task type of models by @mickqian in https://github.com/sgl-project/sglang/pull/13118
[Feature] Trace: Support http/protobuf span exporter protocol by @zhanghaotong in https://github.com/sgl-project/sglang/pull/12396
[sgl-kernel][5/N]Support Expert Specialization Grouped GEMM by @HydraQYH in https://github.com/sgl-project/sglang/pull/12666
feat(engine): add rid parameter to methods in Engine class by @ishandhanani in https://github.com/sgl-project/sglang/pull/13095
[PD] Add custom gpu id to device topo support by @stmatengss in https://github.com/sgl-project/sglang/pull/12817
[CI] Update job dependency and move dpsk v3.2 tests to 8-gpu suite by @Fridge003 in https://github.com/sgl-project/sglang/pull/12942
Fix run suite sanity check by @ispobock in https://github.com/sgl-project/sglang/pull/13133
[router] move radix tree to policy crate and addreses some code styles by @slin1237 in https://github.com/sgl-project/sglang/pull/13131
fix: Remove dupulicated kv_events initialization in scheduler by @wxsms in https://github.com/sgl-project/sglang/pull/13132
Fix strict level setting for Kimi K2 tool calls when not explicitly set by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/13077
[misc] Remove performance and router-benchmark label matching by @CatherineSue in https://github.com/sgl-project/sglang/pull/13135
Fix re-trigger actor of CI rate limit by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13136
[router] Support complex assistant and tool messages in /chat/completions by @hellodanylo in https://github.com/sgl-project/sglang/pull/12860
[router] add minmax m2 reasoning parser by @slin1237 in https://github.com/sgl-project/sglang/pull/13137
fix(tcp-port): replace bind_server_socket to get_zmq_socket(Port conflict) by @jimmy-evo in https://github.com/sgl-project/sglang/pull/11961
fix: duplicate resize images logic of qwen-vl series models by @yangsijia-serena in https://github.com/sgl-project/sglang/pull/12458
[router][grpc] Support vllm backend for grpc router by @CatherineSue in https://github.com/sgl-project/sglang/pull/13120
Dump total_throughput to output-file in bench_serving.py by @Rohan138 in https://github.com/sgl-project/sglang/pull/9790
Fix the Wrong Return Type of Scheduler.recv_requests by @Arist12 in https://github.com/sgl-project/sglang/pull/7886
Update aiter to v0.1.7.post1 by @sogalin in https://github.com/sgl-project/sglang/pull/13149
[RPC] Fix handle_rpc_request with **recv_req.parameters by @CharlieFRuan in https://github.com/sgl-project/sglang/pull/7906
[Ascend]adapt enable-profile-cuda-graph for NPU by @ping1jing2 in https://github.com/sgl-project/sglang/pull/12617
[Feature] Propagate Trace Headers into Root Span for OpenTelemetry Cross-Service Context by @zhanghaotong in https://github.com/sgl-project/sglang/pull/10808
chore: bump SGLang version to 0.5.5.post2 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13129
[Ascend][feature] support L1+ L2 radixcache on ascend by @khalil2ji3mp6 in https://github.com/sgl-project/sglang/pull/12214
[ngram] use SGLANG_NGRAM_FORCE_GREEDY_VERIFY to control verify method by @a4zhangfei in https://github.com/sgl-project/sglang/pull/13153
[Ascend] torch_npu.npu_mrope for MRotaryEmbedding by @Makcum888e in https://github.com/sgl-project/sglang/pull/10907
[VLM] Support PP for Qwen2.5-VL by @yuan-luo in https://github.com/sgl-project/sglang/pull/13075
Fuse routed_scaling_factor to fused_marlin_moe by @ispobock in https://github.com/sgl-project/sglang/pull/12998
[Auto Sync] Update test_deterministic.py (20251112) by @merrymercy in https://github.com/sgl-project/sglang/pull/13128
bugfix: multi-model routing for /generate api by @SYChen123 in https://github.com/sgl-project/sglang/pull/12979
[router] Add comprehensive validation to Responses API by @key4ng in https://github.com/sgl-project/sglang/pull/13127
[router] Fix Flaky test_circuit_breaker_opens_and_recovers by @XinyueZhang369 in https://github.com/sgl-project/sglang/pull/13164
Add job and runner failure monitor workflow for CI by @dougyster in https://github.com/sgl-project/sglang/pull/13104
[DeepseekV32]: use _concat_mla_absorb_q_general to replace torch.cat by @bingps in https://github.com/sgl-project/sglang/pull/12215
Fix nan in global scaling factor for large scale nvfp4 EP by @wenscarl in https://github.com/sgl-project/sglang/pull/13162
[Ascend] LoRA: adding Ascend LoRA backend with using kernels from sgl_kernel_npu by @vlserov in https://github.com/sgl-project/sglang/pull/12288
Add RequestMetricsExporter utility to export request-level metrics by @scottjlee in https://github.com/sgl-project/sglang/pull/10973
Opt kimi_k2_thinking biased topk module by @BBuf in https://github.com/sgl-project/sglang/pull/13150
[router] remove worker url requirement by @slin1237 in https://github.com/sgl-project/sglang/pull/13172
Remove EBNF Composer by @TJ5 in https://github.com/sgl-project/sglang/pull/13163
fix build error in Dockerfile.diffusion by @Sunhaihua1 in https://github.com/sgl-project/sglang/pull/12975
[Ascend] add npu synchronize by @hustmf in https://github.com/sgl-project/sglang/pull/13154
[AMD] Add AITER Custom All-Reduce by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/13102
Revert "fallback to triton mm_persistent kernel when deepGemm fail" by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13178
Remove enable_dp_attention in deepseek nightly tests by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13190
[router] minmax-m2 xml tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/13148
fix: display served_model_name in /v1/models by @Sunhaihua1 in https://github.com/sgl-project/sglang/pull/13155
Replace [silu_and_mul_]scaled_fp4_group_quant by Flashinfer equivalent by @wenscarl in https://github.com/sgl-project/sglang/pull/12376
[FEAT][ROCM] enable fused shared expert for Rocm by @ZLkanyo009 in https://github.com/sgl-project/sglang/pull/12201
Enable Flashinfer TRTLLM-GEN-MoE FP8 blockwise kernel for Qwen3-Next on Blackwell by @samuellees in https://github.com/sgl-project/sglang/pull/12543
[Quantization] Support Quark Dense + MoE FP8 & FP8 PTPC by @BowenBao in https://github.com/sgl-project/sglang/pull/10485
Fix accept rate in speculative decoding metrics by @SiqiLi-Fighting in https://github.com/sgl-project/sglang/pull/13212
Set max parallel for 1-gpu runner by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13215
Add model validation for all GPU runners to prevent cache corruption by @alisonshao in https://github.com/sgl-project/sglang/pull/13171
Bump actions/download-artifact from v4 to v6 for B200 workers by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13220
docs: update fused MoE config path by @edwardzjl in https://github.com/sgl-project/sglang/pull/13211
[PD / HiCache]fix deocde kvcache offload manager memory leak by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/12774
Fix broken Markdown formatting in DeepEP documentation by @Taishi-N324 in https://github.com/sgl-project/sglang/pull/13210
[Feature] Enable CUDA graph for PD-Multiplexing. by @ykcombat in https://github.com/sgl-project/sglang/pull/11595
Fix wrong running_bs in priority scheduling by @dtcccc in https://github.com/sgl-project/sglang/pull/13142
[sgl-kernel] support custom fp8 flashmla kernel by @FlamingoPg in https://github.com/sgl-project/sglang/pull/13087
[router][grpc] Refine docs in minimax_m2 to match other parsers by @CatherineSue in https://github.com/sgl-project/sglang/pull/13218
Update GDN causal conv1d cuda kernel - prepare for new changes by @byjiang1996 in https://github.com/sgl-project/sglang/pull/13188
Use 32x32 black image for VLM server warmup and bring glm4.1v back to UT by @byjiang1996 in https://github.com/sgl-project/sglang/pull/13222
[sgl-kernel] clean up fa fetch in CMakeLists.txt by @FlamingoPg in https://github.com/sgl-project/sglang/pull/12392
[Auto Sync] Update pynccl_wrapper.py, environ.py, registry.... (20251111) by @merrymercy in https://github.com/sgl-project/sglang/pull/13097
[AMD] Fix AITER_MXFP4_MOE_SF setting for gfx950 by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/13239
remove deprecated tile_tokens_dim by @b8zhong in https://github.com/sgl-project/sglang/pull/13186
[feat] make warmup timeout configurable through SGLANG_WARMUP_TIMEOUT by @billishyahao in https://github.com/sgl-project/sglang/pull/13243
fix: fix serve command without diffusion dependency by @mickqian in https://github.com/sgl-project/sglang/pull/13246
docker: Fix apt-add-repository by @kshitij12345 in https://github.com/sgl-project/sglang/pull/13213
[Auto Sync] Update backend.py, forward_batch_info.py, piece... (20251113) by @merrymercy in https://github.com/sgl-project/sglang/pull/13221
Fix nightly tests to fail properly when any job fails by @alisonshao in https://github.com/sgl-project/sglang/pull/13096
Super tiny fix outdated doc by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13255
Remove glm41v from CI to speed up CI by @byjiang1996 in https://github.com/sgl-project/sglang/pull/13257
Update model weight validation logic to handle special weight file naming by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13256
Add 3 models to 2 gpu runner in model downloading from nightly tests by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13261
[Tool Call] Steamline function arguments when tool_choice="auto" for deepseekv31_detector by @Muqi1029 in https://github.com/sgl-project/sglang/pull/11589
[model-gateway] change mg labeler from router to model-gateway by @slin1237 in https://github.com/sgl-project/sglang/pull/13265
ci: speed up b200 ci by @b8zhong in https://github.com/sgl-project/sglang/pull/13237
Extend lint test to test/ directory by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13247
[BugFix] weight load bug when checkpoint expert.gate and exepert.up_proj are not fused by @Yuechguo in https://github.com/sgl-project/sglang/pull/13113
Tiny fix update version logic location by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12620
Tiny enhance dumper with ctx and enable flags by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12622
Enhance dumper comparator with tensor unifier and location finder by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12623
Tiny add utility to parse server logs by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12605
[CPU] Use covt_e4m3_bf16 to optim BF16 to FP8 convert by @wangyxbh in https://github.com/sgl-project/sglang/pull/12191
chore: bump flashinfer v0.5.2 by @zhyncs in https://github.com/sgl-project/sglang/pull/13242
Super tiny fix CI by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13283
Add script to create a model with fewer layers for debugging by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13284
[BugFix] fix bench_serving error when multimodal image is testing by @ZLkanyo009 in https://github.com/sgl-project/sglang/pull/13254
Optimized prefill cache allocation for NPU by @terfendail in https://github.com/sgl-project/sglang/pull/13288
refactor: remove duplicate function _get_bootstrap_info_from_server by @acelyc111 in https://github.com/sgl-project/sglang/pull/13277
LLama4 Attention: Update assertion msg by @kshitij12345 in https://github.com/sgl-project/sglang/pull/12777
[Deepseek V3.2] Clean up MTP by @hlu1 in https://github.com/sgl-project/sglang/pull/13236
Support orion by @ppraneth in https://github.com/sgl-project/sglang/pull/10665
model: support teleflm by @ppraneth in https://github.com/sgl-project/sglang/pull/10573
[Doc] Add item for repetition punishment by @SenmiaoORZ in https://github.com/sgl-project/sglang/pull/13260
Support FP8 Per Token Quant Piecewise by @hebiao064 in https://github.com/sgl-project/sglang/pull/13272
[minor] remove debug code in python/sglang/srt/compilation/weak_ref_tensor_jit.py by @merrymercy in https://github.com/sgl-project/sglang/pull/13235
[NVIDIA] Fix use case of SGLANG_ENABLE_FLASHINFER_GEMM by @kaixih in https://github.com/sgl-project/sglang/pull/13274
Fix NSA indexer nightly test failed issues by @Johnsonms in https://github.com/sgl-project/sglang/pull/13298
Implement nightly test workflow naming conventions by @alisonshao in https://github.com/sgl-project/sglang/pull/13170
Remove nightly b200 tests and revert a change for test file by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13305
[router]Replace requests lib with openai in e2e_response_api by @XinyueZhang369 in https://github.com/sgl-project/sglang/pull/13293
Piecewise Cuda Graph Support for gpt-oss model by @Oasis-Git in https://github.com/sgl-project/sglang/pull/13045
Add missing model in model validate list by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13310
Add missing model for 2-gpu-runner in nightly tests by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13311
[Deterministic] Support Qwen3-Next model deterministic inference by @zminglei in https://github.com/sgl-project/sglang/pull/13100
CI: server performance test for SGLang Diffusion by @adarshxs in https://github.com/sgl-project/sglang/pull/13091
[model-gateway] smg release 0.2.3 by @slin1237 in https://github.com/sgl-project/sglang/pull/13312
Fix syntax errors in cpp_radix_tree by @Missmiaom in https://github.com/sgl-project/sglang/pull/13315
[Misc]Add date to cu13 dev image tag by @Fridge003 in https://github.com/sgl-project/sglang/pull/13316
[Diffusion] switch to local calculate_dimensions by @adarshxs in https://github.com/sgl-project/sglang/pull/13294
Add more statistics for spec decoding by @zhuzilin in https://github.com/sgl-project/sglang/pull/13317
Consolidate similar tests to reduce duplication by @alisonshao in https://github.com/sgl-project/sglang/pull/12871
feat: Add FP4 (E2M1) KV Cache Support for MHA by @JackChuang in https://github.com/sgl-project/sglang/pull/12612
Fix: test_vlm_offline_throughput output throughput by @dougyster in https://github.com/sgl-project/sglang/pull/13279
Add feature flag to broadcast mm inputs processing by @yuan-luo in https://github.com/sgl-project/sglang/pull/13278
re-submit 12911 but relax the requirement for deepgemm by @zminglei in https://github.com/sgl-project/sglang/pull/13226
Revert moe sum reduce for marlin moe by @ispobock in https://github.com/sgl-project/sglang/pull/13314
[RL] support only do cpu backup on draft model by @zhuzilin in https://github.com/sgl-project/sglang/pull/13318
[model-gateway] move python to binding folder by @slin1237 in https://github.com/sgl-project/sglang/pull/13295
[RL] Allow bypassing /health check by @zhuzilin in https://github.com/sgl-project/sglang/pull/13320
Support fast gemm when in batch invariant DeepGEMM fallback by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13259
Support inverse transform ue8m0 scale by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13285
Tiny refactor condition to requant scale ue8m0 by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13286
[RL] support update_weights_from_tensor for mtp by @zhuzilin in https://github.com/sgl-project/sglang/pull/7415
Update marlin moe kernel interface by @ispobock in https://github.com/sgl-project/sglang/pull/13322
[opt kimi k2 1 / n] Add kimi k2 moe fused gate by @BBuf in https://github.com/sgl-project/sglang/pull/13287
Super tiny expose transform_scale_ue8m0 API for RL frameworks by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13323
Update README by @zhyncs in https://github.com/sgl-project/sglang/pull/13326
Fix: add missing get_embed_and_head in MiniMax M2 for Eagle3 by @pyc96 in https://github.com/sgl-project/sglang/pull/13297
[router] Fix flaky router e2e tests by @XinyueZhang369 in https://github.com/sgl-project/sglang/pull/13306
Opt tp: tp attn support tp reduce scattered input by @xu-yfei in https://github.com/sgl-project/sglang/pull/10568
[Diffusion] add health endpoints to diffusion server by @adarshxs in https://github.com/sgl-project/sglang/pull/13329
[model-gateway] remove grpc feature flag and mark as default by @slin1237 in https://github.com/sgl-project/sglang/pull/13330
Temporarily disable test_vision_openai_server_a CI by @ispobock in https://github.com/sgl-project/sglang/pull/13331
[Feature] Spec-Overlap supporting DP-ATTN; PD-Disaggregation; npugraph mode by @iforgetmyname in https://github.com/sgl-project/sglang/pull/12443
[Ascend][Feat] Add Ascend sampling backend by @Alexhaoge in https://github.com/sgl-project/sglang/pull/12692
perf: optimize TypeBasedDispatcher using dict for O(1) lookup by @zhengxle in https://github.com/sgl-project/sglang/pull/12001
tiny fix lint by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13337
[optimize] Provide Usrbio compilation and installation commands by @leihuang-sketch in https://github.com/sgl-project/sglang/pull/12329
chore: bump sgl-kernel version to 0.3.17.post1 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13325
Clean up deprecated tile_tokens_dim for next flashinfer by @vincentzed in https://github.com/sgl-project/sglang/pull/13341
Add FP32 dtype support for RoPE - Part1 by @jinyouzhi in https://github.com/sgl-project/sglang/pull/13181
Add missing models by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13351
[CI] check unit-test-backend-8-gpu-h20 in workflow by @ch-wan in https://github.com/sgl-project/sglang/pull/13355
[Fix] Register custom ops only if they exist by @merrymercy in https://github.com/sgl-project/sglang/pull/13321
[Piecewise CUDA Graph] Support ModelOpt FP4 by @b8zhong in https://github.com/sgl-project/sglang/pull/13101
chore: bump sgl-kernel version to 0.3.17.post1 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13358
[feature] Add layerwise NVTX support by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/11870
[Ascend]support xgrammar backend for ascend npu by @ash-sigh in https://github.com/sgl-project/sglang/pull/12310
[Piecewise CUDA Graph] Support W4A8 by @b8zhong in https://github.com/sgl-project/sglang/pull/13179
[Performance] Move the contiguous to torch compile region by @DarkSharpness in https://github.com/sgl-project/sglang/pull/13199
Fix dpsk-r1-fp4 tp8 by reverting two commits (#13162 and #13341) by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/13348
Add missing models by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13369
[RL] enable offloading hybrid linear attn model by @zhuzilin in https://github.com/sgl-project/sglang/pull/13336
[model-gateway] fix model gateway pypi release workflow path by @slin1237 in https://github.com/sgl-project/sglang/pull/13372
[model-gateway] fix SDist step readme path by @slin1237 in https://github.com/sgl-project/sglang/pull/13373
[diffusion] refactor and added tests for Flux, T2V, TI2V, I2V by @adarshxs in https://github.com/sgl-project/sglang/pull/13344
diffusion: correct check-changes for multimodal_gen by @mickqian in https://github.com/sgl-project/sglang/pull/13375
diffusion: enable fa4 for blackwell by @yhyang201 in https://github.com/sgl-project/sglang/pull/13263
[opt kimi k2 2/n] apply kimi k2 thinking moe_fused_gate by @BBuf in https://github.com/sgl-project/sglang/pull/13332
[2 / 2] apply sgl-kernel weak_ref_tensor by @BBuf in https://github.com/sgl-project/sglang/pull/12978
Add SGLANG_ENABLE_REQ_POOL_LEAK_STRICT_CHECK to bypass mem leak check by @zhuzilin in https://github.com/sgl-project/sglang/pull/13339
Add default enable_memory_saver to HybridLinearKVPool by @zhuzilin in https://github.com/sgl-project/sglang/pull/13371
Cleanup vision attention related codes by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/13228
Remove unused code / testcases in lang by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13335
Tiny deprecate the --range-begin in run_suite.py by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13381
[router] bindings for go by @whybeyoung in https://github.com/sgl-project/sglang/pull/13384
fix generative_models.md table - remove newlines by @netanel-haber in https://github.com/sgl-project/sglang/pull/13385
fix nightly docker build by @b8zhong in https://github.com/sgl-project/sglang/pull/13386
fix import qwenvl error in RL engine by @dangkai4u in https://github.com/sgl-project/sglang/pull/12874
[CI] use cached deepep installation in gb200 CI by @ch-wan in https://github.com/sgl-project/sglang/pull/13388
Support spec decoding when LoRA is applied to target model by @lifuhuang in https://github.com/sgl-project/sglang/pull/12903
[CI] Fix B200 CI by @Fridge003 in https://github.com/sgl-project/sglang/pull/13387
[Tiny]Fix 1-gpu nightly test bugs by @Fridge003 in https://github.com/sgl-project/sglang/pull/13389
chore: bump SGLang version to 0.5.5.post3 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13366
refactor: replace worker pool with semaphore-based concurrency in jobqueue by @RiversJin in https://github.com/sgl-project/sglang/pull/13383
Update docs by @merrymercy in https://github.com/sgl-project/sglang/pull/13391
(1/n)support context parallel with deepseekv3.2-DSA by @lixiaolx in https://github.com/sgl-project/sglang/pull/12065
[RL] re-abort_request when model_update_lock is still locked by @zhuzilin in https://github.com/sgl-project/sglang/pull/13338
[1/N] CI refactor: introduce CI register. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13345
[Doc] Update CI oncall list by @merrymercy in https://github.com/sgl-project/sglang/pull/13396
Update .github/MAINTAINER.md by @Ying1123 in https://github.com/sgl-project/sglang/pull/13398
[HiCache] support memory_pool_host page head layout by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/11644
[HiCache] add GPU id to IB dev topo for mooncake storage backend by @stmatengss in https://github.com/sgl-project/sglang/pull/13112
Support weight update for blackwell DeepGEMM by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13324
refactor linear memory pool by @yizhang2077 in https://github.com/sgl-project/sglang/pull/13004
Remove deprecated scripts by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13399
[model-gateway] update workflow names for gateway and exclude npu by @slin1237 in https://github.com/sgl-project/sglang/pull/13415
[Tiny fix] Fix bench_speculative.py run bug by @BBuf in https://github.com/sgl-project/sglang/pull/13416
[model-gateway] Add Gateway Release Tooling by @slin1237 in https://github.com/sgl-project/sglang/pull/13420
fix uneven PP layer indices by @alpha-baby in https://github.com/sgl-project/sglang/pull/13282
diffusion: fix wan2.2 ti2v num_frames adjust logic by @mickqian in https://github.com/sgl-project/sglang/pull/13379
[PD][bug fix] fix memleak when last_batch is none by @XucSh in https://github.com/sgl-project/sglang/pull/13144
Fix cache_tokens calculate issue when retracted by @QiuMike in https://github.com/sgl-project/sglang/pull/11900
[feature] Custom base path on FastAPI server by @kebyn in https://github.com/sgl-project/sglang/pull/5879
Adding user defined hooks support by @Carlomus in https://github.com/sgl-project/sglang/pull/13217
Fix log time stats by @qhsc in https://github.com/sgl-project/sglang/pull/13418
[Ci tiny fix] Lower score threshold in evaluation test by @BBuf in https://github.com/sgl-project/sglang/pull/13443
diffusion: fix loading with local model_path by @mickqian in https://github.com/sgl-project/sglang/pull/13445
[2/N] CI refactor: sperate some backend-independent CPU tasks. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13447
Temporarily disable model hooks CI by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13450
[Deepseek V3.2] Use torch.compile to speed up torch.cat in nsa by @hlu1 in https://github.com/sgl-project/sglang/pull/13022
Remove verbs from GET endpoint paths to follow REST standards by @slin1237 in https://github.com/sgl-project/sglang/pull/13273
Add missing models by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13456
extend sagemaker.Dockerfile serve script to allow all sglang serve flags by @sirutBuasai in https://github.com/sgl-project/sglang/pull/13173
Fix 8-gpu B200 nightly tests by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13457
Fixes validation errors for Wan-AI models which store model weights in subdirectories by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13461
[Embeddings Performance Testing] Add performance test for embedding models by @vedantjh2 in https://github.com/sgl-project/sglang/pull/12359
[NVIDIA] Fix broken fp8 MoE of deepseek v3 by @kaixih in https://github.com/sgl-project/sglang/pull/13264
Temporarily comment out multimodal gen test to recover runners by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13463
Add interface_v1 option for dynamic HiCache backend by @pansicheng in https://github.com/sgl-project/sglang/pull/13140
Add bfloat16 tuned fused moe config for Dpsk-MTP layer on B200 by @Fridge003 in https://github.com/sgl-project/sglang/pull/13455
fix MambaPool clear method after refactoring by @zminglei in https://github.com/sgl-project/sglang/pull/13449
[AMD CI] Update sgl-router python path in dockerfile. by @saienduri in https://github.com/sgl-project/sglang/pull/13458
[CI] re-enable test_vision_openai_server_a ci by @yhyang201 in https://github.com/sgl-project/sglang/pull/13444
Adding CI Monitor Improvements by @dougyster in https://github.com/sgl-project/sglang/pull/13462
[GLM4.6v] Required changes for bumping up to transformer 5.x by @byjiang1996 in https://github.com/sgl-project/sglang/pull/13229
[GLM4.6v] Relax the constraint of non-user role chat completion message schema for new GLM-v release by @byjiang1996 in https://github.com/sgl-project/sglang/pull/13258
[model-gateway] use worker startup time out for worker registration by @slin1237 in https://github.com/sgl-project/sglang/pull/13473
Support JetVLM by @futrime in https://github.com/sgl-project/sglang/pull/13289
Add an unified server arg for multimodal inputs preprocessing config. by @WingEdge777 in https://github.com/sgl-project/sglang/pull/12149
[PD] Clarify init method docstrings for kvsender and kvreceiver by @ShangmingCai in https://github.com/sgl-project/sglang/pull/13476
Fix lora test by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13479
[Piecewise CUDA Graph] Support ModelOpt FP8 by @b8zhong in https://github.com/sgl-project/sglang/pull/13094
CI: fix NFS EBUSY error in PR test workflow by @alisonshao in https://github.com/sgl-project/sglang/pull/13460
[CI] fix triggered by a non-run-ci label by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13393
[CI] remove auto-labeling run-ci label. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13486
fix: change performance log directory to cache path by @ch-wan in https://github.com/sgl-project/sglang/pull/13482
[CI] Add input for pr-gate by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13491
[opt kimi k2 3/n] opt kimi_k2 moe_fused_gate kernel by @BBuf in https://github.com/sgl-project/sglang/pull/13374
[CI] fix lint yml (syntax error) by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13496
[VLM][feat] Support encoder DP for Qwen2.5-VL by @liusy58 in https://github.com/sgl-project/sglang/pull/13126
[HiCache] Critical fix to host memory double free by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/13501
[BugFix] Accuracy and function Issue when run ptpc quant model by @Yuechguo in https://github.com/sgl-project/sglang/pull/13157
fix: create git tags directly instead of temporary branches by @alisonshao in https://github.com/sgl-project/sglang/pull/13168
Add .github/CI_PERMISSIONS.json to define the CI permissions by @merrymercy in https://github.com/sgl-project/sglang/pull/13509
README.md -> FOLDER_README.md by @merrymercy in https://github.com/sgl-project/sglang/pull/13510
Use slash command to trigger CI by @merrymercy in https://github.com/sgl-project/sglang/pull/13512
Add docs on trigger ci by @merrymercy in https://github.com/sgl-project/sglang/pull/13513
[Feature] Re:Enable hybrid mem saver by @ocss884 in https://github.com/sgl-project/sglang/pull/12962
Trigger CI retry with edit by @merrymercy in https://github.com/sgl-project/sglang/pull/13516
Update docs by @merrymercy in https://github.com/sgl-project/sglang/pull/13519
Add /tag-and-rerun-ci by @sglang-bot in https://github.com/sgl-project/sglang/pull/13521
[CI] update pr-gate to be compatible with new slash triggering mananer. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13522
[CI] fix skipping pr-gate on main by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13525
Small cleanups related to LoRA weight loading by @glenliu21 in https://github.com/sgl-project/sglang/pull/13474
[CI] fix CI skipped on main by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13527
[model-gateway] fix gateway docker build due to recent py code change by @CatherineSue in https://github.com/sgl-project/sglang/pull/13532
[model-gateway] limit opened files in docker build to fix edge case by @CatherineSue in https://github.com/sgl-project/sglang/pull/13536
[docker] fix dockerfile naming for diffusion by @slin1237 in https://github.com/sgl-project/sglang/pull/13534
fix lora test by @gongwei-130 in https://github.com/sgl-project/sglang/pull/13537
Remove jet-ai/Jet-Nemotron-2B in nightly text tests as this is constantly failing by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13540
[Bug] Fixes accuracy issues caused by incorrect use of rope by @Baidu-AIAK in https://github.com/sgl-project/sglang/pull/13495
Flashinfer TRTLLM-GEN-MoE + Qwen3 by @b8zhong in https://github.com/sgl-project/sglang/pull/13489
[chore] Disable ccache for sgl-kernel release by @Fridge003 in https://github.com/sgl-project/sglang/pull/13541
Add Qwen/Qwen1.5-MoE-A2.7B to model list by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13543
[Fix] Fix DeepSeek V3 MTP on B200 by @Fridge003 in https://github.com/sgl-project/sglang/pull/13548
[router][grpc] Support num_reasoning_tokens in haromy models by @CatherineSue in https://github.com/sgl-project/sglang/pull/13047
[feat][Ascend][Mindspore]: support model-impl of mindspore by @chz34 in https://github.com/sgl-project/sglang/pull/9234
[AMD CI] Local cache fallback. by @saienduri in https://github.com/sgl-project/sglang/pull/13452
[CI] fix amd 1 gpu basic test by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13551
[Doc] Update HiCache and Mooncake docs & Mooncake Setup Error Checking by @ykwd in https://github.com/sgl-project/sglang/pull/12740
purge unnecessary env variable set in deterministic test by @zminglei in https://github.com/sgl-project/sglang/pull/13481
chore: bump sgl-kernel version to 0.3.17.post2 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13542
Add lmsys/gpt-oss-20b-bf16 to model validation check by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13557
CI Failure Monitor Improvements by @dougyster in https://github.com/sgl-project/sglang/pull/13558
[RL] Allow passing tensors of different dtypes for FlattenedTensorBucket by @zhuzilin in https://github.com/sgl-project/sglang/pull/13413
[CI] Fix CUDA workflow's dependency. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13568
[NPU] Adapt pr-gate for pr-test workflow & workflows refresh by @iforgetmyname in https://github.com/sgl-project/sglang/pull/13567
Tiny enhance test suites sanity check by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13589
[3/N] CI refactor: move some manually triggered tests. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13448
Support moe topk sigmoid kernel by @rogeryoungh in https://github.com/sgl-project/sglang/pull/13049
Expend compatibility check for all quantized MoE models by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/13465
add https://github.com/netanel-haber to CI_PERMISSIONS.json by @netanel-haber in https://github.com/sgl-project/sglang/pull/13577
chore: bump sgl-kernel version to 0.3.17.post2 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13570
[Auto Sync] Update base_grammar_backend.py, collector.py (20251116) by @merrymercy in https://github.com/sgl-project/sglang/pull/13357
[GDN] Remove unnecessary contiguous() by @byjiang1996 in https://github.com/sgl-project/sglang/pull/13604
[GDN] Remove unnecessary conv state clone by @byjiang1996 in https://github.com/sgl-project/sglang/pull/13603
[VLM] Support Piecewise CUDA Graph for Qwen2.5-VL by @yuan-luo in https://github.com/sgl-project/sglang/pull/13055
CI: improve diffusion CI by @mickqian in https://github.com/sgl-project/sglang/pull/13562
Support external custom models by @zhooooong in https://github.com/sgl-project/sglang/pull/13429
[CI fix] Fix image download failures in VLM CI tests by @BBuf in https://github.com/sgl-project/sglang/pull/13613
[NVIDIA] Add fp8 gemm benchmark on blackwell by @kaixih in https://github.com/sgl-project/sglang/pull/13528
[UT] Destroy process group after broadcast to resolve port occupation issues in multi-server tests by @galeselee in https://github.com/sgl-project/sglang/pull/12379
diffusion: remove PreprocessorConfig by @mickqian in https://github.com/sgl-project/sglang/pull/13248
diffusion: refactor pipeline folders by @mickqian in https://github.com/sgl-project/sglang/pull/13253
Add FP32 dtype support for RoPE - Part2 by @jinyouzhi in https://github.com/sgl-project/sglang/pull/13328
[Fix] Remove multimodal_gen redundant get_bool_env_var func by @shauntajoesph-ops in https://github.com/sgl-project/sglang/pull/13583
Add support for new aiter version (AR accuracy, is_shuffled PR) by @1am9trash in https://github.com/sgl-project/sglang/pull/13554
diffusion: improve baseline performance monitor by @mickqian in https://github.com/sgl-project/sglang/pull/13614
[Feature] Introduce JIT Kernel in sglang (with hicache JIT kernel) by @DarkSharpness in https://github.com/sgl-project/sglang/pull/13453
[CI] Align metric units for CI rate limit by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13633
[ROCM] Optimized deepseek-r1 fp8 model with + triton_gemm_a8w8 + batch_gemm_a8w8 + fused set_mla_kv_buffer kernel by @yctseng0211 in https://github.com/sgl-project/sglang/pull/13617
fix bench_speculative bug by @Lzhang-hub in https://github.com/sgl-project/sglang/pull/13197
Revert "[Feature] Introduce JIT Kernel in sglang (with hicache JIT kernel)" by @merrymercy in https://github.com/sgl-project/sglang/pull/13644
[CI] optimize CI workflow info by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13634
Kill zombie diffusion processes in CI & minor code style fix on rotary embedding fallback by @merrymercy in https://github.com/sgl-project/sglang/pull/13637
[CI] apply pr-gate for XPU by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13663
Add fused_rmsnorm_gated_cpu kernel for CPU to support Qwen3-Next by @yanbing-j in https://github.com/sgl-project/sglang/pull/11577
[10/n] decouple quantization impl from vllm dependency - fix import by @FlamingoPg in https://github.com/sgl-project/sglang/pull/13524
Adding nightly tests as release guard for bot bump workflows by @dougyster in https://github.com/sgl-project/sglang/pull/13655
[DeepseekV3.2] Deepseek fp8 support for MHA path by @YAMY1234 in https://github.com/sgl-project/sglang/pull/12964
Fix launch of Olmo3 by @vincentzed in https://github.com/sgl-project/sglang/pull/13666
[Deepseek V3.2] Change indexer weights_proj to fp32 by @hlu1 in https://github.com/sgl-project/sglang/pull/13459
enable csgmv automatically on cuda by @b8zhong in https://github.com/sgl-project/sglang/pull/13600
Add nightly test CI monitor workflow by @alisonshao in https://github.com/sgl-project/sglang/pull/13038
allow loras to be implicitly evicted and loaded based on max_loaded_loras by @glenliu21 in https://github.com/sgl-project/sglang/pull/11526
Test reorganization: Move tests to manual/ by @alisonshao in https://github.com/sgl-project/sglang/pull/13610
[Piecewise CUDA Graph] Fix recompile issue for Mixtral and Grok2 by @hebiao064 in https://github.com/sgl-project/sglang/pull/13667
Super tiny remove unused MiniMaxM2MLP class by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13659
Update quantization.md with new model resources by @zhaochenyang20 in https://github.com/sgl-project/sglang/pull/13677
[model-gateway] add both python and rust cli alias by @slin1237 in https://github.com/sgl-project/sglang/pull/13678
[diffusion] CI: improve validation method by @mickqian in https://github.com/sgl-project/sglang/pull/13627
[model-gateway] fix gateway cli arg parser to not use = by @CatherineSue in https://github.com/sgl-project/sglang/pull/13685
[CI] Move nightly tests to test/nightly/ by @alisonshao in https://github.com/sgl-project/sglang/pull/13683
[NVIDIA] Add cutedsl e2e test to GB200 CI by @kaixih in https://github.com/sgl-project/sglang/pull/12672
Add sgl-kernel CI test for Blackwell (B200) by @alisonshao in https://github.com/sgl-project/sglang/pull/13301
remove unnecessary starvation check by @glenliu21 in https://github.com/sgl-project/sglang/pull/13619
Fix target MLA with eagle3 support for PD disaggregation by @QiuMike in https://github.com/sgl-project/sglang/pull/13555
[kimi k2 thinking] Avoid useless torch.zeros_ by @BBuf in https://github.com/sgl-project/sglang/pull/13596
[opt kimi k2 4 / n] Delete useless pad kernel in sgl_moe_align_block_size by @BBuf in https://github.com/sgl-project/sglang/pull/13587
[VLM] Support Piecewise CUDA Graph for InternVL by @yuan-luo in https://github.com/sgl-project/sglang/pull/13640
[Piecewise Cuda Graph] rename, refactor and add more logging by @hebiao064 in https://github.com/sgl-project/sglang/pull/13675
difusion: speed up multimodal_gen ci by @yhyang201 in https://github.com/sgl-project/sglang/pull/13665
[diffusion] doc: minor update docs by @mickqian in https://github.com/sgl-project/sglang/pull/13177
Fix ZMQ bind error on non-zero rank nodes when using SGLANG_BLOCK_NONZERO_RANK_CHILDREN=0 by @ishandhanani in https://github.com/sgl-project/sglang/pull/13686
[diffusion] server: use meta to avoid Linear init for TextEncoder by @zyksir in https://github.com/sgl-project/sglang/pull/13564
[Auto Sync] Update http_server.py, io_struct.py, scheduler_... (20251120) by @merrymercy in https://github.com/sgl-project/sglang/pull/13679
[Bugfix] Fix hidden state size in EAGLE PD disaggregation buffers by @michelemarzollo in https://github.com/sgl-project/sglang/pull/13590
[HiCache] fix unit test with changed new APIs by @stmatengss in https://github.com/sgl-project/sglang/pull/13498
[Fix] Qwen3Next lmhead dtype by @ZeldaHuang in https://github.com/sgl-project/sglang/pull/13708
[NPU] chore: bump to CANN 8.3.RC1 and Pytorch 2.8.0 by @iforgetmyname in https://github.com/sgl-project/sglang/pull/13647
[11/N] MoE Refactor: Simplifying SBO Implementation with Dispatcher Hooks by @ch-wan in https://github.com/sgl-project/sglang/pull/13327
[Clean code] Compressed_tensors_moe code clean by @BBuf in https://github.com/sgl-project/sglang/pull/13719
[diffusion] profile: support performance metric dumping and comparison by @mickqian in https://github.com/sgl-project/sglang/pull/13630
[AMD] Enable fused shared expert append and flatten quant for fp8 deepseekR1 model by @yichiche in https://github.com/sgl-project/sglang/pull/13705
[diffusion] doc: add contributing.md by @mickqian in https://github.com/sgl-project/sglang/pull/13649
fix 3fs down, lock schedule main thread by @weibingo in https://github.com/sgl-project/sglang/pull/13407
Fix url: use https://roadmap.sglang.io for roadmap by @merrymercy in https://github.com/sgl-project/sglang/pull/13733
Super tiny delete unused files by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13734
[diffusion] log: minor improve logging by @mickqian in https://github.com/sgl-project/sglang/pull/13735
[CI] minor hot fix of model validation list by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13737
Add to ci permission by @guapisolo in https://github.com/sgl-project/sglang/pull/13739
[Piecewise CUDA Graph] Support Kimi-K2 (non-Thinking) by @b8zhong in https://github.com/sgl-project/sglang/pull/13466
Fix: CI monitor should not exit with error on regressions by @alisonshao in https://github.com/sgl-project/sglang/pull/13694
Revert "enable csgmv automatically on cuda" by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/13707
Support torch 12.9 + DeepEP by removing custom nvshmem by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12949
add some more labels by @b8zhong in https://github.com/sgl-project/sglang/pull/13701
Feat/nemotron nano v3 support by @roikoren755 in https://github.com/sgl-project/sglang/pull/12690
Fix global scaling factor loading hang by @wenscarl in https://github.com/sgl-project/sglang/pull/13484
Fix B200 Nightly tests and move one manual test back to unit test to prevent the same issue by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13746
fix test_lora_update.py starvation message check by @glenliu21 in https://github.com/sgl-project/sglang/pull/13702
Fix model weights validation with automatic cache cleanup by @alisonshao in https://github.com/sgl-project/sglang/pull/13729
[Auto Sync] Update evict_policy.py, radix_cache.py (20251120) by @merrymercy in https://github.com/sgl-project/sglang/pull/13669
[Tiny] Renaming environ for NVFP4 dispatch by @Fridge003 in https://github.com/sgl-project/sglang/pull/13756
modularize gsm8k and mmmu test classes by @netanel-haber in https://github.com/sgl-project/sglang/pull/13506
Use dual stream for DS MoE whenever cuda graph is used (instead of with token threshold) by @trevor-m in https://github.com/sgl-project/sglang/pull/9405
[Ascend] support Kimi-K2-Thinking by @zhuyijie88 in https://github.com/sgl-project/sglang/pull/12759
Refactor eagle bigram key matching by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13714
fix hunyuanvideo and add 2gpu ci testing by @yhyang201 in https://github.com/sgl-project/sglang/pull/13720
Update mem checker during busy by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13704
Tiny support different prompts in send_one.py by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13768
[diffusion] refactor: refactor sampling params by @mickqian in https://github.com/sgl-project/sglang/pull/13706
[VLM] Replace torch.repeat_interleave with faster np.repeat for Qwen-VL series by @yuan-luo in https://github.com/sgl-project/sglang/pull/13736
[Spec v2] Remove allocate_lens and enable over-allocation by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13478
tinyfix: diffusion ci by @yhyang201 in https://github.com/sgl-project/sglang/pull/13769
align code style eagle draft&draft_extend cuda graph runner by @cicirori in https://github.com/sgl-project/sglang/pull/13533
Refactor MHA & MLA KV caches to support FP4 by @JackChuang in https://github.com/sgl-project/sglang/pull/13547
Move unnecessary input_addr capture under debug mode flag for speed-up by @byjiang1996 in https://github.com/sgl-project/sglang/pull/13690
Gather static input buffers for cuda graph by @cctry in https://github.com/sgl-project/sglang/pull/13676
Revert "Fix RMSNorm API CALL mismatch issue. (#10032)" by @ErsongWang in https://github.com/sgl-project/sglang/pull/13727
[model-gateway] update smg code owner by @slin1237 in https://github.com/sgl-project/sglang/pull/13777
[model-gateway] clean up router manager function order by @slin1237 in https://github.com/sgl-project/sglang/pull/13776
Fix typo in docs by @yinpeiqi in https://github.com/sgl-project/sglang/pull/13709
[Feature] HiCache JIT kernel (once again) by @DarkSharpness in https://github.com/sgl-project/sglang/pull/13764
[DeepEP] Add SGLANG_DEEPEP_BF16_DISPATCH env var in Normal mode by @BBuf in https://github.com/sgl-project/sglang/pull/13787
Upgrade flashmla kernel for NSA tp support by @YAMY1234 in https://github.com/sgl-project/sglang/pull/13718
[diffusion] feat: support sp for image models by @mickqian in https://github.com/sgl-project/sglang/pull/13180
[diffusion] CI: add run_suite to multimodal_gen CI by @mickqian in https://github.com/sgl-project/sglang/pull/13791
Fix pagination bug in CI monitor preventing performance-test-2-gpu data collection by @alisonshao in https://github.com/sgl-project/sglang/pull/13781
[Scheduler] Tiny organize code style by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13806
[Deepseek] Refactor deepseek server_args _handle_model_specific_adjustments by @hlu1 in https://github.com/sgl-project/sglang/pull/13687
[CI] Tiny refactoring sgl-kernel tests by @Fridge003 in https://github.com/sgl-project/sglang/pull/13813
Tune fp8_w8a8 fused triton moe for GLM-4.6-FP8 by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/13815
make trtllm attn backend's init_forward_metadat non blocking by @cicirori in https://github.com/sgl-project/sglang/pull/13802
remove package json which is not used by @slin1237 in https://github.com/sgl-project/sglang/pull/13810
[1/2] Refactor DeepGeem requant for FP8 Linear on Blackwell by @Fridge003 in https://github.com/sgl-project/sglang/pull/13601
chore: bump sgl-kernel version to 0.3.18 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13816
xgrammar up version to 0.1.27 by @Swipe4057 in https://github.com/sgl-project/sglang/pull/13650
Fix bug: Incorrect variable used in rem_total_token_offset calculatio… by @liuhuijiayou in https://github.com/sgl-project/sglang/pull/13201
[Doc] Refine fused_moe_triton configs doc by @BBuf in https://github.com/sgl-project/sglang/pull/13820
Update MindSpore documentation by @wangtiance in https://github.com/sgl-project/sglang/pull/13656
Refactor cache init logic by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13800
[Bugfix] Add jit kernel files in packaging by @yuan-luo in https://github.com/sgl-project/sglang/pull/13829
[diffusion] doc: minor update contributing.md with test section by @mickqian in https://github.com/sgl-project/sglang/pull/13792
[misc] Rename minilb install env & remove files & fix lint by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13831
[diffusion] CI: send nightly-test outputs of diffusion to slack for correctness monitoring by @yhyang201 in https://github.com/sgl-project/sglang/pull/13833
[chore]Upgrade flashinfer to 0.5.3 by @Fridge003 in https://github.com/sgl-project/sglang/pull/13751
[Intel XPU]support xgrammar backend for intel xpu by @gaopengff in https://github.com/sgl-project/sglang/pull/13245
[sgl-kernel Code Clean] Remove useless lightning_attention kernel by @BBuf in https://github.com/sgl-project/sglang/pull/13819
[VLM] Revise InternVL Piecewise CUDA Graph Supporting by @yuan-luo in https://github.com/sgl-project/sglang/pull/13846
Fix TorchAO quant in VLM by @zhooooong in https://github.com/sgl-project/sglang/pull/13508
[Fix]: Adjust FutureMap's token_id_bufs Size to Prevent ChunkedPrefill's next_token_ids from Overwriting Previous Prefill Requests' next_token_id by @ant-yy in https://github.com/sgl-project/sglang/pull/13713
Fix: Safe RoPE Cache Expansion to Prevent Position-ID Out-of-Bounds in EAGLE + Long-Sequence Workloads by @YAMY1234 in https://github.com/sgl-project/sglang/pull/11871
[Fix] Fix uvloop get_event_loop() is not suitable for 0.22.x by @tom-jerr in https://github.com/sgl-project/sglang/pull/13612
Tiny unpin uvloop for other backends by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13858
[model-gateway] Refactor router e2e responses tests by @XinyueZhang369 in https://github.com/sgl-project/sglang/pull/13745
[Perf] Optimize DeepSeek-R1 w4afp8 glue kernels by @yuhyao in https://github.com/sgl-project/sglang/pull/10027
Fix quantized moe checker fail for Qwen3 dense fp8 model by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13853
[model-gateway] add grpc server code owner by @slin1237 in https://github.com/sgl-project/sglang/pull/13865
[BugFix] fix outplace_fused_experts missing is_gated by @zminglei in https://github.com/sgl-project/sglang/pull/13864
fix xgrammar_backend crash with malformed inputs by @gongwei-130 in https://github.com/sgl-project/sglang/pull/13752
[Auto Sync] Update schedule_batch.py, schedule_policy.py, b... (20251122) by @merrymercy in https://github.com/sgl-project/sglang/pull/13763
[Doc] Add an Introduction to Expert Parallelism by @ch-wan in https://github.com/sgl-project/sglang/pull/13783
add LoRA warning if loading a preexisting LoRA adapter with a different name by @glenliu21 in https://github.com/sgl-project/sglang/pull/13822
[NPU] Fix NPU CI by @iforgetmyname in https://github.com/sgl-project/sglang/pull/13834
Overlap glm moe gemms in two cuda streams by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/13786
[Performance] Replace preprocess_video logic from GLM multimodal processor with transformer impl for speed up (up to 27% faster) and addressing OOM (up to 50x improvements) by @byjiang1996 in https://github.com/sgl-project/sglang/pull/13487
Add support for bf16 x bf16 cutlass fused MoE by @nvcastet in https://github.com/sgl-project/sglang/pull/10275
[Router bugfix] Fix router_manager selecting the wrong router when enable-igw. by @SYChen123 in https://github.com/sgl-project/sglang/pull/13572
Fix nightly test job to fail when any test fails by @alisonshao in https://github.com/sgl-project/sglang/pull/13871
[diffusion] refactor: remove training-related code by @mickqian in https://github.com/sgl-project/sglang/pull/13860
[CI] fix multimodel-gen-test job by @cyb70289 in https://github.com/sgl-project/sglang/pull/13874
Add validation and cleanup for corrupted safetensors in multimodal loader by @alisonshao in https://github.com/sgl-project/sglang/pull/13870
[CI] fix lint error by @cyb70289 in https://github.com/sgl-project/sglang/pull/13891
fix: draft model revision misuse model revision by @gongwei-130 in https://github.com/sgl-project/sglang/pull/11893
Fix trace publish paths in nightly-test-nvidia workflow by @alisonshao in https://github.com/sgl-project/sglang/pull/13888
Adding nightly tests for Kimi-K2-thinking, Qwen3, minimax-m2, GLM4.6 by @dougyster in https://github.com/sgl-project/sglang/pull/13890
[Fix] JIT kernel dependencies in other platforms by @DarkSharpness in https://github.com/sgl-project/sglang/pull/13889
remove RoPE CPU fp32 tests by @ZailiWang in https://github.com/sgl-project/sglang/pull/13827
Move test_dummy_grok_models.py from manual to srt (temporary) by @alisonshao in https://github.com/sgl-project/sglang/pull/13901
[CI tiny fix] Enhance robustness of vision chunked prefill test with ROUGE-L metric by @BBuf in https://github.com/sgl-project/sglang/pull/13793
update flashinfer_cubin==0.5.3 by @Lzhang-hub in https://github.com/sgl-project/sglang/pull/13848
Add test_dummy_grok_models.py to not_in_ci section by @alisonshao in https://github.com/sgl-project/sglang/pull/13908
fix diffusion profile bugs by @yizhang2077 in https://github.com/sgl-project/sglang/pull/13642
[Fix]: Further fix the buffer len of future map by @ant-yy in https://github.com/sgl-project/sglang/pull/13916
[diffusion] CI: minor refactor CI for less code duplication by @mickqian in https://github.com/sgl-project/sglang/pull/13905
Update release-whl-kernel.yml by @ispobock in https://github.com/sgl-project/sglang/pull/13921
[Ascend] qwen optimization by @Liwansi in https://github.com/sgl-project/sglang/pull/12078
Support piecewise cuda graph for Qwen3-next by @Chen-0210 in https://github.com/sgl-project/sglang/pull/13081
fix nixl prefill crash make decode health check failed by @llc-kc in https://github.com/sgl-project/sglang/pull/13657
[CI] CI registry update by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13927
[CI] rename: per_commit -> registered by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13928
[diffusion] doc: add doc for LoRA usage by @mickqian in https://github.com/sgl-project/sglang/pull/13931
[diffusion] feat: support LoRA by @mickqian in https://github.com/sgl-project/sglang/pull/13859
[CPU] Apply PR gating rule in CI workflow by @ZailiWang in https://github.com/sgl-project/sglang/pull/13933
[misc] add llama3.1 chat template by @slin1237 in https://github.com/sgl-project/sglang/pull/13935
[Minor] Fix lint by @Fridge003 in https://github.com/sgl-project/sglang/pull/13938
[DeepSeekV3.2] Centralize NSA dispatch logic in NativeSparseAttnBackend by @YAMY1234 in https://github.com/sgl-project/sglang/pull/13544
Fix docstrings for v1 HiCacheStorage methods by @ptovam in https://github.com/sgl-project/sglang/pull/13851
Add Llama4 attention backend auto-selection by @janbernloehr in https://github.com/sgl-project/sglang/pull/13421
Improve nightly tests by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13903
[Auto Sync] Improve profilers and simplify bench_one_batch_server.py by @merrymercy in https://github.com/sgl-project/sglang/pull/13866
Fix update weight error for blackwell DeepGEMM by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13910
[chore] update torch version to 2.9 by @FlamingoPg in https://github.com/sgl-project/sglang/pull/12969
chore: bump sgl-kernel version to 0.3.18.post1 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13942
[Tiny]Upgrade README for sgl-kernel by @Fridge003 in https://github.com/sgl-project/sglang/pull/13945
Fix nightly-test-nvidia.yml to have the correct trigger by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13950
chore: bump sgl-kernel version to 0.3.18.post1 by @sglang-bot in https://github.com/sgl-project/sglang/pull/13951
Fix Deepseek v3.1 loading issue by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13954
fix: spec overlap predict shape does not match verify output shapes by @timmy-feng in https://github.com/sgl-project/sglang/pull/12786
[VLM] Support InternVL Vision Encoder Data Parallelism by @yuan-luo in https://github.com/sgl-project/sglang/pull/13925
Support FlashAttention3 page_size > 1 and topk > 1 case with paged attn and spec decode by @yubofredwang in https://github.com/sgl-project/sglang/pull/7725
Fix nightly test failures: NSA indexer dtype and CPP radix cache init by @alisonshao in https://github.com/sgl-project/sglang/pull/13958
update CI permission list by @ZailiWang in https://github.com/sgl-project/sglang/pull/13962
Fix SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION not working by @fzyzcjy in https://github.com/sgl-project/sglang/pull/13961
[feat] support in-flight weight update by @ShawnY112358 in https://github.com/sgl-project/sglang/pull/10071
Turn off PREBUILD aiter in MI355 by @1am9trash in https://github.com/sgl-project/sglang/pull/13963
Support piecewise CUDA graph for embedding models by @zhooooong in https://github.com/sgl-project/sglang/pull/13852
diffusion: fix the issue where the qwen-edit&wan model produces incorrect output during sequence parallelism by @yhyang201 in https://github.com/sgl-project/sglang/pull/13922
[Feature] Initial block diffusion language model support by @ClawSeven in https://github.com/sgl-project/sglang/pull/12588
Use dynamically maintained num_waiting_tokens in get_load() by @vipwangerxiao in https://github.com/sgl-project/sglang/pull/13203
Optimize uneven PP layer distribution logic to improve PP performance by @ShangmingCai in https://github.com/sgl-project/sglang/pull/13977
Fix get_load API by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13991
Rename: --hooks to --forward-hooks by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13994
[Ascend] Support enable-mixed-chunk in non-MLA scenarios by @MichelleWu351 in https://github.com/sgl-project/sglang/pull/12491
fix spec dec request level metrics by @vedantjh2 in https://github.com/sgl-project/sglang/pull/13754
[model-gateway] Add PostgreSQL support to binding by @xuwenyihust in https://github.com/sgl-project/sglang/pull/13766
Put pr-gate after check-changes by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14009
[diffusion] model: support black-forest-labs/FLUX.2-dev by @mickqian in https://github.com/sgl-project/sglang/pull/14000
fix: correct usage of minimax-m2 deepep moe forward by @yuukidach in https://github.com/sgl-project/sglang/pull/13892
Support internvl on Blackwell (which doesn't support fa3): add SingletonCache support to Vision{Sdpa|Triton|Ascend}Attention by @netanel-haber in https://github.com/sgl-project/sglang/pull/13151
[model-gateway] fix xpu ci by @slin1237 in https://github.com/sgl-project/sglang/pull/14012
[ci] mark skip as success instead of failure by @slin1237 in https://github.com/sgl-project/sglang/pull/14014
Revert "Fix nightly test failures: NSA indexer dtype and CPP radix cache init" by @Fridge003 in https://github.com/sgl-project/sglang/pull/14015
[model gateway][grpc] Add tojson filter to override minijinja's tojson by @CatherineSue in https://github.com/sgl-project/sglang/pull/14013
[ci] allow manual label to trigger ci in rust, change ci order by @slin1237 in https://github.com/sgl-project/sglang/pull/14016
[model-gateway][doc] Update transport terminology to protocol in README.md by @xuwenyihust in https://github.com/sgl-project/sglang/pull/13872
Fix Nvidia nightly test trigger params when it is triggered by parent workflow by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/13966
Update CODEOWNERS for layer and executor files by @hebiao064 in https://github.com/sgl-project/sglang/pull/14020
[Code sync] Fix registration of some ops in grok & Fix oss sync scripts by @merrymercy in https://github.com/sgl-project/sglang/pull/13990
Add stress test workflow by @dougyster in https://github.com/sgl-project/sglang/pull/13937
Temporarily disable test_update_weights_from_disk.py in CI by @alisonshao in https://github.com/sgl-project/sglang/pull/14021
[model-gateway] Fix flaky test_circuit_breaker_half_open_failure_reopens by @XinyueZhang369 in https://github.com/sgl-project/sglang/pull/14019
Add adapter_model.safetensors to corruption validation for LoRA by @alisonshao in https://github.com/sgl-project/sglang/pull/14022
fix: cuda graph issue while running longcat_flash by @tianhaoz95 in https://github.com/sgl-project/sglang/pull/14007
Fix nightly test failure: CPP radix cache init by @alisonshao in https://github.com/sgl-project/sglang/pull/14018
Support KTransformers for Qwen3-VL moe by @mrhaoxx in https://github.com/sgl-project/sglang/pull/13983
Add nightly test support to unified run_suite.py by @alisonshao in https://github.com/sgl-project/sglang/pull/13941
Support nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 (and nvidia/C-RADIOv2-H) by @netanel-haber in https://github.com/sgl-project/sglang/pull/12277
[feat] update bucketed weights from distributed by @ShawnY112358 in

Note truncated.

Open source →

0.5.56 Nov 2025Security

Fix(security): block unsafe pickle deserialization to mitigate CVE-2025-10164 by @thelongestusernameofall in https://github.com/sgl-project/sglang/pul…

Highlights

Day 0 support for Kimi-K2-Thinking https://huggingface.co/moonshotai/Kimi-K2-Thinking
Day 0 support for Minimax-M2 https://huggingface.co/MiniMaxAI/MiniMax-M2
Video and image generation support https://lmsys.org/blog/2025-11-07-sglang-diffusion/
Q4 Roadmap: https://github.com/sgl-project/sglang/issues/12780
Blackwell kernel optimizations and MoE runner backend refactor
Overlap spec and prefill cuda graph support more models

What's Changed

[8/n] decouple quantization impl from vllm dependency - gguf srt by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11964
lang: support direct video inference by @mickqian in https://github.com/sgl-project/sglang/pull/9936
Enable Llama 4 + TRTLLM MHA by @b8zhong in https://github.com/sgl-project/sglang/pull/12003
Refactor Triton-kernel MoE runner integration by @Jonahcb in https://github.com/sgl-project/sglang/pull/11795
use flashinfer_trtllm moe runner backend to gain around 10% perf on b200 fp8 dpsk by @b8zhong in https://github.com/sgl-project/sglang/pull/11816
Fix(security): block unsafe pickle deserialization to mitigate CVE-2025-10164 by @thelongestusernameofall in https://github.com/sgl-project/sglang/pull/11909
Revert "lang: support direct video inference" by @merrymercy in https://github.com/sgl-project/sglang/pull/12038
support more model in piecewise cuda graph by @narutolhy in https://github.com/sgl-project/sglang/pull/11745
[Fix] Fix lint to pass CI by @Fridge003 in https://github.com/sgl-project/sglang/pull/12037
Revert "[Fix] Fix lint to pass CI" by @Fridge003 in https://github.com/sgl-project/sglang/pull/12042
fix: fix MMMU loading issue by @ZailiWang in https://github.com/sgl-project/sglang/pull/11759
Opt MHA chunked prefix: merge prefix and extend kv cache to run mha once by @xu-yfei in https://github.com/sgl-project/sglang/pull/10953
Add gguf dependency for cpu/xpu by @ZailiWang in https://github.com/sgl-project/sglang/pull/12041
fix: the hardcode hf repo name comparison for deepseek-ocr by @rainj-me in https://github.com/sgl-project/sglang/pull/12031
Install numactl in Dockerfile for GH200/GB200/GB300 by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11853
[router] Add mTLS Support for Router-to-Worker Communication by @slin1237 in https://github.com/sgl-project/sglang/pull/12019
Tiny cleanup send_single by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12056
Refactoring GLM-4.5 and GLM-4.5V related implementations by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/11800
[Fix] fix missing ipc_name of __getitem__ in some IO structs by @whybeyoung in https://github.com/sgl-project/sglang/pull/12053
fix: bench_serving ITL calculation when using spec-decoding by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/12064
Fix dpsk-r1-fp4 launching crash by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/12063
Revise POINTSV15Chat model by @yuan-luo in https://github.com/sgl-project/sglang/pull/12049
Add 'gguf' to project dependencies by @Muqi1029 in https://github.com/sgl-project/sglang/pull/12046
[Profiler] expand '~' by @Muqi1029 in https://github.com/sgl-project/sglang/pull/11999
[b200] fix piecewise cuda graph launch bug by @BBuf in https://github.com/sgl-project/sglang/pull/12067
Fix multi processing serializer bug by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11958
[Fix]: HiCache hasher failed when EAGLE mode enabled by @leavelet in https://github.com/sgl-project/sglang/pull/12025
adjust dynamic vs static outputs comparison in test_lora_update.py by @glenliu21 in https://github.com/sgl-project/sglang/pull/11884
[router] implement response api get input item function and refactor input/output store by @key4ng in https://github.com/sgl-project/sglang/pull/11924
fix(compile_utils, ep_moe): update environment variable and dtype check by @ishandhanani in https://github.com/sgl-project/sglang/pull/12034
[router] fix ut router config init to use build pattern by @slin1237 in https://github.com/sgl-project/sglang/pull/12084
docs(server-arguments): add allowed options for each argument by @Jonahcb in https://github.com/sgl-project/sglang/pull/11560
[router] migrate app context to builder pattern 1/n by @slin1237 in https://github.com/sgl-project/sglang/pull/12086
[router] migrate app context to builder pattern 2/n by @slin1237 in https://github.com/sgl-project/sglang/pull/12089
[router][grpc] Remove gpt_oss parsers and remove _parser suffix in tool parser files by @CatherineSue in https://github.com/sgl-project/sglang/pull/12091
[1/2] deepseek deterministic: support deterministic inference for deepseek arch models on a single GPU by @zminglei in https://github.com/sgl-project/sglang/pull/12000
Fix: Update blog link by @LucaLow in https://github.com/sgl-project/sglang/pull/12071
perf: trtllm_mla attention backend spec decoding speedup w/ cuda graph by @cicirori in https://github.com/sgl-project/sglang/pull/12093
[2/N]Support DeepSeek-R1 w4a8 low latency deepep by @ayrnb in https://github.com/sgl-project/sglang/pull/8464
Enhance tests in deterministic kernels by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12070
[Doc] Add documentation for DeepSeek V3.2 by @Fridge003 in https://github.com/sgl-project/sglang/pull/11877
[10/N] MoE Refactor: reorganize deepgemm runner in DeepEPMoE by @ch-wan in https://github.com/sgl-project/sglang/pull/12054
Support true on-policy by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12058
[Docs] update sgl-kernel readme by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11379
Fix 'KeyError' for per_token expert distribution recorder by @vipwangerxiao in https://github.com/sgl-project/sglang/pull/9501
Fix kernel version bump file by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12087
[Fix] Set global args in cpu test by @Fridge003 in https://github.com/sgl-project/sglang/pull/12105
chore: bump sgl-kernel version to 0.3.16.post4 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12103
[Auto Sync] Update test_deterministic.py, test_deterministi... (20251024) by @merrymercy in https://github.com/sgl-project/sglang/pull/12083
[router] Refactor data connector architecture with unified storage modules by @key4ng in https://github.com/sgl-project/sglang/pull/12096
fix: release workflow should work on both archs by @ishandhanani in https://github.com/sgl-project/sglang/pull/12110
[bugs] docker file name should be <name>.Dockerfile so it can properly render by @slin1237 in https://github.com/sgl-project/sglang/pull/11869
Clean up server args & Add CI scripts by @merrymercy in https://github.com/sgl-project/sglang/pull/12124
[Misc] Improve the error message of failed import by @DarkSharpness in https://github.com/sgl-project/sglang/pull/12119
[CI] Add ci monitor balance workflow by @BBuf in https://github.com/sgl-project/sglang/pull/11962
Skip TestLlama4LoRA in CI by @lifuhuang in https://github.com/sgl-project/sglang/pull/12098
clean up github tokens by @merrymercy in https://github.com/sgl-project/sglang/pull/12126
Fix Illegal Instruction/IMA errors when using DP attention -- num_tokens_for_logprob calculation by @YAMY1234 in https://github.com/sgl-project/sglang/pull/12115
Fix token for CI monitor by @merrymercy in https://github.com/sgl-project/sglang/pull/12127
Reenable b200 tests by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/11814
Update document index for DeepSeek-v32 docs by @Fridge003 in https://github.com/sgl-project/sglang/pull/12101
Update sgl-kernel version to 0.3.16.post4 by @Fridge003 in https://github.com/sgl-project/sglang/pull/12125
[Doc] Fix format for deepseek v3.2 document by @Fridge003 in https://github.com/sgl-project/sglang/pull/12130
Accelerate deepseek fp4 b200 ci by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/11993
Clean up server launch code and multi tokenizer by @merrymercy in https://github.com/sgl-project/sglang/pull/12132
[Test] Add dsv3.2 nsa backend testing by @Johnsonms in https://github.com/sgl-project/sglang/pull/11936
[docs] upd docker files names everywhere by @vincentzed in https://github.com/sgl-project/sglang/pull/12133
Make bmm batch invariant injection optional by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12118
[Doc] Small update of DeepSeek v3.2 document by @Fridge003 in https://github.com/sgl-project/sglang/pull/12138
docs: update README by @zhyncs in https://github.com/sgl-project/sglang/pull/12139
[router] MCP Manager - Support Connection Pooling, Tool Inventory and Proxy by @slin1237 in https://github.com/sgl-project/sglang/pull/12097
[NVIDIA] Change default quant method for model_opt by @kaixih in https://github.com/sgl-project/sglang/pull/11991
[router] update smg code owners for each component by @slin1237 in https://github.com/sgl-project/sglang/pull/12141
[router] cleaned up all the redundant comments in the config module by @CatherineSue in https://github.com/sgl-project/sglang/pull/12147
Clean up attention backend selection code & Other minor rename by @merrymercy in https://github.com/sgl-project/sglang/pull/12136
[log] Make forward iter count optional by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12116
[misc] depdencies & enviroment flag by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12113
[quantization] AWQ Marlin doesn't work when dtype is bfloat16 by @kevin85421 in https://github.com/sgl-project/sglang/pull/11494
[HiCache]Page head layout IO kernel by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/11615
Do not use MagicMock to mock server_args in tests by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12154
[router][grpc] Fix tool call id in parse_json_schema_response by @CatherineSue in https://github.com/sgl-project/sglang/pull/12152
[router] centralize mcp tool args handling by @slin1237 in https://github.com/sgl-project/sglang/pull/12155
Fix ITL metrics when using openai endpoint with spec by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12156
[Fix] fix allreduce bug in Piecewise Graph by @zyksir in https://github.com/sgl-project/sglang/pull/12106
Support DeepGEMM for deterministic inference by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12142
model: support NVILA and NVILA Lite by @futrime in https://github.com/sgl-project/sglang/pull/10399
Avoid using flashinfer_allreduce_fusion when dp attention is enabled. by @elfiegg in https://github.com/sgl-project/sglang/pull/11632
transfer mrope_position_delta to device when first running by @ash-sigh in https://github.com/sgl-project/sglang/pull/11047
add gitignore for claude code and serena mcp by @slin1237 in https://github.com/sgl-project/sglang/pull/12166
Support MiniMax M2 model by @zhaochenyang20 in https://github.com/sgl-project/sglang/pull/12129
[misc][grpc] Remove duplicate log by @CatherineSue in https://github.com/sgl-project/sglang/pull/12168
[router][grpc] Add ResponsesContext and fix error propagation in responses api by @CatherineSue in https://github.com/sgl-project/sglang/pull/12164
[router] Remove SharedXxxStorage type aliases to make Arc explicit by @CatherineSue in https://github.com/sgl-project/sglang/pull/12171
Remove deprecated --enable-beta-spec argument and fix b200 test by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12167
fix broken deepep/flashmla install in container by adding --no-build-isolation by @ishandhanani in https://github.com/sgl-project/sglang/pull/12170
Remove description for --enable-beta-spec argument by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/12177
chore: bump SGLang version to 0.5.4.post1 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12169
[doc] add example of using w4fp8 for Deepseek by @Kevin-XiongC in https://github.com/sgl-project/sglang/pull/12057
[sgl-route] Optimize the use of constant slices and retain to simplif… by @lengrongfu in https://github.com/sgl-project/sglang/pull/12159
[Fix] Fix cu130 sgl-kernel wheel renaming by @Fridge003 in https://github.com/sgl-project/sglang/pull/12173
docs: update contact by @zhyncs in https://github.com/sgl-project/sglang/pull/12192
[sgl-kernel] feat: Support sm120 cutlass fp8 gemm kernel by @kaln27 in https://github.com/sgl-project/sglang/pull/9403
[sgl-kernel][4/N]Support Expert Specialization Grouped GEMM by @HydraQYH in https://github.com/sgl-project/sglang/pull/12080
GLM-4-0414 and GLM-4.1V Code Refactor by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/12117
Add support for AutoRound quantized models by @WeiweiZhang1 in https://github.com/sgl-project/sglang/pull/10153
Optimize triton_mrope with torch compile by @yuan-luo in https://github.com/sgl-project/sglang/pull/12112
Fix crash after flush cache by @cctry in https://github.com/sgl-project/sglang/pull/12107
[Detokenizer Manager] Cleanup state when reqs are finished by @Muqi1029 in https://github.com/sgl-project/sglang/pull/12205
fix(metrics): double times add_latency for DECODE_BOOTSTRAP by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/12209
improve mimax-m2 rmsnorm precision by @haichao592 in https://github.com/sgl-project/sglang/pull/12186
check_offload_progress more frequently by @pansicheng in https://github.com/sgl-project/sglang/pull/11656
[Feature] PD-Multiplexing Context and Scheduler. by @ykcombat in https://github.com/sgl-project/sglang/pull/11592
rope xpu: fix missing argument 'fused_set_kv_buffer_arg' and replace native with sgl_kernel_xpu impl by @chunyuan-w in https://github.com/sgl-project/sglang/pull/12006
Add support for Matryoshka embeddings (#126) by @satyamk7054 in https://github.com/sgl-project/sglang/pull/11142
fix: AttributeError: 'NixlKVManager' object has no attribute 'prefill_tp_size_table' by @gongwei-130 in https://github.com/sgl-project/sglang/pull/12234
Compiling rope while preserving true on policy by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12161
[Auto Sync] Update scheduler.py, spec_info.py, run_suite.py... (20251027) by @zhyncs in https://github.com/sgl-project/sglang/pull/12235
Support running FP4 Deepseek on SM120. by @weireweire in https://github.com/sgl-project/sglang/pull/11708
Add env var to control custom Triton kernel cache and set CSGMV as default backend. by @lifuhuang in https://github.com/sgl-project/sglang/pull/12176
Use explicit uint64 dtype for Tensor data_ptr() to avoid overflow by @jianan-gu in https://github.com/sgl-project/sglang/pull/11994
Update openai package version to 2.6.1 by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/12222
[2/2] Use moe_sum_reduce cuda kernel by @yuan-luo in https://github.com/sgl-project/sglang/pull/10654
docker: add CUDA13 support in dockerfile and update GDRCopy/NVSHMEM for blackwell support by @ishandhanani in https://github.com/sgl-project/sglang/pull/11517
[router] remove code duplication by @slin1237 in https://github.com/sgl-project/sglang/pull/12245
[DeepseekV32] Enable flashmla_prefill kernel with fp8 kvcache by @hlu1 in https://github.com/sgl-project/sglang/pull/11655
Add per-request retraction count by @scottjlee in https://github.com/sgl-project/sglang/pull/11177
Opt fused triton moe: add tma for down proj kernel by @xu-yfei in https://github.com/sgl-project/sglang/pull/10567
Support releasing CUDA graph memory when paused by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7873
[router] use mcp struct from sdk and clean up code across codebase by @slin1237 in https://github.com/sgl-project/sglang/pull/12249
[router] configure workflow retries and timeout based on routerConfig by @slin1237 in https://github.com/sgl-project/sglang/pull/12252
Feature/Add GET endpoint to query loaded LoRA adapters by @ConnorLi96 in https://github.com/sgl-project/sglang/pull/12229
[hotfix] Incorrect CombineOverlapArgs in SBO by @ch-wan in https://github.com/sgl-project/sglang/pull/12230
[Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/10804
[Bug fix] [PP] fix wrong dtype for quantified model by @XucSh in https://github.com/sgl-project/sglang/pull/12247
Fix potential eos bug on decode instance when PD is enabled by @ShangmingCai in https://github.com/sgl-project/sglang/pull/12206
Revert "[Feature] PD-Multiplexing Context and Scheduler." by @zhyncs in https://github.com/sgl-project/sglang/pull/12267
chore: cleanup quant deps by @zhyncs in https://github.com/sgl-project/sglang/pull/12268
[router] Fix type unmatch during validation by @key4ng in https://github.com/sgl-project/sglang/pull/12257
Modify rocm.Dockerfile by @sogalin in https://github.com/sgl-project/sglang/pull/12274
[router] upgrade grpc dependency and py 3.13 3.14 support by @slin1237 in https://github.com/sgl-project/sglang/pull/12284
Fix 'BypassedTopKOutput' object has no attribute 'topk_weights' for DeepEP by @trevor-m in https://github.com/sgl-project/sglang/pull/12231
Tiny fix sgl-kernel related CI installing the wrong binary by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12283
doc for logit_bias by @whybeyoung in https://github.com/sgl-project/sglang/pull/12188
Use Flashinfer TRT-LLM as Llama 4 compatible MoE backend by @b8zhong in https://github.com/sgl-project/sglang/pull/11928
[rust][ci] Add end-to-end tests for Oracle history backend by @key4ng in https://github.com/sgl-project/sglang/pull/12233
[router] support arm, windows, mac, linux, reduce wheel size and number by @slin1237 in https://github.com/sgl-project/sglang/pull/12285
fix seqlen bug for trtllm_mla's draft_extend by @bmac3 in https://github.com/sgl-project/sglang/pull/12295
Update deepseek_v32.md by @hlu1 in https://github.com/sgl-project/sglang/pull/12296
Super tiny fix expert distribution dump error by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12271
[router][grpc] Fix inconsistent behavior of conversation_id not found by @CatherineSue in https://github.com/sgl-project/sglang/pull/12299
fix: Llama 4 BF16 load on Blackwell by @b8zhong in https://github.com/sgl-project/sglang/pull/12308
Add continuous_usage_stats support for streaming responses by @BBuf in https://github.com/sgl-project/sglang/pull/12241
[hotfix] missing w13_weight_fp8 and w2_weight_fp8 in UE8M0 requantization by @ch-wan in https://github.com/sgl-project/sglang/pull/12259
[hotfix] Fix pytest not found in CI by @Fridge003 in https://github.com/sgl-project/sglang/pull/12311
a tiny fix for support deepseek bf16 weights by @Gao016 in https://github.com/sgl-project/sglang/pull/12313
[metrics][EPLB]: Support selected count of physical experts on each GPU by @acelyc111 in https://github.com/sgl-project/sglang/pull/9825
doc: improve modelopt error description by @lianakoleva in https://github.com/sgl-project/sglang/pull/12269
EPLB: prefer to use physical experts in the same gpu or node by @acelyc111 in https://github.com/sgl-project/sglang/pull/10874
Add Batch‑Invariant RMSNorm by @zyzshishui in https://github.com/sgl-project/sglang/pull/12144
followup fix for llama 4 trtllm flashinfer backend by @b8zhong in https://github.com/sgl-project/sglang/pull/12314
[Deepseek V3.2] Enable flashmla_auto with MTP by @hlu1 in https://github.com/sgl-project/sglang/pull/12294
feat: preview filename from tuning_fused_moe_triton.py by @lianakoleva in https://github.com/sgl-project/sglang/pull/12276
[ci] Try fixing broken CIs by @Fridge003 in https://github.com/sgl-project/sglang/pull/12317
Refactor abortion in event loop by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12312
[Test] Fix session control test by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12336
Eagle3 DP attention for Qwen3 MoE by @qhsc in https://github.com/sgl-project/sglang/pull/12002
feat: return partial generation results when aborting requests in waiting queue by @guoyuhong in https://github.com/sgl-project/sglang/pull/11673
[Bug fix] trace: fix import error in mini_lb if sgl-router image does not install sglang by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/12338
[router] fix router release workflow and add build test in PR by @CatherineSue in https://github.com/sgl-project/sglang/pull/12315
Triton fused_moe_kernel support ep moe tuning by @BBuf in https://github.com/sgl-project/sglang/pull/12343
[Fix] fix type issue of env flag value MODELOPT_MAX_TOKENS_PER_EXPERT by @zejunchen-zejun in https://github.com/sgl-project/sglang/pull/11709
[bug] fix router pypi license file by @slin1237 in https://github.com/sgl-project/sglang/pull/12345
fix: llama 4 + trtllm gen + fp8 kv cache incompatibility by @b8zhong in https://github.com/sgl-project/sglang/pull/12347
[2/2] Deepseek deterministic: support deepseek v3 deterministic inference on 8 x H200 by @zminglei in https://github.com/sgl-project/sglang/pull/12095
Fix Flashinfer Backend for SM120 Usage by @weireweire in https://github.com/sgl-project/sglang/pull/12325
[router] refactor mcp to use LRU and fix pooling bug by @CatherineSue in https://github.com/sgl-project/sglang/pull/12346
support cutlass fp4 kernel in sm120 by @AichenF in https://github.com/sgl-project/sglang/pull/11737
[bug] fix router installation to include additional dependency by @slin1237 in https://github.com/sgl-project/sglang/pull/12348
[router] update router docker to use maturin and build from local by @CatherineSue in https://github.com/sgl-project/sglang/pull/12350
Fix Duplicate Classmethod in spec_info.py by @hebiao064 in https://github.com/sgl-project/sglang/pull/12354
[CI] Add Llama 3.1 8B FP4 to B200 CI by @b8zhong in https://github.com/sgl-project/sglang/pull/12182
Fuse wk and weight_proj in Indexer for DeepSeekV3.2-FP4 by @trevor-m in https://github.com/sgl-project/sglang/pull/12094
[router] Harmony Pipeline: Chat Completion & Responses API with MCP Support by @slin1237 in https://github.com/sgl-project/sglang/pull/12153
[bugfix] fix deepseekvl2 and deepseek_ocr model type conflict by @leihuang-sketch in https://github.com/sgl-project/sglang/pull/12050
[Ckpt Engine] feat: new sglang entrypoint support for update by @stmatengss in https://github.com/sgl-project/sglang/pull/12216
[Perf] Optimize multimodal mm_inputs process in scheduler by @yuan-luo in https://github.com/sgl-project/sglang/pull/11910
[NPU] fix pp_size>1 by @Makcum888e in https://github.com/sgl-project/sglang/pull/12195
Super tiny add tag for benchmark scripts by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12340
Allow benchmarking tool to handle empty response by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12174
Super tiny fix AMD ci by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12378
Import flash_mla from sgl-kernel by @Fridge003 in https://github.com/sgl-project/sglang/pull/12135
[Bug fix][PP] fix deadlock with tie_word_embeddings by @XucSh in https://github.com/sgl-project/sglang/pull/12362
[fix] added image token as prefix for deepseek-ocr by @Tushar-ml in https://github.com/sgl-project/sglang/pull/12358
Fix DeepSeek chat templates to handle tool call arguments type checking (#11700) by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12123
[Feature] Initial eagle3 support for Deepseek-like models by @JensenFire in https://github.com/sgl-project/sglang/pull/12319
Enable fast silu-and-mul-and-quant fused kernel by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11806
[Test] Enhance radix cache test for spec cases by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12394
[NPU] bugfix for Qwen3-Next and performance update by @iforgetmyname in https://github.com/sgl-project/sglang/pull/11969
[Feature] Support DeepSeek MTP on NPU by @iforgetmyname in https://github.com/sgl-project/sglang/pull/11897
Revert "Triton fused_moe_kernel support ep moe tuning" by @BBuf in https://github.com/sgl-project/sglang/pull/12377
[sgl-kernel] upd deepgemm hash to rebased commit by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11960
[router] harmony responses api streaming support by @slin1237 in https://github.com/sgl-project/sglang/pull/12395
[docker] clean up main dockerfile for router and dev configurations by @CatherineSue in https://github.com/sgl-project/sglang/pull/12364
feat: add EP support in tuning by @Chen-0210 in https://github.com/sgl-project/sglang/pull/12012
[router] use safety_identifier replace user on chat history storage by @lengrongfu in https://github.com/sgl-project/sglang/pull/12185
[CI Monitor] Fix ci_monitor perf analyzer bug by @BBuf in https://github.com/sgl-project/sglang/pull/12281
[router] Fix safety_identifier missing by @key4ng in https://github.com/sgl-project/sglang/pull/12404
[ci] Fix ci_install_deepep by @Fridge003 in https://github.com/sgl-project/sglang/pull/12375
Update news section in README.md by @merrymercy in https://github.com/sgl-project/sglang/pull/12409
[router] Function call support for openai router Responses API by @key4ng in https://github.com/sgl-project/sglang/pull/12386
minor code sync by @merrymercy in https://github.com/sgl-project/sglang/pull/12403
[Bug fix][PD Dissaggregation] fix prefill hanging issue with PP and DP Attention, by @popsiclexu in https://github.com/sgl-project/sglang/pull/12368
[NVIDIA] Add CI workloads for GB200 by @kaixih in https://github.com/sgl-project/sglang/pull/12242
[router] web_search_preview tool basic implementation by @key4ng in https://github.com/sgl-project/sglang/pull/12290
[router] 0.2.2 release by @slin1237 in https://github.com/sgl-project/sglang/pull/12399
enable cudaProfilerApi for one batch benchmarking by @lpc0220 in https://github.com/sgl-project/sglang/pull/11116
[Refactor] tuning_fused_moe for MLLM and small refactor by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/11224
[DeepSeekV32] Bug fix to ensure page_table and result in same type by @Johnsonms in https://github.com/sgl-project/sglang/pull/12300
[CI] fix tests' time estimation by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12401
Reserved abortion API when retracting by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12425
Fix the shared expert & routed expert overlap in Llama 4 by @b8zhong in https://github.com/sgl-project/sglang/pull/12405
feat: Add Non-intrusive Tensor Dumping for Model Inference by @guoyuhong in https://github.com/sgl-project/sglang/pull/10566
feat: support trtllm_mha FP8 query attention kernel by @elvischenv in https://github.com/sgl-project/sglang/pull/12307
[Bugfix]: distinguish processors for deepseek_vl2 and deepseek_ocr to p… by @bppps in https://github.com/sgl-project/sglang/pull/12384
[ci] install released version router by @key4ng in https://github.com/sgl-project/sglang/pull/12410
Revert "fix llama4 kv cache layout" by @b8zhong in https://github.com/sgl-project/sglang/pull/12437
Add trait for BasePrefixCache by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12436
[CI] Add more bins for 1-gpu CI test by @Fridge003 in https://github.com/sgl-project/sglang/pull/12422
[bugfix] set is_prefill_only=false when mixed_chunk by @Bruce-x-1997 in https://github.com/sgl-project/sglang/pull/10889
Clean up sgl kernel by @merrymercy in https://github.com/sgl-project/sglang/pull/12413
[CI] fix possible port conflicts. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12452
Fix ci install to allow prerelease by @merrymercy in https://github.com/sgl-project/sglang/pull/12449
fix: Add default value for backend in sample_mmmu_requests by @ZailiWang in https://github.com/sgl-project/sglang/pull/12256
Enable bailing_moe to support TP=16 by @guoyuhong in https://github.com/sgl-project/sglang/pull/12369
fix:watchdog thread exception by @Kindyaa in https://github.com/sgl-project/sglang/pull/12328
Simplify watchdog by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12463
[Bug fix] Fix severe memory waste issue with torch.empty pin_memory by @sjtushenhai in https://github.com/sgl-project/sglang/pull/12266
Feat: deepseek-ocr logits processor by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/12415
Fix lint in deepseek-ocr by @ispobock in https://github.com/sgl-project/sglang/pull/12470
[Test] Add Functional Tests for Penalty Parameters by @neelabhsinha in https://github.com/sgl-project/sglang/pull/11931
[Bug] OOM (Out-of-Memory) errors for extreme testing scenarios (min_tokens=2) by @LuYanFCP in https://github.com/sgl-project/sglang/pull/11757
[Feature] PD-Multiplexing Context and Scheduler, lazy import spatial. by @ykcombat in https://github.com/sgl-project/sglang/pull/12275
[VLM] Optimize async mm data process mechanism by @yuan-luo in https://github.com/sgl-project/sglang/pull/12066
fix default env var for mooncake store by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/12429
add served model name in bench serving by @carolove in https://github.com/sgl-project/sglang/pull/12428
Tiny assert no running requests when releasing memory to avoid IMA by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12341
fix: dummy health check server not accessible on non-zero rank nodes by @ishandhanani in https://github.com/sgl-project/sglang/pull/12297
Fix run benchmark by @ispobock in https://github.com/sgl-project/sglang/pull/12473
Add env var to disable FA4 warmup by @cicirori in https://github.com/sgl-project/sglang/pull/12430
Try to allow NCCL cumem for multi node nvlink case by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11987
Support Kimi Linear by @ispobock in https://github.com/sgl-project/sglang/pull/12469
[CI] Fix kernel installation on aarch runners by @Fridge003 in https://github.com/sgl-project/sglang/pull/12475
fa3 & trtllm_mha spec overlap by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/11874
chore: bump SGLang version to 0.5.4.post2 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12439
Tiny fix eos handling for PD disaggregation by @ShangmingCai in https://github.com/sgl-project/sglang/pull/12334
Forward unknown tool calls instead of dropping by @Surya-Gunukula in https://github.com/sgl-project/sglang/pull/12226
Use sgl fp4 quant kernel by default by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/12482
[hot fix] Remove from python.sglang.xxx by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12483
perf: trtllm mla performance minor improvements by @cicirori in https://github.com/sgl-project/sglang/pull/12435
Filter tokenizer warning for kimi models by @ispobock in https://github.com/sgl-project/sglang/pull/12485
[CI] Build aarch64 kernels for sgl-kernel test by @Fridge003 in https://github.com/sgl-project/sglang/pull/12480
[Hotfix] Remove extra comment in sgl-kernel README by @Fridge003 in https://github.com/sgl-project/sglang/pull/12500
[feat] Add SGLANG_TOOL_STRICT_LEVEL for tool-call behavior control by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/12423
Reduce docker image size. mount cache when use pip/cargo build by @whybeyoung in https://github.com/sgl-project/sglang/pull/12238
[HICache / PD]: Support offloading incremental KV cache in decode side. by @hzh0425 in https://github.com/sgl-project/sglang/pull/11966
[Deterministic] add deepseek v3 deterministic inference CI test by @zminglei in https://github.com/sgl-project/sglang/pull/12412
[Bug] test_flashattn_mla_backend errors in Hopper #12487 by @Johnsonms in https://github.com/sgl-project/sglang/pull/12488
Update Mooncake EP's a2a interface by @UNIDY2002 in https://github.com/sgl-project/sglang/pull/12391
[CI][NPU] remove pypi mirror site that hangs ci dependency installation by @iforgetmyname in https://github.com/sgl-project/sglang/pull/12499
[Ascend] Add Ascend NPU support for sglang.check_env & rework proposal by @Alexhaoge in https://github.com/sgl-project/sglang/pull/11052
[Feature] Qwen3-Next & FLA: Support MTP topk>1; Up to 6% faster by @byjiang1996 in https://github.com/sgl-project/sglang/pull/11133
[CI] Move some Lora/Deterministic CI tests to nightly by @Fridge003 in https://github.com/sgl-project/sglang/pull/12507
Migrate weak_ref_tensor to sgl-kernel by @BBuf in https://github.com/sgl-project/sglang/pull/12505
feat: Add FP4 (E2M1) KV Cache Support with Quantization Utilities for MLA by @JackChuang in https://github.com/sgl-project/sglang/pull/10078
chore: bump sgl-kernel version to 0.3.16.post5 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12511
[FEAT] Shared mem pool based cuda ipc for multi-modal data transport by @kousakawang in https://github.com/sgl-project/sglang/pull/11917
Add prefix for torch symm mem by @yuan-luo in https://github.com/sgl-project/sglang/pull/12506
[ServerArgs] allow --mamba-ssm-dtype extend by @hanming-lu in https://github.com/sgl-project/sglang/pull/12481
[Fix] concat_mla_absorb_q_kernel fails for long inputs by @bingps in https://github.com/sgl-project/sglang/pull/12453
Super tiny fix naming in bench serving scripts by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12515
move all get_stream in sgl_kernel to c++ to reduce the launch overhead by @merrymercy in https://github.com/sgl-project/sglang/pull/12521
[Refact] Remove hardcoded KV cache dimension in MLATokenToKVPool by @Johnsonms in https://github.com/sgl-project/sglang/pull/12502
[Bug] Fix Intern-S1 model accuracy and support /generate interface with input_ids by @hhaAndroid in https://github.com/sgl-project/sglang/pull/12367
chore: upgrade flashinfer 0.5.0 by @zhyncs in https://github.com/sgl-project/sglang/pull/12523
[hotfix] Remove flashinfer-jit-cache from pyproject by @Fridge003 in https://github.com/sgl-project/sglang/pull/12530
fix: move dummy format loader check before quantization checks by @cicirori in https://github.com/sgl-project/sglang/pull/12532
chore: upgrade mooncake 0.3.7.post1 by @ShangmingCai in https://github.com/sgl-project/sglang/pull/12541
fix: Fix KTransformers hybrid inference with int8 quantization and format by @Atream in https://github.com/sgl-project/sglang/pull/12536
Conditionally recapture cuda graph after model weight update from disk by @harrisonlimh in https://github.com/sgl-project/sglang/pull/12060
[spec v2] Fix output repetition by speculative sampling error by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12561
[hot-fix] Fix broken CI by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12564
fix: fix the bug which leads qwen2_5_vl to crash with mixed_chunk by @PanJason in https://github.com/sgl-project/sglang/pull/11330
Fix error when calling quantization by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12548
[Test] Add parameters to SRTRunner by @Jonahcb in https://github.com/sgl-project/sglang/pull/12227
[ROCm] Update Mooncake to v0.3.7.post1 and add -DUSE_HIP=ON to rocm.Dockerfile by @yeahdongcn in https://github.com/sgl-project/sglang/pull/12560
Reduce the overhead of nccl symmetric memory by @merrymercy in https://github.com/sgl-project/sglang/pull/12524
tiny optimize for bench serving by @yizhang2077 in https://github.com/sgl-project/sglang/pull/12553
Super tiny allow profile activities in bench_serving by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12549
Super tiny dump server info such as args in bench for post analysis by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12550
update usage of trtllm_fp8_per_tensor_scale_moe by @b8zhong in https://github.com/sgl-project/sglang/pull/12569
[router][grpc] Consolidate error messages build in error.rs by @CatherineSue in https://github.com/sgl-project/sglang/pull/12301
Remove the dependency of nccl.h in symmetric memory by @merrymercy in https://github.com/sgl-project/sglang/pull/12571
[chore] Fix update_kernel_whl_index script for multiple cuda version by @Fridge003 in https://github.com/sgl-project/sglang/pull/12519
Enable mixed type LayerNorm kernel for NSA indexer by @akhilg-nv in https://github.com/sgl-project/sglang/pull/12044
Super tiny add UT for copy_to_gpu_no_ce by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12270
[Doc] fix miss index for production request trace by @stmatengss in https://github.com/sgl-project/sglang/pull/12547
[GDN/SWA] mamba and swa radix cache edge case fix by @hanming-lu in https://github.com/sgl-project/sglang/pull/12111
[Qwen3 VL] Add LoRA support for Qwen 3 VL by @Jonahcb in https://github.com/sgl-project/sglang/pull/12165
test: support return logprobs in bench_offline_throughput test by @aftersnow in https://github.com/sgl-project/sglang/pull/12462
Tiny fix ExpertDistributionReq error by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11760
fix: respect --ignore-eos in PD case for benchmarking by @ishandhanani in https://github.com/sgl-project/sglang/pull/12597
Improve the metrics for PD by @merrymercy in https://github.com/sgl-project/sglang/pull/12580
Enable memory saver for hybrid model by @ocss884 in https://github.com/sgl-project/sglang/pull/11974
Restore torch defaults between sgl-kernel tests by @benbarsdell in https://github.com/sgl-project/sglang/pull/11131
feat: limit peak memory usage when computing logprobs by @aftersnow in https://github.com/sgl-project/sglang/pull/6318
[router][grpc] Restructure modules and code clean up by @CatherineSue in https://github.com/sgl-project/sglang/pull/12598
Add --speculative-moe-runner-backend server arg by @trevor-m in https://github.com/sgl-project/sglang/pull/10183
[Deterministic] Optimize bmm_batch_invariant op by @zminglei in https://github.com/sgl-project/sglang/pull/12522
chore: bump mooncake version to 0.3.7.post2 by @ShangmingCai in https://github.com/sgl-project/sglang/pull/12599
[sepc-v2] Fix imcompatibility with constrained decoding by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12615
Support aggregating engine metrics in sgl-router by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11456
Ensure GPU work is finished when release memory occupation call is finished by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12592
Add sanity checks when a test file is not added to CI (reland) by @fzyzcjy in https://github.com/sgl-project/sglang/pull/12594
[router][grpc] Fix model validation, tool call check, streaming logic and misc in responses by @CatherineSue in https://github.com/sgl-project/sglang/pull/12616
[HotFix] Disable torch dynamo for mrope_triton kernel by @yuan-luo in https://github.com/sgl-project/sglang/pull/12593
Fix skip layer in get_quant_method by @ispobock in https://github.com/sgl-project/sglang/pull/12632
[Test] Merge all constrained decoding tests. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12633
Add io struct naming check back by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12634
Fix output_ids inconsistency by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12628
fix: Lazy import mooncake-ep to fix extra gpu contexts being created by @trevor-m in https://github.com/sgl-project/sglang/pull/12641
[hotfix] Fix deepep w4a8 bug by @Fridge003 in https://github.com/sgl-project/sglang/pull/12642
[Auto Sync] Update scheduler_metrics_mixin.py, collector.py (20251104) by @merrymercy in https://github.com/sgl-project/sglang/pull/12647
[Bug] Fix NSA Backend KV-Buffer Shape Mismatch in DeepSeek-V3.2 by @Johnsonms in https://github.com/sgl-project/sglang/pull/12645
[NVIDIA] Fix wrong symmetric sizes for fp4 cases by @kaixih in https://github.com/sgl-project/sglang/pull/12640
[router][grpc] Fix index issues in reasoning content and missing streaming events by @CatherineSue in https://github.com/sgl-project/sglang/pull/12650
Revert "Enable memory saver for hybrid model" by @Fridge003 in https://github.com/sgl-project/sglang/pull/12648
Add multi-GPU configurations to nightly-test.yml by @alisonshao in https://github.com/sgl-project/sglang/pull/12585
[fix] Handle escaped characters in GLM tool call parser to prevent double serialization by @soaringk in https://github.com/sgl-project/sglang/pull/12456
[router][grpc] Emit OutputItemDone event and store output item array by @CatherineSue in https://github.com/sgl-project/sglang/pull/12656
Register allgather/reducescatter buffers with symm memory by @nvcastet in https://github.com/sgl-project/sglang/pull/12572
chore: bump SGLang version to 0.5.4.post3 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12639
[NVIDIA] Fix cutedsl backend of MoE by @kaixih in https://github.com/sgl-project/sglang/pull/12353
[PD-Disagg] Check finish after pop tranferred by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12638
fix typo of args description in sglang.profiler by @ai-easy-cpu in https://github.com/sgl-project/sglang/pull/12486
[Dockerfile] Speed up docker image building by @acelyc111 in https://github.com/sgl-project/sglang/pull/8784
Fix VLLM dependency test by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12670
[Feature] add --lora-request-distribution arg to bench_serving.py and support skewed and distinct workloads by @glenliu21 in https://github.com/sgl-project/sglang/pull/12175
[router][grpc] Implement tool_choice support for Responses API by @CatherineSue in https://github.com/sgl-project/sglang/pull/12668
Expand and update test coverage for AMD CI by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/10044
fix: add seed bench_serving to cache key, remove redundant function definition. by @cicirori in https://github.com/sgl-project/sglang/pull/12680
[Profiler] Add SGLANG_PROFILE_RECORD_SHAPES for recording shapes when profiling by @zejunchen-zejun in https://github.com/sgl-project/sglang/pull/11641
fix trtllm_mla attention backend when disabling cuda graph. by @cicirori in https://github.com/sgl-project/sglang/pull/12687
Refactor --debug-tensor-dump-layers to list by @guoyuhong in https://github.com/sgl-project/sglang/pull/12691
[Grammar Fix] GLM-4-MOE self.first_k_dense_replace is undefined. by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/12455
add Kimi k2 reasoning parser by @MoyanZitto in https://github.com/sgl-project/sglang/pull/12702
Commented out b200 tests due to runner shortage by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12609
[CI] Fix qwen3-vl lora nightly ci by @Fridge003 in https://github.com/sgl-project/sglang/pull/12708
Fix server args for gpt oss so users can override the moe runner backend by @merrymercy in https://github.com/sgl-project/sglang/pull/12696
[router][grpc] Support streaming parsing with Tool Choice in chat completions API by @CatherineSue in https://github.com/sgl-project/sglang/pull/12677
feat: initial multimodal-gen support by @mickqian in https://github.com/sgl-project/sglang/pull/12484
Enable Aiter Attention for VL model by @Yuechguo in https://github.com/sgl-project/sglang/pull/12699
[router] fix: validate HTTP status codes in health check by @wyx-0203 in https://github.com/sgl-project/sglang/pull/12631
Support Expert Deferral Mechanism in KTransformers by @Atream in https://github.com/sgl-project/sglang/pull/12586
Add mm_fp4 trtllm backend by @wenscarl in https://github.com/sgl-project/sglang/pull/12406
[NVIDIA] Fix unit test of MoE and add it to nightly ci by @kaixih in https://github.com/sgl-project/sglang/pull/12709
[misc] Add labeler for automatic labeling by @CatherineSue in https://github.com/sgl-project/sglang/pull/12710
[router][ci] speed up python binding to 1.5 min by @key4ng in https://github.com/sgl-project/sglang/pull/12673
Fix CI and style by @merrymercy in https://github.com/sgl-project/sglang/pull/12658
Revert "Commented out b200 tests due to runner shortage (#12609)" by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12712
[misc] Change sync-labels to false by @CatherineSue in https://github.com/sgl-project/sglang/pull/12714
[router][grpc] Make harmony parser checks recipient first before channel by @CatherineSue in https://github.com/sgl-project/sglang/pull/12713
[router][quick fix] Add minimal option for reasoning effort in spec by @key4ng in https://github.com/sgl-project/sglang/pull/12711
[router] add basic ci tests for gpt-oss model support by @key4ng in https://github.com/sgl-project/sglang/pull/12651
fix labeler by @key4ng in https://github.com/sgl-project/sglang/pull/12718
[ci] fix permission by @key4ng in https://github.com/sgl-project/sglang/pull/12729
[chore]Remove dockerfile from target file of bump kernel version by @Fridge003 in https://github.com/sgl-project/sglang/pull/12728
[CPU] Upgrade default PT version to 2.9 by @ZailiWang in https://github.com/sgl-project/sglang/pull/12611
Revert "[ci] fix permission" by @key4ng in https://github.com/sgl-project/sglang/pull/12732
Revert "[router] web_search_preview tool basic implementation" by @key4ng in https://github.com/sgl-project/sglang/pull/12716
fix sgl-kernel version by @gongwei-130 in https://github.com/sgl-project/sglang/pull/12723
[chore] SGLang tag management in Dockerfile by @Fridge003 in https://github.com/sgl-project/sglang/pull/12734
Add nightly test multi gpu configs by @alisonshao in https://github.com/sgl-project/sglang/pull/12721
DeepSeek-V3.2: Add Adaptive MHA Attention Pathway for Short-Sequence Prefill by @YAMY1234 in https://github.com/sgl-project/sglang/pull/11892
Temporarily fix missing routed_scaling_factor for CompressedTensorsWNA16MoEMethod by @Atream in https://github.com/sgl-project/sglang/pull/12738
[chore] Fix triton installation for cu13 image by @Fridge003 in https://github.com/sgl-project/sglang/pull/12742
keep attention backend document up to date by @b8zhong in https://github.com/sgl-project/sglang/pull/12741
[Fix]Tiny fix in Dockerfile by @Fridge003 in https://github.com/sgl-project/sglang/pull/12748
[router][grpc] Support mixin tool calls in Responses API by @CatherineSue in https://github.com/sgl-project/sglang/pull/12736
fix: tiny fix cli by @mickqian in https://github.com/sgl-project/sglang/pull/12744
[router][ci] Disable cache by @key4ng in https://github.com/sgl-project/sglang/pull/12752
fix mamba prefix cache leak caused by abort by @yizhang2077 in https://github.com/sgl-project/sglang/pull/12693
[BUGFIX] fix output_ids in abort by @yizhang2077 in https://github.com/sgl-project/sglang/pull/12737
[GDN] Fuse b.sigmoid(), fused_gdn_gating and unsqueeze into one kernel: up to 0.85% e2e speedup by @byjiang1996 in https://github.com/sgl-project/sglang/pull/12508
[VLM] Optimize qwen_vl preprocess_video by @yuan-luo in https://github.com/sgl-project/sglang/pull/12240
Add timing metrics for requests by @cicirori in https://github.com/sgl-project/sglang/pull/12646
fix qwen3-omni audio length < 30s by @jiapingW in https://github.com/sgl-project/sglang/pull/12674
docs: document video-capable multimodal models by @WazupSteve in https://github.com/sgl-project/sglang/pull/12565
fix ci by @key4ng in https://github.com/sgl-project/sglang/pull/12760
[Refactor] Refactor fused_moe_triton tuning tools: extract shared utils, add EP/MLLM support, reduce overhead by @BBuf in https://github.com/sgl-project/sglang/pull/12440
Update dsv3 quantization auto setting for sm100 by @ispobock in https://github.com/sgl-project/sglang/pull/12778
chore: bump SGLang version to 0.5.5 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12739

New Contributors

@thelongestusernameofall made their first contribution in https://github.com/sgl-project/sglang/pull/11909
@LucaLow made their first contribution in https://github.com/sgl-project/sglang/pull/12071
@vipwangerxiao made their first contribution in https://github.com/sgl-project/sglang/pull/9501
@Johnsonms made their first contribution in https://github.com/sgl-project/sglang/pull/11936
@ash-sigh made their first contribution in https://github.com/sgl-project/sglang/pull/11047
@Kevin-XiongC made their first contribution in https://github.com/sgl-project/sglang/pull/12057
@kaln27 made their first contribution in https://github.com/sgl-project/sglang/pull/9403
@haichao592 made their first contribution in https://github.com/sgl-project/sglang/pull/12186
@satyamk7054 made their first contribution in https://github.com/sgl-project/sglang/pull/11142
@weireweire made their first contribution in https://github.com/sgl-project/sglang/pull/11708
@bmac3 made their first contribution in https://github.com/sgl-project/sglang/pull/12295
@Gao016 made their first contribution in https://github.com/sgl-project/sglang/pull/12313
@lianakoleva made their first contribution in https://github.com/sgl-project/sglang/pull/12269
@zyzshishui made their first contribution in https://github.com/sgl-project/sglang/pull/12144
@zejunchen-zejun made their first contribution in https://github.com/sgl-project/sglang/pull/11709
@AichenF made their first contribution in https://github.com/sgl-project/sglang/pull/11737
@JensenFire made their first contribution in https://github.com/sgl-project/sglang/pull/12319
@Chen-0210 made their first contribution in https://github.com/sgl-project/sglang/pull/12012
@popsiclexu made their first contribution in https://github.com/sgl-project/sglang/pull/12368
@lpc0220 made their first contribution in https://github.com/sgl-project/sglang/pull/11116
@elvischenv made their first contribution in https://github.com/sgl-project/sglang/pull/12307
@sjtushenhai made their first contribution in https://github.com/sgl-project/sglang/pull/12266
@LuYanFCP made their first contribution in https://github.com/sgl-project/sglang/pull/11757
@carolove made their first contribution in https://github.com/sgl-project/sglang/pull/12428
@Surya-Gunukula made their first contribution in https://github.com/sgl-project/sglang/pull/12226
@Alexhaoge made their first contribution in https://github.com/sgl-project/sglang/pull/11052
@JackChuang made their first contribution in https://github.com/sgl-project/sglang/pull/10078
@bingps made their first contribution in https://github.com/sgl-project/sglang/pull/12453
@hhaAndroid made their first contribution in https://github.com/sgl-project/sglang/pull/12367
@yeahdongcn made their first contribution in https://github.com/sgl-project/sglang/pull/12560
@akhilg-nv made their first contribution in https://github.com/sgl-project/sglang/pull/12044
@alisonshao made their first contribution in https://github.com/sgl-project/sglang/pull/12585
@soaringk made their first contribution in https://github.com/sgl-project/sglang/pull/12456
@ai-easy-cpu made their first contribution in https://github.com/sgl-project/sglang/pull/12486
@MoyanZitto made their first contribution in https://github.com/sgl-project/sglang/pull/12702
@wyx-0203 made their first contribution in https://github.com/sgl-project/sglang/pull/12631
@WazupSteve made their first contribution in https://github.com/sgl-project/sglang/pull/12565

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.4...v0.5.5

Open source →

0.5.424 Oct 2025Deprecated

[8/N] MoE Refactor: deprecate EPMoE by @ch-wan in https://github.com/sgl-project/sglang/pull/11211

Highlights

AMD AI Dev Day 2025 SGLang (slide), PyTorch Conference 2025 SGLang (slide)
Model gateway v0.2 release: https://docs.sglang.ai/advanced_features/router.html
[beta] Overlap scheduler for speculative decoding: https://github.com/sgl-project/sglang/issues/11762
[beta] Piecewise CUDA graph for prefill: https://github.com/sgl-project/sglang/issues/11490
Prefix cache for qwen3 next and GDN/mamba models: https://github.com/sgl-project/sglang/pull/11214
Fullset optimizations for DeepSeek-V3.2 (MTP, PD-Disagg, Function Calling) (https://docs.sglang.ai/basic_usage/deepseek_v32.html, https://github.com/sgl-project/sglang/issues/11989)
Various Blackwell kernel optimizations
DGX Spark Support: https://lmsys.org/blog/2025-10-13-nvidia-dgx-spark/
KTransformer integration: https://lmsys.org/blog/2025-10-22-KTransformers/
New model support: Nemotron, DeepSeek OCR, Qwen3-Omni, Olmo 3
Native ModelOpt quantization support

What's Changed

[router] add ipv6 support across all components by @slin1237 in https://github.com/sgl-project/sglang/pull/11219
Remove env var warnings for release by @merrymercy in https://github.com/sgl-project/sglang/pull/11262
Enable native ModelOpt quantization support (1/3) by @Edwardf0t1 in https://github.com/sgl-project/sglang/pull/7149
[router][tool call] Clean up redundant detect_format and has_tool_markers by @CatherineSue in https://github.com/sgl-project/sglang/pull/11270
disable sm100 for FlashMLA and fast-hadamard-transform in cuda12.6.1 by @gongwei-130 in https://github.com/sgl-project/sglang/pull/11274
docker: add manifest to versioned docker releases by @ishandhanani in https://github.com/sgl-project/sglang/pull/11268
[Bug] Fix incorrect assertion in FA4 and add UT. by @lifuhuang in https://github.com/sgl-project/sglang/pull/11182
[router][grpc] Refine streaming processes by @CatherineSue in https://github.com/sgl-project/sglang/pull/11277
Fix code sync scripts by @merrymercy in https://github.com/sgl-project/sglang/pull/11276
[Auto Sync] Update test_utils.py (20251006) by @merrymercy in https://github.com/sgl-project/sglang/pull/11280
Rename max_micro_batch_size -> pp_max_micro_batch_size by @merrymercy in https://github.com/sgl-project/sglang/pull/11279
Reverse the AMD CI test back to 1200s and split the 8-gpu deepseek job into two. by @sunxxuns in https://github.com/sgl-project/sglang/pull/11238
Fix LoRA support for multimodal models (VLMs) by implementing a consistent pattern for skipping vision components by @ConnorLi96 in https://github.com/sgl-project/sglang/pull/11261
fix: correct scale parameter remapping logic in Llama4ForConditionalGeneration by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/11282
docs: update sgl-kernel README by @zhyncs in https://github.com/sgl-project/sglang/pull/11286
chore: bump sgl-kernel version to 0.3.15 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11281
[router][grpc] Fix proto3 default value mismatches and cleanup unused fields by @CatherineSue in https://github.com/sgl-project/sglang/pull/11283
convert test_deterministic into unit tests by @skyzh in https://github.com/sgl-project/sglang/pull/11095
Feature/longbench v2 evaluation utils by @alhridoy in https://github.com/sgl-project/sglang/pull/10949
[ci] fix pp test by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11294
EAGLE cache fix for SWARadixCache by @ispobock in https://github.com/sgl-project/sglang/pull/11231
Remove overlap thread by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11210
[router] add reasoning and tool parser argument in router by @slin1237 in https://github.com/sgl-project/sglang/pull/11290
Remove sampling info events and overlap thread file by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11300
Introduce future indices by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11301
[sgl-kernel] Support float64 moe_sum_reduce cuda kernel by @yuan-luo in https://github.com/sgl-project/sglang/pull/11068
[Docs] [Router] Update Observability and Common Issues Section by @xuwenyihust in https://github.com/sgl-project/sglang/pull/11302
[router] add get server info and get model info in grpc server by @slin1237 in https://github.com/sgl-project/sglang/pull/11303
[router][grpc] Refactor chat template content format detection by @CatherineSue in https://github.com/sgl-project/sglang/pull/11288
[Doc] HiCache Design Documents by @ykwd in https://github.com/sgl-project/sglang/pull/11027
[Doc]: Best Practice for HICache by @hzh0425 in https://github.com/sgl-project/sglang/pull/11001
[router] fix grpc connection conversion and add optimization by @slin1237 in https://github.com/sgl-project/sglang/pull/11305
[router][grpc] Fix sampling_params.stop_strs is None by @CatherineSue in https://github.com/sgl-project/sglang/pull/11306
Update tool parser and related documentation by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/11223
[router][grpc] Fix error message format in grpc chat handler by @CatherineSue in https://github.com/sgl-project/sglang/pull/11307
[quantization] Properly ignore quantization for layers excluded in quant_config by @BowenBao in https://github.com/sgl-project/sglang/pull/11205
[router] support Openai router conversation API CRUD by @key4ng in https://github.com/sgl-project/sglang/pull/11297
[router][grpc] Fix request_id extraction when n > 1 by @CatherineSue in https://github.com/sgl-project/sglang/pull/11311
[router] cleanup worker health check to return early by @slin1237 in https://github.com/sgl-project/sglang/pull/11310
[oai serving chat] Add argument --sampling-defaults and fix ChatCompletionRequest defaults by @CatherineSue in https://github.com/sgl-project/sglang/pull/11304
Clean match_prefix and prepare_for_extend for mem cache V2 by @cctry in https://github.com/sgl-project/sglang/pull/11200
ci: unify the model launch method of nightly ci by @mickqian in https://github.com/sgl-project/sglang/pull/11230
[Chore] Update xgrammar 0.1.24 -> 0.1.25 by @DarkSharpness in https://github.com/sgl-project/sglang/pull/10710
update sampling_params documentation with defaults by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/11315
Optimize copy_kv_cache for spec decoding by @YAMY1234 in https://github.com/sgl-project/sglang/pull/11126
Rename ngram_utils -> ngram_info by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11316
[router][grpc] Refactor chat handler in grpc/ to use centralized orchestrator by @CatherineSue in https://github.com/sgl-project/sglang/pull/11314
[Feature] Add /tokenize and /detokenize OpenAI compatible endpoints by @adarshxs in https://github.com/sgl-project/sglang/pull/9545
[8/N] MoE Refactor: deprecate EPMoE by @ch-wan in https://github.com/sgl-project/sglang/pull/11211
Skip weight loading in deepgemm compilation by @ch-wan in https://github.com/sgl-project/sglang/pull/11312
[2/2] Support MHA prefill with FlashAttention 4. by @lifuhuang in https://github.com/sgl-project/sglang/pull/10937
[Doc] Update mooncake nvlink transport doc for PD disaggregation by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11321
fix(decode): adjust ServerArgs import to explicit module path by @xiaguan in https://github.com/sgl-project/sglang/pull/11007
Support LoRA in bench_serving oai interface by @lifuhuang in https://github.com/sgl-project/sglang/pull/11318
benchmark: enhance configurable multimodal benchmarking in bench_serving by @AlienKevin in https://github.com/sgl-project/sglang/pull/9812
[CI] improve disaggregation CI. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11264
model: Support Hybrid Mamba2 NemotronHForCausalLM (nvidia/NVIDIA-Nemotron-Nano-9B-v2) by @netanel-haber in https://github.com/sgl-project/sglang/pull/10909
[router] refactor generate to use new pipeline arch by @slin1237 in https://github.com/sgl-project/sglang/pull/11323
[router] improve reasoning parser lock and reduce req cloning by @slin1237 in https://github.com/sgl-project/sglang/pull/11336
[router][grpc] Cleanup debug logs in grpc_server and grpc_router by @CatherineSue in https://github.com/sgl-project/sglang/pull/11340
[router] Fix all unused_qualifications by @CatherineSue in https://github.com/sgl-project/sglang/pull/11341
[router] Support history management using conversation by @key4ng in https://github.com/sgl-project/sglang/pull/11339
[router][grpc] Add dependencies in Cargo.toml to support chat template rendering by @CatherineSue in https://github.com/sgl-project/sglang/pull/11342
fix: fix revision for sgl-flash-attn in sgl-kernel by @mickqian in https://github.com/sgl-project/sglang/pull/11327
[Auto Sync] Update scheduler.py (20251009) by @zhyncs in https://github.com/sgl-project/sglang/pull/11350
[Generative Score API] Multi-Item scoring with custom attention mask. by @sundar24295s in https://github.com/sgl-project/sglang/pull/10979
[router][grpc] disable health check generation and increase timeout by @slin1237 in https://github.com/sgl-project/sglang/pull/11353
[router] Refactor OpenAI router: split monolithic file and move location by @key4ng in https://github.com/sgl-project/sglang/pull/11359
[router][lint] Add unused_qualifications to cargo lint warnings by @CatherineSue in https://github.com/sgl-project/sglang/pull/11366
[DeepSeek-V3.2] Include indexer kv cache when estimating kv cache size by @trevor-m in https://github.com/sgl-project/sglang/pull/11309
[router][grpc] Fix tool call streaming bugs: empty tool names, state pollution, and panics by @CatherineSue in https://github.com/sgl-project/sglang/pull/11373
add code pp support for nixl by @shaharmor98 in https://github.com/sgl-project/sglang/pull/11375
fix bench_serving mishandling of internal states by @shaharmor98 in https://github.com/sgl-project/sglang/pull/11376
[router][grpc] Replace fake health check with correct ones by @CatherineSue in https://github.com/sgl-project/sglang/pull/11387
[router] change grpc client from mutable to clone by @slin1237 in https://github.com/sgl-project/sglang/pull/11394
chore: upgrade flashinfer 0.4.0 by @zhyncs in https://github.com/sgl-project/sglang/pull/11364
[router] conversation item API: create, retrieve and delete by @key4ng in https://github.com/sgl-project/sglang/pull/11369
chore: bump SGLang version to 0.5.3.post1 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11324
move more files under srt/utils by @merrymercy in https://github.com/sgl-project/sglang/pull/11285
[grammar] Avoid server crash when grammar backend is None by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/11401
fix: fix gpu-proc affinity set incorrectly when pp_size > 1 by @acelyc111 in https://github.com/sgl-project/sglang/pull/11389
[Bug Fix] prevent lora adapter from being loaded into LoRAManager if it is already loaded by @glenliu21 in https://github.com/sgl-project/sglang/pull/11365
[CI] Refactor PD disaggregation test suite by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11363
Replace pad with cat for better performance by @yuan-luo in https://github.com/sgl-project/sglang/pull/11388
fix: reinstall torch in deps install by @zhyncs in https://github.com/sgl-project/sglang/pull/11414
feat(hicache): Support passing prefix keys for l3 store. by @hzh0425 in https://github.com/sgl-project/sglang/pull/9045
fix file and object naming scheme in HiCacheNixl to avoid data corruption by @ziruiliu in https://github.com/sgl-project/sglang/pull/10969
Dedicated toml files for CPU/XPU by @ZailiWang in https://github.com/sgl-project/sglang/pull/10734
Add metrics for speculative decoding (acceptance rate, average acceptance length) by @scottjlee in https://github.com/sgl-project/sglang/pull/11144
chore: update pyproject by @zhyncs in https://github.com/sgl-project/sglang/pull/11420
fix: fix video input for qwen3-vl by @mickqian in https://github.com/sgl-project/sglang/pull/11361
perf: optimize qwen-vl with symm mem allreduce by @yuan-luo in https://github.com/sgl-project/sglang/pull/11381
[HiCache] feat: add multi tenant with prefix tag by @stmatengss in https://github.com/sgl-project/sglang/pull/9256
[CI] Merge build-dev into workflow matrix by @csahithi in https://github.com/sgl-project/sglang/pull/11345
Revert "perf: optimize qwen-vl with symm mem allreduce" by @ch-wan in https://github.com/sgl-project/sglang/pull/11436
Revert "fix: fix video input for qwen3-vl" by @merrymercy in https://github.com/sgl-project/sglang/pull/11437
Revert "Add metrics for speculative decoding (acceptance rate, average acceptance length)" by @scottjlee in https://github.com/sgl-project/sglang/pull/11433
[router] Fix ci nvcc not found error by @key4ng in https://github.com/sgl-project/sglang/pull/11411
feat(mooncake): support GB suffix for global_segment_size by @xiaguan in https://github.com/sgl-project/sglang/pull/10745
Separate allocation logic from scheduler by @cctry in https://github.com/sgl-project/sglang/pull/11313
[router] disable rate limiter by default by @slin1237 in https://github.com/sgl-project/sglang/pull/11435
[router] leverage RAII to actively cancel request during client disconnect by @slin1237 in https://github.com/sgl-project/sglang/pull/11399
[router][grpc] Consolidate parser checks for chat completions by @CatherineSue in https://github.com/sgl-project/sglang/pull/11439
Reorder PD disagg CI tests by @merrymercy in https://github.com/sgl-project/sglang/pull/11438
fix: Change dsv32 hack temporary path to use system temp directory by @wxsms in https://github.com/sgl-project/sglang/pull/11445
Fix batch invariant ops by @hebiao064 in https://github.com/sgl-project/sglang/pull/11368
[BugFix] test_mla_fp8.py fails on Cublas 12.9 by @Liu-congo in https://github.com/sgl-project/sglang/pull/11360
[DPSKv3.2] Rewrite nsa tilelang act_quant kernel to triton by @byjiang1996 in https://github.com/sgl-project/sglang/pull/11450
Remove tilelang dependency in Dockerfile by @Fridge003 in https://github.com/sgl-project/sglang/pull/11455
Enable native ModelOpt quantization support (2/3) by @Edwardf0t1 in https://github.com/sgl-project/sglang/pull/9991
Reland [1/2] Optimizations and refactors about quant kernel by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10312
Super tiny delete unused openai router in sgl-router by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11448
Adjust logits metada init for target verify by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11467
[Documentation][Configuration] Server args and documentation of PD-Multiplexing. by @ykcombat in https://github.com/sgl-project/sglang/pull/11427
Fix enable_v2 in int8 quant by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11470
[Fix] Fix split prefill with fa3. by @ykcombat in https://github.com/sgl-project/sglang/pull/11428
fix stop when stream by @whybeyoung in https://github.com/sgl-project/sglang/pull/11462
Add option to disable any_whitespace for xgrammar and llguidance backends. by @lulor in https://github.com/sgl-project/sglang/pull/8919
[7/n] decouple quantization impl from vllm dependency - gguf kernel by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11019
fix Xeon CI by @ZailiWang in https://github.com/sgl-project/sglang/pull/11454
[CI] Add nightly builds to dockerhub by @csahithi in https://github.com/sgl-project/sglang/pull/9804
[Feature] support regex strings as a stopping condition by @glenliu21 in https://github.com/sgl-project/sglang/pull/10635
Beta spec-overlap for EAGLE by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11398
Piecewise CUDA Graph Support & Torch Compile Backend by @Oasis-Git in https://github.com/sgl-project/sglang/pull/10062
[Router]: Small Typo in a comment within tree.rs by @xuwenyihust in https://github.com/sgl-project/sglang/pull/11489
chore: bump sgl-kernel version to 0.3.16 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11476
[smol] [perf] Qwen3-VL in place op. by @vincentzed in https://github.com/sgl-project/sglang/pull/11481
[chore][1/N] Avoid using default mutable parameters by @kevin85421 in https://github.com/sgl-project/sglang/pull/11478
[bugfix]: use correct causality condition for flashattention, flashinfer, and triton backends by @MahmoudAshraf97 in https://github.com/sgl-project/sglang/pull/10172
[ perf ] Replace json-> orjson in hot path by @vincentzed in https://github.com/sgl-project/sglang/pull/11221
[chore][2/N] Avoid using default mutable parameters by @kevin85421 in https://github.com/sgl-project/sglang/pull/11479
Fix the GPT function calling regex to allow dash in the name by @antoine-roux in https://github.com/sgl-project/sglang/pull/10577
bailingMoE: Fix Key error of deepep_mode by @QiuMike in https://github.com/sgl-project/sglang/pull/11465
Fix CI break by express-laned PRs. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11499
Move args from global_config to environ by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11332
move fla env check position by @yizhang2077 in https://github.com/sgl-project/sglang/pull/11500
Temporarily remove b200 tests by @merrymercy in https://github.com/sgl-project/sglang/pull/11501
Fix port conflicts in CI by @merrymercy in https://github.com/sgl-project/sglang/pull/11497
temporarily remove b200 tests by @merrymercy in https://github.com/sgl-project/sglang/pull/11502
Fix unit tests by @merrymercy in https://github.com/sgl-project/sglang/pull/11503
Bugfix: Fix Type consistency for KV indices in SWARadixCache by @hzh0425 in https://github.com/sgl-project/sglang/pull/11452
doc: add doc for adding new models into nightly-ci by @mickqian in https://github.com/sgl-project/sglang/pull/11443
[CI] fix lint by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11509
Deprecate global_server_args_dict by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11331
chore: remove flashinfer cleanup cache by @zhyncs in https://github.com/sgl-project/sglang/pull/11514
fix: revert temporarily remove b200 tests by @zhyncs in https://github.com/sgl-project/sglang/pull/11515
[Fix] Improve longbench prompt and other logics by @byjiang1996 in https://github.com/sgl-project/sglang/pull/11474
Sync changes on io_struct.py and deterministic ops by @merrymercy in https://github.com/sgl-project/sglang/pull/11498
[lint] Fix the lint issue by @ch-wan in https://github.com/sgl-project/sglang/pull/11516
Revert "Deprecate global_server_args_dict" by @ch-wan in https://github.com/sgl-project/sglang/pull/11520
Improve dp attention port assignment scheme by @jokerwyt in https://github.com/sgl-project/sglang/pull/5889
[router] openai router: support grok model by @key4ng in https://github.com/sgl-project/sglang/pull/11511
docs(router): add token-bucket rate limiting to the docs by @Jonahcb in https://github.com/sgl-project/sglang/pull/11485
[sgl-kernel][1/N]Support Expert Specialization Grouped GEMM by @HydraQYH in https://github.com/sgl-project/sglang/pull/11432
Update DeepSeek-R1-FP4 default config on blackwell by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/11512
[Fix]: add missing device attribute to ChunkCache by @leavelet in https://github.com/sgl-project/sglang/pull/11493
[Feature] Support mamba radix cache v0 by @yizhang2077 in https://github.com/sgl-project/sglang/pull/11214
ci: improve nightly-ci by @mickqian in https://github.com/sgl-project/sglang/pull/11385
[CI monitor] Improve CI analyzer: fix job failure tracking and add CUDA-focused filtering by @BBuf in https://github.com/sgl-project/sglang/pull/11505
[HICache]: Support 3FS-Store with page_first_direct layout by @hzh0425 in https://github.com/sgl-project/sglang/pull/11460
Tiny fix test run estimated time by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11544
[Reland] perf: optimize qwen-vl with symm mem allreduce by @yuan-luo in https://github.com/sgl-project/sglang/pull/11457
Depreate global_server_args_dict by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11528
[Fix] Add per_channel_quant parameter to MoE config functions by @mmangkad in https://github.com/sgl-project/sglang/pull/11201
[router][ci] Add Nightly Release Workflow for SGLang Router by @slin1237 in https://github.com/sgl-project/sglang/pull/11527
[router] add tokenizer path to be dir by @slin1237 in https://github.com/sgl-project/sglang/pull/11530
Remove tp_worker.worker by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11548
fix: fix video input for qwen3-vl by @mickqian in https://github.com/sgl-project/sglang/pull/11442
[NVIDIA] BUMP FA3 by @johnnynunez in https://github.com/sgl-project/sglang/pull/11444
[Fix] Include grpc reflection runtime dependency by @ai-jz in https://github.com/sgl-project/sglang/pull/11419
Adjust overlap event loop by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11507
Move deep gemm related arguments to sglang.srt.environ by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11547
[router][grpc] Further delegate non-stream processing to processing.rs by @CatherineSue in https://github.com/sgl-project/sglang/pull/11553
[router] allow user to specify chat template path by @slin1237 in https://github.com/sgl-project/sglang/pull/11549
Minor: improve sampler & remove unused fields from model_config.py by @merrymercy in https://github.com/sgl-project/sglang/pull/11531
[router] Add Rust CLI flags for queue size, timeout, and rate limit for token bucket rate limiter by @Jonahcb in https://github.com/sgl-project/sglang/pull/11483
Add metrics for speculative decoding (acceptance rate, average acceptance length) by @scottjlee in https://github.com/sgl-project/sglang/pull/11441
Fix DeepSeek-v3.2 default config (ValueError: not enough values to unpack (expected 4, got 3)) by @trevor-m in https://github.com/sgl-project/sglang/pull/11557
[CI] Add Basic Test for DeepSeek V3.2 by @Fridge003 in https://github.com/sgl-project/sglang/pull/11308
[router][grpc] Add error handling to generate_tool_constraints by @CatherineSue in https://github.com/sgl-project/sglang/pull/11562
[NVIDIA] update pyproject.toml to support cu130 option by @johnnynunez in https://github.com/sgl-project/sglang/pull/11521
[CI Monitor] Ci monitor only deal with main branch in default by @BBuf in https://github.com/sgl-project/sglang/pull/11538
Tiny cleanup fp4 gemm calls by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11537
[router][grpc] Add serve_grpc to launch_server and log id for HealthCheck by @CatherineSue in https://github.com/sgl-project/sglang/pull/11564
[router] Add BRANCH_TYPE=local support to Dockerfile.router for local builds by @YouNeedCryDear in https://github.com/sgl-project/sglang/pull/11571
[sgl-kernel][2/N]Support Expert Specialization Grouped GEMM by @HydraQYH in https://github.com/sgl-project/sglang/pull/11534
chore: bump sgl-kernel version to 0.3.16.post1 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11573
Fix accept rate in speculative decoding metrics by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/11572
Compilation Folder Reset by @Oasis-Git in https://github.com/sgl-project/sglang/pull/11539
[FEATURE] Add Profile Trace Merger for Distributed Traces by @neelabhsinha in https://github.com/sgl-project/sglang/pull/11413
[DSv32] Use torch.compile for _get_logits_head_gate by @trevor-m in https://github.com/sgl-project/sglang/pull/11565
Make DeepEP combine recv do not overlap by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11535
bench_serving support PD Disaggregation by @BBuf in https://github.com/sgl-project/sglang/pull/11542
Implement LRU eviction policy for LoRA adapters by @ConnorLi96 in https://github.com/sgl-project/sglang/pull/11041
Revert "[NVIDIA] BUMP FA3 (#11444)" by @zhyncs in https://github.com/sgl-project/sglang/pull/11582
chore: bump sgl-kernel version to 0.3.16.post2 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11583
[Auto Sync] Update model_config.py (20251014) by @merrymercy in https://github.com/sgl-project/sglang/pull/11580
Add fused_moe_triton config: triton_3_4_0/E=256,N=256,device_name=NVIDIA_B200.json by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/11587
[router][protocols] Add Axum validate extractor and use it for /v1/chat/completions endpoint by @CatherineSue in https://github.com/sgl-project/sglang/pull/11588
[router] update generate spec to align with sgl io struct by @slin1237 in https://github.com/sgl-project/sglang/pull/11591
[router] change worker api to async instead of sync by @slin1237 in https://github.com/sgl-project/sglang/pull/11566
Update news section in README.md by @merrymercy in https://github.com/sgl-project/sglang/pull/11598
[router] delete useless table content comment in spec by @slin1237 in https://github.com/sgl-project/sglang/pull/11597
[router] allow router launch server to use grpc mode by @slin1237 in https://github.com/sgl-project/sglang/pull/11600
[Docs] [Router]: Update sg-router doc on circuit breaker by @xuwenyihust in https://github.com/sgl-project/sglang/pull/11449
[router] when given both local tokenizer and chat template, log all by @slin1237 in https://github.com/sgl-project/sglang/pull/11601
[AMD CI] Add image and weights caching. by @saienduri in https://github.com/sgl-project/sglang/pull/11593
Update release-docker-dev.yml by @sglang-bot in https://github.com/sgl-project/sglang/pull/11603
Optimize Triton Draft Backend by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11556
Refactor spec decoding metrics calculation into separate TokenizerManager utility function by @scottjlee in https://github.com/sgl-project/sglang/pull/11586
make radix cache deterministic by @skyzh in https://github.com/sgl-project/sglang/pull/10721
move eagle draft post process to cuda graph by @cicirori in https://github.com/sgl-project/sglang/pull/11434
Reduce one step decode for draft model. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11561
[router] add py binding and readme for openai router and history backend by @key4ng in https://github.com/sgl-project/sglang/pull/11453
[router] cleanup app context and move to startup by @slin1237 in https://github.com/sgl-project/sglang/pull/11617
[router] add chang and keyang to sgl router author by @slin1237 in https://github.com/sgl-project/sglang/pull/11620
use non_blocking h2d in ForwardBatch.prepare_mlp_sync_batch. by @strgrb in https://github.com/sgl-project/sglang/pull/11605
[router] update router readme to latest features by @slin1237 in https://github.com/sgl-project/sglang/pull/11619
Fix log for chunked prefix cache by @Fridge003 in https://github.com/sgl-project/sglang/pull/11624
[Auto Sync] Update scheduler.py, server_args.py (20251014) by @merrymercy in https://github.com/sgl-project/sglang/pull/11623
[Auto Sync] Update collector.py (20251014) by @merrymercy in https://github.com/sgl-project/sglang/pull/11625
[Minor] Update xgrammar dependency by @DarkSharpness in https://github.com/sgl-project/sglang/pull/11622
Update install.md by @merrymercy in https://github.com/sgl-project/sglang/pull/11631
fix: Update SGL_KERNEL_VERSION to 0.3.15 by @zhyncs in https://github.com/sgl-project/sglang/pull/11633
[router][grpc] add warm up to grpc server by @slin1237 in https://github.com/sgl-project/sglang/pull/11627
Refactor kv cache free by @cctry in https://github.com/sgl-project/sglang/pull/11351
[router] update router doc to latest features by @slin1237 in https://github.com/sgl-project/sglang/pull/11639
fix: upgrade transformers to 4.57.1 by @csahithi in https://github.com/sgl-project/sglang/pull/11628
[router] add worker self discovery for metadata by @slin1237 in https://github.com/sgl-project/sglang/pull/11638
[router] upgrade to 0.2.0 by @slin1237 in https://github.com/sgl-project/sglang/pull/11642
[1/N] Introduce Mooncake Backend and Mooncake EP to Support Elastic EP by @UNIDY2002 in https://github.com/sgl-project/sglang/pull/10423
[1/N]Support DeepSeek-R1 w4a8 normal deepep by @ayrnb in https://github.com/sgl-project/sglang/pull/8247
[Fix] Fix accuracy bug in CSGMV kernel caching key. by @lifuhuang in https://github.com/sgl-project/sglang/pull/11579
feat: add add_chunked_prefix_cache_attention_backend by @zhyncs in https://github.com/sgl-project/sglang/pull/11636
Super tiny improve FA3 import error message by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11590
[BugFix][Qwen3-VL]: fix cu_seqlens in qwen3-vl by @ZhengWG in https://github.com/sgl-project/sglang/pull/11458
[Doc] Update support matrix for attn and hybrid attn by @b8zhong in https://github.com/sgl-project/sglang/pull/11293
Clean up some Qwen3-Next and deterministic code by @hebiao064 in https://github.com/sgl-project/sglang/pull/11585
docs: update sglang installation guide by @zhyncs in https://github.com/sgl-project/sglang/pull/11659
Tiny cleanup some eagle unused codes by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11660
Fix 1-step draft model forward by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11653
[tool call] Fix prev_tool_call_arr management in base_format_detector.py by @CatherineSue in https://github.com/sgl-project/sglang/pull/11367
[router] Fix response api related spec by @key4ng in https://github.com/sgl-project/sglang/pull/11621
Fix missing json imports in serving_responses.py by @CatherineSue in https://github.com/sgl-project/sglang/pull/11681
[sgl-kernel][3/N]Support Expert Specialization Grouped GEMM by @HydraQYH in https://github.com/sgl-project/sglang/pull/11674
[sgl-kernel] Optimize gguf test by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11667
[router][grpc] Simplify model_id determination by @CatherineSue in https://github.com/sgl-project/sglang/pull/11684
[router] Refactor StopSequenceDecoder to Use Sequence for Incremental Decoding by @slin1237 in https://github.com/sgl-project/sglang/pull/11676
chore: bump SGLang version to 0.5.3.post2 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11680
[CI][XPU]enable sglang CI on Intel XPU by @DiweiSun in https://github.com/sgl-project/sglang/pull/9493
enable rmsnorm on XPU by @huaiyuzh in https://github.com/sgl-project/sglang/pull/10248
Sync code and test CI; rename some env vars by @merrymercy in https://github.com/sgl-project/sglang/pull/11686
docs: Add Contributor Covenant Code of Conduct by @zhyncs in https://github.com/sgl-project/sglang/pull/11689
[Mamba] Increase default mamba_full_memory_ratio to 0.9 by @hanming-lu in https://github.com/sgl-project/sglang/pull/11679
[PD] Add PD support for hybrid model (Qwen3-Next, DeepSeek V3.2 Exp) by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10912
[sgl-kernel] support hadamard by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11663
Fix missing a2a backend init of GLM4.5 MoE Block by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11692
Split test_intel_amx_attention_backend.py to pass CI of timeout by @yanbing-j in https://github.com/sgl-project/sglang/pull/11370
Set csgmv as default lora backend. by @lifuhuang in https://github.com/sgl-project/sglang/pull/11488
[Bugfix] Fix Qwen3/DSV3/DSV3.2 model support by @iforgetmyname in https://github.com/sgl-project/sglang/pull/11510
[CI] Add GLM4MoE model test by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11706
[router] fix get_models endpoint for openai router by @key4ng in https://github.com/sgl-project/sglang/pull/11687
[ci]use H20 to run disaggregation test by @HanHan009527 in https://github.com/sgl-project/sglang/pull/11543
chore: bump SGLang version to 0.5.3.post3 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11693
model: qwen3-omni (thinker-only) by @mickqian in https://github.com/sgl-project/sglang/pull/10911
[Router] Refactor protocol definitions: split spec.rs into modular files by @key4ng in https://github.com/sgl-project/sglang/pull/11677
[router] fix p and d worker filtering and bootstrap port handling by @slin1237 in https://github.com/sgl-project/sglang/pull/11729
[router][grpc] add dissag info to warm up in grpc server by @slin1237 in https://github.com/sgl-project/sglang/pull/11727
[router] Fix tool_choice normalization in ChatCompletionRequest and fix ut by @CatherineSue in https://github.com/sgl-project/sglang/pull/11731
Revert "make radix cache deterministic" by @Fridge003 in https://github.com/sgl-project/sglang/pull/11728
Reduce the image processing latency in VLM by @zhooooong in https://github.com/sgl-project/sglang/pull/11541
[router] add spec.rs to enables tests under spec folder by @key4ng in https://github.com/sgl-project/sglang/pull/11734
[router] Add rustfmt and set group imports by default by @CatherineSue in https://github.com/sgl-project/sglang/pull/11732
Revert "[router] fix get_models endpoint for openai router (#11687)" by @key4ng in https://github.com/sgl-project/sglang/pull/11740
[router][CI] Clean up deprecated fields in pr-test-pd-router.yml by @CatherineSue in https://github.com/sgl-project/sglang/pull/11739
[CI] Fix broken event loop creation by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11746
[overlap-spec] Make plan stream an option by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11724
ci: reduce and refactor vlm ut and combine test files by @mickqian in https://github.com/sgl-project/sglang/pull/11062
Abstraction for spec worker and code cleanup by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11643
add tuned fuse moe kernel for qwen3 235b fp8 on h200 by @pdasgup in https://github.com/sgl-project/sglang/pull/11730
Revert "Set csgmv as default lora backend. (#11488)" by @zhyncs in https://github.com/sgl-project/sglang/pull/11735
[router] Fix UTF-8 Boundary Panic in Stop Sequence Decoder by @slin1237 in https://github.com/sgl-project/sglang/pull/11766
[router] fix grpc client time out to 1h by @slin1237 in https://github.com/sgl-project/sglang/pull/11768
[doc] update router document by @key4ng in https://github.com/sgl-project/sglang/pull/11767
[Feature] Reuse flashinfer workspace for PD-Multiplexing. by @ykcombat in https://github.com/sgl-project/sglang/pull/11540
Turn on shm_allreduce and shm_allgather for fp16 by @chunyuan-w in https://github.com/sgl-project/sglang/pull/10725
[Auto Sync] Update scheduler.py (20251017) by @zhyncs in https://github.com/sgl-project/sglang/pull/11738
[router][grpc] Remove timeout for connections and remove max_tokens deprecation warning log by @CatherineSue in https://github.com/sgl-project/sglang/pull/11775
Cleaning indexer for DeepSeek V3.2 by @Fridge003 in https://github.com/sgl-project/sglang/pull/11682
[minor] sync code on python/sglang/test/test_deterministic.py and improve ci tests by @merrymercy in https://github.com/sgl-project/sglang/pull/11777
[Auto Sync] Update common.py (20251017) by @merrymercy in https://github.com/sgl-project/sglang/pull/11782
[Fix] Skip visual layers when applying LoRA to Qwen2VL modules by @anvdn in https://github.com/sgl-project/sglang/pull/11519
[Lint] Add python/sglang to ruff F401 checks and remove unused imports in files by @CatherineSue in https://github.com/sgl-project/sglang/pull/11685
Super tiny fix missing input throughput by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11607
Support shared experts overlap in cutlass moe by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11611
Support casting bf16 NextN moe to fp8 by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11613
Manually flip deepep_mode for cuda_graph by @zhuzilin in https://github.com/sgl-project/sglang/pull/11666
Set CUDA_VISIBLE_DEVICES to achieve one GPU per process by @merrymercy in https://github.com/sgl-project/sglang/pull/9170
Super tiny fix CI by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11788
Make single-batch overlap compatible with offloading by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11614
completely remove mixed mode deterministic test as prefix mode could cover it by @zminglei in https://github.com/sgl-project/sglang/pull/11783
[Refactor] move deep_gemm_wrapper out of quantization by @ch-wan in https://github.com/sgl-project/sglang/pull/11784
Enable lint on main by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11794
[router][grpc] Support parallel queue puts in grpc_request_manager and remove mutex for grpc_client by @CatherineSue in https://github.com/sgl-project/sglang/pull/11798
Try add back no-commit-to-branch by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11799
fix(glm45): disable reduce scatter by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/11665
fix command line usage of profiling by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/11793
[RL] support weight update with DP attention by @zhuzilin in https://github.com/sgl-project/sglang/pull/11669
[RL] use cpu group to prepare_mlp_sync_batch_raw when the server is offloaded by @zhuzilin in https://github.com/sgl-project/sglang/pull/10152
set default attention backend for deterministic inference by @zminglei in https://github.com/sgl-project/sglang/pull/11801
Eager Compiler for Torch Compile by @Oasis-Git in https://github.com/sgl-project/sglang/pull/11803
Fix install instructions and pyproject.tomls by @merrymercy in https://github.com/sgl-project/sglang/pull/11781
Bump torch_memory_saver to avoid installing pre-release versions by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11797
[HiCache] feat: add more eviction policy by @stmatengss in https://github.com/sgl-project/sglang/pull/11506
[overlap-spec] support page size > 1 by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11772
support server arg override KV cache to bf16 to avoid slow cases by @b8zhong in https://github.com/sgl-project/sglang/pull/11749
feat(example/fastapi): support --startup-timeout using Qwen3-Next-80B-A3B-Instruct as example by @Kindyaa in https://github.com/sgl-project/sglang/pull/11710
ci: update lmms-eval to speed up multimodal CI by @b8zhong in https://github.com/sgl-project/sglang/pull/11000
Use cutlass fp4 gemm by default by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/11813
Fix Dockerfile not installing correct version of DeepEP for arm build by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/11773
[router] Add Configurable L0 and L1 Tokenizer Caching by @slin1237 in https://github.com/sgl-project/sglang/pull/11688
[2/2] [feature] support openai like classification api in router by @whybeyoung in https://github.com/sgl-project/sglang/pull/11670
[1/2][feature] support openai like classification api by @whybeyoung in https://github.com/sgl-project/sglang/pull/11618
make sure logit bias is applied during eagle spec decoding verification by @petricevich in https://github.com/sgl-project/sglang/pull/11555
fix: do not wrap invalid grammar objects during constrained generation by @tazjin in https://github.com/sgl-project/sglang/pull/11328
Improve send_sone script by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11817
Fix: Dynamic RoPE Cache Expansion to Prevent Position-ID Out-of-Bounds in EAGLE + Long-Sequence Workloads by @YAMY1234 in https://github.com/sgl-project/sglang/pull/10788
Update CODEOWNERS for layer quantization path by @merrymercy in https://github.com/sgl-project/sglang/pull/11818
support tokenized batch request by @narutolhy in https://github.com/sgl-project/sglang/pull/11091
Tiny add hints when users send requests to wrong place by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11808
Make single-batch overlap compatible with NextN by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11804
Support not officially supported high sgl-kernel version with low srt version by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11786
Avoid generation gets hanging when user specifies multiple event loops by @fzyzcjy in https://github.com/sgl-project/sglang/pull/5162
Change bf16 to fp8 for some gemms in attention for DeepSeek ckpt v2 by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11805
Revert "Fix: Dynamic RoPE Cache Expansion to Prevent Position-ID Out-of-Bounds in EAGLE + Long-Sequence Workloads" by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11827
[overlap-spec] fix stop condition and trimming by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11819
[Spec Decoding] Support MTP for dsv3.2 by @Paiiiiiiiiiiiiii in https://github.com/sgl-project/sglang/pull/11652
[CI] always print back trace in retry() by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11834
[Test] Add basic matched stop for beta eagle by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11833
Deterministic Mode: Add 1-stage triton kernel for prefill by @hebiao064 in https://github.com/sgl-project/sglang/pull/11147
[logprobs] Enable local deterministic logrprobs testing with strict threshold by @PrinsYin in https://github.com/sgl-project/sglang/pull/10994
[CI] Add CI test for DeepSeek V3.2 MTP by @Fridge003 in https://github.com/sgl-project/sglang/pull/11835
[NVIDIA] FA3/FA4 Fix by @johnnynunez in https://github.com/sgl-project/sglang/pull/11606
[DeepseekV32] Add fast_topk_transform_ragged_fused kernel by @hlu1 in https://github.com/sgl-project/sglang/pull/11815
Fix triton_kernels import error on some hardwares by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11831
Tiny bump DeepEP version in ARM blackwell by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11810
[BugFix] replace the input_to_float8 used in dsv2 by @Liu-congo in https://github.com/sgl-project/sglang/pull/11612
[Doc] Update documents for FA4 by @Fridge003 in https://github.com/sgl-project/sglang/pull/11778
fix(ci): Fix CI Monitor limit parameter and add CI Analysis to summary by @BBuf in https://github.com/sgl-project/sglang/pull/11832
Fix version bump script to handle TOML files with outdated versions by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/11787
Improve Kernel Build Time by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/11508
check master server for mooncake store by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/10510
chore: bump sgl-kernel version to 0.3.16.post3 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11733
Recapture cuda graph after model weight update to resolve IMA error by @harrisonlimh in https://github.com/sgl-project/sglang/pull/11780
[Feature] Use current greenctx stream to communicate in PD-Multiplexing. by @ykcombat in https://github.com/sgl-project/sglang/pull/11594
Support mrope triton kernel and add unit test by @yuan-luo in https://github.com/sgl-project/sglang/pull/11722
[PD] Improve eagle acceptance rate by transferring draft model hidden states by @ZeldaHuang in https://github.com/sgl-project/sglang/pull/10801
Tiny clean up for PD module and doc by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11747
Revert "[CI Monitor] Ci monitor only deal with main branch in default" by @BBuf in https://github.com/sgl-project/sglang/pull/11846
[Model] Add Olmo 3 model support by @2015aroras in https://github.com/sgl-project/sglang/pull/11396
Update amd gpu install docs. by @saienduri in https://github.com/sgl-project/sglang/pull/11849
[AMD CI] Populate image cache in nightly docker release. by @saienduri in https://github.com/sgl-project/sglang/pull/11822
fix(server_args): handle tokenizer init conflicts by @ishandhanani in https://github.com/sgl-project/sglang/pull/11776
[Feature] New structural tag support by @DarkSharpness in https://github.com/sgl-project/sglang/pull/10691
Tiny fix main lint by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11862
[9/N] MoE Refactor: cleanup dispatcher interfaces by @ch-wan in https://github.com/sgl-project/sglang/pull/11847
Fix acc len and gen throughput metrics when enabling overlap-spec by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/11823
Replace function call with set literal by @penguin-wwy in https://github.com/sgl-project/sglang/pull/11867
Support mixing cutedsl and deepgemm backend by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11807
[router] Worker Management Workflow Engine by @slin1237 in https://github.com/sgl-project/sglang/pull/11868
[router] remove encoding header for oai router by @slin1237 in https://github.com/sgl-project/sglang/pull/11881
[Auto Sync] Update scheduler.py, server_args.py (20251020) by @merrymercy in https://github.com/sgl-project/sglang/pull/11875
[router][grpc] Remove continue_final_message in ChatTemplateParams and add minijinja-contrib by @CatherineSue in https://github.com/sgl-project/sglang/pull/11882
fix(sql-router): fix conflict port in test by @htiennv in https://github.com/sgl-project/sglang/pull/11826
[router] clean up workflow logs to debug for implementation details logs by @slin1237 in https://github.com/sgl-project/sglang/pull/11886
[code move] move pp into a separate mixin by @merrymercy in https://github.com/sgl-project/sglang/pull/11838
[router][grpc] Fix wram-up random token ids for small models by @CatherineSue in https://github.com/sgl-project/sglang/pull/11887
Revise MRotaryEmbedding's forward by @yuan-luo in https://github.com/sgl-project/sglang/pull/11859
piecewise cuda graph support qwen3-moe by @BBuf in https://github.com/sgl-project/sglang/pull/11845
Fix RotaryEmbedding for fp32 input by @zhangdonghao-zdh in https://github.com/sgl-project/sglang/pull/11843
Init attention backend for Intel XPU by @airMeng in https://github.com/sgl-project/sglang/pull/10656
Use trtllm_mla decode kernel for draft extend in speculative decoding by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/11664
[router] release router 0.2.1 by @slin1237 in https://github.com/sgl-project/sglang/pull/11885
[AMD] Update wave-lang to 3.8.0 by @xintin in https://github.com/sgl-project/sglang/pull/11878
init support for KTransformers Heterogeneous Computing by @Atream in https://github.com/sgl-project/sglang/pull/11487
[FEATURE] Add OpenAI-Compatible LoRA Adapter Selection by @neelabhsinha in https://github.com/sgl-project/sglang/pull/11570
[fix] fix ci uv install dependency by @HanHan009527 in https://github.com/sgl-project/sglang/pull/11895
Support Thinking Budget (via custom_logit_processor for OpenAI API) [Fix #6572] by @whybeyoung in https://github.com/sgl-project/sglang/pull/11416
Simplify multi-tokenizer by @zhengkezhou1 in https://github.com/sgl-project/sglang/pull/11295
[CI] disable glm4.1v and fix the flashinfer installation by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11902
vlm: enforce pybase64 for image and str encode/decode by @b8zhong in https://github.com/sgl-project/sglang/pull/10700
[smol] [perf] Inverse perm improvement by @vincentzed in https://github.com/sgl-project/sglang/pull/11482
[quantization][MoE] fix the check for tp_size / moe_ep_size / moe_intermediate_size / weight_block_size_n by @kevin85421 in https://github.com/sgl-project/sglang/pull/11702
[CI] Fix b200 flashinfer installation by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11915
Fix flush cache API for spec v2 by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11918
[NVIDIA] Add new SMs support for Spark & Thor by @Kh4L in https://github.com/sgl-project/sglang/pull/11287
Update sgl-kernel and remove fast hadamard depedency by @Fridge003 in https://github.com/sgl-project/sglang/pull/11844
Rename flashmla kernel options of nsa backend for better readability by @Fridge003 in https://github.com/sgl-project/sglang/pull/11876
chore: upgrade flashinfer 0.4.1 by @zhyncs in https://github.com/sgl-project/sglang/pull/11933
[BugFix][Qwen3-VL]: add metadata for video in qwen3-vl by @ZhengWG in https://github.com/sgl-project/sglang/pull/11377
[Auto Sync] Update forward_batch_info.py (20251021) by @zhyncs in https://github.com/sgl-project/sglang/pull/11934
Fix openai input_text type compatibility by @key4ng in https://github.com/sgl-project/sglang/pull/11935
fix: resolve flashinfer 0.4.1 import by @zhyncs in https://github.com/sgl-project/sglang/pull/11940
[router][grpc] Support v1/responses API by @CatherineSue in https://github.com/sgl-project/sglang/pull/11926
[router] Add gRPC E2E test suite by @key4ng in https://github.com/sgl-project/sglang/pull/11790
[router][grpc] Fix background tasks stored with wrong id by @CatherineSue in https://github.com/sgl-project/sglang/pull/11945
[lint] improve ruff check by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11922
[sgl-kernel] support flashmla libtorch by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11717
[NVIDIA] upstream FA4 and fix cccl path by @johnnynunez in https://github.com/sgl-project/sglang/pull/11929
Enable native ModelOpt quantization support (3/3) by @Edwardf0t1 in https://github.com/sgl-project/sglang/pull/10154
Fix mooncake dispatcher by @UNIDY2002 in https://github.com/sgl-project/sglang/pull/11908
[2/N] Added the core structure of elastic EP and the eplb algorithm with faulty rank by @HanHan009527 in https://github.com/sgl-project/sglang/pull/10606
[model] Support POINTSV15Chat model by @josephydu in https://github.com/sgl-project/sglang/pull/9651
Fix flaky hicache test with mooncake backend by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11953
[Fix] Remove unused import from triton_kernels_moe.py by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11967
[router] Support multiple worker URLs for OpenAI router by @key4ng in https://github.com/sgl-project/sglang/pull/11723
[Documentation] add doc for deterministic inference by @zminglei in https://github.com/sgl-project/sglang/pull/11956
[6/n]decouple quantization implementation from vLLM dependency by @Hongbosherlock in https://github.com/sgl-project/sglang/pull/10750
[BUG] AttributeError: 'DeepEPMoE' object has no attribute 'use_w4a… by @yuho8818 in https://github.com/sgl-project/sglang/pull/11977
Revert "Recapture cuda graph after model weight update to resolve IMA error " by @merrymercy in https://github.com/sgl-project/sglang/pull/11980
[NVIDIA] Update to leverage flashinfer trtllm FP4 MOE throughput kernel by @jiahanc in https://github.com/sgl-project/sglang/pull/11563
[router] create worker removal step and clean up worker manager by @slin1237 in https://github.com/sgl-project/sglang/pull/11921
Implement BGE-M3 Sparse Embeddings in SGLang by @approximated-intelligence in https://github.com/sgl-project/sglang/pull/10869
[Doc] Update deterministic inference flag in server_arguments.md by @Fridge003 in https://github.com/sgl-project/sglang/pull/11978
[grpc] Support gRPC standard health check by @CatherineSue in https://github.com/sgl-project/sglang/pull/11955
[AMD] Support a new flag to disable quant on parallelLinear layer if required by @yichiche in https://github.com/sgl-project/sglang/pull/11811
[ROCm] Remove vLLM rope dependency & use AITER impl by @b8zhong in https://github.com/sgl-project/sglang/pull/11322
[NVIDIA] Build CUDA 13 by @johnnynunez in https://github.com/sgl-project/sglang/pull/11299
Bump grace blackwell DeepEP version by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11990
[CPU] misc updates by @ZailiWang in https://github.com/sgl-project/sglang/pull/11906
fix(deepep): resolve benchmark failure on 4×IB-card setup by aligning tuning config with DeepEP commit bdd119f8 by @zheng1 in https://github.com/sgl-project/sglang/pull/11965
[CPU] Optimize FP16 decode_attention_cpu by @blzheng in https://github.com/sgl-project/sglang/pull/10652
Allow to disable batch decoding. by @LorrinWWW in https://github.com/sgl-project/sglang/pull/11944
Fix incorrect KV indices creation when page_size=32 in TRTLLM MLA backend by @cicirori in https://github.com/sgl-project/sglang/pull/11985
aiter update to v0.1.6.post1 by @HaiShaw in https://github.com/sgl-project/sglang/pull/12004
Support overlap-spec-v2 with trtllm_mla attention backend by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/11821
Support nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8/NVFP4 by @netanel-haber in https://github.com/sgl-project/sglang/pull/11866
[router] Add comprehensive E2E tests for Response API by @key4ng in https://github.com/sgl-project/sglang/pull/11988
[Router] Consolidate ConnectionMode enum to core module by @YouNeedCryDear in https://github.com/sgl-project/sglang/pull/11937
Move memory runtime checker to mixin class by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12014
Revert "Support nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8/NVFP4" by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12015
[Fix] memory leak by overlap + retract by @cctry in https://github.com/sgl-project/sglang/pull/11981
[Feature] Support loading weights from ckpt engine worker by @stmatengss in https://github.com/sgl-project/sglang/pull/11755
[router] change ci names and update log level in ci by @slin1237 in https://github.com/sgl-project/sglang/pull/12021
Feature/nano v2 offline modelopt fp8 and nvfp4 by @netanel-haber in https://github.com/sgl-project/sglang/pull/12018
[Auto Sync] Update test_deterministic_utils.py (20251023) by @merrymercy in https://github.com/sgl-project/sglang/pull/12022
ci: fix night-ci with push retry mechanism by @mickqian in https://github.com/sgl-project/sglang/pull/11765
[router][CI] Clean up imports and prints statements in sgl-router/py_test by @CatherineSue in https://github.com/sgl-project/sglang/pull/12024
Add AWQ quantization support for NPU. by @ErvinXie in https://github.com/sgl-project/sglang/pull/10158
model: support deepseek-ocr by @mickqian in https://github.com/sgl-project/sglang/pull/11891
Log iteration # for prefill and decode by @nvcastet in https://github.com/sgl-project/sglang/pull/9366
Revert "[ROCm] Remove vLLM rope dependency & use AITER impl" by @b8zhong in https://github.com/sgl-project/sglang/pull/12028
Fix mamba radix cache eviction logic in alloc_req_slots by @rogeryoungh in https://github.com/sgl-project/sglang/pull/11616
Update Github action title for kernel build by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/12029
[router] Add builder pattern for RouterConfig with zero duplication by @slin1237 in https://github.com/sgl-project/sglang/pull/12030
Fixed aarch64 flash-mla by @nvjullin in https://github.com/sgl-project/sglang/pull/12009
chore: bump SGLang version to 0.5.4 by @sglang-bot in https://github.com/sgl-project/sglang/pull/12027

New Contributors

@xuwenyihust made their first contribution in https://github.com/sgl-project/sglang/pull/11302
@ziruiliu made their first contribution in https://github.com/sgl-project/sglang/pull/10969
@scottjlee made their first contribution in https://github.com/sgl-project/sglang/pull/11144
@Liu-congo made their first contribution in https://github.com/sgl-project/sglang/pull/11360
@lulor made their first contribution in https://github.com/sgl-project/sglang/pull/8919
@antoine-roux made their first contribution in https://github.com/sgl-project/sglang/pull/10577
@QiuMike made their first contribution in https://github.com/sgl-project/sglang/pull/11465
@ai-jz made their first contribution in https://github.com/sgl-project/sglang/pull/11419
@neelabhsinha made their first contribution in https://github.com/sgl-project/sglang/pull/11413
@UNIDY2002 made their first contribution in https://github.com/sgl-project/sglang/pull/10423
@zhooooong made their first contribution in https://github.com/sgl-project/sglang/pull/11541
@pdasgup made their first contribution in https://github.com/sgl-project/sglang/pull/11730
@anvdn made their first contribution in https://github.com/sgl-project/sglang/pull/11519
@Kindyaa made their first contribution in https://github.com/sgl-project/sglang/pull/11710
@petricevich made their first contribution in https://github.com/sgl-project/sglang/pull/11555
@tazjin made their first contribution in https://github.com/sgl-project/sglang/pull/11328
@Paiiiiiiiiiiiiii made their first contribution in https://github.com/sgl-project/sglang/pull/11652
@2015aroras made their first contribution in https://github.com/sgl-project/sglang/pull/11396
@zhangdonghao-zdh made their first contribution in https://github.com/sgl-project/sglang/pull/11843
@xintin made their first contribution in https://github.com/sgl-project/sglang/pull/11878
@zhengkezhou1 made their first contribution in https://github.com/sgl-project/sglang/pull/11295
@Kh4L made their first contribution in https://github.com/sgl-project/sglang/pull/11287
@yuho8818 made their first contribution in https://github.com/sgl-project/sglang/pull/11977
@jiahanc made their first contribution in https://github.com/sgl-project/sglang/pull/11563
@approximated-intelligence made their first contribution in https://github.com/sgl-project/sglang/pull/10869
@zheng1 made their first contribution in https://github.com/sgl-project/sglang/pull/11965
@ErvinXie made their first contribution in https://github.com/sgl-project/sglang/pull/10158
@rogeryoungh made their first contribution in https://github.com/sgl-project/sglang/pull/11616
@nvjullin made their first contribution in https://github.com/sgl-project/sglang/pull/12009

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.3...v0.5.4

Open source →

0.5.36 Oct 2025Deprecated

[HiCache] Cleaning the deprecated host memory state by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/10778

Highlights

Day 0 Support for DeepSeek-V3.2 with Sparse Attention: https://lmsys.org/blog/2025-09-29-deepseek-V32/
Deterministic inference on multiple attention backends: https://lmsys.org/blog/2025-09-22-sglang-deterministic/
Integration of FlashAttention 4 prefill kernels
Enhancing support of Qwen3-Next with MTP, DP, optimized kernels and multiple hardware platforms
Support models including Qwen3-VL series, dots.vlm1, Ling-V2, Apertus, SOLAR

What's Changed

[Auto Sync] Update server_args.py (20250912) by @merrymercy in https://github.com/sgl-project/sglang/pull/10347
[CPU][doc] add torch.compile param in example commands by @ZailiWang in https://github.com/sgl-project/sglang/pull/10349
[router][ci] Add gpu utilization analyze with nvml by @key4ng in https://github.com/sgl-project/sglang/pull/10345
[NVIDIA] [3/N] Nvfp4 Masked Gemm: Add flashinfer grouped_gemm_nt_masked by @wenscarl in https://github.com/sgl-project/sglang/pull/9199
fix: flashinfer_cutlass_moe: Use max of global expert scales instead of local for input scale by @trevor-m in https://github.com/sgl-project/sglang/pull/10296
model: support Apertus by @EduardDurech in https://github.com/sgl-project/sglang/pull/9774
fix dual stream bug by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10352
[router] Basic OAI Response api by @key4ng in https://github.com/sgl-project/sglang/pull/10346
Implement Standalone gRPC Server for SGLang Python Scheduler by @CatherineSue in https://github.com/sgl-project/sglang/pull/10283
support memory_pool_host page first direct layout by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/10031
fix the break in FlashInferFusedMoE by @chenqianfzh in https://github.com/sgl-project/sglang/pull/10356
fix: resolve transfer_kv_all_layer_direct_lf_pf import error by @zhyncs in https://github.com/sgl-project/sglang/pull/10360
Support LingV2 model by @strgrb in https://github.com/sgl-project/sglang/pull/10359
Fix Bailing MoE model bugs by @yuan-luo in https://github.com/sgl-project/sglang/pull/10362
Revert add mainprocess's proctitle by @whybeyoung in https://github.com/sgl-project/sglang/pull/10351
model: support dots.vlm1 model by @yonghenglh6 in https://github.com/sgl-project/sglang/pull/8778
Support loading weights from remote instance by @amysaq2023 in https://github.com/sgl-project/sglang/pull/8215
add qwen3-next ut by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10355
Fix chunked prefix cache for nvfp4 by @wenscarl in https://github.com/sgl-project/sglang/pull/10180
Fix FA4 import cause moe_fused_gate output be illegal memory by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10368
Fix global input scale incompatible with CuTe DSL moe by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10370
[router] Add Rerank Routing Logic in Regular Router by @fangjian601 in https://github.com/sgl-project/sglang/pull/10219
[router] enable sccache in ci and local build by @slin1237 in https://github.com/sgl-project/sglang/pull/10099
fix: add fast path for function call by @yizhang2077 in https://github.com/sgl-project/sglang/pull/9023
[Auto Sync] Update base_grammar_backend.py, llguidance_back... (20250911) by @merrymercy in https://github.com/sgl-project/sglang/pull/10333
fix: resolve gb200 image link by @zhyncs in https://github.com/sgl-project/sglang/pull/10343
fix: exclude protobuf generated code by @zhyncs in https://github.com/sgl-project/sglang/pull/10388
[bug] fix ci syntax by @slin1237 in https://github.com/sgl-project/sglang/pull/10390
Fix GPU fault issue when run dsv3 with dp mode and enable torch-compile by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/10361
feat: add deepseek v3 fp4 ut by @zhyncs in https://github.com/sgl-project/sglang/pull/10391
Add sentencepiece to project dependencies by @mmangkad in https://github.com/sgl-project/sglang/pull/10386
[router] allow one router to support different model families and serving mode by @slin1237 in https://github.com/sgl-project/sglang/pull/10244
[router] Add get and cancel method for response api by @key4ng in https://github.com/sgl-project/sglang/pull/10387
Benchmark: Support API_KEY without 'bearer' by @Muqi1029 in https://github.com/sgl-project/sglang/pull/10380
Support Qwen3-Next on Ascend NPU by @iforgetmyname in https://github.com/sgl-project/sglang/pull/10379
[HiCache] fix mooncake config in different tp size by @stmatengss in https://github.com/sgl-project/sglang/pull/10377
[HiCache] doc: update deployment in readme by @stmatengss in https://github.com/sgl-project/sglang/pull/10332
[router] add not implemented functions for multi model trait by @slin1237 in https://github.com/sgl-project/sglang/pull/10394
[Auto Sync] Update xgrammar_backend.py (20250913) by @merrymercy in https://github.com/sgl-project/sglang/pull/10395
fix probs name which without temp scaling name by @narutolhy in https://github.com/sgl-project/sglang/pull/9984
Fix the style of sgl kernel by @merrymercy in https://github.com/sgl-project/sglang/pull/10398
fix: tool parse in large streaming chunk beginning with normal content by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10397
[Fix] Init mamba related memory pools with torch.zeros by @byjiang1996 in https://github.com/sgl-project/sglang/pull/10400
support qwen3_next blackwell by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10403
[Fix] Support qwen3-next MTP+DP by @byjiang1996 in https://github.com/sgl-project/sglang/pull/10392
Update ROCm docker image to add sgl-router support by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/10406
[Performance] Dynamic Batch Tokenizer by @sundar24295s in https://github.com/sgl-project/sglang/pull/9382
[Generative Score API] Scoring(Prefill-only) optimizations. by @sundar24295s in https://github.com/sgl-project/sglang/pull/9748
Remove repeatedly lists adding in init_incremental_detokenization by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10412
[Hack] Add pd-disaggregation decode polling interval by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10411
fix duplicated logger in eager_utils by @lj970926 in https://github.com/sgl-project/sglang/pull/10410
Fix cutlass moe accuracy drop caused by attention UB from DP padding mode by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10414
Add self.capture_aux_hidden_states For GLM-4.5V by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/10228
Add h200 fused moe config for Qwen3-Next by @Ximingwang-09 in https://github.com/sgl-project/sglang/pull/10404
Auto determine sgl kernel version in blackwell CI by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10318
Fix the global scale fix does not support EPLB and improve enabling condition by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10369
Let sgl-kernel changes be tested on srt by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10313
[2/2] Speed up prefill mla attention concat by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10157
Support offloading in fp8 by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9948
Support global scale in addition to per expert scale for cutedsl moe by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10270
Support profile args in Engine API by @fzyzcjy in https://github.com/sgl-project/sglang/pull/6539
Fix sgl-kernel + srt CI by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10419
[PD metrics] Fix some uncompleted PD related metrics by @acelyc111 in https://github.com/sgl-project/sglang/pull/8627
Typo: in --enable-custom-logit-processor: agree with cli arg by @thalahors in https://github.com/sgl-project/sglang/pull/10076
[Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 1 by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/9962
fix: use latest flashinfer by @zhyncs in https://github.com/sgl-project/sglang/pull/10428
fix: enable cu124 and cu128 build on main push by @zhyncs in https://github.com/sgl-project/sglang/pull/10431
[Fix] MoE: fix w8a8_fp8 MoE and add tests to cover this code path by @ch-wan in https://github.com/sgl-project/sglang/pull/10429
Add split tile size for Triton attention by @ispobock in https://github.com/sgl-project/sglang/pull/10425
Fix correction bias undefined behavior for nvfp4 models by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10426
feat: add dsv3 fp4 cutlass moe etp ut by @zhyncs in https://github.com/sgl-project/sglang/pull/10433
router: Add Embedding routing logic by @tao12345666333 in https://github.com/sgl-project/sglang/pull/10129
Revert "Fix FA4 import cause moe_fused_gate output be illegal memory" by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10432
[4/N]DP refactor: support watching mode get_load and shortest queue strategy by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10201
automatically label pr for ci by @merrymercy in https://github.com/sgl-project/sglang/pull/10435
Refactor TopK to ensure readability and extensibility by @ch-wan in https://github.com/sgl-project/sglang/pull/9338
Tiny fix wrong naming by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10437
Fix label pr for ci by @merrymercy in https://github.com/sgl-project/sglang/pull/10441
metrics: support customer labels specified in request header by @acelyc111 in https://github.com/sgl-project/sglang/pull/10143
[docs / oneliner] update mmmu docs instruction by @vincentzed in https://github.com/sgl-project/sglang/pull/9768
Add reasoning examples for GPT-OSS in Markdown examples by @vincentzed in https://github.com/sgl-project/sglang/pull/9626
Fix label PR by @merrymercy in https://github.com/sgl-project/sglang/pull/10445
Update permissions in label-pr.yml by @merrymercy in https://github.com/sgl-project/sglang/pull/10450
[CI] Fix token key in label-pr.yml workflow by @merrymercy in https://github.com/sgl-project/sglang/pull/10452
fix: fix max_new_tokens uninitialized error by @mickqian in https://github.com/sgl-project/sglang/pull/9343
[router] fix service discovery and mcp ut by @slin1237 in https://github.com/sgl-project/sglang/pull/10449
fix(server_args): Skip chunked_prefill_size validation when disaggregation mode is decode by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/10358
[router] add dependency for router by @ooapex in https://github.com/sgl-project/sglang/pull/10401
[router] fix logger ordering git ctx by @CatherineSue in https://github.com/sgl-project/sglang/pull/10457
Update GITHUB_TOKEN secret for documentation push by @merrymercy in https://github.com/sgl-project/sglang/pull/10458
[HotFix]: Hot fix import path in 3fs_bench_client.py by @hzh0425 in https://github.com/sgl-project/sglang/pull/10463
Add rtx5880 moe triton by @Jimmy-L99 in https://github.com/sgl-project/sglang/pull/10439
Run tests based on labels by @merrymercy in https://github.com/sgl-project/sglang/pull/10456
Fix trtllm_moe wrong correction bias by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10440
feat: support pip install sglang by @zhyncs in https://github.com/sgl-project/sglang/pull/10465
chore: bump v0.5.3rc0 by @zhyncs in https://github.com/sgl-project/sglang/pull/10468
[router] minor code clean up in server startup by @CatherineSue in https://github.com/sgl-project/sglang/pull/10470
[bugfix] fix typo by @1195343015 in https://github.com/sgl-project/sglang/pull/10471
[PD metrics] Add latency Histogram metrics of each stage for generate requests by @acelyc111 in https://github.com/sgl-project/sglang/pull/8710
[CI] Fix runner for sgl-kernel by @csahithi in https://github.com/sgl-project/sglang/pull/9887
fix(internvl): fix accuracy issue of normalization by @KEVINTUAN12 in https://github.com/sgl-project/sglang/pull/10375
fix: gpt-oss streaming dropping normal content when tools are provided but not used by @jonaslsaa in https://github.com/sgl-project/sglang/pull/9657
model: support solar by @ppraneth in https://github.com/sgl-project/sglang/pull/8189
fix: resolve sgl-kernel ut by @zhyncs in https://github.com/sgl-project/sglang/pull/10476
[1/2] Speed up trtllm_mla attention backend (>10% e2e) by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10473
Fix --dataset-path in bench_one_batch_server by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10475
[Env] minimal version for organizing envs by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10479
chore: bump v0.3.10 sgl-kernel by @zhyncs in https://github.com/sgl-project/sglang/pull/10478
[router] multi model registration fix by @CatherineSue in https://github.com/sgl-project/sglang/pull/10481
[2/4] Introduce Chunked-SGMV kernels and corresponding LoRA backend for improved performance by @lifuhuang in https://github.com/sgl-project/sglang/pull/10286
[Auto Sync] Update registry.py (20250915) by @merrymercy in https://github.com/sgl-project/sglang/pull/10484
[router] fix worker registration in multi model mode by @CatherineSue in https://github.com/sgl-project/sglang/pull/10486
fix crash of DeepSeek-V3 update_weights_from_disk by @scut-cbq in https://github.com/sgl-project/sglang/pull/8863
Temporay work-around for rocm 7.0.0 alpha with enabling data-parallel issue by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/10434
[Hicache] Evaluate Per-Round Metrics in Multiturn Bench by @ykwd in https://github.com/sgl-project/sglang/pull/10203
[ModelOpt] Respect kv_cache_quant_algo in ModelOpt checkpoints by @brayden-hai in https://github.com/sgl-project/sglang/pull/10336
Add Logprobs unit test with a loose threshold by @PrinsYin in https://github.com/sgl-project/sglang/pull/10230
[router] add router db connector for responses api by @slin1237 in https://github.com/sgl-project/sglang/pull/10487
Remove wrong imports from sglang.python by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10493
[router] fix router manager and router init in server by @CatherineSue in https://github.com/sgl-project/sglang/pull/10499
Cache the result of is_blackwell platform check by @b8zhong in https://github.com/sgl-project/sglang/pull/10498
feat: update support for qwen3next model by @cao1zhg in https://github.com/sgl-project/sglang/pull/10466
Minor fix lint introduced by #10466 by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10507
chore: upgrade sgl-kernel 0.3.10 by @zhyncs in https://github.com/sgl-project/sglang/pull/10500
Update CUTLASS. Refine KernelSchedule for fp8 (grouped) gemm. by @HydraQYH in https://github.com/sgl-project/sglang/pull/10491
Fix CI when sgl-kernel is changed but srt is not changed by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10515
Support sgl-router parallel_batch in bench_one_batch_server by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10506
[CPU] fix CPU backend sel. issue for Llama4 by @ZailiWang in https://github.com/sgl-project/sglang/pull/10511
adjust import setuptools_rust by @whybeyoung in https://github.com/sgl-project/sglang/pull/10524
Fix formatting in long code blocks by @philipkiely-baseten in https://github.com/sgl-project/sglang/pull/10528
skip vision_model for lora by @gongwei-130 in https://github.com/sgl-project/sglang/pull/10530
[2/2] Speed up trtllm_mla attention backend by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10474
support using fa4 on deepseek on blackwell by @cicirori in https://github.com/sgl-project/sglang/pull/9928
[Auto Sync] Update scheduler_profiler_mixin.py, rpd_utils.p... (20250916) by @merrymercy in https://github.com/sgl-project/sglang/pull/10494
[Auto Sync] Update activation.py, chunk_cache.py, utils.py (20250917) by @merrymercy in https://github.com/sgl-project/sglang/pull/10538
feat: add priority based scheduling with priority based request acceptance and preemption by @harrisonlimh in https://github.com/sgl-project/sglang/pull/8746
Fix decord dependency for aarch64 docker build by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/10529
enable prefix cache with dp by @wenscarl in https://github.com/sgl-project/sglang/pull/10459
[bugfix]hicache bench_long_context.py run failed by @zhannngchen in https://github.com/sgl-project/sglang/pull/10523
Remove duplicated code by @oraluben in https://github.com/sgl-project/sglang/pull/10545
CUDA Arch Independent by @EduardDurech in https://github.com/sgl-project/sglang/pull/8813
[bench] Fix random seed in bench_one_batch_server by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10548
[HiCache] Add tests for hicache storage mooncake backend by @stmatengss in https://github.com/sgl-project/sglang/pull/10171
[BugFix] Fix incorrect hidden_states_tensor in pd disaggregation + eagle by @ZeldaHuang in https://github.com/sgl-project/sglang/pull/9976
fix: update dsv3 fp4 ut by @zhyncs in https://github.com/sgl-project/sglang/pull/10584
vlm: remove redundant d2h movement of mm feature tensors by @AlienKevin in https://github.com/sgl-project/sglang/pull/9987
Enable trtllm mla prefix extend by @wenscarl in https://github.com/sgl-project/sglang/pull/10526
[ROCm] Fix fp8 quantization accuracy issue. by @sogalin in https://github.com/sgl-project/sglang/pull/10558
[HICache] introduce evict policy by @XucSh in https://github.com/sgl-project/sglang/pull/10190
aiter v0.1.5.post2 by @HaiShaw in https://github.com/sgl-project/sglang/pull/10563
[PD] Improve disaggregation common backend and refactor mooncake backend by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10273
chore: upgrade mooncake 0.3.6 by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10596
[improvement] add average input/output token length for hicache benchmark stats output by @zhannngchen in https://github.com/sgl-project/sglang/pull/10525
Scale kkt after reduction by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10604
fix deepep assert when PD disaggregation == null by @alpha-baby in https://github.com/sgl-project/sglang/pull/8274
[RL] Add destroy process group api by @penguin-wwy in https://github.com/sgl-project/sglang/pull/9979
Feat/add heartbeat mechanism for nixl conn by @shaharmor98 in https://github.com/sgl-project/sglang/pull/10222
update deepep version for qwen3-next deepep moe by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10624
support qwen3-next-fp8 deepep by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10622
Fix sgl_kernel import failure on devices other than CUDA by @ZailiWang in https://github.com/sgl-project/sglang/pull/10610
[Performance] qwen3-next improve causal conv1d in prefill phase by @liz-badada in https://github.com/sgl-project/sglang/pull/10595
Fix bias handling in TritonMoeQuantInfo within quantization/mxfp4.py by @yhyang201 in https://github.com/sgl-project/sglang/pull/10579
feat: Add FlexAttention Backend for Efficient Sparse Attention by @yukiy927 in https://github.com/sgl-project/sglang/pull/9947
Garbage collector regression in the online server by @brayden-hai in https://github.com/sgl-project/sglang/pull/10621
[router] refactor worker to builder pattern 1/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10628
refactor: use registry for _get_attention_backend_from_str by @zhyncs in https://github.com/sgl-project/sglang/pull/10629
[Feature] Speculative decoding support lookahead by @a4zhangfei in https://github.com/sgl-project/sglang/pull/9873
[Performance] Qwen3-Next: replace arange to cached query_start_loc_li… by @byjiang1996 in https://github.com/sgl-project/sglang/pull/10553
[Performance] Qwen3-Next: speed up update_mamba_state_after_mtp_verify by 10x; e2e up to 3.54% faster by @byjiang1996 in https://github.com/sgl-project/sglang/pull/10586
model support: Sarashina2VisionForCausalLM by @CatherineSue in https://github.com/sgl-project/sglang/pull/10632
feat: add fused moe config for Qwen3-Next-80B-A3B-Instruct on B200 by @zixuanzhang226 in https://github.com/sgl-project/sglang/pull/10631
chore: bump sgl-kernel 0.3.11 by @zhyncs in https://github.com/sgl-project/sglang/pull/10630
Hicache L3 backend mooncake optimization configuration reading method by @leihuang-sketch in https://github.com/sgl-project/sglang/pull/10319
[router] refactor worker to builder pattern 2/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10633
[Feature]feat(get_ip): unify get_ip_xxx by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/10081
[router] refactor worker to builder pattern 3/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10647
[1/2][sgl-kernel] Support moe_sum_reduce cuda kernel by @yuan-luo in https://github.com/sgl-project/sglang/pull/10321
[router] refactor worker to builder pattern 4/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10650
Fix fast decode plan for flashinfer v0.4.0rc1 and upgrade sgl-kernel 0.3.11 by @Fridge003 in https://github.com/sgl-project/sglang/pull/10634
[router] refactor worker to builder pattern 5/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10653
[HiCacheStorage]support page_first_direct layout for generic set&get by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/10522
[router] preserve order of json params using preserve_order feature by @fgebhart in https://github.com/sgl-project/sglang/pull/10661
[router] refactor router and worker management 1/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10664
fix: resolve sync issue by @zhyncs in https://github.com/sgl-project/sglang/pull/10668
[Auto Sync] Update .clang-format (20250919) by @zhyncs in https://github.com/sgl-project/sglang/pull/10670
[router] refactor router and worker management 2/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10666
router-spec: Reorder ChatCompletionRequest and fix validation logic by @CatherineSue in https://github.com/sgl-project/sglang/pull/10675
chore: cleanup docker image by @zhyncs in https://github.com/sgl-project/sglang/pull/10671
limit sgl-kernel causal conv1d to cuda only by @liz-badada in https://github.com/sgl-project/sglang/pull/10648
[Auto Sync] Update model_runner.py (20250920) by @zhyncs in https://github.com/sgl-project/sglang/pull/10679
[router] refactor router and worker management 2.5/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10677
[1/2] Support deterministic inference with flashinfer attention backend by @Fridge003 in https://github.com/sgl-project/sglang/pull/10645
[Auto Sync] Update deepseek_v2.py (20250920) by @zhyncs in https://github.com/sgl-project/sglang/pull/10683
chore: upgrade mooncake 0.3.6.post1 to fix gb200 dockerfile by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10681
[Performance] Qwen3-Next: optimize causal_conv1d_fn triton kernel - up to 9% faster by @byjiang1996 in https://github.com/sgl-project/sglang/pull/10680
Replace os.environ in layernorm.py by @Fridge003 in https://github.com/sgl-project/sglang/pull/10684
fix(disagg): fix sending KV cache in case of MLA for NIXL backend by @dmitrygx in https://github.com/sgl-project/sglang/pull/10673
fix: update run_suite by @zhyncs in https://github.com/sgl-project/sglang/pull/10685
fix: remove awq_dequantize deps by @zhyncs in https://github.com/sgl-project/sglang/pull/10686
[Auto Sync] Update modelopt_quant.py (20250920) by @zhyncs in https://github.com/sgl-project/sglang/pull/10688
[Feature] Support deterministic inference with FA3 backend by @hebiao064 in https://github.com/sgl-project/sglang/pull/10651
feat: update server args by @zhyncs in https://github.com/sgl-project/sglang/pull/10696
Super tiny fix extra logs by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10697
[3/4] Speed up CSGMV backend perf by 10% through dynamic chunking + kernel optimization by @lifuhuang in https://github.com/sgl-project/sglang/pull/10592
Update release-docs.yml by @sglang-bot in https://github.com/sgl-project/sglang/pull/10706
Refactors radix cache for extra key support by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10317
[Router]fix: fix get_load missing api_key by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/10385
fix: disable gpt-oss b200 ut by @zhyncs in https://github.com/sgl-project/sglang/pull/10716
Optimize cutlass int8 gemm kernel for large M on SM89 Ada GPU by @HydraQYH in https://github.com/sgl-project/sglang/pull/10714
[Auto Sync] Update deepseek_v2.py (20250922) by @zhyncs in https://github.com/sgl-project/sglang/pull/10717
Support deterministic inference with triton backend (Hardware test: NV and AMD GPUs) by @yushengsu-thu in https://github.com/sgl-project/sglang/pull/10694
[deterministic inference] Move batch invariant pkg to sglang by @hebiao064 in https://github.com/sgl-project/sglang/pull/10695
[2/2] Support deterministic inference for temperature > 0 by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/10678
[Ascend] codeowner updates for ascend related files by @ping1jing2 in https://github.com/sgl-project/sglang/pull/10699
[4/4] Introduce CachedKernel to reduce CSGMV kernel launch overheads by 60% by @lifuhuang in https://github.com/sgl-project/sglang/pull/10709
Convert FLASHINFER_WORKSPACE_SIZE to integer by @reyoung in https://github.com/sgl-project/sglang/pull/10731
EPLB: prefer to use physical experts in the same node by @acelyc111 in https://github.com/sgl-project/sglang/pull/9849
fix capture_bs when speculative decoding enabled by @feng397 in https://github.com/sgl-project/sglang/pull/10730
Fix flaky logprobs test by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10728
Fix CI TestChunkedSGMV by @lifuhuang in https://github.com/sgl-project/sglang/pull/10737
[Docs, minor] Fix LLM doc matrix by @adarshxs in https://github.com/sgl-project/sglang/pull/10753
Add warnings and remove dependency for deterministic inference by @Fridge003 in https://github.com/sgl-project/sglang/pull/10724
bugfix: Fix get_worker_urls_for_model in http/router.rs by @CatherineSue in https://github.com/sgl-project/sglang/pull/10754
[router] refactor router and worker management 3/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10727
[router] update ci so only execute benchmarks when labels are added by @slin1237 in https://github.com/sgl-project/sglang/pull/10757
Fix MTP MoE weight loading with NVFP4 target model. by @LorrinWWW in https://github.com/sgl-project/sglang/pull/10758
chore: bump sgl-kernel v0.3.12 by @zhyncs in https://github.com/sgl-project/sglang/pull/10732
[Generative Score API] Added test_scores_api.py to github CICD to run per commit by @vedantjh2 in https://github.com/sgl-project/sglang/pull/10755
refactor zero copy by @pansicheng in https://github.com/sgl-project/sglang/pull/10300
Fix multimodal registry and code sync scripts by @merrymercy in https://github.com/sgl-project/sglang/pull/10759
Enables TRT-LLM backend to be used for target_verify by @pranavm-nvidia in https://github.com/sgl-project/sglang/pull/10281
fix: kv events with tp > 1 by @ishandhanani in https://github.com/sgl-project/sglang/pull/10541
[Auto Sync] Update flashattention_backend.py (20250922) by @zhyncs in https://github.com/sgl-project/sglang/pull/10762
[Feature] Add MLAProcess for DeepSeek MLA on NPU by @iforgetmyname in https://github.com/sgl-project/sglang/pull/10130
[Ascend] optimize Qwen-vl on Ascend by @ping1jing2 in https://github.com/sgl-project/sglang/pull/10556
[Ascend]optimize Qwen3 on Ascend by @ping1jing2 in https://github.com/sgl-project/sglang/pull/10574
[Auto Sync] Update configurer.py (20250923) by @merrymercy in https://github.com/sgl-project/sglang/pull/10765
[router] refactor router and worker management 4/n by @slin1237 in https://github.com/sgl-project/sglang/pull/10756
[router] remove pd router draining channel by @slin1237 in https://github.com/sgl-project/sglang/pull/10767
[router] fix logger type mismatch by @CatherineSue in https://github.com/sgl-project/sglang/pull/10774
Use simulate acc len from sglang.environ by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10771
Fix trtllm_mla slow concat kernel in MTP by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10777
Move cached kernel to srt.utils by @lifuhuang in https://github.com/sgl-project/sglang/pull/10776
feat: unify dockerfiles by @ishandhanani in https://github.com/sgl-project/sglang/pull/10705
Introduce FutureMap by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10715
chore: upgrade sgl-kernel 0.3.12 by @zhyncs in https://github.com/sgl-project/sglang/pull/10782
followup: clean up dockerfiles and release yamls by @ishandhanani in https://github.com/sgl-project/sglang/pull/10783
Clean up server args by @merrymercy in https://github.com/sgl-project/sglang/pull/10770
move environ into sglang.srt to avoid break SRT auto sync. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10791
Fix hicache mooncake backend CI by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10792
[router] fix cache aware lock by @slin1237 in https://github.com/sgl-project/sglang/pull/10773
[router] responses api POST and GET with local storage by @slin1237 in https://github.com/sgl-project/sglang/pull/10581
model: support qwen3-vl series by @zju-stu-lizheng in https://github.com/sgl-project/sglang/pull/10323
[fix][pd-disag]no need set next batch sampling info done in prefill by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/10259
[ROCm] Update aiter to v0.1.5.post3 by @sogalin in https://github.com/sgl-project/sglang/pull/10812
[router] use dashmap for radix tree instead of hash for multi model by @slin1237 in https://github.com/sgl-project/sglang/pull/10814
router(grpc): Implement route for chat_cmpl endpoint by @CatherineSue in https://github.com/sgl-project/sglang/pull/10761
fix ceval by @ZhengHSI in https://github.com/sgl-project/sglang/pull/10504
Remove duplicate code in qwen2 model by @Lzhang-hub in https://github.com/sgl-project/sglang/pull/10540
[router] fix axum default body limit by @CatherineSue in https://github.com/sgl-project/sglang/pull/10818
Fix latest main ci by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10799
add tunning files for QWEN-3-NEXT by @yiakwy-xpu-ml-framework-team in https://github.com/sgl-project/sglang/pull/10794
[Auto Sync] Update protocol.py (20250923) by @zhyncs in https://github.com/sgl-project/sglang/pull/10820
fix: draft model IMA by overide max_positional_embeddings by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10787
[Auto Sync] Update elementwise.py (20250923) by @merrymercy in https://github.com/sgl-project/sglang/pull/10823
[Auto Sync] Update simple_eval_common.py (20250923) by @merrymercy in https://github.com/sgl-project/sglang/pull/10824
[router] Support streaming for Openai Router Response api by @key4ng in https://github.com/sgl-project/sglang/pull/10822
[router] add auth middleware for api key auth by @CatherineSue in https://github.com/sgl-project/sglang/pull/10826
[Auto Sync] Update load_config.py, model_config.py, configu... (20250923) by @merrymercy in https://github.com/sgl-project/sglang/pull/10825
Revert "[fix][pd-disag]no need set next batch sampling info done in prefill" by @merrymercy in https://github.com/sgl-project/sglang/pull/10828
Add CI timeout guidelines by @merrymercy in https://github.com/sgl-project/sglang/pull/10829
feat: add cache_salt support to request by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10718
fix bailing_moe with enable_dp_attention by @GuoweiWangU in https://github.com/sgl-project/sglang/pull/10860
ci: free space on workers for build by @ishandhanani in https://github.com/sgl-project/sglang/pull/10786
router-grpc: Support jinja chat template content format detection by @CatherineSue in https://github.com/sgl-project/sglang/pull/10832
[router] select first healthy worker on proxied get requests by @lun-4 in https://github.com/sgl-project/sglang/pull/10827
chore: Initial support for input config files by @kushanam in https://github.com/sgl-project/sglang/pull/10534
router-grpc: Add tools processing and other paramters for apply_chat_template by @CatherineSue in https://github.com/sgl-project/sglang/pull/10877
[router] consolidate health endpoints and flush cache by @slin1237 in https://github.com/sgl-project/sglang/pull/10876
Restruct sgl-kernel benchmark by @BBuf in https://github.com/sgl-project/sglang/pull/10861
[Bug] Fix Issue#10215 by @yuhyao in https://github.com/sgl-project/sglang/pull/10572
[router] consolidate worker get loads by @slin1237 in https://github.com/sgl-project/sglang/pull/10880
[router] Support Oracle DB(ATP) Data Connector by @key4ng in https://github.com/sgl-project/sglang/pull/10845
[router] simplify tokenizer dev doc by @slin1237 in https://github.com/sgl-project/sglang/pull/10895
[Auto Sync] Update model_config.py (20250925) by @merrymercy in https://github.com/sgl-project/sglang/pull/10885
[ci feature] add ci monitor by @BBuf in https://github.com/sgl-project/sglang/pull/10872
[HiCache] Cleaning the deprecated host memory state by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/10778
integrate AIBrix KVcache by @yapple in https://github.com/sgl-project/sglang/pull/10376
Add fuse_moe per-channel tune by @sleepcoo in https://github.com/sgl-project/sglang/pull/10915
[router] consolidate worker load monitoring by @slin1237 in https://github.com/sgl-project/sglang/pull/10894
router: Fix constraint proto and build_constraint in grpc router by @CatherineSue in https://github.com/sgl-project/sglang/pull/10881
Refactor kv_cache_scheme handling for quantization by @mmangkad in https://github.com/sgl-project/sglang/pull/10132
refactor: Move grpc/client.rs to grpc_client/sglang_scheduler.rs by @CatherineSue in https://github.com/sgl-project/sglang/pull/10924
fix env flashinfer by @Swipe4057 in https://github.com/sgl-project/sglang/pull/10910
[minor] Remove deprecated function get_ip by @merrymercy in https://github.com/sgl-project/sglang/pull/10883
Rename customer label -> custom label by @merrymercy in https://github.com/sgl-project/sglang/pull/10899
[router] change log level to warning by @slin1237 in https://github.com/sgl-project/sglang/pull/10926
[router][refactor] Clean up protobuf fields by @CatherineSue in https://github.com/sgl-project/sglang/pull/10923
Replace the Kimi-K2 generated tool call idx with history tool call count by @eraser00 in https://github.com/sgl-project/sglang/pull/10612
[ci] add ci-monitor workflow by @BBuf in https://github.com/sgl-project/sglang/pull/10898
Remove pull_request trigger from CI monitor workflow by @merrymercy in https://github.com/sgl-project/sglang/pull/10932
router: Support parallel sampling num > 1 in grpc_server and non-stream handling by @CatherineSue in https://github.com/sgl-project/sglang/pull/10929
Revert "Refactor kv_cache_scheme handling for quantization (#10132)" by @zhyncs in https://github.com/sgl-project/sglang/pull/10935
Update CODEOWNERS to include JustinTong0323 in FC by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10939
[PD-HiCache]: Support Async Offloading KVCache In Decode Side by @hzh0425 in https://github.com/sgl-project/sglang/pull/10192
CI: Fix docker manifest build by @csahithi in https://github.com/sgl-project/sglang/pull/10936
[router] update owners for router components by @slin1237 in https://github.com/sgl-project/sglang/pull/10927
Fuse write kv buffer into rope for qwen3 moe & bailing moe by @yuan-luo in https://github.com/sgl-project/sglang/pull/10749
[router] add grpc client get and set by @slin1237 in https://github.com/sgl-project/sglang/pull/10955
[router]fix code owner syntax error by @slin1237 in https://github.com/sgl-project/sglang/pull/10956
[router] move grpc client from router to worker and builder by @slin1237 in https://github.com/sgl-project/sglang/pull/10958
[router] add move grpc worker management from router to worker manager by @slin1237 in https://github.com/sgl-project/sglang/pull/10960
[router] grpc router regular mode import cleanup by @slin1237 in https://github.com/sgl-project/sglang/pull/10963
[router] remove old/oudated/useless comments by @slin1237 in https://github.com/sgl-project/sglang/pull/10967
[router] remove old/oudated/useless comments across code base by @slin1237 in https://github.com/sgl-project/sglang/pull/10968
ci: fix rate-limit of huggingface with hf auth login by @mickqian in https://github.com/sgl-project/sglang/pull/10947
Update label field comment to indicate deprecation by @merrymercy in https://github.com/sgl-project/sglang/pull/10970
Restruct gpu_memory_settings in a unify function and relax max_cuda_graph_bs by @BBuf in https://github.com/sgl-project/sglang/pull/10372
ci: refactor nightly test by @mickqian in https://github.com/sgl-project/sglang/pull/10495
refactor loading weights from remote instance coding format by @amysaq2023 in https://github.com/sgl-project/sglang/pull/10941
[router][grpc] Add helpfer functions for decoder in router.rs and fix specs by @CatherineSue in https://github.com/sgl-project/sglang/pull/10971
Add simple docker file for B300 by @hlu1 in https://github.com/sgl-project/sglang/pull/10944
Ci monitor support performance by @BBuf in https://github.com/sgl-project/sglang/pull/10965
[HiCache]: Support dynamic loading backends for hicache by @hzh0425 in https://github.com/sgl-project/sglang/pull/10551
[Bugfix][Minor][Benchmark] Fix some bugs due to PR #10495 by @Muqi1029 in https://github.com/sgl-project/sglang/pull/10982
[router][grpc] Support E2E non-stream chat completions by @CatherineSue in https://github.com/sgl-project/sglang/pull/10980
fix: fp8 quantization failure of qwen 2.5 VL 7B model by @PanJason in https://github.com/sgl-project/sglang/pull/10112
[Fix] RuntimeError: get_cfg Unsupported input_type:Float4_e2m1fn_x2 in using aiter-mxfp4-moe by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/10981
fix: make inference deterministic for large TP by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10930
Add auth to get server info by @Muqi1029 in https://github.com/sgl-project/sglang/pull/10751
Add support for topk metadata transferring for PD by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10616
[PD] Extract the PP transfer layer calculate logic from Mooncake to Common backend by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10565
Use jsonschema to constrain required or specific tool choice by @TJ5 in https://github.com/sgl-project/sglang/pull/10550
Fix profiler by @merrymercy in https://github.com/sgl-project/sglang/pull/10997
[router][tool parser] Modify tool parser to return both normal text and tool calls (non-stream) by @CatherineSue in https://github.com/sgl-project/sglang/pull/10995
[router] basic mcp support for openai router response api by @key4ng in https://github.com/sgl-project/sglang/pull/10978
[router] fix chat template loading and tokenizer path by @slin1237 in https://github.com/sgl-project/sglang/pull/10999
Fix CI failure of TypeError: RotaryEmbedding.forward_cpu() got an unexpected keyword argument 'fused_set_kv_buffer_arg' by @yanbing-j in https://github.com/sgl-project/sglang/pull/11009
[bugfix]Add empty_context import to two_batch_overlap.py by @wejoncy in https://github.com/sgl-project/sglang/pull/10964
prepare for sglang+verl by @lbk-sys in https://github.com/sgl-project/sglang/pull/10555
[sgl-kernel] Optimize concat_mla_k kernel by @yuan-luo in https://github.com/sgl-project/sglang/pull/10543
[HiCache] bug: fix mooncake store batch set v1 by @stmatengss in https://github.com/sgl-project/sglang/pull/11013
Fix FusedSetKVBufferArg in RotaryEmbedding by @merrymercy in https://github.com/sgl-project/sglang/pull/11003
Update GLM-4.5 Model Doc by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/11017
[router] migrate to rust python module for pythonic parser by @slin1237 in https://github.com/sgl-project/sglang/pull/11033
fix: show failed models in nightly ci by @mickqian in https://github.com/sgl-project/sglang/pull/10986
[router][tool call] Support normal content extraction before tool call (streaming) by @CatherineSue in https://github.com/sgl-project/sglang/pull/11038
[router] add harmony tool parser base structure and interface by @slin1237 in https://github.com/sgl-project/sglang/pull/11036
Unify SGL Kernel Releases by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/10701
[1/2] Support FA4 for MHA Prefill in sgl-kernel by @lifuhuang in https://github.com/sgl-project/sglang/pull/10940
fix: check if weights are already local before downloading by @mickqian in https://github.com/sgl-project/sglang/pull/11015
[HiCacheStorage] mooncake store support page_first_direct layout by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/10591
[speculative decoding] rename lookahead to ngram by @a4zhangfei in https://github.com/sgl-project/sglang/pull/11010
Fix gemma 3 launch with transformers: the error: AttributeError: 'TransformersForCausalLM' object has no attribute 'tp_size' by @vincentzed in https://github.com/sgl-project/sglang/pull/9614
Fix sgl-kernel benchmark dead code by @BBuf in https://github.com/sgl-project/sglang/pull/11022
[router][tool call] Improve normal content extraction and error handling (non-stream) by @CatherineSue in https://github.com/sgl-project/sglang/pull/11050
chore: upgrade cutedsl 4.2.1 by @zhyncs in https://github.com/sgl-project/sglang/pull/11054
[Ci Monitor] Auto uploaded performance data to sglang_ci_data repo by @BBuf in https://github.com/sgl-project/sglang/pull/10976
chore: upgrade sgl-kernel 0.3.13 by @zhyncs in https://github.com/sgl-project/sglang/pull/11056
[router] add n to generate sampling params by @slin1237 in https://github.com/sgl-project/sglang/pull/11069
Use more general heuristics to set the default value of --mem-fraction-static by @merrymercy in https://github.com/sgl-project/sglang/pull/10975
[router][tool call] Separate JsonParser and LlamaParser by @CatherineSue in https://github.com/sgl-project/sglang/pull/11073
Fix mem fraction static for nightly tests by @merrymercy in https://github.com/sgl-project/sglang/pull/11076
fix: fp8 mllama4 without vision modules being quantized by @mickqian in https://github.com/sgl-project/sglang/pull/10611
Use get_pooled in process_single_choice by @CatherineSue in https://github.com/sgl-project/sglang/pull/11079
[router][grpc] Add logprobs support to router by @CatherineSue in https://github.com/sgl-project/sglang/pull/11082
feat(reasoning): improve enable thinking from request by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/10875
[Profile] dump memory trace when cuda graph profile is enabled by @ch-wan in https://github.com/sgl-project/sglang/pull/11083
Remove hybrid_linear_attn attention backend and refactor attention registry by @samuellees in https://github.com/sgl-project/sglang/pull/10816
[model] added support for w8a8int8 used by neuralmagic/Qwen2-0.5B-Ins… by @DevashishLal-CB in https://github.com/sgl-project/sglang/pull/9642
Enable optional FP32 compute for LM Head by @narutolhy in https://github.com/sgl-project/sglang/pull/10729
Update CODEOWNERS for attention/ascend_backend.py by @merrymercy in https://github.com/sgl-project/sglang/pull/11092
[router] grpc router generate endpoint support by @slin1237 in https://github.com/sgl-project/sglang/pull/11070
[router][tool call] Full support for ToolChoice by @CatherineSue in https://github.com/sgl-project/sglang/pull/11085
Fix spec filter batch when target extend by @ispobock in https://github.com/sgl-project/sglang/pull/10991
[Fix] Resolve performance drop in speculative decoding aiter backend by @yichiche in https://github.com/sgl-project/sglang/pull/11087
[Auto Sync] Update fused_moe_triton_config.py (20250930) by @merrymercy in https://github.com/sgl-project/sglang/pull/11099
chore: bump sgl-kernel v0.3.14 by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11067
[router][grpc-server] Fix gRPC server shutdown by @slin1237 in https://github.com/sgl-project/sglang/pull/11094
Fix eagle radix cache by @ispobock in https://github.com/sgl-project/sglang/pull/10846
[Eval] Add --repeat in run_eval by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11101
[CPU] Adding Memory Capacity Acquisition Functionality by @ZailiWang in https://github.com/sgl-project/sglang/pull/11102
Fix DSR1 accuracy for flashinfer_trtllm MoE with FP8 quantization by @trevor-m in https://github.com/sgl-project/sglang/pull/11081
Support Dots.ocr model by @albaNnaksqr in https://github.com/sgl-project/sglang/pull/11071
[router][bugfix] Fix input_logprobs handling with None value and logprob_start_len = -1 by @CatherineSue in https://github.com/sgl-project/sglang/pull/11113
Feature/make PEFT adapter module format compatibile by @ConnorLi96 in https://github.com/sgl-project/sglang/pull/11080
fix: KimiK2Detector Improve tool call ID parsing with regex by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10972
[router] add mcp list and mcp call in output array by @key4ng in https://github.com/sgl-project/sglang/pull/11112
Organize spec-related data structures by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10735
[AMD] Add Tilelang and Fast Hadamard Transform builds to Dockerfile.rocm by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/11114
[Auto Sync] Update base_grammar_backend.py, xgrammar_backen... (20250930) by @merrymercy in https://github.com/sgl-project/sglang/pull/11115
[Doc] Update multimodal language models documentation by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/11111
Quick Fix: fix Qwen3-VL launch failure caused by MRotaryEmbedding arg by @yhyang201 in https://github.com/sgl-project/sglang/pull/10985
docker: x86 dev builds for hopper and blackwell by @ishandhanani in https://github.com/sgl-project/sglang/pull/11075
Refactor AMD CI. by @saienduri in https://github.com/sgl-project/sglang/pull/11128
feat: add fast_decode_plan from flashinfer, flashinfer to 0.4.0rc3 by @yyihuang in https://github.com/sgl-project/sglang/pull/10760
[HiCache]bug fix: fixed blank item in host_mem_release_queue by @zhangzuo21 in https://github.com/sgl-project/sglang/pull/11005
[Feature] Add EIC as sglang HiCache Storage backend by @mss1213 in https://github.com/sgl-project/sglang/pull/10271
[HiCache] Configurable and Dynamic Prefetch Timeout by @ykwd in https://github.com/sgl-project/sglang/pull/10512
[router] add pd service in grpc router for pd by @slin1237 in https://github.com/sgl-project/sglang/pull/11120
[router] Add multi-turn tool calling loop support for MCP integration by @key4ng in https://github.com/sgl-project/sglang/pull/11143
Fix metrics and request tracing (TimeStats) by @merrymercy in https://github.com/sgl-project/sglang/pull/11123
Remove debug print statement from scheduler output by @merrymercy in https://github.com/sgl-project/sglang/pull/11145
Intoduce cpu tensor as metadata to avoid blocking gpu kernel launch by @AHEADer in https://github.com/sgl-project/sglang/pull/10720
Fix ngram spec with page size > 1 by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11135
[ROCm] To reduce the compiling time when using torch compile. by @sogalin in https://github.com/sgl-project/sglang/pull/10559
Fix DeepSeek chunked prefill memory issue by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11149
Clean up parallel_state.py by @merrymercy in https://github.com/sgl-project/sglang/pull/11148
Tiny improve dumper by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11132
Tiny fix missing alt stream in nextn layer by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10768
Fuse quantize and rope in trtllm_mla MTP by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10779
Tiny detect slow ranks by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10508
Remove unused pack .item() in paged allocator. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11156
Support dispatch low latency by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10263
Support single batch overlap by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10422
[router][grpc] Support tool call parser in streaming by @CatherineSue in https://github.com/sgl-project/sglang/pull/11160
[model] Add mamba2 and Falcon-H1 support. by @ilyasch2 in https://github.com/sgl-project/sglang/pull/10988
Clean up ascend allocator by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11152
fix cpp JIT compilation issue of ngram speculative decoding by @b8zhong in https://github.com/sgl-project/sglang/pull/10837
Tiny cleanup deepseek_v2.py by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11163
Tiny fix ep_gather behavior different in CI by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11130
Tiny remove duplicated code by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11164
[proto] Add script to compile python protos by @CatherineSue in https://github.com/sgl-project/sglang/pull/11171
Unify forward output datastructure by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11124
[grpc] style fix for grpc compilation. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11175
Remove dp balance metadata and minimul token balance. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11170
Minor fixes for server_args, parallel_state, and test_deterministic.py by @merrymercy in https://github.com/sgl-project/sglang/pull/11159
fix: shoudn't include CUDA_ARCH 100 and 120 for cuda12.6.1 by @gongwei-130 in https://github.com/sgl-project/sglang/pull/11176
[router][grpc] Support streaming for v1/chat/completions by @CatherineSue in https://github.com/sgl-project/sglang/pull/11179
Allow use of TRTLLM_MHA backend for hybrid attention on Blackwell by @DomBrown in https://github.com/sgl-project/sglang/pull/11138
Introduce naming convention in io_struct and base sglang io classes. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10133
[Generative Scores API] add performance tests to CICD by @vedantjh2 in https://github.com/sgl-project/sglang/pull/10830
[1/n] Enable DCA CUDA graph capture by @b8zhong in https://github.com/sgl-project/sglang/pull/9537
[Fix] Update to v0.1.5.post4 and refine HIP attention backend selection by @yichiche in https://github.com/sgl-project/sglang/pull/11161
[CI]] Tee server logs to both file and stdout/stderr using PIPE by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11185
fix: radix cache memory accounting by @skyzh in https://github.com/sgl-project/sglang/pull/10637
Tiny add PD disaggregation + DP attention test by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11167
[router] Steaming support for MCP Tool Calls in OpenAI Router by @key4ng in https://github.com/sgl-project/sglang/pull/11173
[Feature] Option to save model weights to CPU when memory saver mode is enabled by @mattnappo in https://github.com/sgl-project/sglang/pull/10873
Add --thinking-mode to run_eval by @hlu1 in https://github.com/sgl-project/sglang/pull/11189
[hot-fix] Fix CI break which caused by adding thinking_mode in eval by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11192
Tiny move files to utils folder by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11166
Fix CUDA illegal memory access issues in speculative decoding by @ur4t in https://github.com/sgl-project/sglang/pull/10892
Fix [test]: Env:SGLANG_TORCH_PROFILER_DIR for pytest. by @singhalshubham03 in https://github.com/sgl-project/sglang/pull/10780
Optimize debug log position of PD abort request by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11090
fix 3fs indices by @pansicheng in https://github.com/sgl-project/sglang/pull/10855
model: support starcoder2 by @ppraneth in https://github.com/sgl-project/sglang/pull/10609
[Test] Initialize mem_fraction_static in setUpClass to fix pytest VLM test crashes. by @vshekhawat-hlab in https://github.com/sgl-project/sglang/pull/10859
fix xeon ci check by @DiweiSun in https://github.com/sgl-project/sglang/pull/10838
fix qwen2 eagle3 runtime error by @jiapingW in https://github.com/sgl-project/sglang/pull/10517
[minor] fix the lint by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11198
[Fix] Fix the bug of the calculation of base_gpu_id (dp offset) in data_parallel_controller.py by @XSongQ in https://github.com/sgl-project/sglang/pull/10741
[fix]missing prefix_lens_cpu init when p/d disaggregation by @HanHan009527 in https://github.com/sgl-project/sglang/pull/11196
fix self.enable_kv_cache_events by @narutolhy in https://github.com/sgl-project/sglang/pull/11178
[HICache]: Refactor HiCache CI by @hzh0425 in https://github.com/sgl-project/sglang/pull/11011
fix sampling_seed handling when deterministic is enabled by @skyzh in https://github.com/sgl-project/sglang/pull/11096
[fix]enable flashmla when using draft model P/D attention select by @HanHan009527 in https://github.com/sgl-project/sglang/pull/11012
[router] fix get load response parsing by @slin1237 in https://github.com/sgl-project/sglang/pull/11213
[router] add grpc router pd mode for chat and generate by @slin1237 in https://github.com/sgl-project/sglang/pull/11140
EAGLE cache fix for HiCache by @ispobock in https://github.com/sgl-project/sglang/pull/11215
Add --max-new-tokens CLI flag for MMMU evaluation by @yhyang201 in https://github.com/sgl-project/sglang/pull/11217
Add DeepSeek-V3.2 Tool Call Template by @Xu-Wenqing in https://github.com/sgl-project/sglang/pull/11063
Tiny skip_sample adjust by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11225
[Feature] Add a fast-topk to sgl-kernel for DeepSeek v3.2 by @DarkSharpness in https://github.com/sgl-project/sglang/pull/11194
Update v1/responses to be more OpenAI-compatible. by @vincentzed in https://github.com/sgl-project/sglang/pull/9624
chore: bump sgl-kernel v0.3.14.post1 by @FlamingoPg in https://github.com/sgl-project/sglang/pull/11137
Update DeepGEMM repository tag to specific commit by @merrymercy in https://github.com/sgl-project/sglang/pull/11229
[Feat] Support Torch Symm Mem AllReduce by @yuan-luo in https://github.com/sgl-project/sglang/pull/10571
Refactor and optimize mooncake CI by @ShangmingCai in https://github.com/sgl-project/sglang/pull/11162
[Fix AMD CI] VRAM cleanup by @sunxxuns in https://github.com/sgl-project/sglang/pull/11174
Update transformers package version to 4.57.0 by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/11222
Remove gdrcopy check in ci_install_deepep.sh by @ch-wan in https://github.com/sgl-project/sglang/pull/11237
Rename runner labels by @merrymercy in https://github.com/sgl-project/sglang/pull/11228
[Auto Sync] Update io_struct.py (20251004) by @merrymercy in https://github.com/sgl-project/sglang/pull/11206
Create two new GH workflows to automatically bump SGLang and Kernel version by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/10996
Fix spec_utils.py by @sglang-bot in https://github.com/sgl-project/sglang/pull/11247
ci: make find_local_hf_snapshot_dir more robust by @mickqian in https://github.com/sgl-project/sglang/pull/11248
[quantization] Fix scale remapping for mllama4 by @BowenBao in https://github.com/sgl-project/sglang/pull/10042
[quantization] Enable aiter mxfp4 fused_moe for Quark by @BowenBao in https://github.com/sgl-project/sglang/pull/10048
Use cu128 for torch audio to fix some CI tests by @merrymercy in https://github.com/sgl-project/sglang/pull/11251
Bump torch_memory_saver 0.0.9rc2 by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11252
update sgl kernel version to 0.3.14.post1 by @merrymercy in https://github.com/sgl-project/sglang/pull/11242
Update condition for sgl-kernel-benchmark-test by @merrymercy in https://github.com/sgl-project/sglang/pull/11254
feat: add shortcut detection for multimodal templates in Jinja format by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/11209
Improve bot release workflow by @Kangyan-Zhou in https://github.com/sgl-project/sglang/pull/11240
Add flashmla and fast hadamard transform to Dockerfile by @Fridge003 in https://github.com/sgl-project/sglang/pull/11235
Support DeepSeek V3.2 Exp by @fzyzcjy in https://github.com/sgl-project/sglang/pull/11061
chore: bump SGLang version to 0.5.3rc2 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11259
chore: bump SGLang version to 0.5.3 by @sglang-bot in https://github.com/sgl-project/sglang/pull/11263

New Contributors

@chenqianfzh made their first contribution in https://github.com/sgl-project/sglang/pull/10356
@yonghenglh6 made their first contribution in https://github.com/sgl-project/sglang/pull/8778
@amysaq2023 made their first contribution in https://github.com/sgl-project/sglang/pull/8215
@lj970926 made their first contribution in https://github.com/sgl-project/sglang/pull/10410
@thalahors made their first contribution in https://github.com/sgl-project/sglang/pull/10076
@sufeng-buaa made their first contribution in https://github.com/sgl-project/sglang/pull/9962
@tao12345666333 made their first contribution in https://github.com/sgl-project/sglang/pull/10129
@ooapex made their first contribution in https://github.com/sgl-project/sglang/pull/10401
@Jimmy-L99 made their first contribution in https://github.com/sgl-project/sglang/pull/10439
@1195343015 made their first contribution in https://github.com/sgl-project/sglang/pull/10471
@csahithi made their first contribution in https://github.com/sgl-project/sglang/pull/9887
@scut-cbq made their first contribution in https://github.com/sgl-project/sglang/pull/8863
@brayden-hai made their first contribution in https://github.com/sgl-project/sglang/pull/10336
@PrinsYin made their first contribution in https://github.com/sgl-project/sglang/pull/10230
@philipkiely-baseten made their first contribution in https://github.com/sgl-project/sglang/pull/10528
@zhannngchen made their first contribution in https://github.com/sgl-project/sglang/pull/10523
@alpha-baby made their first contribution in https://github.com/sgl-project/sglang/pull/8274
@yukiy927 made their first contribution in https://github.com/sgl-project/sglang/pull/9947
@a4zhangfei made their first contribution in https://github.com/sgl-project/sglang/pull/9873
@leihuang-sketch made their first contribution in https://github.com/sgl-project/sglang/pull/10319
@fgebhart made their first contribution in https://github.com/sgl-project/sglang/pull/10661
@dmitrygx made their first contribution in https://github.com/sgl-project/sglang/pull/10673
@sglang-bot made their first contribution in https://github.com/sgl-project/sglang/pull/10706
@yushengsu-thu made their first contribution in https://github.com/sgl-project/sglang/pull/10694
@reyoung made their first contribution in https://github.com/sgl-project/sglang/pull/10731
@feng397 made their first contribution in https://github.com/sgl-project/sglang/pull/10730
@LorrinWWW made their first contribution in https://github.com/sgl-project/sglang/pull/10758
@vedantjh2 made their first contribution in https://github.com/sgl-project/sglang/pull/10755
@zju-stu-lizheng made their first contribution in https://github.com/sgl-project/sglang/pull/10323
@ZhengHSI made their first contribution in https://github.com/sgl-project/sglang/pull/10504
@GuoweiWangU made their first contribution in https://github.com/sgl-project/sglang/pull/10860
@lun-4 made their first contribution in https://github.com/sgl-project/sglang/pull/10827
@eraser00 made their first contribution in https://github.com/sgl-project/sglang/pull/10612
@TJ5 made their first contribution in https://github.com/sgl-project/sglang/pull/10550
@wejoncy made their first contribution in https://github.com/sgl-project/sglang/pull/10964
@lbk-sys made their first contribution in https://github.com/sgl-project/sglang/pull/10555
@Kangyan-Zhou made their first contribution in https://github.com/sgl-project/sglang/pull/10701
@samuellees made their first contribution in https://github.com/sgl-project/sglang/pull/10816
@albaNnaksqr made their first contribution in https://github.com/sgl-project/sglang/pull/11071
@ConnorLi96 made their first contribution in https://github.com/sgl-project/sglang/pull/11080
@zhangzuo21 made their first contribution in https://github.com/sgl-project/sglang/pull/11005
@AHEADer made their first contribution in https://github.com/sgl-project/sglang/pull/10720
@ilyasch2 made their first contribution in https://github.com/sgl-project/sglang/pull/10988
@DomBrown made their first contribution in https://github.com/sgl-project/sglang/pull/11138
@skyzh made their first contribution in https://github.com/sgl-project/sglang/pull/10637
@mattnappo made their first contribution in https://github.com/sgl-project/sglang/pull/10873
@ur4t made their first contribution in https://github.com/sgl-project/sglang/pull/10892
@singhalshubham03 made their first contribution in https://github.com/sgl-project/sglang/pull/10780
@XSongQ made their first contribution in https://github.com/sgl-project/sglang/pull/10741
@sunxxuns made their first contribution in https://github.com/sgl-project/sglang/pull/11174
@BowenBao made their first contribution in https://github.com/sgl-project/sglang/pull/10042

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.2...v0.5.3

Open source →

0.5.211 Sept 2025Security

[Vulnerability]feat(conn): set bootstrap server host by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/9931

Highlights

SGLang HiCache: Fast Hierarchical KV Caching with Your Favorite Storage Backends: https://lmsys.org/blog/2025-09-10-sglang-hicache/

What's Changed

feat: allow use local branch to build image by @gongwei-130 in https://github.com/sgl-project/sglang/pull/9546
[readme] Include additional resources for the SGLang x AMD SF Meetup event by @wisclmy0611 in https://github.com/sgl-project/sglang/pull/9547
[doc] deepseekv31 support by @XiaotongJiang in https://github.com/sgl-project/sglang/pull/9544
fix(grok): remove duplicate replicate_lm_head configuration by @vincentzed in https://github.com/sgl-project/sglang/pull/9549
chore: update configurer by @zhyncs in https://github.com/sgl-project/sglang/pull/9557
chore: bump v0.5.1.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/9558
[router] add right rustls dependency in sgl-router cargo.toml by @Bruce-x-1997 in https://github.com/sgl-project/sglang/pull/9498
fix: use sgl-kernel 0.3.5 by @zhyncs in https://github.com/sgl-project/sglang/pull/9565
Add target module validation for init adapters by @Beichen-Ma in https://github.com/sgl-project/sglang/pull/9429
fix: Update OpenAI client base URL in documentation by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9576
[PD] Improve disaggregation metrics output: update the metrics to keep reflecting real stats by @SCDESPERTATE in https://github.com/sgl-project/sglang/pull/7317
remove redundant rank0_log function. by @miter6 in https://github.com/sgl-project/sglang/pull/9560
Update CUTLASS 4.2 & Enable K-Major Scale Factor for SM90 FP8 Blockwise Group GEMM by @HydraQYH in https://github.com/sgl-project/sglang/pull/9559
Reintroduce memory usage fix by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9535
Offload tensors by sharding on GPU by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9536
bugfix for undefined logging functions in HarmonyBrowserTool & HarmonyPythonTool by @CiaranZhou in https://github.com/sgl-project/sglang/pull/9229
chore: upgrade flashinfer 0.2.14.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/9578
fix: revert #8593 by @zhyncs in https://github.com/sgl-project/sglang/pull/9581
fix: resolve tuning fused moe issue by @zhyncs in https://github.com/sgl-project/sglang/pull/9587
Tiny fix wrong comments by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9589
chore: update config by @zhyncs in https://github.com/sgl-project/sglang/pull/9591
chore: bump v0.5.1.post2 by @zhyncs in https://github.com/sgl-project/sglang/pull/9592
[Doc] add LWS(LeaderWorkerSet) use case in sgl-router README by @Bruce-x-1997 in https://github.com/sgl-project/sglang/pull/9568
[Performance] Batch Send from Tokenizer Manager. by @sundar24295s in https://github.com/sgl-project/sglang/pull/9436
Fix GLM45 tool call multi-turn bug by @byjiang1996 in https://github.com/sgl-project/sglang/pull/9500
Fix GLM45v launch server cuda torch compile bug by @byjiang1996 in https://github.com/sgl-project/sglang/pull/9554
Fix Harmony reasoning parser for and auto-separation for gpt-oss models by @jonaslsaa in https://github.com/sgl-project/sglang/pull/9190
[docs] Refactor, remove compiled results and add gpt-oss by @zhaochenyang20 in https://github.com/sgl-project/sglang/pull/9613
[Fix] HiCache Bugfix & Mooncake Error Handling Enhance by @ykwd in https://github.com/sgl-project/sglang/pull/8901
Improve bench_one_batch_server script by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9608
[router] add mistral tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9622
[router] add qwen tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9623
[router] add pythonic parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9628
[router] add llama tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9629
[router] add ut for mistral, llama, pythonic, and streaming tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9632
[new feat] ascend backend support fia fusion kernel by @ZhengdQin in https://github.com/sgl-project/sglang/pull/8328
model: Support nvidia/Llama-3_1-Nemotron-Ultra-253B-v1 by @netanel-haber in https://github.com/sgl-project/sglang/pull/9301
Fix lint for router by @hebiao064 in https://github.com/sgl-project/sglang/pull/9636
[docs] Update README with additional highlights and resources for SGLang x AMD SF Meetup by @wisclmy0611 in https://github.com/sgl-project/sglang/pull/9640
Add reasoning_effort param in TiktokenTokenizer.apply_chat_template by @lshmouse in https://github.com/sgl-project/sglang/pull/9630
fix: allow user to specify function as role by @GavinZhu-GMI in https://github.com/sgl-project/sglang/pull/9635
Fix kimi k2 function calling format by @XiaotongJiang in https://github.com/sgl-project/sglang/pull/9606
[router] address worker load tracking consistency by @slin1237 in https://github.com/sgl-project/sglang/pull/9523
[router] add token bucket rate limiter by @CatherineSue in https://github.com/sgl-project/sglang/pull/9656
[doc] add kimik2 --tool-call-parser by @XiaotongJiang in https://github.com/sgl-project/sglang/pull/9647
Install py-spy by default for containers for easier debugging by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9649
BugFix(hicache): Fix host indices out of bound error by @hzh0425 in https://github.com/sgl-project/sglang/pull/9637
HiCache Storage fix host memory leak by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9648
add response_format support for completion API by @cicirori in https://github.com/sgl-project/sglang/pull/9665
Fix FA3 swa spec verify topk>1 by @ispobock in https://github.com/sgl-project/sglang/pull/9658
[RL] fix register the same ops multiple times by @hebiao064 in https://github.com/sgl-project/sglang/pull/9564
chore: enhance bench_serving for vlms with a new dataset of configurable image count and resolution by @mickqian in https://github.com/sgl-project/sglang/pull/9583
refactor(hicache): Introduce generic HiCacheStorageConfig for improved configuration management by @hzh0425 in https://github.com/sgl-project/sglang/pull/9555
feat: (chat-template matching) enhance multimodal model detection with config.json by @KEVINTUAN12 in https://github.com/sgl-project/sglang/pull/9597
[docs] Instructions for bench_serving.py by @yhyang201 in https://github.com/sgl-project/sglang/pull/9071
Support DeepSeek-V3.1 tool call by @Xu-Wenqing in https://github.com/sgl-project/sglang/pull/9446
Add A100 fused MoE kernel configs for Dpsk by @ehuaa in https://github.com/sgl-project/sglang/pull/9677
support cuda 13.0 and trtllm kernel by @rainj-me in https://github.com/sgl-project/sglang/pull/9495
fix: HiRadixCache: fix prefetch completion race by @pabloiyu in https://github.com/sgl-project/sglang/pull/9397
fix mooncake store mla zero copy meta by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/9678
move is_sm90_supported/is_sm100_supported to python/sglang/srt/utils.py by @merrymercy in https://github.com/sgl-project/sglang/pull/9679
[router] restructure tool parser module folder by @slin1237 in https://github.com/sgl-project/sglang/pull/9693
[router] add deepseek tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9694
Quick fix for loading processor for supporting internvl3_5 series by @yilian49 in https://github.com/sgl-project/sglang/pull/9676
Fix get_ip when no external network by @whybeyoung in https://github.com/sgl-project/sglang/pull/9700
Sets default model name in request classes by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9683
[router] add step3 tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9695
[router] add kimi-k2 tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9702
[router] add gpt-oss and glm4 tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9703
[sgl-kernel] misc: update deepgemm version for sgl-kernel by @FlamingoPg in https://github.com/sgl-project/sglang/pull/9340
chore: upgrade sgl-kernel 0.3.7 by @zhyncs in https://github.com/sgl-project/sglang/pull/9708
chore: bump v0.5.1.post3 by @zhyncs in https://github.com/sgl-project/sglang/pull/9716
[router] upgrade kernel version in pd ci by @CatherineSue in https://github.com/sgl-project/sglang/pull/9720
[Sync] Update mxfp4.py (20250827) by @merrymercy in https://github.com/sgl-project/sglang/pull/9724
[router] fix error response in pd_router by @Bruce-x-1997 in https://github.com/sgl-project/sglang/pull/9505
[router] Add MCP Tool Handler by @key4ng in https://github.com/sgl-project/sglang/pull/9615
gpt-oss blog reproduction document by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9728
[router] additional pythonic parser unit test by @slin1237 in https://github.com/sgl-project/sglang/pull/9730
[router] additional llama32 parser unit test and multi json support by @slin1237 in https://github.com/sgl-project/sglang/pull/9732
support mooncake store dp attention by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/9684
add support for nvidia/gpt-oss-120b-Eagle3 by @zyksir in https://github.com/sgl-project/sglang/pull/9739
Move git clone command up from README by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9740
[feat] Reduce GPU memory overhead by using weakref by @yhyang201 in https://github.com/sgl-project/sglang/pull/9673
Support speculative decoding in hybrid attention backend by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/9573
[router] add llama3.2 multi json streaming parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9735
Support compile sgl-kernel on cuda 13.0 by @rainj-me in https://github.com/sgl-project/sglang/pull/9721
[Sync] Update server_args.py (20250828) by @merrymercy in https://github.com/sgl-project/sglang/pull/9745
[router] grpc router bootstraps by @slin1237 in https://github.com/sgl-project/sglang/pull/9759
[AMD] Support Hierarchical Caching on AMD GPUs by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/8236
feat: add tuned fused moe config for GLM-4.5-Air-FP8 tp = 4 on B200 by @zixuanzhang226 in https://github.com/sgl-project/sglang/pull/9770
[Feature] Support NPUGraph for DeepSeek on Ascend NPU by @chenxu140 in https://github.com/sgl-project/sglang/pull/9355
feat(draft_model): support draft_model for RemoteModelLoader by @DellCurry in https://github.com/sgl-project/sglang/pull/6407
fix: fix MLA for ShardedModelLoader/RemoteModelLoader by @DellCurry in https://github.com/sgl-project/sglang/pull/6287
Optimize prefill performance on cpu backend by @mingfeima in https://github.com/sgl-project/sglang/pull/8750
[HiCache] change the default policy to write through by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9772
bugfix(hicache): Move exists check before key suffixing by @hzh0425 in https://github.com/sgl-project/sglang/pull/9749
Skip some tests on Blackwell by @hlu1 in https://github.com/sgl-project/sglang/pull/9777
Raise error when topk>1 and page>1 for paged attention backends. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9784
ROCm 7.0 update by @sogalin in https://github.com/sgl-project/sglang/pull/9757
add bench_mix.py by @pansicheng in https://github.com/sgl-project/sglang/pull/9788
Make sm100 fp8 kernels available on sm103 by @hlu1 in https://github.com/sgl-project/sglang/pull/9789
accomendate json schema in the "schema" field, not in "json_schema" field of response_format by @gongwei-130 in https://github.com/sgl-project/sglang/pull/9786
[PD] Support get_model_info interface for mini_lb by @XucSh in https://github.com/sgl-project/sglang/pull/9792
[HiCache] resolve conflict between chunked-prefill and hicache hit count by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9776
feat(hicache-3fs): 3FS-Store Backup Optimizations For MLA Model. by @hzh0425 in https://github.com/sgl-project/sglang/pull/9692
support enable in the reasoning field to enable thingking for thinkin… by @gongwei-130 in https://github.com/sgl-project/sglang/pull/9715
feat: Add flexible validation for partial weight updates by @GeLee-Q in https://github.com/sgl-project/sglang/pull/9663
feat: add original logprobs to response by @narutolhy in https://github.com/sgl-project/sglang/pull/8375
[feat] Support EAGLE3 for Qwen2 by @KerwinKai in https://github.com/sgl-project/sglang/pull/9216
chore: upgrade flashinfer 0.3.0rc1 by @zhyncs in https://github.com/sgl-project/sglang/pull/9793
[ModelOpt] Fix Weight Loading for DSR1-FP4 Quantization by @pavanimajety in https://github.com/sgl-project/sglang/pull/9712
Fix TRTLLM MLA Cuda KV Blocks Causing accuracy drop by @farazkh80 in https://github.com/sgl-project/sglang/pull/9675
[NVIDIA] [2/N] Optimize silu_and_mul_scaled_fp4_grouped_quant perf by @kaixih in https://github.com/sgl-project/sglang/pull/9556
Adds initialize_moe_config to bench_one_batch so MOE backend is respected by @pranavm-nvidia in https://github.com/sgl-project/sglang/pull/9670
Small bug fix in transformers model implementation by @yilian49 in https://github.com/sgl-project/sglang/pull/9809
feature(eplb): add min-rebalancing-utilization-threshold for eplb by @hzh0425 in https://github.com/sgl-project/sglang/pull/8345
Make fp4_quantize kernels work on sm103 by @hlu1 in https://github.com/sgl-project/sglang/pull/9807
fix: dsv3 lite q_lora_rank none by @zhyncs in https://github.com/sgl-project/sglang/pull/9815
Fix memory leak when aborting decode request in PD-Disagg by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9817
chore: fix cuda driver api issue and bump sgl-kernel 0.3.7.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/9746
chore: update Dockerfile by @zhyncs in https://github.com/sgl-project/sglang/pull/9820
Fix typo in warning message about DeepGEMM JIT by @mmangkad in https://github.com/sgl-project/sglang/pull/9802
chore: upgrade sgl-kernel 0.3.7.post1 with deepgemm fix by @zhyncs in https://github.com/sgl-project/sglang/pull/9822
[sgl-kernel] fix: fix missing FetchContent_Populate for fmt by @FlamingoPg in https://github.com/sgl-project/sglang/pull/9826
chore: upgrade transformers 4.56.0 by @zhyncs in https://github.com/sgl-project/sglang/pull/9827
[Auto Sync] Update parallel_state.py (20250830) by @merrymercy in https://github.com/sgl-project/sglang/pull/9828
[CI] Fix the trigger condition for PR test workflows by @merrymercy in https://github.com/sgl-project/sglang/pull/9761
[CI] Code sync tools by @merrymercy in https://github.com/sgl-project/sglang/pull/9830
Update guidelines for syncing code between repos by @merrymercy in https://github.com/sgl-project/sglang/pull/9831
hot fix for mooncake batch set api by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9836
[router] add reasoning parser readme by @slin1237 in https://github.com/sgl-project/sglang/pull/9837
Tool parser.benchmark by @CatherineSue in https://github.com/sgl-project/sglang/pull/9835
[Model] Support Meituan LongCat-Flash && LongCat-Flash-MTP by @Orchard-DT in https://github.com/sgl-project/sglang/pull/9824
[router] global tool parser registry by @CatherineSue in https://github.com/sgl-project/sglang/pull/9840
[feat]Ascend NPU Gemma-3-12b and Gemma-3-27b support by @VDV1985 in https://github.com/sgl-project/sglang/pull/8909
[Performance] Improve Qwen RMSNorm by replacing with native RMSNorm op by @vincentzed in https://github.com/sgl-project/sglang/pull/9709
[HiCache] Clear kvcache in storage backend with fastAPI by @stmatengss in https://github.com/sgl-project/sglang/pull/9750
Fix input logprob index for a batch that includes both requests with input logprob and requests with input logprob. by @merrymercy in https://github.com/sgl-project/sglang/pull/9841
Fuse gate_proj and up_proj in Qwen 2.5 VL's vision MLP by @AlienKevin in https://github.com/sgl-project/sglang/pull/9661
[HiCache] Storage Refactoring by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9797
fix set_interal_state API by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9850
fix inconsistent arguments for generated shared prefix bench by @pbkowalski in https://github.com/sgl-project/sglang/pull/9073
fix(hicahce-long-bench): adjust context workload generator to use full query set by @hzh0425 in https://github.com/sgl-project/sglang/pull/9847
Disable radix cache in test_lora_update.py for better stability by @Fridge003 in https://github.com/sgl-project/sglang/pull/9852
Tiny allow DeepGEMM on cu12.9 by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9858
Update docker build workflows for gfx942 ROCm 7.0. by @saienduri in https://github.com/sgl-project/sglang/pull/9794
Support Multi Process Tokenizer Manager(#6555) by @whybeyoung in https://github.com/sgl-project/sglang/pull/8964
chore: upgrade flashinfer 0.3.0 by @zhyncs in https://github.com/sgl-project/sglang/pull/9864
chore: bump v0.5.2rc0 by @zhyncs in https://github.com/sgl-project/sglang/pull/9862
Mooncake store get zero copy meta optimization by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/9857
[router] add tokenizer download support from hf hub by @CatherineSue in https://github.com/sgl-project/sglang/pull/9882
support fp8 kvcache for hybrid attn backend on GPT-OSS by @rainj-me in https://github.com/sgl-project/sglang/pull/9783
[HiCacheStorage] fix abort request host memory leaks by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/9874
[HiCacheStorage]: Improve 3fs kvstore‘s performance and resolve mla issues by @hzh0425 in https://github.com/sgl-project/sglang/pull/9876
[router] Fix short timeout for the prefill client by @LukasBluebaum in https://github.com/sgl-project/sglang/pull/9803
[code style] restruct fused_moe to avoid very long single file by @BBuf in https://github.com/sgl-project/sglang/pull/9878
[router] add grpc pd and regular router init by @CatherineSue in https://github.com/sgl-project/sglang/pull/9893
[router] fix FunctionCallResponse proto, support arguments is null by @Bruce-x-1997 in https://github.com/sgl-project/sglang/pull/9875
[feat] Support tp mode for DeepSeek-R1-W4AFP8 by @chenxijun1029 in https://github.com/sgl-project/sglang/pull/8118
Move multi-tokenizer event loop to better place by @ShangmingCai in https://github.com/sgl-project/sglang/pull/9902
[chore] fix dead links in doc by @lifuhuang in https://github.com/sgl-project/sglang/pull/9913
Change tensor alignment method to mn major by @mmangkad in https://github.com/sgl-project/sglang/pull/9844
chore: bump v0.3.8 sgl-kernel by @zhyncs in https://github.com/sgl-project/sglang/pull/9907
[Fix] fix the issue encountered when inference LongCat-Flash/MTP EP MoE on b200 by @Orchard-DT in https://github.com/sgl-project/sglang/pull/9916
fix parallel_state.py current_platform bug by @BBuf in https://github.com/sgl-project/sglang/pull/9919
[feat] apply deep_gemm compile_mode to skip launch by @Alcanderian in https://github.com/sgl-project/sglang/pull/9879
fix: update router deps by @zhyncs in https://github.com/sgl-project/sglang/pull/9921
chore: bump v0.5.2rc1 by @zhyncs in https://github.com/sgl-project/sglang/pull/9920
[Hicache] Generic page get bugfix by @ykwd in https://github.com/sgl-project/sglang/pull/9909
Support the internvl3.5 family models in sglang by @yilian49 in https://github.com/sgl-project/sglang/pull/9705
[router] include rust benchamrks by @slin1237 in https://github.com/sgl-project/sglang/pull/9932
Fix the key passing issue in page first layout. by @hzh0425 in https://github.com/sgl-project/sglang/pull/9929
[router] fix grpc client url normalzation and health check by @CatherineSue in https://github.com/sgl-project/sglang/pull/9939
[model] support MiniCPM-V 4.0 by @tc-mb in https://github.com/sgl-project/sglang/pull/8747
[HiCache] Minor fix on file storage backend by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9869
Move parsers under a single folder by @merrymercy in https://github.com/sgl-project/sglang/pull/9912
[Fix] DeepSeek EP accuracy issue on B200 GPUs by @alhridoy in https://github.com/sgl-project/sglang/pull/9946
fix(cache): move ongoing_prefetch pop after validation to prevent leak by @xiaguan in https://github.com/sgl-project/sglang/pull/9927
Remove annoying warnings in sgl kernel build by @merrymercy in https://github.com/sgl-project/sglang/pull/9905
Update tool_chat_template_deepseekv31.jinja by @WangJianQ-0118 in https://github.com/sgl-project/sglang/pull/9895
Qwen FP8/NVFP4 ModelOPT Quantization support by @jingyu-ml in https://github.com/sgl-project/sglang/pull/7912
Optimized deepseek-v3/r1 model performance on mxfp4 run by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/9671
add proctitle for tokenizers by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9952
[feat] Add P/D attention select for draft model by @Ximingwang-09 in https://github.com/sgl-project/sglang/pull/9755
Revert "[Fix] DeepSeek EP accuracy issue on B200 GPUs (#9946)" by @zhyncs in https://github.com/sgl-project/sglang/pull/9955
Revert "Optimized deepseek-v3/r1 model performance on mxfp4 run (#9671)" by @zhyncs in https://github.com/sgl-project/sglang/pull/9959
[benchmark] add flashinfer_allreduce_fusion benchmark by @BBuf in https://github.com/sgl-project/sglang/pull/9937
[1/N][Bug] Fix w4afp8 MoE NaN issue (sgl-kernel) by @yuhyao in https://github.com/sgl-project/sglang/pull/9953
[router] Add Rerank API Specification by @fangjian601 in https://github.com/sgl-project/sglang/pull/9906
[router] add chat_template_kwargs in ChatCompletionRequest by @tonyluj in https://github.com/sgl-project/sglang/pull/9958
Remove mrope position sync by @timmy-feng in https://github.com/sgl-project/sglang/pull/9460
fix swa clear(): rename is_in_free_group to is_not_in_free_group by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9914
Triton 3.4.0 MoE config for Deepseek TP16 H100 by @SzymonOzog in https://github.com/sgl-project/sglang/pull/9978
nsys profile output kernel classifier by @gracehonv in https://github.com/sgl-project/sglang/pull/9314
Minor update regarding issue #9704 by @elfiegg in https://github.com/sgl-project/sglang/pull/9733
[Auto Sync] Update parallel_state.py, few_shot_gsm8k.py (20250903) by @merrymercy in https://github.com/sgl-project/sglang/pull/9986
feat: add gpt oss b200 ci by @zhyncs in https://github.com/sgl-project/sglang/pull/9988
[router] move tokenizer, reasoning, tool initialization to server by @slin1237 in https://github.com/sgl-project/sglang/pull/9996
[router] clean up dependency injector to use ctx by @slin1237 in https://github.com/sgl-project/sglang/pull/10000
[router] fix grpc connection mode detection by @slin1237 in https://github.com/sgl-project/sglang/pull/9999
[Fix] gpt-oss mxfp4 model run failed on ROCm platform by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/9994
Fix Llama 4 with MXFP4 dynamic quant on MI35x by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/9993
[Bugfix] fix pd chat completion protocol for batching support by @tonyluj in https://github.com/sgl-project/sglang/pull/10016
fix: health_generate endpoint in mini_lb by @wxsms in https://github.com/sgl-project/sglang/pull/9997
[1/N] DP-refactor: move dp balance code into scheduler's mixin class by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10004
Ensure chunked request extension length respects both rem_chunk_tokens and rem_total_tokens limits by @pansicheng in https://github.com/sgl-project/sglang/pull/10003
feat(hicache): Add generic hicache ci e2e test and benchmark test by @hzh0425 in https://github.com/sgl-project/sglang/pull/9846
Optimize Qwen3-moe model by using flashinfer fused allreduce by @yuan-luo in https://github.com/sgl-project/sglang/pull/9973
[Doc] Fix SGLang tool parser doc by @PopSoda2002 in https://github.com/sgl-project/sglang/pull/9886
metrics: support customer buckets for prompt/generation_tokens_histogram by @acelyc111 in https://github.com/sgl-project/sglang/pull/9634
fix 3fs zerocopy by @pansicheng in https://github.com/sgl-project/sglang/pull/9938
Save memory for expert model parallel by @ch-wan in https://github.com/sgl-project/sglang/pull/9957
[Hicache] Mooncake API Fix & Test, and Improved Readme by @ykwd in https://github.com/sgl-project/sglang/pull/9951
Optimized deepseek-v3/r1 model performance on mxfp4 run by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/10008
Fix accuracy drop of dsv3 run in dp enablement by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/8677
chore: bump v0.5.2rc2 by @zhyncs in https://github.com/sgl-project/sglang/pull/10050
fix: update gb200 dep by @zhyncs in https://github.com/sgl-project/sglang/pull/10052
Simplify Router arguments passing and build it in docker image by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9964
[router] fix release workflow to include protobuf by @CatherineSue in https://github.com/sgl-project/sglang/pull/10055
fix MultiTokenizerWrapper name by @LLLL114 in https://github.com/sgl-project/sglang/pull/10049
Integrate trtllm ragged attention for prefill self-attention by @elfiegg in https://github.com/sgl-project/sglang/pull/9801
[Vulnerability]feat(conn): set bootstrap server host by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/9931
Fix typo in scheduler by @JamesLim-sy in https://github.com/sgl-project/sglang/pull/9934
[1/2] Optimizations and refactors about quant kernel by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9534
Tiny support setting numa nodes for different ranks by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10006
[Fix] Add speculative_draft_model_revision to server_args by @DevashishLal-CB in https://github.com/sgl-project/sglang/pull/5255
Forbid DeepEP racing condition when too many tokens by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9567
Support simple evals in text comparator by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8867
Fix and enhance dumper by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8725
Tiny let DeepGEMM scale checks cover more cases by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7182
Support copying tensor from cpu to gpu without using copy engines by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10007
[router] add py binding unit tests to coverage 80% by @key4ng in https://github.com/sgl-project/sglang/pull/10043
[router] add rust cache for rust unit test by @key4ng in https://github.com/sgl-project/sglang/pull/10079
[router] add rust cache by @slin1237 in https://github.com/sgl-project/sglang/pull/10080
enable aiter gemm_a8w8_bpreshuffle for ptpc gemm by @Yuechguo in https://github.com/sgl-project/sglang/pull/8555
[bugfix]: use correct cache location for cross attention in torch native backend by @MahmoudAshraf97 in https://github.com/sgl-project/sglang/pull/8622
Update flashinfer to 0.3.1 for B300 support by @hlu1 in https://github.com/sgl-project/sglang/pull/10087
[Bug Fix] Fix Glm4vVisionBlock norm by @sdpkjc in https://github.com/sgl-project/sglang/pull/9884
Update wave-lang to 3.7.0 and unify Wave kernel buffer options by @yichiche in https://github.com/sgl-project/sglang/pull/10069
Add storage read/write bandwidth logs to monitor kvcache performance by @pansicheng in https://github.com/sgl-project/sglang/pull/9965
[Minor] Refactors KV memory pool by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9842
support Llama4 with non uniformed intermediate size across layers for… by @gongwei-130 in https://github.com/sgl-project/sglang/pull/10047
[router] move to mcp sdk instead by @slin1237 in https://github.com/sgl-project/sglang/pull/10057
[router] Introduce router integration tests by @key4ng in https://github.com/sgl-project/sglang/pull/10086
Add lora_path argument to bench_multiturn.py by @Fridge003 in https://github.com/sgl-project/sglang/pull/10092
[HiStorage] Remove delete and clear as necessary methods by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/10039
Modify ci workflow for auto-partitioning in 2-GPU backend tests by @hzh0425 in https://github.com/sgl-project/sglang/pull/10029
Revert "[1/N][Bug] Fix w4afp8 MoE NaN issue (sgl-kernel) (#9953)" by @zhyncs in https://github.com/sgl-project/sglang/pull/10097
Fix RMSNorm API CALL mismatch issue. by @sogalin in https://github.com/sgl-project/sglang/pull/10032
fix double sparsity initialization by @shadowpa0327 in https://github.com/sgl-project/sglang/pull/6905
[Fix] illegal sync based on undefined behaviour by @DevashishLal-CB in https://github.com/sgl-project/sglang/pull/9620
[7/N] MoE Refactor: the implementation of new framework by @ch-wan in https://github.com/sgl-project/sglang/pull/9269
[NVIDIA] Remove unused get_fused_moe_impl_class function by @kaixih in https://github.com/sgl-project/sglang/pull/9764
[NVIDIA] disable chunked prefix cache when dp is used by @kaixih in https://github.com/sgl-project/sglang/pull/9861
perf: Avoid unnecessary data type conversions for DeepSeek-V3 on Blackwell by @jinyangyuan-nvidia in https://github.com/sgl-project/sglang/pull/9834
[Fix] Compatibility between DP attention and pipeline parallelism by @ch-wan in https://github.com/sgl-project/sglang/pull/10100
Fix circular import by @ch-wan in https://github.com/sgl-project/sglang/pull/10107
Disable kernel cutlass_mla_decode on SM103 by @hlu1 in https://github.com/sgl-project/sglang/pull/10058
Remove non-accelerated targets(100 and up) from cmake by @hlu1 in https://github.com/sgl-project/sglang/pull/10041
[chore] Remove unused ep_moe cuda kernels by @hlu1 in https://github.com/sgl-project/sglang/pull/9956
[CI] Refactor disaggregation tests by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10068
increase the rust e2e timeout by @key4ng in https://github.com/sgl-project/sglang/pull/10116
[router] Improve the e2e tests by @key4ng in https://github.com/sgl-project/sglang/pull/10102
[Auto Sync] Update server_args.py (20250906) by @merrymercy in https://github.com/sgl-project/sglang/pull/10117
Optimize moe_sum_reduce_kernel by @yuan-luo in https://github.com/sgl-project/sglang/pull/9477
[Feature] LMCache Connector Integration by @Oasis-Git in https://github.com/sgl-project/sglang/pull/9741
CUTLASS fp8 blockwise gemm support of sm120 by @jianyingzhu in https://github.com/sgl-project/sglang/pull/9969
Optimize nvfp4 block scaled gemm kernel when M is small. by @HydraQYH in https://github.com/sgl-project/sglang/pull/10101
Fix cuda graph mode in flashinfer attn backend by @benbarsdell in https://github.com/sgl-project/sglang/pull/10056
[HiCache] fix: check clear() method for storage backend by @stmatengss in https://github.com/sgl-project/sglang/pull/10096
add dataset_path for bench_one_batch_server.py by @miter6 in https://github.com/sgl-project/sglang/pull/10113
[Auto Sync] Update parallel_state.py (20250907) by @merrymercy in https://github.com/sgl-project/sglang/pull/10126
[Minor] fix lint in main by @DarkSharpness in https://github.com/sgl-project/sglang/pull/10128
[1/2] Refactor multi-tokenizer manager by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10074
Fix flashinfer version in sgl-kernel by @merrymercy in https://github.com/sgl-project/sglang/pull/10135
[DOC]: some minor updates by @yyihuang in https://github.com/sgl-project/sglang/pull/10134
[BUG FIX] add fail check when get fail in case wait complete block by @mss1213 in https://github.com/sgl-project/sglang/pull/9971
[MoE] fix: incorrect weight initialization for cutlass_fused_experts_fp8 by @ch-wan in https://github.com/sgl-project/sglang/pull/10144
Enables GLM4.1V server testing & fix video processing by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10095
Fix slow fused add RMSNorm by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10141
fix the fp8 topk_config.correction_bias is none bug by @rainj-me in https://github.com/sgl-project/sglang/pull/10040
Qwen2.5-VL eagle3 infer by @Lzhang-hub in https://github.com/sgl-project/sglang/pull/8801
Fix run time error in dsv3-fp8 model on mi35x by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/10104
Standalone speculative decoding by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/10090
Add graph runner support with torch compile on CPU by @CaoE in https://github.com/sgl-project/sglang/pull/7843
move compile threads to an option to avoid OOM on low memory host by @rainj-me in https://github.com/sgl-project/sglang/pull/10123
[1/N][Bug] Fix w4afp8 MoE NaN issue (sgl-kernel, fixed) by @yuhyao in https://github.com/sgl-project/sglang/pull/10108
[Bugfix] Retract not releasing enough memory when page size > 1 by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9989
Add speculator attention backend switch by @cicirori in https://github.com/sgl-project/sglang/pull/9981
Fix: (glm4v) Add missing field by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10147
[Bugfix] Qwen3MoE aclrtMemcpy failed with NPUGraph by @iforgetmyname in https://github.com/sgl-project/sglang/pull/10013
enable auto-round quantization model by @WeiweiZhang1 in https://github.com/sgl-project/sglang/pull/6226
Revert "enable auto-round quantization model (#6226)" by @zhyncs in https://github.com/sgl-project/sglang/pull/10148
enable llama3.1-8B on xpu by @huaiyuzh in https://github.com/sgl-project/sglang/pull/9434
[Bug fix] Fix Gemma 2 and fix Gemma 3 multimodal with bs > 1 on NPU by @ssshinigami in https://github.com/sgl-project/sglang/pull/9871
update xgrammar 0.1.24 and transformers 4.56.1 by @Swipe4057 in https://github.com/sgl-project/sglang/pull/10155
[2/N] DP-Refactor: move communicators into tokenizer_communicator_mixin by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10028
[Hicache]: Add E2E CI For 3FS-KVStore by @hzh0425 in https://github.com/sgl-project/sglang/pull/10131
Monkey patch uvicorn multi worker is_alive timeout by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10159
[CI] fix ambiguous argument in testing hybrid attentions. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10161
[1/2] Speed up prefill mla attention by @fzyzcjy in https://github.com/sgl-project/sglang/pull/10156
[Bug fix] Fix ascend mla in aclgraph by @alanhe151220037 in https://github.com/sgl-project/sglang/pull/9925
pref: Add H20 fp8 fused MoE kernel configs for Qwen3 by @Zhiy-Zhang in https://github.com/sgl-project/sglang/pull/10166
[fix] Relax white space rules in EBNFComposer by @LukasBluebaum in https://github.com/sgl-project/sglang/pull/9595
Revert "[ModelOpt] Fix Weight Loading for DSR1-FP4 Quantization (#9712)" by @zhyncs in https://github.com/sgl-project/sglang/pull/10176
[Bench] feat: mooncake trace integration by @stmatengss in https://github.com/sgl-project/sglang/pull/9839
fix: resolve lint issue by @zhyncs in https://github.com/sgl-project/sglang/pull/10181
fix the cutlass moe tests by @rainj-me in https://github.com/sgl-project/sglang/pull/10182
gb200: update dockerfile to latest kernel by @ishandhanani in https://github.com/sgl-project/sglang/pull/9522
Cleaning codes for speculative attention mode by @Fridge003 in https://github.com/sgl-project/sglang/pull/10149
Revert "feat: add fused moe config for Qwen3-30B-A3B on B200" by @rainj-me in https://github.com/sgl-project/sglang/pull/10185
[Fix] Orphan process in data parallel by @Capronir in https://github.com/sgl-project/sglang/pull/7995
Update link for EAGLE speculative decoding by @gerayking in https://github.com/sgl-project/sglang/pull/10191
[CPU] Fix phi4-mm prompt issue in bench_serving by @blzheng in https://github.com/sgl-project/sglang/pull/9900
Updated Nvidia Jetson docs by @shahizat in https://github.com/sgl-project/sglang/pull/4422
[3/N]DP refactor: Improve dp rank scheduling in PD disaggregation mode. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10169
Support opt model by @wenhuipeng in https://github.com/sgl-project/sglang/pull/10165
feat: use sgl-kernel cu129 as default by @zhyncs in https://github.com/sgl-project/sglang/pull/10188
[Refactor] Remove Hicache Load & Write threads by @DarkSharpness in https://github.com/sgl-project/sglang/pull/10127
Explictly export CMAKE_BUILD_PARALLEL_LEVEL by @key4ng in https://github.com/sgl-project/sglang/pull/10193
[CPU] Add gelu_and_mul kernel in sgl-kernel and add ut by @blzheng in https://github.com/sgl-project/sglang/pull/9300
feat: support fa cute in sgl-kernel by @zhyncs in https://github.com/sgl-project/sglang/pull/10205
Refactor fused_add_rmsnorm import logic by @ShangmingCai in https://github.com/sgl-project/sglang/pull/10207
tool-call(dsv3): Fixed a parse problem when there are multiple function definitions in tool_calls by @Missmiaom in https://github.com/sgl-project/sglang/pull/10209
[Auto Sync] Update sampling_batch_info.py (20250909) by @merrymercy in https://github.com/sgl-project/sglang/pull/10212
chore: bump v0.3.9 sgl-kernel by @zhyncs in https://github.com/sgl-project/sglang/pull/10208
add variable TP Decode > Prefill size support by @shaharmor98 in https://github.com/sgl-project/sglang/pull/9960
[Fix] KV-cache eviction mismatch across PP ranks in DeepSeek V3/R1 by @qhsc in https://github.com/sgl-project/sglang/pull/10214
chore: upgrade v0.3.9 sgl-kernel by @zhyncs in https://github.com/sgl-project/sglang/pull/10220
Revert the changes on NCCL symmetric memory by @merrymercy in https://github.com/sgl-project/sglang/pull/10210
Revert "Revert the changes on NCCL symmetric memory" by @merrymercy in https://github.com/sgl-project/sglang/pull/10238
[HiCache] feat: add mooncake backend extra config by @stmatengss in https://github.com/sgl-project/sglang/pull/10213
Add mamba kernel by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10234
[Auto Sync] Update io_struct.py (20250909) by @merrymercy in https://github.com/sgl-project/sglang/pull/10236
[Auto Sync] Update collector.py, startup_func_log_and_timer... (20250910) by @merrymercy in https://github.com/sgl-project/sglang/pull/10242
Revert "chore: upgrade v0.3.9 sgl-kernel" by @merrymercy in https://github.com/sgl-project/sglang/pull/10245
refactor(InternVL): Use gpu to preprocess the input image by @KEVINTUAN12 in https://github.com/sgl-project/sglang/pull/9795
make --speculative-draft-model an alias of --speculative-draft-model-path by @merrymercy in https://github.com/sgl-project/sglang/pull/10246
[UT for RL] Add UT to cover release/resume memory case for moe model by @ryang-max in https://github.com/sgl-project/sglang/pull/8803
[Benchmark] Prefil-only benchmark scripts by @sundar24295s in https://github.com/sgl-project/sglang/pull/10240
[doc] add walkthrough for implementing and hosting a simple llama wrapper m… by @glenliu21 in https://github.com/sgl-project/sglang/pull/10093
Fix: the default choice is wrong for flashinfer mxfp4 moe precision by @LauYeeYu in https://github.com/sgl-project/sglang/pull/10253
Page first direct IO kernel by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/10060
support vlm model spec bench by @Lzhang-hub in https://github.com/sgl-project/sglang/pull/10173
Fix assertion typo in tp_worker.py by @sgncho in https://github.com/sgl-project/sglang/pull/9954
[Auto Sync] Update io_struct.py (20250910) by @merrymercy in https://github.com/sgl-project/sglang/pull/10262
Fix potential flakiness in test_lora_qwen3 by @lifuhuang in https://github.com/sgl-project/sglang/pull/10250
[router][ci] Add PD router mmlu test by @key4ng in https://github.com/sgl-project/sglang/pull/10256
[1/2] Refactor LoRA to support backend-specific batch preprocessing. by @lifuhuang in https://github.com/sgl-project/sglang/pull/10251
[Bugfix] Fix Weightloading for the original nvidia/Deepseek-R1-FP4 checkpoint by @pavanimajety in https://github.com/sgl-project/sglang/pull/9940
add dual stream for qwen2_moe by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10252
Add tests to AMD CI for MI35x by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/9662
pass a_scale from fp8 quant result instead of hard code to 1.0f by @rainj-me in https://github.com/sgl-project/sglang/pull/10241
Feat: support disable tool parser by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/10184
[Auto Sync] Update serving_base.py, serving_chat.py, servin... (20250910) by @merrymercy in https://github.com/sgl-project/sglang/pull/10282
Revert "[1/2] Optimizations and refactors about quant kernel (#9534)" by @zhyncs in https://github.com/sgl-project/sglang/pull/10292
chore: bump sgl-kernel 0.3.9.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/10294
[Feature] Support DeepEP normal & Redundant Experts on NPU by @iforgetmyname in https://github.com/sgl-project/sglang/pull/9881
add flash linear attention triton kernel by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10239
[chore]Add sgl-router to npu images by @BourneSun0527 in https://github.com/sgl-project/sglang/pull/10229
[CPU] fix OOM when mem-fraction is not set by @ZailiWang in https://github.com/sgl-project/sglang/pull/9090
[fix CI] Fix logical condition in fused MoE layer for compressed tensor quantization by @BBuf in https://github.com/sgl-project/sglang/pull/10299
Revert "Fix flashinfer version in sgl-kernel (#10135)" by @zhyncs in https://github.com/sgl-project/sglang/pull/10310
chore: bump sgl-kernel 0.3.9.post2 by @zhyncs in https://github.com/sgl-project/sglang/pull/10311
[CI] add pyproject.toml to deepseek w4a8 ci by @HanHan009527 in https://github.com/sgl-project/sglang/pull/10314
chore: upgrade v0.3.9.post2 sgl-kernel by @zhyncs in https://github.com/sgl-project/sglang/pull/10297
Qwen3-Next support by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10233
[Auto Sync] Update parallel_state.py (20250911) by @merrymercy in https://github.com/sgl-project/sglang/pull/10326
[Minor] Improve the style of server args by @merrymercy in https://github.com/sgl-project/sglang/pull/10328
[bugfix] fix norm type error in qwen3_next model by @cao1zhg in https://github.com/sgl-project/sglang/pull/10322
[Qwen3-Next] switch to triton and cache conv states to accelerate MTP from 300 tok/s to 341 tok/s by @hebiao064 in https://github.com/sgl-project/sglang/pull/10335
[router] add benchmark for regular router and pd router by @key4ng in https://github.com/sgl-project/sglang/pull/10280
add h20 qwen3 next config by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10264
[router] Add OpenAI backend support - core function by @key4ng in https://github.com/sgl-project/sglang/pull/10254
[router][ci] add gpu process check and free port before start server by @key4ng in https://github.com/sgl-project/sglang/pull/10338
add qwen3-next doc by @yizhang2077 in https://github.com/sgl-project/sglang/pull/10327
fix: trtllm-gen attention take zero-init workspace by @yyihuang in https://github.com/sgl-project/sglang/pull/10330
Fix errors of hicache kernels in sgl-kernel for ROCm by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/10339
update GLM nightly test threshold by @zminglei in https://github.com/sgl-project/sglang/pull/10331
[LongCat] Optimize zero_experts_compute_triton by changing mask by @zk-lover in https://github.com/sgl-project/sglang/pull/10303
add try catch for quant config hf download by @gongwei-130 in https://github.com/sgl-project/sglang/pull/10340
chore: bump v0.5.2 by @zhyncs in https://github.com/sgl-project/sglang/pull/10221

New Contributors

@Beichen-Ma made their first contribution in https://github.com/sgl-project/sglang/pull/9429
@SCDESPERTATE made their first contribution in https://github.com/sgl-project/sglang/pull/7317
@CiaranZhou made their first contribution in https://github.com/sgl-project/sglang/pull/9229
@jonaslsaa made their first contribution in https://github.com/sgl-project/sglang/pull/9190
@ykwd made their first contribution in https://github.com/sgl-project/sglang/pull/8901
@ZhengdQin made their first contribution in https://github.com/sgl-project/sglang/pull/8328
@lshmouse made their first contribution in https://github.com/sgl-project/sglang/pull/9630
@GavinZhu-GMI made their first contribution in https://github.com/sgl-project/sglang/pull/9635
@cicirori made their first contribution in https://github.com/sgl-project/sglang/pull/9665
@KEVINTUAN12 made their first contribution in https://github.com/sgl-project/sglang/pull/9597
@rainj-me made their first contribution in https://github.com/sgl-project/sglang/pull/9495
@pabloiyu made their first contribution in https://github.com/sgl-project/sglang/pull/9397
@KerwinKai made their first contribution in https://github.com/sgl-project/sglang/pull/9216
@mmangkad made their first contribution in https://github.com/sgl-project/sglang/pull/9802
@Orchard-DT made their first contribution in https://github.com/sgl-project/sglang/pull/9824
@pbkowalski made their first contribution in https://github.com/sgl-project/sglang/pull/9073
@LukasBluebaum made their first contribution in https://github.com/sgl-project/sglang/pull/9803
@chenxijun1029 made their first contribution in https://github.com/sgl-project/sglang/pull/8118
@tc-mb made their first contribution in https://github.com/sgl-project/sglang/pull/8747
@alhridoy made their first contribution in https://github.com/sgl-project/sglang/pull/9946
@xiaguan made their first contribution in https://github.com/sgl-project/sglang/pull/9927
@WangJianQ-0118 made their first contribution in https://github.com/sgl-project/sglang/pull/9895
@jingyu-ml made their first contribution in https://github.com/sgl-project/sglang/pull/7912
@fangjian601 made their first contribution in https://github.com/sgl-project/sglang/pull/9906
@SzymonOzog made their first contribution in https://github.com/sgl-project/sglang/pull/9978
@gracehonv made their first contribution in https://github.com/sgl-project/sglang/pull/9314
@JamesLim-sy made their first contribution in https://github.com/sgl-project/sglang/pull/9934
@DevashishLal-CB made their first contribution in https://github.com/sgl-project/sglang/pull/5255
@MahmoudAshraf97 made their first contribution in https://github.com/sgl-project/sglang/pull/8622
@sdpkjc made their first contribution in https://github.com/sgl-project/sglang/pull/9884
@shadowpa0327 made their first contribution in https://github.com/sgl-project/sglang/pull/6905
@jinyangyuan-nvidia made their first contribution in https://github.com/sgl-project/sglang/pull/9834
@Oasis-Git made their first contribution in https://github.com/sgl-project/sglang/pull/9741
@jianyingzhu made their first contribution in https://github.com/sgl-project/sglang/pull/9969
@benbarsdell made their first contribution in https://github.com/sgl-project/sglang/pull/10056
@mss1213 made their first contribution in https://github.com/sgl-project/sglang/pull/9971
@WeiweiZhang1 made their first contribution in https://github.com/sgl-project/sglang/pull/6226
@huaiyuzh made their first contribution in https://github.com/sgl-project/sglang/pull/9434
@ssshinigami made their first contribution in https://github.com/sgl-project/sglang/pull/9871
@alanhe151220037 made their first contribution in https://github.com/sgl-project/sglang/pull/9925
@Zhiy-Zhang made their first contribution in https://github.com/sgl-project/sglang/pull/10166
@gerayking made their first contribution in https://github.com/sgl-project/sglang/pull/10191
@wenhuipeng made their first contribution in https://github.com/sgl-project/sglang/pull/10165
@Missmiaom made their first contribution in https://github.com/sgl-project/sglang/pull/10209
@shaharmor98 made their first contribution in https://github.com/sgl-project/sglang/pull/9960
@qhsc made their first contribution in https://github.com/sgl-project/sglang/pull/10214
@glenliu21 made their first contribution in https://github.com/sgl-project/sglang/pull/10093
@LauYeeYu made their first contribution in https://github.com/sgl-project/sglang/pull/10253
@sgncho made their first contribution in https://github.com/sgl-project/sglang/pull/9954
@BourneSun0527 made their first contribution in https://github.com/sgl-project/sglang/pull/10229
@zk-lover made their first contribution in https://github.com/sgl-project/sglang/pull/10303

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.1...v0.5.2

Open source →

0.5.123 Aug 2025Deprecated

Improve LoRA Perf by Deprecating FlashInfer and Eliminating Redundant Tensor Ops by @lifuhuang in https://github.com/sgl-project/sglang/pull/8940

What's Changed

[PD] Use batch transfer for rdma transport and add notes for mnnvl usage by @ShangmingCai in https://github.com/sgl-project/sglang/pull/8595
[bugifx] QWen-1M context support[2/3] using current cuda stream in the DCA's kernel for bugfix. by @sighingnow in https://github.com/sgl-project/sglang/pull/8611
Fix hf3fs_fuse import error by @ispobock in https://github.com/sgl-project/sglang/pull/8623
Update step3v default config by @ispobock in https://github.com/sgl-project/sglang/pull/8626
[ci] fix genai-bench execution cmd by @slin1237 in https://github.com/sgl-project/sglang/pull/8629
[router] update router pypi version by @slin1237 in https://github.com/sgl-project/sglang/pull/8628
[Optimization][Perf] Disable the GC during CUDA graph capture to speed up by up to 3x by @b8zhong in https://github.com/sgl-project/sglang/pull/8577
Fix typos in py_test/test_launch_server.py by @windsonsea in https://github.com/sgl-project/sglang/pull/6227
misc: Remove debug print to logger.info by @CatherineSue in https://github.com/sgl-project/sglang/pull/8633
SGLang HiCache NIXL Connector by @vvenkates27 in https://github.com/sgl-project/sglang/pull/8488
[bug] remove pdlb from minilb since its no longer available by @slin1237 in https://github.com/sgl-project/sglang/pull/8634
[bugfix] Fix flashinfer cutlass EP moe after MoE refactor by @trevor-m in https://github.com/sgl-project/sglang/pull/8630
Conditionally import HiCacheHF3FS by @pansicheng in https://github.com/sgl-project/sglang/pull/8598
TRTLLM Gen MLA Decode Kernel Integration (same as #7938) by @farazkh80 in https://github.com/sgl-project/sglang/pull/8632
Fix nan value generated after custom all reduce by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/8532
Revert "Fix nan value generated after custom all reduce (#8532)" by @zhyncs in https://github.com/sgl-project/sglang/pull/8642
Feature/modelscope model download by @yrk111222 in https://github.com/sgl-project/sglang/pull/8083
chore: speedup NPU CI by cache by @pkking in https://github.com/sgl-project/sglang/pull/8270
[Bugfix] fix w8a8_int8 load issue by @iforgetmyname in https://github.com/sgl-project/sglang/pull/8308
[bugfix] fix router python parser for pd urls by @slin1237 in https://github.com/sgl-project/sglang/pull/8644
[router] add basic usage doc by @slin1237 in https://github.com/sgl-project/sglang/pull/8640
[router] upgrade router version to 0.1.8 by @slin1237 in https://github.com/sgl-project/sglang/pull/8645
[NVIDIA] Enable Flashinfer MoE blockscale fp8 backend for TP MoE by @kaixih in https://github.com/sgl-project/sglang/pull/8450
HiCache, fixing hash value indexing by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8636
Interface change for kvcache io to support page first layout by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8318
Update batch size limitation of dsv3_router_gemm kernel to 16 by @Fridge003 in https://github.com/sgl-project/sglang/pull/8051
chore: bump v0.4.10.post1 by @ispobock in https://github.com/sgl-project/sglang/pull/8652
Add hf3fs_utils.cpp to package-data by @pansicheng in https://github.com/sgl-project/sglang/pull/8653
Fix chat template handling for OpenAI serving by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8635
Bug: apply final_hidden_states*=self.routed_scaling_factor at MoE lay… by @byjiang1996 in https://github.com/sgl-project/sglang/pull/8511
[5/N] MoE Refactor: Update MoE parallelism arguments by @ch-wan in https://github.com/sgl-project/sglang/pull/8658
Increase tolerance to address CI failures by @lifuhuang in https://github.com/sgl-project/sglang/pull/8643
[Kimi K2] dsv3_router_gemm supports NUM_EXPERTS == 384 by @panpan0000 in https://github.com/sgl-project/sglang/pull/8013
[Doc] fix: Update README for cu126 sgl-kernel compile problem by @Hongbosherlock in https://github.com/sgl-project/sglang/pull/8665
fix per token cuda kernel hidden dim cannot divide by 16 by @hebiao064 in https://github.com/sgl-project/sglang/pull/8543
fix arg typo for --disaggregation-transfer-backend by @ZacWang in https://github.com/sgl-project/sglang/pull/8664
[fix] fix pd disagg error of vlms by @ccw1996 in https://github.com/sgl-project/sglang/pull/8094
Disable tp for shared experts under expert parallelism for GLM4.5 model (#8647) by @zminglei in https://github.com/sgl-project/sglang/pull/8647
[bugfix] Fix page size for create_flashmla_kv_indices_triton() for cutlass mla by @trevor-m in https://github.com/sgl-project/sglang/pull/8685
[bug] limit bootstrap room to to [0, 2^63 - 1] by @slin1237 in https://github.com/sgl-project/sglang/pull/8684
Update CODEOWNERS by @merrymercy in https://github.com/sgl-project/sglang/pull/8686
Fix deepgemm masked grouped gemm jit compile by @ispobock in https://github.com/sgl-project/sglang/pull/8679
Fix FP8 block quantization when N or K is not multiples of 128 by @yanbing-j in https://github.com/sgl-project/sglang/pull/8648
bugfix(hicache): Fix 'MooncakeStore' not defined error. by @hzh0425 in https://github.com/sgl-project/sglang/pull/8668
upgrade xgrammar 0.1.22 by @Swipe4057 in https://github.com/sgl-project/sglang/pull/8522
[bugfix] Add 'disaggregation_mode' parameter to warmup function when compile deep_gemm manually by @lbh2001 in https://github.com/sgl-project/sglang/pull/8618
Add support for NCCL symmetric memory for TP allreduces by @nvcastet in https://github.com/sgl-project/sglang/pull/8238
[1/2] sgl-kernel: Fuse routed scaling factor into select_experts by @trevor-m in https://github.com/sgl-project/sglang/pull/8364
chore(gb200): update dockerfile to handle fp4 disaggregation by @ishandhanani in https://github.com/sgl-project/sglang/pull/8694
[bugfix] Apply routed scaling factor to cutlass_fused_experts_fp8 by @trevor-m in https://github.com/sgl-project/sglang/pull/8688
Fix: resolve prefill of retracted request out-of-memory issue when ignore_eos is enabled by @GaoYusong in https://github.com/sgl-project/sglang/pull/7434
model: adapt mllama4 to VisionAttention by @wenchen76 in https://github.com/sgl-project/sglang/pull/8512
Add tensor.detach() back to update weight util by @hebiao064 in https://github.com/sgl-project/sglang/pull/8691
[Doc] Polish sgl-kernel readme for cu126 build error by @FlamingoPg in https://github.com/sgl-project/sglang/pull/8704
Revert "[1/2] sgl-kernel: Fuse routed scaling factor into select_experts" by @hnyls2002 in https://github.com/sgl-project/sglang/pull/8706
[router] minor code clean up and and refactoring by @slin1237 in https://github.com/sgl-project/sglang/pull/8711
[Bug] fix green context's incompatibility with cuda < 12.4 by @hnyls2002 in https://github.com/sgl-project/sglang/pull/8701
chore: bump sgl-kernel v0.2.9 by @zhyncs in https://github.com/sgl-project/sglang/pull/8713
Remove assertions about per group quant fp8 by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8717
[FIX] Fix the nightly CI by disabling swa mem pool for gemma2 by @merrymercy in https://github.com/sgl-project/sglang/pull/8693
Fix triton moe error caused by TopK refactor by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8705
[router] Implement HTTP Dependency Injection Pattern for Router System by @slin1237 in https://github.com/sgl-project/sglang/pull/8714
[Feature] Radix Tree in C++ by @DarkSharpness in https://github.com/sgl-project/sglang/pull/7369
[Perf]Use Cooperative Schedule for H100 & H200 & H800 in fp8_blockwise_scaled_grouped_mm by @HydraQYH in https://github.com/sgl-project/sglang/pull/8722
Fix fused MoE when routed_scaling_factor is None by @hnyls2002 in https://github.com/sgl-project/sglang/pull/8709
Tiny fix CI pytest error by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8524
[hotfix] fix mixtral with tensor-level compressed-tensor quantization by @ch-wan in https://github.com/sgl-project/sglang/pull/8721
Support limiting max loaded loras in CPU. by @lifuhuang in https://github.com/sgl-project/sglang/pull/8650
Reduce memory accumulation in long-running server by @Edenzzzz in https://github.com/sgl-project/sglang/pull/8306
HiCache storage, style change and bug fix by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8719
[feat] support minimum token load balance in dp attention by @WANG-GH in https://github.com/sgl-project/sglang/pull/7379
Do layernorm before allgather for DP attention by @trevor-m in https://github.com/sgl-project/sglang/pull/8631
[fix] Fix divide by zero error for llama4. by @shenoyvvarun in https://github.com/sgl-project/sglang/pull/8683
feat: Add new moe triton for NVIDIA RTX 6000 Ada by @17Reset in https://github.com/sgl-project/sglang/pull/8547
[Improvements] Merge health check route by @whybeyoung in https://github.com/sgl-project/sglang/pull/8444
chore: bump sgl-kernel 0.3.0 with torch 2.8.0 by @zhyncs in https://github.com/sgl-project/sglang/pull/8718
Save cuda graph memory for fa3 by @ch-wan in https://github.com/sgl-project/sglang/pull/8567
[CUDA Graph] save cuda graph memory by using next_token_logits_buffer by @ch-wan in https://github.com/sgl-project/sglang/pull/8579
[DP] fix the compatibility issue between DP attention and --attention-backend triton by @ch-wan in https://github.com/sgl-project/sglang/pull/8723
chore: bump v0.4.10.post2 by @zhyncs in https://github.com/sgl-project/sglang/pull/8727
feat: Support DP Attention for step3_vl by @yhyang201 in https://github.com/sgl-project/sglang/pull/8699
[RL] fix update weight for FusedMoE with EP by @zhuzilin in https://github.com/sgl-project/sglang/pull/8676
use fp32 for e_score_correction_bias in GLM-4.5 by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/8729
Fix triton kernels topk with keyword arguments by @ispobock in https://github.com/sgl-project/sglang/pull/8732
feat: support cutlass_moe_fp8 kernel for fusedmoe in sm90 by @TianQiLin666666 in https://github.com/sgl-project/sglang/pull/8678
Fix the missing 'lof' choice of --schedule-policy server args by @acelyc111 in https://github.com/sgl-project/sglang/pull/7114
fix args typo in memory_pool_host by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/8662
[CI] Do not trigger pd-disaggregation CI in draft PR by @hnyls2002 in https://github.com/sgl-project/sglang/pull/8737
[MoE] Enable renormalize=False in Triton kernels by @ch-wan in https://github.com/sgl-project/sglang/pull/8735
Replace torch.jit.script with torch.compile in get_masked_input_and_mask to fix benchmark underreporting by @YyWangCS in https://github.com/sgl-project/sglang/pull/8733
Fix bug of refactoring TopKOutput in w4afp8 by @yuan-luo in https://github.com/sgl-project/sglang/pull/8745
Rename lora_path to lora_id in batches by @Fridge003 in https://github.com/sgl-project/sglang/pull/8437
[sgl-kernel] avoid per_token_quant_fp8.cu hardcode sm_count by @BBuf in https://github.com/sgl-project/sglang/pull/8738
[CI] Ascend NPU CI enhancement by @iforgetmyname in https://github.com/sgl-project/sglang/pull/8294
[bugfix] fix import path in HiCacheController by @lbh2001 in https://github.com/sgl-project/sglang/pull/8749
[NVIDIA] Add Low Latency NVFP4 decode kernels from Flashinfer by @azhurkevich in https://github.com/sgl-project/sglang/pull/8552
[router] introduce dp worker abstraction by @slin1237 in https://github.com/sgl-project/sglang/pull/8639
[bugfix] Fix typo in modelopt quant: 'FusedMoE' object has no attribute 'local_num_experts' by @trevor-m in https://github.com/sgl-project/sglang/pull/8768
Integrate triton_kernels in sgl-kernel by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/8762
chore: bump sgl-kernel v0.3.1 by @zhyncs in https://github.com/sgl-project/sglang/pull/8771
[NVIDIA] Fix breakage of using trtllm-gen fp8 moe by @kaixih in https://github.com/sgl-project/sglang/pull/8773
[Fix] Fix several issues preventing gemma3n LoRA support. by @lifuhuang in https://github.com/sgl-project/sglang/pull/8776
Support OCP MXFP4 quantization on AMD GPUs by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/8255
[CPU][sgl-kernel] biased_grouped_topk: fix correction_bias dtype to float32 by @chunyuan-w in https://github.com/sgl-project/sglang/pull/8212
[PD] Refactor parallel sizes and add pp support for mooncake by @ShangmingCai in https://github.com/sgl-project/sglang/pull/8571
[pd-router] Add Configurable Retry Logic for reduce backend pressure by @slin1237 in https://github.com/sgl-project/sglang/pull/8744
chore: upgrade flashinfer v0.2.9 by @zhyncs in https://github.com/sgl-project/sglang/pull/8780
[NVIDIA]Fix local_num_experts for EP by @wenscarl in https://github.com/sgl-project/sglang/pull/8779
[feat] Add detail in image_data by @yuhyao in https://github.com/sgl-project/sglang/pull/8596
Revert "[NVIDIA]Fix local_num_experts for EP (#8779)" by @zhyncs in https://github.com/sgl-project/sglang/pull/8797
feat: support sgl-kernel cu129 by @zhyncs in https://github.com/sgl-project/sglang/pull/8800
chore: bump sgl-kernel v0.3.2 by @zhyncs in https://github.com/sgl-project/sglang/pull/8802
feat: add trtllm-gen mha from direct call by @yyihuang in https://github.com/sgl-project/sglang/pull/8782
GLM-4.5 and GLM-4.5-Air both support by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/8804
fix: update cmake by @zhyncs in https://github.com/sgl-project/sglang/pull/8817
chore: upgrade transformers 4.55.0 by @zhyncs in https://github.com/sgl-project/sglang/pull/8823
chore: upgrade flashinfer 0.2.10 by @zhyncs in https://github.com/sgl-project/sglang/pull/8827
Fix potential memory fault issue and ncclSystemError in CI test by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/8681
feat: use py312 by @zhyncs in https://github.com/sgl-project/sglang/pull/8832
fix: remove unused import by @zhyncs in https://github.com/sgl-project/sglang/pull/8809
Add initial support for gpt-oss by @Ying1123 in https://github.com/sgl-project/sglang/pull/8824
chore: upgrade torch 2.8.0 by @zhyncs in https://github.com/sgl-project/sglang/pull/8836
[router] complete router oai spec by @slin1237 in https://github.com/sgl-project/sglang/pull/8828
Turn off hybrid cache by default by @ispobock in https://github.com/sgl-project/sglang/pull/8839
Support bailing moe by @ppraneth in https://github.com/sgl-project/sglang/pull/8680
[Feature] improve TBO: two chunk overlap by @House-West in https://github.com/sgl-project/sglang/pull/8144
[router] PD Router Simplification and Reorganization by @slin1237 in https://github.com/sgl-project/sglang/pull/8838
[1/3] Optimize Slime Update Weights: Remove QWen3MOE Load Weight Overhead by @hebiao064 in https://github.com/sgl-project/sglang/pull/8751
[2/3] Optimize Slime Update Weights: Avoid GPU-to-CPU Device Sync when update expert weights by @hebiao064 in https://github.com/sgl-project/sglang/pull/8753
Support mxfp4 for GPT-OSS by @Ying1123 in https://github.com/sgl-project/sglang/pull/8843
Add unit test for triton swa kernel by @ispobock in https://github.com/sgl-project/sglang/pull/8853
fix: resolve ci issue by @zhyncs in https://github.com/sgl-project/sglang/pull/8859
fix benchmark fp8 blockwise group gemm by @yuan-luo in https://github.com/sgl-project/sglang/pull/8815
Refine naming by @ispobock in https://github.com/sgl-project/sglang/pull/8868
Optimize triton swa kernel by skipping computation by @ispobock in https://github.com/sgl-project/sglang/pull/8860
Support B200 in CI by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8861
chore: update Dockerfile by @mickqian in https://github.com/sgl-project/sglang/pull/8872
[NVIDIA] Fix num_experts in modelopt_quant by @wenscarl in https://github.com/sgl-project/sglang/pull/8811
[CI] fix pip upgrade by @ch-wan in https://github.com/sgl-project/sglang/pull/8881
chore: use torch 2.8 stable by @zhyncs in https://github.com/sgl-project/sglang/pull/8880
Support v1/responses and use harmony in serving_chat by @CatherineSue in https://github.com/sgl-project/sglang/pull/8837
Use reduce scatter for DP by @trevor-m in https://github.com/sgl-project/sglang/pull/8539
add flashinfer mxfp4 by @BBuf in https://github.com/sgl-project/sglang/pull/8847
fix glm4 moe by @ch-wan in https://github.com/sgl-project/sglang/pull/8883
feat: openai oss attention sink support with trtllm-gen backend #8825 by @yyihuang in https://github.com/sgl-project/sglang/pull/8834
Support GPU pinning for LoRA by @lifuhuang in https://github.com/sgl-project/sglang/pull/8697
Enables force reasoning based on chat template for Qwen3-Thinking by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8369
[AMD] Pull latest SGLang version for AMD CI by @michael-amd in https://github.com/sgl-project/sglang/pull/8787
[Feature][Multimodal] Implement LRU cache for multimodal embeddings by @ZhengWG in https://github.com/sgl-project/sglang/pull/8292
[router] fix req handling order, improve serialization, remove retry by @slin1237 in https://github.com/sgl-project/sglang/pull/8888
[Feat] QWen-1M context support[2/2]: Update block sparse attention backend by @FlamingoPg in https://github.com/sgl-project/sglang/pull/5949
[CPU] Fix fallback allgather issue by @blzheng in https://github.com/sgl-project/sglang/pull/8041
Disable gemma3 for SWA due to CUDA illegal memory access error by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8895
[Perf] Auto enable best flashinfer mxfp4 kernel in b200 by @BBuf in https://github.com/sgl-project/sglang/pull/8898
Fix sgl-kernel arch and missing package in CI by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8869
refactor(sgl-router): Replace once_cell with LazyLock in worker.rs and remove once_cell dependency from Cargo.toml by @htiennv in https://github.com/sgl-project/sglang/pull/8698
[router] re-enable pd router benchmark CI by @slin1237 in https://github.com/sgl-project/sglang/pull/8912
[router] update pd router ci summary step with new threshold by @slin1237 in https://github.com/sgl-project/sglang/pull/8916
[router] upgrade router version to 0.1.9 by @slin1237 in https://github.com/sgl-project/sglang/pull/8844
Fix hopper launch gpt-oss model illegal memory by @BBuf in https://github.com/sgl-project/sglang/pull/8908
fix: use openai 1.99.1 by @zhyncs in https://github.com/sgl-project/sglang/pull/8927
codeowner updates for modelopt related files by @Edwardf0t1 in https://github.com/sgl-project/sglang/pull/8925
chore: support blackwell cu129 image by @zhyncs in https://github.com/sgl-project/sglang/pull/8928
docs: update README by @zhyncs in https://github.com/sgl-project/sglang/pull/8929
remove vllm fp8quant from fp8.py by @hebiao064 in https://github.com/sgl-project/sglang/pull/8937
fix: reasoning parser when request have enable_thinking flag by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8933
correct the tp_plan logic by @hebiao064 in https://github.com/sgl-project/sglang/pull/8850
[router] dedicated prefill HTTP client and request-path optimizations by @slin1237 in https://github.com/sgl-project/sglang/pull/8923
Enhancements for bench_one_batch by @ZailiWang in https://github.com/sgl-project/sglang/pull/8703
refactor: Move scalar_types.py to sgl-kernel to avoid circular import by @Hongbosherlock in https://github.com/sgl-project/sglang/pull/8720
Fix enable flashinfer mxfp4 moe bf16 check by @BBuf in https://github.com/sgl-project/sglang/pull/8950
Reduce scheduler recv requests overhead by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8947
Better optimization log for gpt-oss model by @BBuf in https://github.com/sgl-project/sglang/pull/8953
minor: global workspace buffer for trtllm-gen mha from flashinfer by @yyihuang in https://github.com/sgl-project/sglang/pull/8952
bench: add attention sink op benchmark, triton and trtllm-gen [B200] by @yyihuang in https://github.com/sgl-project/sglang/pull/8932
Fix typos and unify size(s)/stride(s) API calls by @triple-Mu in https://github.com/sgl-project/sglang/pull/8799
Expert Parallelism for GPT-OSS by @ch-wan in https://github.com/sgl-project/sglang/pull/8944
Add ernie4.py for ERNIE-4.5 by @solrex in https://github.com/sgl-project/sglang/pull/7657
[NVIDIA] Fix missing get_col_major_tma_aligned_tensor for Blackwell deepgemm in EpMoE by @kaixih in https://github.com/sgl-project/sglang/pull/8955
chore: bump sgl-kernel v0.3.3 by @zhyncs in https://github.com/sgl-project/sglang/pull/8957
add zai-org/GLM-4.5-Air-FP8 model into nightly CI by @zminglei in https://github.com/sgl-project/sglang/pull/8894
Support Multi Process Tokenizer Manager by @whybeyoung in https://github.com/sgl-project/sglang/pull/6555
Simple prefetch policy by @pansicheng in https://github.com/sgl-project/sglang/pull/8692
chore: update flashinfer by @zhyncs in https://github.com/sgl-project/sglang/pull/8958
Revert "Support Multi Process Tokenizer Manager" by @merrymercy in https://github.com/sgl-project/sglang/pull/8960
[RL] fix skip_server_warmup and rl health_generate logic by @zhuzilin in https://github.com/sgl-project/sglang/pull/8757
chore: bump v0.5.0rc0 by @zhyncs in https://github.com/sgl-project/sglang/pull/8959
[router] router circuit breaker core by @slin1237 in https://github.com/sgl-project/sglang/pull/8941
refine aiter_backend for mtp by @valarLip in https://github.com/sgl-project/sglang/pull/7279
[router] harden retries + metrics; fix streaming load; header filtering by @slin1237 in https://github.com/sgl-project/sglang/pull/8972
Fix kimi k2 function call format by @merrymercy in https://github.com/sgl-project/sglang/pull/8968
[router] add metrics for worker and policy by @tonyluj in https://github.com/sgl-project/sglang/pull/8971
chore(gb200): update to CUDA 12.9 and improve build process by @ishandhanani in https://github.com/sgl-project/sglang/pull/8772
chore(ci): update Python version from 3.9 to 3.10 in sgl-kernel workflow by @ishandhanani in https://github.com/sgl-project/sglang/pull/8981
[router] reduce radix tree contention, fix radix tree double-count race by @slin1237 in https://github.com/sgl-project/sglang/pull/8978
[router] fix radix tree integration issues in PD router by @slin1237 in https://github.com/sgl-project/sglang/pull/8982
Update qwen3_coder_detector.py for streaming by @maocheng23 in https://github.com/sgl-project/sglang/pull/8371
[bug fix] Ensure local token and global token buffers are pointing to different storage by @elfiegg in https://github.com/sgl-project/sglang/pull/8785
Create cancel-all-pr-test-runs by @merrymercy in https://github.com/sgl-project/sglang/pull/8986
[Fix] Add a workflow to cancel all pending CI runs by @merrymercy in https://github.com/sgl-project/sglang/pull/8988
Minor Optimizations in Schedule Batch by @merrymercy in https://github.com/sgl-project/sglang/pull/8724
[1/2][resubmit] sgl-kernel: Fuse routed scaling factor into moe_fused_gate (select_experts) by @trevor-m in https://github.com/sgl-project/sglang/pull/8770
Add unit test for flashinfer fp4 moe by @trevor-m in https://github.com/sgl-project/sglang/pull/8330
[AMD] Update SGLang image fallback logic for AMD CI by @michael-amd in https://github.com/sgl-project/sglang/pull/8980
Clean up server_args.py to have a dedicated function for model specific adjustments by @merrymercy in https://github.com/sgl-project/sglang/pull/8983
Molly/ci gnr server by @DiweiSun in https://github.com/sgl-project/sglang/pull/8667
[Fix] Fix wrong backend chosen in hybrid backend by @DarkSharpness in https://github.com/sgl-project/sglang/pull/8989
Revert "[bug fix] Ensure local token and global token buffers are pointing to different storage " by @ch-wan in https://github.com/sgl-project/sglang/pull/8993
[hotfix] use the original implementation in 8785 by @ch-wan in https://github.com/sgl-project/sglang/pull/8994
Fix incorrect default get_hidden_dim logic by @lifuhuang in https://github.com/sgl-project/sglang/pull/8987
optimize: reduce shulffle and quantization overhead in cutlass_moe sm90 by @TianQiLin666666 in https://github.com/sgl-project/sglang/pull/8962
chore(deps): update minimum python to 3.10 by @ishandhanani in https://github.com/sgl-project/sglang/pull/8984
Add CI for gpt-oss model on hopper by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8851
Fix redundant kernel in sink dtype conversion by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8966
Fix qwen2 audio not working bug by @byjiang1996 in https://github.com/sgl-project/sglang/pull/8600
feat: update flashinfer ar oneshot params by @yyihuang in https://github.com/sgl-project/sglang/pull/8687
Support glm4.1v and glm4.5v by @byjiang1996 in https://github.com/sgl-project/sglang/pull/8798
feature(hicache): Support hf3fs-hicache reusing kvcache across different instances by @hzh0425 in https://github.com/sgl-project/sglang/pull/8673
Tiny Llama4 type error in constructor by @b8zhong in https://github.com/sgl-project/sglang/pull/6752
HiCache Storage tp fix by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8878
chore: upgrade sgl-kernel 0.3.3 by @zhyncs in https://github.com/sgl-project/sglang/pull/8998
[DP] fix: engine crash when decode batch is padded by @ch-wan in https://github.com/sgl-project/sglang/pull/8995
[bugfix] Fix missing args in bench one batch by @trevor-m in https://github.com/sgl-project/sglang/pull/8877
[Feature] Optimize DeepSeek's DeepEP on Ascend NPU by @iforgetmyname in https://github.com/sgl-project/sglang/pull/8355
Enable TBO on ROCm by @lcskrishna in https://github.com/sgl-project/sglang/pull/8329
fix nvshmem cu126 by @zhyncs in https://github.com/sgl-project/sglang/pull/9001
[perf] add kimi-k2 b200 fused moe config by @Alcanderian in https://github.com/sgl-project/sglang/pull/9010
fix: fix obsolete qwen-audio processor arg by @mickqian in https://github.com/sgl-project/sglang/pull/9003
Fix CI by @merrymercy in https://github.com/sgl-project/sglang/pull/9012
fix flashinfer allreduce fusion import bug by @BBuf in https://github.com/sgl-project/sglang/pull/9007
Fix CI by @merrymercy in https://github.com/sgl-project/sglang/pull/9013
[hicache] Optimization for DMA copy by @cctry in https://github.com/sgl-project/sglang/pull/8245
fix page first per layer pf2lf kernel by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/8915
[Fix] Fix hicache backend by @DarkSharpness in https://github.com/sgl-project/sglang/pull/8991
[Fix] Fix flashinfer cpu <-> gpu synchronization by @DarkSharpness in https://github.com/sgl-project/sglang/pull/8340
[router] upgrade to latest sgl kernel for router ci by @slin1237 in https://github.com/sgl-project/sglang/pull/9019
[router] upgrade rand to latest version by @slin1237 in https://github.com/sgl-project/sglang/pull/9017
[router] upgrade kube version to latest by @slin1237 in https://github.com/sgl-project/sglang/pull/9018
Optimize: Cache CUDA device to reduce redundant calls during tensor l… by @GeLee-Q in https://github.com/sgl-project/sglang/pull/8996
Improve LoRA Perf by Deprecating FlashInfer and Eliminating Redundant Tensor Ops by @lifuhuang in https://github.com/sgl-project/sglang/pull/8940
[router] update pyo3 version to 0.25.1 by @slin1237 in https://github.com/sgl-project/sglang/pull/9022
[RL] Add test for /abort_request by @hebiao064 in https://github.com/sgl-project/sglang/pull/7626
Simplify frontend language by @merrymercy in https://github.com/sgl-project/sglang/pull/9029
Reorganize CI and test files by @merrymercy in https://github.com/sgl-project/sglang/pull/9027
Reduce CI duration of test_lora_update. by @lifuhuang in https://github.com/sgl-project/sglang/pull/9024
[Optimization] Update estimated_num_new_pages logic in TokenToKVPoolAllocator by @YiXR in https://github.com/sgl-project/sglang/pull/8794
Support Flatten Tensor Update Weights to speed up MOE Update Weights by 20% by @hebiao064 in https://github.com/sgl-project/sglang/pull/8079
Simplify memory pool by @merrymercy in https://github.com/sgl-project/sglang/pull/9033
Revert "[1/2][resubmit] sgl-kernel: Fuse routed scaling factor into m… by @zhyncs in https://github.com/sgl-project/sglang/pull/9035
Simplify health check by @merrymercy in https://github.com/sgl-project/sglang/pull/9034
chore: upgrade flashinfer 0.2.11 by @zhyncs in https://github.com/sgl-project/sglang/pull/9036
Update release-docs.yml by @merrymercy in https://github.com/sgl-project/sglang/pull/9037
Refactor the docs by @merrymercy in https://github.com/sgl-project/sglang/pull/9031
Improve docs and developer guide by @merrymercy in https://github.com/sgl-project/sglang/pull/9044
Update REVIEWERS.md by @merrymercy in https://github.com/sgl-project/sglang/pull/9046
[router] regular router circuit breaker by @slin1237 in https://github.com/sgl-project/sglang/pull/8997
REVIEWERS.md typo fix by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9048
Revert "feat: update flashinfer ar oneshot params (#8687)" by @zhyncs in https://github.com/sgl-project/sglang/pull/9054
[CI] Fix CI tests by @ch-wan in https://github.com/sgl-project/sglang/pull/9050
Revert "chore: upgrade flashinfer 0.2.11 (#9036)" by @zhyncs in https://github.com/sgl-project/sglang/pull/9057
bugfix: Fix output_ids extraction in detokenizer_manager by @CatherineSue in https://github.com/sgl-project/sglang/pull/9047
[pd-router] add retry and circuit breakfor for pd router by @slin1237 in https://github.com/sgl-project/sglang/pull/9051
Support radix cache for Lora feature by @Fridge003 in https://github.com/sgl-project/sglang/pull/7216
update deepep commit to support qwen3-coder by @yizhang2077 in https://github.com/sgl-project/sglang/pull/9066
chore(gb200): remove ToT flashinfer installation by @ishandhanani in https://github.com/sgl-project/sglang/pull/9079
Update REVIEWERS by @HaiShaw in https://github.com/sgl-project/sglang/pull/9063
Fix chunked prefill size validation for disabled state by @chi2liu in https://github.com/sgl-project/sglang/pull/8973
Fix broken Kimi models HuggingFace link by @Hangzhi in https://github.com/sgl-project/sglang/pull/9080
[PD]decode: add CLIP_MAX_NEW_TOKEN for pop_preallocated by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/8866
Fix docs for clip max new tokens by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9082
refactor(pd-router): extract common patterns to reduce code duplication by @slin1237 in https://github.com/sgl-project/sglang/pull/9081
fix: w4afp8 accuracy problem and rebase by @yangsijia-serena in https://github.com/sgl-project/sglang/pull/8752
Update hyperparameter_tuning.md by @merrymercy in https://github.com/sgl-project/sglang/pull/9083
fuse allreduce and residual_rmsnorm by @BBuf in https://github.com/sgl-project/sglang/pull/8731
TRTLLM-MLA FP8 path by @farazkh80 in https://github.com/sgl-project/sglang/pull/8638
HiCache Storage: generate hash when inserting new nodes by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9053
[fix] Set Radix tree root node hash to None - Nvidia Dynamo Integration by @faradawn in https://github.com/sgl-project/sglang/pull/9030
HiCache, add bench long context plus minor fixs by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/9086
(gpt-oss, oai, chat): Remove Harmony Integration and Implement Native GPT-OSS Tool Call Support by @CatherineSue in https://github.com/sgl-project/sglang/pull/9043
[router] Add Rust Binary Entrypoint for SGLang Router by @slin1237 in https://github.com/sgl-project/sglang/pull/9089
[CI]Test BM.A10.4 runner by @key4ng in https://github.com/sgl-project/sglang/pull/8992
Fix race condition in async lora unload by @lifuhuang in https://github.com/sgl-project/sglang/pull/9084
Fix broken CI TestRequestLengthValidation by @lifuhuang in https://github.com/sgl-project/sglang/pull/9095
Optimization for AscendPagedTokenToKVPoolAllocator by @Makcum888e in https://github.com/sgl-project/sglang/pull/8293
feat: add fused moe config for Qwen3-30B-A3B on B200 by @zixuanzhang226 in https://github.com/sgl-project/sglang/pull/9087
Fix mismatch between padded_scales shape and reshape dimensions in modelopt quantization by @ovowei in https://github.com/sgl-project/sglang/pull/8766
[Fix] Fix dual chunk model default behavior by @DarkSharpness in https://github.com/sgl-project/sglang/pull/9032
bugfix: Fix the commentary msg extraction in GptOssDetector by @CatherineSue in https://github.com/sgl-project/sglang/pull/9097
docs: fix broken links in README.md by @xxrjun in https://github.com/sgl-project/sglang/pull/9075
Fuse two kernels of hidden states padding into quantization kernel by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9005
update support new models doc by @yichaolemon in https://github.com/sgl-project/sglang/pull/9096
Fuse writing KV buffer into rope kernel (part 1: sgl-kernel) by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9077
chore: bump sgl-kernel v0.3.4 by @zhyncs in https://github.com/sgl-project/sglang/pull/9103
Runtime check CUDA driver version to avoid unresolved green context symbols by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9021
[Bugfix] Fix accuracy-test-1-gpu failure caused by builtin_tools by @CatherineSue in https://github.com/sgl-project/sglang/pull/9114
Fix typo in REVIEWERS by @ShangmingCai in https://github.com/sgl-project/sglang/pull/9113
[5/n] DP Enhancement: Correct num_token_non_padded by @ch-wan in https://github.com/sgl-project/sglang/pull/9107
router: Fix user guide link README.md by @CatherineSue in https://github.com/sgl-project/sglang/pull/9122
fix(docker): update sgl_kernel version to 0.3.4 in Dockerfile.gb200 by @ishandhanani in https://github.com/sgl-project/sglang/pull/9118
Fuse writing KV buffer into rope kernel (part 2: srt) by @JeremieMelo in https://github.com/sgl-project/sglang/pull/9014
[router] update router documentation by @slin1237 in https://github.com/sgl-project/sglang/pull/9121
fix: update Dockerfile by @zhyncs in https://github.com/sgl-project/sglang/pull/9125
[feat] add ascend readme and docker release by @pkking in https://github.com/sgl-project/sglang/pull/8700
[feat] Enable Ascend profiling on SGLang by @ping1jing2 in https://github.com/sgl-project/sglang/pull/8610
[Quantization] Supported w8a8 int8 quantized Gemma3 and Qwen-VL models by @ichernob in https://github.com/sgl-project/sglang/pull/8619
Fix typos in supported models documentation by @Hangzhi in https://github.com/sgl-project/sglang/pull/9119
[AMD] Support Wave attention backend with AMD GPU optimizations by @yichiche in https://github.com/sgl-project/sglang/pull/8660
fix: update Dockerfile by @zhyncs in https://github.com/sgl-project/sglang/pull/9129
chore: use cp310 by @zhyncs in https://github.com/sgl-project/sglang/pull/9130
Support page first layout zero copy for mooncake store by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/8651
[Feature] Support custom set kv buffer kernel by @DarkSharpness in https://github.com/sgl-project/sglang/pull/8884
fix: wrong docker hub org name by @pkking in https://github.com/sgl-project/sglang/pull/9137
Use FlashInfer's TRTLLM FP8 Blockscale GEMM by @elfiegg in https://github.com/sgl-project/sglang/pull/8588
[1/2][resubmit again] sgl-kernel: Fuse routed scaling factor into moe_fused_gate by @trevor-m in https://github.com/sgl-project/sglang/pull/9088
Support Triton FP8 Gemm can handle hidden_dim not divisible by 16 by @hebiao064 in https://github.com/sgl-project/sglang/pull/9093
Fix gpt-oss ~2x memory consumption issue by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9146
Update docker file for MI35x base image update to support gpt-oss mxfp4 model by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/9111
Double vision prefill throughput by defaulting to optimal vision attention backend by @AlienKevin in https://github.com/sgl-project/sglang/pull/8484
Update fa3 interface and add unit test by @ispobock in https://github.com/sgl-project/sglang/pull/9150
feat: update fa3 by @zhyncs in https://github.com/sgl-project/sglang/pull/9126
[router] optimize Rust compilation and development workflow by @slin1237 in https://github.com/sgl-project/sglang/pull/9133
[PD] optimize kv cache transfer directly using batch transfer by @ssssnow in https://github.com/sgl-project/sglang/pull/9149
[PD] feat: mooncake use batch reg/dereg by @stmatengss in https://github.com/sgl-project/sglang/pull/8910
Support FA3 backend for gpt-oss by @ispobock in https://github.com/sgl-project/sglang/pull/9028
chore: bump v0.5.0rc1 by @zhyncs in https://github.com/sgl-project/sglang/pull/9069
[Model] Support Qwen3ForSequenceClassification for Qwen3-Embed Model by @nysa-liu in https://github.com/sgl-project/sglang/pull/7957
Swap xeon ci to gnr server by @DiweiSun in https://github.com/sgl-project/sglang/pull/9042
Clean up allocators by @merrymercy in https://github.com/sgl-project/sglang/pull/9134
[Generative Score API] Optimization to Remove Decode. by @sundar24295s in https://github.com/sgl-project/sglang/pull/8840
Fix broken trtllm_mha attn backend with gpt-oss by @nvcastet in https://github.com/sgl-project/sglang/pull/9161
Replace sglang.srt.layers.quantization.scalar_types with sgl_kernel.scalar_type by @Hongbosherlock in https://github.com/sgl-project/sglang/pull/8951
[AMD] Update fallback images for AMD CI by @michael-amd in https://github.com/sgl-project/sglang/pull/9159
[Bugfix] Avoid unnecessary reduce-scatter call in prepare_mlp by @changhuaixin in https://github.com/sgl-project/sglang/pull/9169
Fix docker container DeepEP error on Blackwell by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9171
[DP Attention] Refactor: adding some utility functions by @ch-wan in https://github.com/sgl-project/sglang/pull/9136
Faster weight processing (trtllm-gen moe nvfp4) by @aleozlx in https://github.com/sgl-project/sglang/pull/9162
Feature: support qwen and llama4 reducescatter for dp attention padding by @Misaka9468 in https://github.com/sgl-project/sglang/pull/9101
fix io group by @pansicheng in https://github.com/sgl-project/sglang/pull/9154
[Perf] Tunings for SM100 FP8 CUTLASS kernel by @hhzguo in https://github.com/sgl-project/sglang/pull/8818
Add A800 fused MoE kernel tuning configs for GLM4.5 and GLM4.5-Air by @lambert0312 in https://github.com/sgl-project/sglang/pull/8808
Add H200 fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct by @forestlee95 in https://github.com/sgl-project/sglang/pull/8852
Add Triton Fused MoE kernel config for E=16 on B200 by @b8zhong in https://github.com/sgl-project/sglang/pull/7004
Add H20 fused MoE kernel configs for Dpsk & Qwen3 by @M0gician in https://github.com/sgl-project/sglang/pull/7631
Add H200 fused MoE kernel configs for DeepSeek-V3 in triton 3.3.1 by @junliu-mde in https://github.com/sgl-project/sglang/pull/7687
add w8a8-fp8-block-wise H20-3e triton config by @sleepcoo in https://github.com/sgl-project/sglang/pull/8018
fix: zero_init buffer by @yyihuang in https://github.com/sgl-project/sglang/pull/9065
[2/n]decouple quantization implementation from vLLM dependency by @AniZpZ in https://github.com/sgl-project/sglang/pull/8112
chore: bump sgl-kernel v0.3.5 by @zhyncs in https://github.com/sgl-project/sglang/pull/9185
[sgl-kernel] 1/N Refactor sglang cutlass 3x - gemm fp8 blockwise sm90 by @yuan-luo in https://github.com/sgl-project/sglang/pull/8913
[sgl-kernel] Support FlashInfer top_k_top_p_sampling_from_logits by @yuan-luo in https://github.com/sgl-project/sglang/pull/9060
refine mxfp4 shuffling log by @BBuf in https://github.com/sgl-project/sglang/pull/9194
[4/n]decouple quantization implementation from vLLM dependency by @Hongbosherlock in https://github.com/sgl-project/sglang/pull/9191
feat: Add model version tracking with API endpoints and response metadata by @yitianlian in https://github.com/sgl-project/sglang/pull/8795
[VLM] Improving multimodal tensor hash kernel by @adarshxs in https://github.com/sgl-project/sglang/pull/9008
chore: upgrade transformers 4.55.2 by @zhyncs in https://github.com/sgl-project/sglang/pull/9197
feat: update model config by @zhyncs in https://github.com/sgl-project/sglang/pull/9202
chore: bump v0.5.0rc2 by @zhyncs in https://github.com/sgl-project/sglang/pull/9203
feat: add fused moe config for Qwen3-235B-A22B-FP8 on B200 by @zixuanzhang226 in https://github.com/sgl-project/sglang/pull/9204
[typo fix] Fix a typo in communicator.py by @LPhgh in https://github.com/sgl-project/sglang/pull/9183
Minor fix docker container DeepEP on multi platforms by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9205
fix: fix unsupported palette mode of images in bench_serving for mmmu by @mickqian in https://github.com/sgl-project/sglang/pull/9206
[6/N] MoE Refactor: Cleanup MoE-related configs by @ch-wan in https://github.com/sgl-project/sglang/pull/8849
use fast math for per_token_group_quant_8bit. by @strgrb in https://github.com/sgl-project/sglang/pull/9177
feat: remove sm75 by @zhyncs in https://github.com/sgl-project/sglang/pull/9207
feat(hicache-3fs): 3FS-SGLang Hierarchical Cache Deployment Guide by @hzh0425 in https://github.com/sgl-project/sglang/pull/9213
fix: the store_dtype typo for ascend mla by @shilinlee in https://github.com/sgl-project/sglang/pull/9208
Fix the deprecation warning for enable_flashinfer_mxfp4_moe by @ch-wan in https://github.com/sgl-project/sglang/pull/9214
Tiny update tmux history limit on dev container by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9218
[Eagle Warning fix] replace the deprecated 'and' with & by @XucSh in https://github.com/sgl-project/sglang/pull/9215
[Misc] feat: Deepgemm update for sgl-kernel by @FlamingoPg in https://github.com/sgl-project/sglang/pull/8790
Fp4 MOE quant kernel optimization by @jy-song-hub in https://github.com/sgl-project/sglang/pull/8777
[CI] Fix sgl-router disaggregation test by @ShangmingCai in https://github.com/sgl-project/sglang/pull/9222
Cleanup MoE Refactor by @ch-wan in https://github.com/sgl-project/sglang/pull/9223
chore: bump sgl-kernel v0.3.6 by @zhyncs in https://github.com/sgl-project/sglang/pull/9220
Optional extension for green context by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9231
[router] allow more health check configuration by @slin1237 in https://github.com/sgl-project/sglang/pull/9198
[router] clean up lint warnings with clippy execution by @jeffdn in https://github.com/sgl-project/sglang/pull/9201
[router] preserve original worker response header in router by @slin1237 in https://github.com/sgl-project/sglang/pull/9236
chore(docker): update sgl_kernel version to 0.3.6 in Dockerfile.gb200 by @ishandhanani in https://github.com/sgl-project/sglang/pull/9243
[AMD] Expand test coverage for AMD CI and enable apply_token_bitmask_inplace_cuda in sgl-kernel by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/8268
Fix nan value generated after custom all reduce by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/8663
Revert "chore(docker): update sgl_kernel version to 0.3.6 in Dockerfi… by @zhyncs in https://github.com/sgl-project/sglang/pull/9246
Revert "chore: bump sgl-kernel v0.3.6 (#9220)" by @zhyncs in https://github.com/sgl-project/sglang/pull/9247
Add fp4 quantize before all-gather for Flashinfer cutlass MoE DP (max throughput) by @trevor-m in https://github.com/sgl-project/sglang/pull/7667
Fix DP load for embedding by @b8zhong in https://github.com/sgl-project/sglang/pull/9165
[CI] add deepseek w4a8 test on h20 ci by @HanHan009527 in https://github.com/sgl-project/sglang/pull/7758
Fix Custom All Reduce CI job. by @saienduri in https://github.com/sgl-project/sglang/pull/9258
[feature] Ascend NPU graph support by @VDV1985 in https://github.com/sgl-project/sglang/pull/8027
fix unexcepted answer in EAGLE mode by @zyksir in https://github.com/sgl-project/sglang/pull/9252
[PD] Support PD disaggregation with Prefill PP by @ShangmingCai in https://github.com/sgl-project/sglang/pull/8846
Combine fp4.py and mxfp4.py into one file and support dynamic mxfp4 quantization in mxfp4.py by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/9049
Bug fix: use correct mm_items in embed_mm_inputs by @byjiang1996 in https://github.com/sgl-project/sglang/pull/8893
ci: simplify multi-modality tests by using mixins by @mickqian in https://github.com/sgl-project/sglang/pull/9006
[Bugfix] Change vLLM install order & Add A2 support by @iforgetmyname in https://github.com/sgl-project/sglang/pull/9232
[router] fix pd prefill http request complinace issue by @slin1237 in https://github.com/sgl-project/sglang/pull/9237
Quick Fix GLM by @hebiao064 in https://github.com/sgl-project/sglang/pull/9264
model: support nvidia/Llama-3_3-Nemotron-Super-49B-v1 by @netanel-haber in https://github.com/sgl-project/sglang/pull/9067
from python.sglang.srt -> from sglang.srt by @netanel-haber in https://github.com/sgl-project/sglang/pull/9268
Revert "[Misc] feat: Deepgemm update for sgl-kernel (#8790)" to fix kernel CI by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9260
[router] add cargo clippy in CI and fix-up linting errors by @jeffdn in https://github.com/sgl-project/sglang/pull/9242
[chore] Clean up redundant lora_weight_names concept to simplify code by @lifuhuang in https://github.com/sgl-project/sglang/pull/9131
Fix swa eagle verify accuracy for Triton backend by @ispobock in https://github.com/sgl-project/sglang/pull/9279
Fix memory pool leak error by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9271
[fix]: fix cutlass moe ut and and Opt H20 cutlass groupGemm performance by @kousakawang in https://github.com/sgl-project/sglang/pull/9272
Tiny make fp4 moe method parameters more static by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8520
[router] introduce prefill response draining for http compliance by @slin1237 in https://github.com/sgl-project/sglang/pull/9281
[CPU] Fix TP padding issue on Phi-4 by @blzheng in https://github.com/sgl-project/sglang/pull/8289
chore: bump sgl-kernel v0.3.6.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/9286
[router] introducing tokenizer trait by @slin1237 in https://github.com/sgl-project/sglang/pull/9287
Set the default attention backend for GLM-4.5v to fa3 by @zifeitong in https://github.com/sgl-project/sglang/pull/9245
[Fix] Add undefined update_tensor_inplace function by @b8zhong in https://github.com/sgl-project/sglang/pull/6307
[router] tokenizer factory, hf tokenizer, and stop sequence detector by @slin1237 in https://github.com/sgl-project/sglang/pull/9293
Fix triton_fused_moe unit test and benchmark by @yuan-luo in https://github.com/sgl-project/sglang/pull/9276
Further fix memory pool leak error by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9298
[router] add tokenizer metrics by @slin1237 in https://github.com/sgl-project/sglang/pull/9307
[router] add reasoning parser base structure by @slin1237 in https://github.com/sgl-project/sglang/pull/9310
Minor style fixes for sgl-kernel by @merrymercy in https://github.com/sgl-project/sglang/pull/9289
[fix] fix enable_pdl for blackwell by @Alcanderian in https://github.com/sgl-project/sglang/pull/9011
Modelopt quant config adaptation by @Edwardf0t1 in https://github.com/sgl-project/sglang/pull/8829
should return invalide request for empty prompt by @gongwei-130 in https://github.com/sgl-project/sglang/pull/9315
[MISC] use dynamic choices for tool-call-parser argument by @key4ng in https://github.com/sgl-project/sglang/pull/9316
[Docs] Correct and clarify notes in Engine docstring by @JiangJiaWei1103 in https://github.com/sgl-project/sglang/pull/9313
upgrade xgrammar 0.1.23 and openai-harmony 0.0.4 by @Swipe4057 in https://github.com/sgl-project/sglang/pull/9284
[PD] Propagate internal server errors from aborted requests to clients instead of blindly returning 200's by @datdo-msft in https://github.com/sgl-project/sglang/pull/8936
[GLM4.1V and GLM4.5V] Add vision transformer num_dummy_head support: max tp=4 -> max tp=8 by @byjiang1996 in https://github.com/sgl-project/sglang/pull/9059
[AMD] Reorganize hip-related header files in sgl-kernel by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/9320
Tiny fix CI by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9306
[router] Add spec for sglang scheduler by @CatherineSue in https://github.com/sgl-project/sglang/pull/9322
support for interns1-mini by @CUHKSZzxy in https://github.com/sgl-project/sglang/pull/9299
[Bug] Fix input arguments of flashinfer_trtllm_moe by @JeremieMelo in https://github.com/sgl-project/sglang/pull/9317
[router]restructure protocol modules for better organization by @key4ng in https://github.com/sgl-project/sglang/pull/9321
Add CMakeLists.txt binary_dir by @EduardDurech in https://github.com/sgl-project/sglang/pull/7019
enable marlin fp8 blockwise by @qeternity in https://github.com/sgl-project/sglang/pull/8990
docs: fix spec by @zhyncs in https://github.com/sgl-project/sglang/pull/9326
[Minor] Fix the style of sgl-kernel by @merrymercy in https://github.com/sgl-project/sglang/pull/9332
[Bugfix] fix kv buffer register & dp attention & deepepmoe by @chenxu140 in https://github.com/sgl-project/sglang/pull/9327
Revert "[feature] Ascend NPU graph support (#8027)" by @iforgetmyname in https://github.com/sgl-project/sglang/pull/9348
[router] add dsr1, kimi, and qwen reasoning parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9353
fix: enable multi-GPU Triton fused MoE tuning by @mpashkovskiy in https://github.com/sgl-project/sglang/pull/6295
[router] add tiktokenizer and sequence in router by @slin1237 in https://github.com/sgl-project/sglang/pull/9354
[CI] Fix lint issues by @CatherineSue in https://github.com/sgl-project/sglang/pull/9361
[Router] Add validation module for API parameters by @key4ng in https://github.com/sgl-project/sglang/pull/9335
[router] adds reasoning parser pooling and thread-safe by @slin1237 in https://github.com/sgl-project/sglang/pull/9360
[router] Implement gRPC SGLangSchedulerClient by @CatherineSue in https://github.com/sgl-project/sglang/pull/9364
[router] add tokenizer chat template support by @slin1237 in https://github.com/sgl-project/sglang/pull/9370
[router] Implement OpenAI Responses API specification by @key4ng in https://github.com/sgl-project/sglang/pull/9367
Fix mini lb timeout issue by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9369
Fix triton backend eagle illegal memory access by @ispobock in https://github.com/sgl-project/sglang/pull/9344
Fix gpt-oss response api streaming issue by @key4ng in https://github.com/sgl-project/sglang/pull/9368
[feature] Rework Ascend NPU graph support by @iforgetmyname in https://github.com/sgl-project/sglang/pull/9350
[minor] Sync style changes by @merrymercy in https://github.com/sgl-project/sglang/pull/9376
[readme] Add SGLang x AMD SF meetup information by @wisclmy0611 in https://github.com/sgl-project/sglang/pull/9380
[CI] Fix disaggregation failure tolerance CI by @ShangmingCai in https://github.com/sgl-project/sglang/pull/9378
[Docs] Update contribution guide by @merrymercy in https://github.com/sgl-project/sglang/pull/9383
Revert "[feature] Rework Ascend NPU graph support" by @iforgetmyname in https://github.com/sgl-project/sglang/pull/9385
Reduce overhead for fa by not calling heavy CUDA property check by @oraluben in https://github.com/sgl-project/sglang/pull/7375
Add PDL support for quant kernel and rope kernel by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9106
Fix the --allow-auto-truncate argument in tokenizer manager. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9391
Refactor allreduce add rmsnorm pattern by @BBuf in https://github.com/sgl-project/sglang/pull/9278
[2/2] Fuse routed scaling factor into select_experts by @trevor-m in https://github.com/sgl-project/sglang/pull/8690
Fix FlashInfer GPU <-> CPU sync by @thecodingwizard in https://github.com/sgl-project/sglang/pull/9409
Support pinning adapter via server args. by @lifuhuang in https://github.com/sgl-project/sglang/pull/9249
Fix incorrect logic in chat template handling. by @lifuhuang in https://github.com/sgl-project/sglang/pull/9336
Support DP attention with GPT-OSS by @nvcastet in https://github.com/sgl-project/sglang/pull/9359
Fixed the issue where eagle3 TPOT was not as good as without eagle3. by @jiapingW in https://github.com/sgl-project/sglang/pull/9404
fix: InternS1 don't recognize image, updates image token for InternVL processor by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9381
misc: parse bench_serving result as markdown table by @mickqian in https://github.com/sgl-project/sglang/pull/9377
Add support for Qwen3-seq-cls by @nathanrchn in https://github.com/sgl-project/sglang/pull/9357
Support trtllm_allreduce_fusion in flashinfer for cuda<12.8 by @strgrb in https://github.com/sgl-project/sglang/pull/9339
[router] Add IGW (Inference Gateway) Feature Flag by @key4ng in https://github.com/sgl-project/sglang/pull/9371
[router] add tokenizer integration test with real mini tokenizer by @CatherineSue in https://github.com/sgl-project/sglang/pull/9413
[router] add glm and step3 reasoning parser by @CatherineSue in https://github.com/sgl-project/sglang/pull/9415
Fix max_seq_len_k in trtllm_mha attention backend by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/9416
Fix biased_grouped_topk_cpu by @CaoE in https://github.com/sgl-project/sglang/pull/9420
[PD] Fix nvlink transport accuracy through transferring metadata with tcp by @ShangmingCai in https://github.com/sgl-project/sglang/pull/9261
[bug] fix errors related to context length in SD by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9388
feat: Add Triton fallback option and SM120 MoE configs for FP8 models by @voipmonitor in https://github.com/sgl-project/sglang/pull/9251
[feature] Ascend NPU graph support by @VDV1985 in https://github.com/sgl-project/sglang/pull/9399
Fix FP4 inference corruption issue in glm4.5-air model by @Azure-Tang in https://github.com/sgl-project/sglang/pull/9346
Fix tiny misalign with previous truncation setting in tokenizer_manager by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9430
[NVIDIA] Fix trtllm fp4 moe backend when used in MTP by @kaixih in https://github.com/sgl-project/sglang/pull/9384
Enables speculative decoding for the trtllm_mla attention backend by @pranavm-nvidia in https://github.com/sgl-project/sglang/pull/9238
ci: enhance xeon ci by @DiweiSun in https://github.com/sgl-project/sglang/pull/9395
[Bug] Fix w4afp8 moe kernel by @yuhyao in https://github.com/sgl-project/sglang/pull/9392
Refactor weight offloading logic by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8521
Fix quant kernel test errors and benchmark wrong output speeds by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7604
[fix] Fix mxfp4 weight loading bug with TP sharding in GPT-OSS by @hlu1 in https://github.com/sgl-project/sglang/pull/9433
[router] add tokenizer benchmark by @slin1237 in https://github.com/sgl-project/sglang/pull/9427
[5/n]decouple quantization implementation from vLLM dependency by @Hongbosherlock in https://github.com/sgl-project/sglang/pull/9454
accomendate reasoning_effort set in chat_template_kwargs by @gongwei-130 in https://github.com/sgl-project/sglang/pull/9458
fix: should return a invalid request response when schema missing by @gongwei-130 in https://github.com/sgl-project/sglang/pull/9461
fix: support fb fp8 by @zhyncs in https://github.com/sgl-project/sglang/pull/9462
Add deepseek v3.1 thinking parser support and update docs by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9464
feat: add fused moe config for GLM-4.5-Air-FP8 on B200 by @zixuanzhang226 in https://github.com/sgl-project/sglang/pull/9463
[FA3] Init Spec Page Table only when Spec is enabled to save ~40MB by @hebiao064 in https://github.com/sgl-project/sglang/pull/9455
fix: tmp revert gpt oss tp sharding on hopper by @zhyncs in https://github.com/sgl-project/sglang/pull/9469
feat: update auto_choose_speculative_params by @zhyncs in https://github.com/sgl-project/sglang/pull/9470
Revert "bugfix: Fix output_ids extraction in detokenizer_manager" by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9467
Update reasoning parser doc by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9468
Add Support for Page Size greater than 1 for Flashinfer MLA Backend by @pavanimajety in https://github.com/sgl-project/sglang/pull/8593
[AMD] Remove the deprecated C10_WARP_SIZE by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/9356
Support MHA with chunked prefix cache for flashinfer/flashmla backend, support page size > 1 for MHA chunked prefix by @xu-yfei in https://github.com/sgl-project/sglang/pull/8616
[router] remove all tokenizer metrics for performance by @CatherineSue in https://github.com/sgl-project/sglang/pull/9474
[code clean] add H20 cutlass groupGemm default config by @kousakawang in https://github.com/sgl-project/sglang/pull/9333
[docs]: fix reasoning context in docs by @zhaochenyang20 in https://github.com/sgl-project/sglang/pull/9483
[Docs]Update reasoning parser doc & fix outdated link by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9492
[router] add tool parser base structure and partial json parser by @CatherineSue in https://github.com/sgl-project/sglang/pull/9482
[router] fix router load guard tracking for streaming by @slin1237 in https://github.com/sgl-project/sglang/pull/9491
torch.compile() mrope by @timmy-feng in https://github.com/sgl-project/sglang/pull/9487
Add trtllm_mla and cutlass_mla for ragged fmha for chunked prefill by @elfiegg in https://github.com/sgl-project/sglang/pull/9480
chore: bump sgl-kernel v0.3.6.post2 by @zhyncs in https://github.com/sgl-project/sglang/pull/9475
Update docker file for supporting PD-Disaggregation on MI300x by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/9494
[Docs] Add doc and quick demo for gpt-oss responses api & buildin tools by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/9497
Support speculative decoding in the trtllm_mha attention backend by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/9331
minor: determine mm attn backend based on platforms by @mickqian in https://github.com/sgl-project/sglang/pull/9303
Disable torch.compile for get_last_loc_large_page_size_large_top_k by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/9507
[bugfix] Make --enable-hierarchical-cache and --disable-radix-cache mutually exclusive by @XucSh in https://github.com/sgl-project/sglang/pull/9452
3fs zerocopy by @pansicheng in https://github.com/sgl-project/sglang/pull/9109
[HiCacheStorage] backup optimization for MLA model by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/8865
Use Tensor Core Decode when gqa group size >= 4 by @Edenzzzz in https://github.com/sgl-project/sglang/pull/8624
[router] tokenizer arch doc by @slin1237 in https://github.com/sgl-project/sglang/pull/9513
[MTP] Force greedy sampling on AMD by @datdo-msft in https://github.com/sgl-project/sglang/pull/9127
[router] add json tool parser by @slin1237 in https://github.com/sgl-project/sglang/pull/9516
[NVIDA] [1/N] Nvfp4 Masked Gemm: Add quant op for the flashinfer grouped gemm by @kaixih in https://github.com/sgl-project/sglang/pull/9200
[AMD] Fix Llama 4 FP8 accuracy issues on MI300X by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/7699
Add Qwen3-30B-A3B-Thinking-2507 support on AMD GPUs. by @sogalin in https://github.com/sgl-project/sglang/pull/9456
[router] Move all protocols to spec.rs file by @key4ng in https://github.com/sgl-project/sglang/pull/9519
[router] ignore client error when record failure in pd_router by @Bruce-x-1997 in https://github.com/sgl-project/sglang/pull/9503
Add support for extensions of interface and pre-registrations to NIXL HiCache by @mkhazraee in https://github.com/sgl-project/sglang/pull/9211
Support GC Freezing to improve latency & throughput by @chanh in https://github.com/sgl-project/sglang/pull/9241
Add enable_flashinfer_mxfp4_bf16_moe for higher precision and slower moe backend by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9004
[benchmark] Add benchmark scripts for ceval and boolq by @yuxingcyx in https://github.com/sgl-project/sglang/pull/8946
fix: blackwell dsv3 fp8 issue temporary solution by @zhyncs in https://github.com/sgl-project/sglang/pull/9530
tool-call(dsv3): Improve deepseek-v3 chat template and tool_choice = required by @CatherineSue in https://github.com/sgl-project/sglang/pull/9525
[fix] Fix mxfp4 triton MoE tp bug by @hlu1 in https://github.com/sgl-project/sglang/pull/9473
Overlapped weight offload by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8034
Tiny make device_loading_context more static by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9478
Partially unify triton per token group quant kernels by @fzyzcjy in https://github.com/sgl-project/sglang/pull/9485
feat(hicache): Supports 3fs-hicache compatibility with dp-attention by @hzh0425 in https://github.com/sgl-project/sglang/pull/9372
Update grok.py and tiktoken tokenizer by @merrymercy in https://github.com/sgl-project/sglang/pull/9532
Release 0.5.1 by @merrymercy in https://github.com/sgl-project/sglang/pull/9533

New Contributors

@sighingnow made their first contribution in https://github.com/sgl-project/sglang/pull/8611
@vvenkates27 made their first contribution in https://github.com/sgl-project/sglang/pull/8488
@farazkh80 made their first contribution in https://github.com/sgl-project/sglang/pull/8632
@yrk111222 made their first contribution in https://github.com/sgl-project/sglang/pull/8083
@pkking made their first contribution in https://github.com/sgl-project/sglang/pull/8270
@ZacWang made their first contribution in https://github.com/sgl-project/sglang/pull/8664
@lbh2001 made their first contribution in https://github.com/sgl-project/sglang/pull/8618
@wenchen76 made their first contribution in https://github.com/sgl-project/sglang/pull/8512
@WANG-GH made their first contribution in https://github.com/sgl-project/sglang/pull/7379
@shenoyvvarun made their first contribution in https://github.com/sgl-project/sglang/pull/8683
@17Reset made their first contribution in https://github.com/sgl-project/sglang/pull/8547
@TianQiLin666666 made their first contribution in https://github.com/sgl-project/sglang/pull/8678
@YyWangCS made their first contribution in https://github.com/sgl-project/sglang/pull/8733
@azhurkevich made their first contribution in https://github.com/sgl-project/sglang/pull/8552
@yuhyao made their first contribution in https://github.com/sgl-project/sglang/pull/8596
@House-West made their first contribution in https://github.com/sgl-project/sglang/pull/8144
@ZhengWG made their first contribution in https://github.com/sgl-project/sglang/pull/8292
@htiennv made their first contribution in https://github.com/sgl-project/sglang/pull/8698
@triple-Mu made their first contribution in https://github.com/sgl-project/sglang/pull/8799
@tonyluj made their first contribution in https://github.com/sgl-project/sglang/pull/8971
@maocheng23 made their first contribution in https://github.com/sgl-project/sglang/pull/8371
@cctry made their first contribution in https://github.com/sgl-project/sglang/pull/8245
@chi2liu made their first contribution in https://github.com/sgl-project/sglang/pull/8973
@Hangzhi made their first contribution in https://github.com/sgl-project/sglang/pull/9080
@jinmingyi1998 made their first contribution in https://github.com/sgl-project/sglang/pull/8866
@Makcum888e made their first contribution in https://github.com/sgl-project/sglang/pull/8293
@ovowei made their first contribution in https://github.com/sgl-project/sglang/pull/8766
@xxrjun made their first contribution in https://github.com/sgl-project/sglang/pull/9075
@yichaolemon made their first contribution in https://github.com/sgl-project/sglang/pull/9096
@JeremieMelo made their first contribution in https://github.com/sgl-project/sglang/pull/9014
@ichernob made their first contribution in https://github.com/sgl-project/sglang/pull/8619
@nysa-liu made their first contribution in https://github.com/sgl-project/sglang/pull/7957
@changhuaixin made their first contribution in https://github.com/sgl-project/sglang/pull/9169
@aleozlx made their first contribution in https://github.com/sgl-project/sglang/pull/9162
@Misaka9468 made their first contribution in https://github.com/sgl-project/sglang/pull/9101
@hhzguo made their first contribution in https://github.com/sgl-project/sglang/pull/8818
@forestlee95 made their first contribution in https://github.com/sgl-project/sglang/pull/8852
@LPhgh made their first contribution in https://github.com/sgl-project/sglang/pull/9183
@shilinlee made their first contribution in https://github.com/sgl-project/sglang/pull/9208
@jy-song-hub made their first contribution in https://github.com/sgl-project/sglang/pull/8777
@jeffdn made their first contribution in https://github.com/sgl-project/sglang/pull/9201
@VDV1985 made their first contribution in https://github.com/sgl-project/sglang/pull/8027
@netanel-haber made their first contribution in https://github.com/sgl-project/sglang/pull/9067
@kousakawang made their first contribution in https://github.com/sgl-project/sglang/pull/9272
@gongwei-130 made their first contribution in https://github.com/sgl-project/sglang/pull/9315
@datdo-msft made their first contribution in https://github.com/sgl-project/sglang/pull/8936
@CUHKSZzxy made their first contribution in https://github.com/sgl-project/sglang/pull/9299
@EduardDurech made their first contribution in https://github.com/sgl-project/sglang/pull/7019
@chenxu140 made their first contribution in https://github.com/sgl-project/sglang/pull/9327
@mpashkovskiy made their first contribution in https://github.com/sgl-project/sglang/pull/6295
@oraluben made their first contribution in https://github.com/sgl-project/sglang/pull/7375
@thecodingwizard made their first contribution in https://github.com/sgl-project/sglang/pull/9409
@jiapingW made their first contribution in https://github.com/sgl-project/sglang/pull/9404
@nathanrchn made their first contribution in https://github.com/sgl-project/sglang/pull/9357
@CaoE made their first contribution in https://github.com/sgl-project/sglang/pull/9420
@voipmonitor made their first contribution in https://github.com/sgl-project/sglang/pull/9251
@Azure-Tang made their first contribution in https://github.com/sgl-project/sglang/pull/9346
@pranavm-nvidia made their first contribution in https://github.com/sgl-project/sglang/pull/9238
@hlu1 made their first contribution in https://github.com/sgl-project/sglang/pull/9433
@timmy-feng made their first contribution in https://github.com/sgl-project/sglang/pull/9487
@Bruce-x-1997 made their first contribution in https://github.com/sgl-project/sglang/pull/9503
@mkhazraee made their first contribution in https://github.com/sgl-project/sglang/pull/9211
@yuxingcyx made their first contribution in https://github.com/sgl-project/sglang/pull/8946

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.4.10...v0.5.1

Open source →

0.4.1031 Jul 2025

This is a regular release with many new optimizations, features, and fixes. Please checkout the following exciting roadmaps and blogs

Highlights

This is a regular release with many new optimizations, features, and fixes. Please checkout the following exciting roadmaps and blogs

Please check the 2025 H2 roadmap https://github.com/sgl-project/sglang/issues/7736
GLM-4.5 Meets SGLang: Reasoning, Coding, and Agentic Abilities https://lmsys.org/blog/2025-07-31-glm4-5/
SpecForge: Accelerating Speculative Decoding Training for SGLang https://lmsys.org/blog/2025-07-25-spec-forge/
Deploying Kimi K2 with PD Disaggregation and Large-Scale Expert Parallelism on 128 H200 GPUs https://lmsys.org/blog/2025-07-20-k2-large-scale-ep/
Accelerating SGLang with Multiple Token Prediction https://lmsys.org/blog/2025-07-17-mtp/
How to support new VLMs into SGLang: A Case Study with NVILA https://lmsys.org/blog/2025-07-16-nvila/
Cost Effective Deployment of DeepSeek R1 with Intel® Xeon® 6 CPU on SGLang https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/
slime: An SGLang-Native Post-Training Framework for RL Scaling https://lmsys.org/blog/2025-07-09-slime/

What's Changed

[AMD] add aiter fused moe in DeepEP path by @alexsun07 in https://github.com/sgl-project/sglang/pull/7268
enable aiter_biased_grouped_topk kernel by @valarLip in https://github.com/sgl-project/sglang/pull/7423
[PD Disaggregation] replace transfer with batch transfer for better performance by @ssssnow in https://github.com/sgl-project/sglang/pull/7236
Remove cumsum_buffer initilization by @ispobock in https://github.com/sgl-project/sglang/pull/7439
[benchmark] fbgemm benchmark support bandwidth report and support fbgemm_cutlass_gmm by @BBuf in https://github.com/sgl-project/sglang/pull/7422
Support multi-thread model weight loading by @xianzhiT in https://github.com/sgl-project/sglang/pull/7277
[PD] NIXL: Register kv args in advance and cleanup finished requests by @trevor-m in https://github.com/sgl-project/sglang/pull/6717
fix: Add --model as an alias for --model-path in server_args by @CatherineSue in https://github.com/sgl-project/sglang/pull/7505
misc: Improvement to serving_chat.py and add more ut by @CatherineSue in https://github.com/sgl-project/sglang/pull/7489
Fuse sorted_token_ids padding to moe_align_block_size kernel by @ispobock in https://github.com/sgl-project/sglang/pull/7437
[OAI] patch origin request_id logic by @whybeyoung in https://github.com/sgl-project/sglang/pull/7508
[PD][Spec] Fix hidden state transfer for spec decode by @ShangmingCai in https://github.com/sgl-project/sglang/pull/7516
EPLB support for MTP by @yilian49 in https://github.com/sgl-project/sglang/pull/7510
clean duplicate code by @habaohaba in https://github.com/sgl-project/sglang/pull/7512
[ci] add router benchmark script and CI by @slin1237 in https://github.com/sgl-project/sglang/pull/7498
fix: force synchronization between TP workers when update_weights by @dangkai4u in https://github.com/sgl-project/sglang/pull/6626
[CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model by @chunyuan-w in https://github.com/sgl-project/sglang/pull/6641
[CI] Upgrade mooncake to v0.3.4.post2 to fix potential slice failed bug by @ShangmingCai in https://github.com/sgl-project/sglang/pull/7522
npu fused op by @ll819214 in https://github.com/sgl-project/sglang/pull/7386
feat: send kvmetrics from sglang scheduler by @zixuanzhang226 in https://github.com/sgl-project/sglang/pull/6721
[PD] Add different TP sizes support for no-MLA models by @Hongbosherlock in https://github.com/sgl-project/sglang/pull/6793
enable aiter fp8 blockscale quant by @valarLip in https://github.com/sgl-project/sglang/pull/7520
take aiter get_rope back by @valarLip in https://github.com/sgl-project/sglang/pull/7521
Fix typo of flash_cache by @hebiao064 in https://github.com/sgl-project/sglang/pull/7513
feat: add return hidden_states at async generation by @yyihuang in https://github.com/sgl-project/sglang/pull/7507
minor: 'role' must be system/assistant/tool, but case insensitive for now by @minleminzui in https://github.com/sgl-project/sglang/pull/7499
Fix FP8 KV Cache Support in FA3 Backend by @guoyuhong in https://github.com/sgl-project/sglang/pull/7148
Fix gathered_buffer issues in tbo by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/7531
[PD] Raise error for incompatible mooncake version and some minor fixes by @ShangmingCai in https://github.com/sgl-project/sglang/pull/7527
[CMake] Fix sgl-kernel CMakeLists for Blackwell by @MasterJH5574 in https://github.com/sgl-project/sglang/pull/7543
Add Tencent HunYuanMoEV1 model support by @mpjlu in https://github.com/sgl-project/sglang/pull/7549
Update seed in CPU UTs to avoid flaky failure with single test by @yanbing-j in https://github.com/sgl-project/sglang/pull/7544
chore: improve ci bug reporting by @mickqian in https://github.com/sgl-project/sglang/pull/7542
chore: remove vlm unnecessary import by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7541
chore: bump v0.4.8.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/7559
[PD][NIXL] Set is_sorted=False to fix NIXL_ERR_NOT_FOUND by @trevor-m in https://github.com/sgl-project/sglang/pull/7330
[Fix] incorrect assert in EPLB by @ch-wan in https://github.com/sgl-project/sglang/pull/7575
Updates Gemma3n MLP layer to adapt latest transformers version by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7573
Fix MTP error when enabling two-batch overlap by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7569
Add e2e test for multi instance multi stage memory release/resume occupuation by @MrAta in https://github.com/sgl-project/sglang/pull/7208
[CI] Add CI Testing for Prefill-Decode Disaggregation with Router by @key4ng in https://github.com/sgl-project/sglang/pull/7540
Updates transformers and timm dependencies by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7577
feat: support compatibility between MTP and two-batch-overlap by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/7225
Move multimodal processors into a separate folder by @merrymercy in https://github.com/sgl-project/sglang/pull/7581
Fix broken CI TestVILAServer by @lifuhuang in https://github.com/sgl-project/sglang/pull/7610
[router] add centralized configuration module for sgl-router by @slin1237 in https://github.com/sgl-project/sglang/pull/7588
Fix: Minicpm by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7612
Hybrid kv cache for LLaMA4 by @tarinkk in https://github.com/sgl-project/sglang/pull/6563
[CPU] add optimizations for INT8 and FP8 DeepSeek by @chunyuan-w in https://github.com/sgl-project/sglang/pull/6769
Tiny add logs for expert location updater by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7308
Fix flakiness in LoRA batch test. by @lifuhuang in https://github.com/sgl-project/sglang/pull/7552
[BUG] fix local_rank in initialize_dp_attention by @TomQuartz in https://github.com/sgl-project/sglang/pull/7584
Support dynamic LoRA loading / unloading in engine/server API by @lifuhuang in https://github.com/sgl-project/sglang/pull/7446
[PD] Respect sampling_params.max_new_tokens when PD disaggregation is activated by @ShangmingCai in https://github.com/sgl-project/sglang/pull/7598
fix unit tests by @zhyncs in https://github.com/sgl-project/sglang/pull/7618
Let ep_scatter support arbitrary strides / ue8m0 format by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7309
Let EP prefill support new DeepGEMM by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7310
docs: add gb200 nvl72 and a16z grant by @zhyncs in https://github.com/sgl-project/sglang/pull/7620
Adds support for OpenAI chat completions API in bench_serving by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7036
[bugfix] Remove PR comment posting from Rust benchmark workflow by @slin1237 in https://github.com/sgl-project/sglang/pull/7625
[Minor] clean up multimodal processor and tokenizer manager by @merrymercy in https://github.com/sgl-project/sglang/pull/7624
Add dsv3 fused a gemm to sgl-kernel by @ispobock in https://github.com/sgl-project/sglang/pull/7630
Add @mickqian as the CODEOWNERS of multimodal by @merrymercy in https://github.com/sgl-project/sglang/pull/7636
Fix stream reasoning parser and Adds Kimi reasoning parser by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7432
Fix sgl-router startup crash by @finetunej in https://github.com/sgl-project/sglang/pull/7619
[bugfix] fix runtime dropping panic in editable by @slin1237 in https://github.com/sgl-project/sglang/pull/7628
Move files related to EPLB by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7580
[misc] reduce weird rope_scaling_factor warning by @Alcanderian in https://github.com/sgl-project/sglang/pull/7176
[AMD] Add unit-test-sgl-kernel-amd to AMD CI by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/7539
Update CODEOWNERS by @merrymercy in https://github.com/sgl-project/sglang/pull/7640
[EAGLE] remove a wrong adjustment for page_size > 1 & topk > 1 in server_args.py by @merrymercy in https://github.com/sgl-project/sglang/pull/7643
[CPU] add c++ kernel to bind CPU cores and memory node by @chunyuan-w in https://github.com/sgl-project/sglang/pull/7524
Improve streaming, log_level, memory report, weight loading, and benchmark script by @merrymercy in https://github.com/sgl-project/sglang/pull/7632
Add dsv3 router gemm kernel by @Fridge003 in https://github.com/sgl-project/sglang/pull/7627
chore: upgrade flashinfer v0.2.7 jit by @zhyncs in https://github.com/sgl-project/sglang/pull/7663
[doc] update lws doc for pd by @whybeyoung in https://github.com/sgl-project/sglang/pull/7318
Fix: sync prepare_fp8_layer_for_marlin with latest vllm changes by @narutolhy in https://github.com/sgl-project/sglang/pull/7648
Add small requirements for benchmark/parse_result tools by @BBuf in https://github.com/sgl-project/sglang/pull/7671
[CPU] remove process_group from inputs of shm_allreduce and shm_allgather by @chunyuan-w in https://github.com/sgl-project/sglang/pull/7486
chore: bump sgl-kernel v0.2.1 by @zhyncs in https://github.com/sgl-project/sglang/pull/7675
support llama4 eagle3 by @sleepcoo in https://github.com/sgl-project/sglang/pull/6985
Refactor mm processors and Enable mixed modality processing by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7629
upgrade sgl kernel to 0.2.1 for main by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/7676
add description for llama4 eagle3 by @yizhang2077 in https://github.com/sgl-project/sglang/pull/7688
fix(model loader): use safe_open to prevent file handle leaks. by @SimonCqk in https://github.com/sgl-project/sglang/pull/7684
chore: upgrade flashinfer v0.2.7.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/7698
Improve error handling for requests with unloaded LoRA path(s) by @lifuhuang in https://github.com/sgl-project/sglang/pull/7642
Apply dsv3_fused_a_gemm kernel by @ispobock in https://github.com/sgl-project/sglang/pull/7635
Fix GPTQMarlinMoE by @lkm2835 in https://github.com/sgl-project/sglang/pull/7697
[1/n] apply wna16marlin kernel in moe weight only quantization by @AniZpZ in https://github.com/sgl-project/sglang/pull/7683
Apply dsv3 router gemm kernel for deepseek-r1 fp4 by @Fridge003 in https://github.com/sgl-project/sglang/pull/7677
[AMD] Temporarily disable test_no_overlap_scheduler and test_vision_chunked_prefill by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/7717
[RL] add --skip-warmup by @zhuzilin in https://github.com/sgl-project/sglang/pull/7416
[RL] support update_weights_from_distributed with different group and multiple weights by @zhuzilin in https://github.com/sgl-project/sglang/pull/7292
[router] add --log-level to sgl-router by @zhuzilin in https://github.com/sgl-project/sglang/pull/6512
[b200] support trt-llm allreduce fuse rms_norm_add kernel by @BBuf in https://github.com/sgl-project/sglang/pull/7621
[CPU] Bind threads and numa node for each TP rank by @chunyuan-w in https://github.com/sgl-project/sglang/pull/6549
Support non-contiguous query input for extend/decode attention by @yanbing-j in https://github.com/sgl-project/sglang/pull/7462
Support updating weights at once by stopping all requests by @tianyuzhou95 in https://github.com/sgl-project/sglang/pull/6698
Fix num_tokens_pre_allocated in disaggregation log by @ZeldaHuang in https://github.com/sgl-project/sglang/pull/7714
[CPU] [sgl-kernel] set dispatch key of initialize to CatchAll by @chunyuan-w in https://github.com/sgl-project/sglang/pull/7734
[CPU] fix all_reduce and all_gather by @chunyuan-w in https://github.com/sgl-project/sglang/pull/6770
fix awq and dsv3 fused gemm compatible by @AniZpZ in https://github.com/sgl-project/sglang/pull/7735
[CI][Router] Fix bench_one_batch_server for pd router test by @ShangmingCai in https://github.com/sgl-project/sglang/pull/7731
Add CUTLASS FP8 Blockscale MoE kernel for Hopper architecture by @ayrnb in https://github.com/sgl-project/sglang/pull/7278
fix dsv3 fused proj check by @AniZpZ in https://github.com/sgl-project/sglang/pull/7738
Ascend attention backend(PA&MLA) by @ping1jing2 in https://github.com/sgl-project/sglang/pull/7722
[fix] fix dsv3_router_gemm filter by @Alcanderian in https://github.com/sgl-project/sglang/pull/7750
[CPU] refine CPU integration code by @chunyuan-w in https://github.com/sgl-project/sglang/pull/7647
[CPU] support the case where num_attention_heads or intermediate_size is not divisible by the TP size by @chunyuan-w in https://github.com/sgl-project/sglang/pull/6771
support qwen3 dense model dp attention by @yizhang2077 in https://github.com/sgl-project/sglang/pull/7681
[optimize] add two stream norm for qwen3 by @yizhang2077 in https://github.com/sgl-project/sglang/pull/7740
feat: use D2D instead of H2H in pp by @TianyuZhang1214 in https://github.com/sgl-project/sglang/pull/7673
[Bug] add flashinfer bool check for fusedmoe in Qwen moe models by @yilian49 in https://github.com/sgl-project/sglang/pull/7723
[fix] put cpu in the first priority in get_device() by @Alcanderian in https://github.com/sgl-project/sglang/pull/7752
[optimize] fuse renormalize into moe_topk_softmax by @yizhang2077 in https://github.com/sgl-project/sglang/pull/7744
chore: bump sgl-kernel 0.2.2 by @zhyncs in https://github.com/sgl-project/sglang/pull/7755
fix CI: update native api ipynb by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7754
fuse renormal into moe topk softmax kernel python code by @yizhang2077 in https://github.com/sgl-project/sglang/pull/7751
Remove type conversion and fix id map in topk by @ispobock in https://github.com/sgl-project/sglang/pull/7759
Add V2-lite model test by @yanbing-j in https://github.com/sgl-project/sglang/pull/7390
refactor llama4 dp attention logic by @yizhang2077 in https://github.com/sgl-project/sglang/pull/7729
fix(docs): fix the broken link in docs/references/production_metrics.md by @rudeigerc in https://github.com/sgl-project/sglang/pull/7741
[fix] update bench_speculative.py for compatibility by @yankay in https://github.com/sgl-project/sglang/pull/7764
Move mem_fraction_static adjustment for multimodal models to server_args.py & Fix session control & Other cleanups by @merrymercy in https://github.com/sgl-project/sglang/pull/7748
[RL] Add --nccl-port to prevent port conflict by @zhuzilin in https://github.com/sgl-project/sglang/pull/7418
[RL] add pause and continue generation for async rl training by @zhuzilin in https://github.com/sgl-project/sglang/pull/7419
[Fix] Alloc return type error by @Capronir in https://github.com/sgl-project/sglang/pull/7778
[feat] Support EAGLE3 for Qwen by @Ximingwang-09 in https://github.com/sgl-project/sglang/pull/7745
saving hidden_states.clone() by @ch-wan in https://github.com/sgl-project/sglang/pull/7705
[1/n]: add cutlass W4A8 moe kernel for hopper architecture by @yangsijia-serena in https://github.com/sgl-project/sglang/pull/7772
add model: qwen2-audio by @leng-yue in https://github.com/sgl-project/sglang/pull/7596
Optimize Hopper CUTLASS FP8 Blockwise Grouped GEMM Kernel in Small K Scenario by @HydraQYH in https://github.com/sgl-project/sglang/pull/7782
Embedding parallel by attn_tp by @MoonBall in https://github.com/sgl-project/sglang/pull/7623
fix: fix apply_shuffle_mul_sum by @mickqian in https://github.com/sgl-project/sglang/pull/7444
chore: bump sgl-kernel v0.2.3 by @zhyncs in https://github.com/sgl-project/sglang/pull/7784
fix: use nvidia-nccl-cu12 2.27.5 by @zhyncs in https://github.com/sgl-project/sglang/pull/7787
DP Attention with Auto DeepEP Dispatch by @ch-wan in https://github.com/sgl-project/sglang/pull/7222
chore: upgrade sgl-kernel v0.2.3 by @zhyncs in https://github.com/sgl-project/sglang/pull/7786
Fix incorrect spec_num_draft_tokens in draft_extend by @ch-wan in https://github.com/sgl-project/sglang/pull/7757
[fix] fix misusing of is_cuda by @Alcanderian in https://github.com/sgl-project/sglang/pull/7790
Add treemask mode to build_eagle_tree & release sgl-kernel 0.2.3 by @merrymercy in https://github.com/sgl-project/sglang/pull/7756
chore: bump sgl-kernel v0.2.4 by @zhyncs in https://github.com/sgl-project/sglang/pull/7800
ci: fix port args by @mickqian in https://github.com/sgl-project/sglang/pull/7792
Fix CI test OOM issue. by @lifuhuang in https://github.com/sgl-project/sglang/pull/7799
chore: upgrade sgl-kernel v0.2.4 by @zhyncs in https://github.com/sgl-project/sglang/pull/7801
chore: bump v0.4.9 by @zhyncs in https://github.com/sgl-project/sglang/pull/7802
[misc] remove pdlb rust by @slin1237 in https://github.com/sgl-project/sglang/pull/7796
fix: free disk space by @zhyncs in https://github.com/sgl-project/sglang/pull/7803
fix: disable dsv3_router_gemm in dsv3_nextn by @Alcanderian in https://github.com/sgl-project/sglang/pull/7793
Support logprobs in two-batch overlap by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7709
Fix division-by-zero bug in LoRA triton kernels. by @lifuhuang in https://github.com/sgl-project/sglang/pull/7785
[AMD] Add test_fused_moe.py and test_rope_rocm.py to AMD CI by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/5246
[RL] Fix illegal memory for _import_static_state by @hebiao064 in https://github.com/sgl-project/sglang/pull/7733
Fix _import_static_state issue by @nanjiangwill in https://github.com/sgl-project/sglang/pull/7812
Optimize moe align block size kernel by @ispobock in https://github.com/sgl-project/sglang/pull/7794
Log the timestamps of each prefill/decode iteration by @yuhsuan-t in https://github.com/sgl-project/sglang/pull/6094
[bugfix] Fix sgl-router get_server_info endpoint compatibility issue by @slin1237 in https://github.com/sgl-project/sglang/pull/7813
Integrate triton moe kernel by @yuan-luo in https://github.com/sgl-project/sglang/pull/7689
Kernels for efficient KV cache IO by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/7313
[docs] update router readme by @slin1237 in https://github.com/sgl-project/sglang/pull/7797
[misc] release new router version by @slin1237 in https://github.com/sgl-project/sglang/pull/7798
fix duplicate args in schedule_batch by @ZeldaHuang in https://github.com/sgl-project/sglang/pull/7816
[AMD] Fail gracefully when AITER is unavailable gfx90a GPUs by @haohui in https://github.com/sgl-project/sglang/pull/7187
docs: update README by @zhyncs in https://github.com/sgl-project/sglang/pull/7821
feat: support DeepSeek-R1-W4AFP8 model with ep-moe mode by @yangsijia-serena in https://github.com/sgl-project/sglang/pull/7762
Enable ModelOpt Llama4 fp8 checkpoint deployment in SGLang by @Edwardf0t1 in https://github.com/sgl-project/sglang/pull/7129
[Minor] Fix sporadic CI timeout caused by underestimated tests. by @lifuhuang in https://github.com/sgl-project/sglang/pull/7850
[Bugfix] Fix two batch overlap with auto DeepEP Dispatch by @ShangmingCai in https://github.com/sgl-project/sglang/pull/7853
Fix cache modules of triton import error by @kkHuang-amd in https://github.com/sgl-project/sglang/pull/7832
[router] forward stream_options in request by @ZhangShuaiyi in https://github.com/sgl-project/sglang/pull/7860
Fix illegal memory in trtllm allreduce fusion by @BBuf in https://github.com/sgl-project/sglang/pull/7864
Fix llama4 vision by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7840
Support Mimo-VL by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7579
fix: Handles input_embeds in GenerateReqInput when n>1 by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7830
[Multimodal][Perf] Use pybase64 instead of base64 by @b8zhong in https://github.com/sgl-project/sglang/pull/7724
Bump xgrammar's version to 0.1.20 by @whybeyoung in https://github.com/sgl-project/sglang/pull/7866
[CPU]convert topk_weights to fp32 for INT8 and FP8 paths (for llama4) and fix LmHead weight pack by @chunyuan-w in https://github.com/sgl-project/sglang/pull/7818
[PD] Add guidance for prefill bootstrap timeout by @ShangmingCai in https://github.com/sgl-project/sglang/pull/7846
Update native_api doc to match the change in the get_model_info endpoint by @Arist12 in https://github.com/sgl-project/sglang/pull/7660
Revert "Embedding parallel by attn_tp (#7623)" by @zhyncs in https://github.com/sgl-project/sglang/pull/7880
chore: bump v0.4.9.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/7882
Fixes typo in assertion message by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/7895
[CI] Add deepep tests to CI by @ch-wan in https://github.com/sgl-project/sglang/pull/7872
[CPU] [FP8] set SGLANG_CPU_FP8_CVT_FTZ in CMakeLists.txt by @chunyuan-w in https://github.com/sgl-project/sglang/pull/7885
[CPU][Qwen3 MoE] Enable fused_topk CPU fusion and enhance FP8 TP padding by @jianan-gu in https://github.com/sgl-project/sglang/pull/7838
Remove unused imports by @almaslof in https://github.com/sgl-project/sglang/pull/7898
[router] Update metrics when request completes by @ZhangShuaiyi in https://github.com/sgl-project/sglang/pull/7899
[feature] Add start step profile argument in /start_profile by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/7608
[bugfix] add pd router policy validation by @slin1237 in https://github.com/sgl-project/sglang/pull/7904
vlm: support video as an input modality by @mickqian in https://github.com/sgl-project/sglang/pull/5888
Feat: Support Phi-3.5-MoE in SGLang by @byjiang1996 in https://github.com/sgl-project/sglang/pull/7907
add sentencepiece as dependency explicitly by @ZailiWang in https://github.com/sgl-project/sglang/pull/7922
Fix bug of deepseek-v3 under DP+EP mode with large batchsize/seqlen by @likesen-alibaba in https://github.com/sgl-project/sglang/pull/6449
[feature]Ascend quantization support by @ping1jing2 in https://github.com/sgl-project/sglang/pull/7791
[ready b200] fuse allreduce+add_rmsnorm in prepare_attention + mlp module by @BBuf in https://github.com/sgl-project/sglang/pull/7775
Support Kimi K2 by @Atream in https://github.com/sgl-project/sglang/pull/7940
[feature] kv transfer support of ascend npu by @ping1jing2 in https://github.com/sgl-project/sglang/pull/7795
fix: minor fix for modelopt weight load compatibility by @AniZpZ in https://github.com/sgl-project/sglang/pull/7953
temporarily disable deepep-8-gpu and activate two small tests by @ch-wan in https://github.com/sgl-project/sglang/pull/7961
[fix]Update unitest for fp8_blockwise_scaled_grouped_mm kernel by @HydraQYH in https://github.com/sgl-project/sglang/pull/7932
chore: bump sgl-kernel v0.2.5 by @zhyncs in https://github.com/sgl-project/sglang/pull/7964
Revert "[PD Disaggregation] replace transfer with batch transfer for better performance (#7236)" by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7968
chore: upgrade xgrammar 0.1.21 by @zhyncs in https://github.com/sgl-project/sglang/pull/7962
delete uselese code caused by fuse allreduce+add_rmsnorm pr by @BBuf in https://github.com/sgl-project/sglang/pull/7970
Fix wrong gemm branch cause 250us slower by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7969
[router] add worker abstraction by @slin1237 in https://github.com/sgl-project/sglang/pull/7960
chore: upgrade sgl-kernel 0.2.5 by @zhyncs in https://github.com/sgl-project/sglang/pull/7971
chore: bump v0.4.9.post2 by @zhyncs in https://github.com/sgl-project/sglang/pull/7963
[minor fix] llama4 hybrid memory by @Ying1123 in https://github.com/sgl-project/sglang/pull/7950
[minor fix] SWA missing methods by @Ying1123 in https://github.com/sgl-project/sglang/pull/7972
[script] update loogle test by @Ying1123 in https://github.com/sgl-project/sglang/pull/7975
docs: update README by @zhyncs in https://github.com/sgl-project/sglang/pull/7985
Overlap the gating function with shared experts in DeepSeek by @ch-wan in https://github.com/sgl-project/sglang/pull/7978
[BugFix] fix pre_reorder_triton_kernel default int32 issue by @Yuechguo in https://github.com/sgl-project/sglang/pull/7814
[minor] Add server_args check for Llama4 with hybrid by @Ying1123 in https://github.com/sgl-project/sglang/pull/7988
Tiny fix mooncake log warning wrong output by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7952
[BugFix] add verify logit_bias to avoid crash because of IndexError by @ehuaa in https://github.com/sgl-project/sglang/pull/7749
SWA Prefix Cache by @hanming-lu in https://github.com/sgl-project/sglang/pull/7367
chore: remove unnecessary limits on quantization methods in test script by @AniZpZ in https://github.com/sgl-project/sglang/pull/7997
Refactor dynamic LoRA update to fix incorrect handling of variant weight shapes by @lifuhuang in https://github.com/sgl-project/sglang/pull/7844
Support for Phi-1.5 & Phi-2 models by @ppraneth in https://github.com/sgl-project/sglang/pull/7862
[Dockerfile] Multi-arch support for ROCm by @mqhc2020 in https://github.com/sgl-project/sglang/pull/7902
[CPU] fix no attribute 'can_fuse_mlp_allreduce' error by @chunyuan-w in https://github.com/sgl-project/sglang/pull/8010
perf: add kimi k2 fused_moe tuning config for h30_3e by @GaoYusong in https://github.com/sgl-project/sglang/pull/8021
[ci] CI supports use cached models by @HanHan009527 in https://github.com/sgl-project/sglang/pull/7874
[Minor] Remove redundant print by @merrymercy in https://github.com/sgl-project/sglang/pull/8005
[Feature]TP Group Switching for PD-Multiplexing by @ykcombat in https://github.com/sgl-project/sglang/pull/7653
[Feature] CUDA Green Context Support by @ykcombat in https://github.com/sgl-project/sglang/pull/7649
Fix flaky CI: test_vlm_models by @lifuhuang in https://github.com/sgl-project/sglang/pull/8006
Fix Bug 'get_cpu_copy not Implemented' in pd offloading mode by @hzh0425 in https://github.com/sgl-project/sglang/pull/7982
prevent server crash from potential invalid grammar by @ehuaa in https://github.com/sgl-project/sglang/pull/7897
Setup workflow for releasing mi300x and mi350x dockers. by @saienduri in https://github.com/sgl-project/sglang/pull/8035
fix: modality length mismatch with image_data by @Yangruipis in https://github.com/sgl-project/sglang/pull/7887
Update CODEOWNERS by @CatherineSue in https://github.com/sgl-project/sglang/pull/8044
[feat]Support fusion kernel for constructing quant input and scale factor for fp8_blockwise_scaled_grouped_mm by @HydraQYH in https://github.com/sgl-project/sglang/pull/8023
feat: update multimodal data handling in engine entrypoint by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8002
fix: remove redundant rotary embedding cache recomputation in MiniCPM by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8022
Fix the input tools format and history tool_calls in OpenAI API by @chen700564 in https://github.com/sgl-project/sglang/pull/6556
fix: resolve arm build issue by @zhyncs in https://github.com/sgl-project/sglang/pull/8052
concurrently load weights of DeepseekV2ForCausalLM by @tianyuzhou95 in https://github.com/sgl-project/sglang/pull/7943
H20 tune config for Kimi by @artetaout in https://github.com/sgl-project/sglang/pull/8047
Update amd docker image. by @saienduri in https://github.com/sgl-project/sglang/pull/8045
feat: replace Decord with video_reader-rs by @kozoy in https://github.com/sgl-project/sglang/pull/5163
remove kv_a.congigous in DeepseekV2AttentionMLA by @strgrb in https://github.com/sgl-project/sglang/pull/8058
update transformers to 4.53.2 by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8029
Fix different device type adjustment in PP by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/7760
Use device_group for all_gather when disabling overlap scheduling by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/8001
Revert "feat: replace Decord with video_reader-rs" by @mickqian in https://github.com/sgl-project/sglang/pull/8077
Fix CI xeon test with triton 3.3.1 by @yanbing-j in https://github.com/sgl-project/sglang/pull/8086
fix greenctx stream compability by @AniZpZ in https://github.com/sgl-project/sglang/pull/8090
[misc] update nvshmem and pin deepEP commit hash by @slin1237 in https://github.com/sgl-project/sglang/pull/8098
[Feature] Layer-wise Prefill by @jason-fxz in https://github.com/sgl-project/sglang/pull/7634
[1/n] chore: decouple quantization implementation from vLLM dependency by @AniZpZ in https://github.com/sgl-project/sglang/pull/7992
refactor: unify names of the feature field of MultimodalDataItem by @mickqian in https://github.com/sgl-project/sglang/pull/8075
feat: add tp_rank, pp_rank and dp_rank labels for scheduler metrics by @acelyc111 in https://github.com/sgl-project/sglang/pull/7597
[ci] limit cmake build nproc by @slin1237 in https://github.com/sgl-project/sglang/pull/8100
[ci] disable memory imbalance check for draft worker by @ch-wan in https://github.com/sgl-project/sglang/pull/8108
[Fix] ensure DeepGEMM is only enabled for FP8_W8A8 models by @hzh0425 in https://github.com/sgl-project/sglang/pull/8110
[ci] recover 8-gpu deepep test by @ch-wan in https://github.com/sgl-project/sglang/pull/8105
Refactor: move all quantization-related code to srt/layer/quantization by @ch-wan in https://github.com/sgl-project/sglang/pull/7989
[kernel] opt moe align block kernel by block/warp scan algorithm by @yuan-luo in https://github.com/sgl-project/sglang/pull/7884
Super tiny fix typo by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8046
fix: update HostKVCache init to report correct msg when available memory is not enough by @ziqifan617 in https://github.com/sgl-project/sglang/pull/8102
[Hunyuan]: Fix Dense Model Support by @kzjeef in https://github.com/sgl-project/sglang/pull/8117
feat: add production metric for retracted requests due to insufficient kvcache by @aftersnow in https://github.com/sgl-project/sglang/pull/7030
refactor: simply MultimodalTokens logic by @mickqian in https://github.com/sgl-project/sglang/pull/7924
[Fix][Ready]Fix register spilling in cutlass nvfp4 gemm kernel on Blackwell by @HydraQYH in https://github.com/sgl-project/sglang/pull/8127
Feat: Support Granite 3.0 MoE in SGLang by @zminglei in https://github.com/sgl-project/sglang/pull/7959
load draft model fix by @yilian49 in https://github.com/sgl-project/sglang/pull/7506
[CPU][Llama4] Fix Llama4 MoE inputs with "apply_router_weight_on_input" by @jianan-gu in https://github.com/sgl-project/sglang/pull/7889
[Quantization][w8a8_int8] Fix weight loading issue for w8a8_int8 path with "ignore" layer list in quantization config by @jianan-gu in https://github.com/sgl-project/sglang/pull/7820
Hicache Storage Layer Prototype by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/7704
Revert "Fix different device type adjustment in PP" by @saienduri in https://github.com/sgl-project/sglang/pull/8141
feat: enchance green context stream creation robust with backward compatibility by @AniZpZ in https://github.com/sgl-project/sglang/pull/8136
fix compressed tensors WNA16 imports by @qeternity in https://github.com/sgl-project/sglang/pull/8142
[Bugfix] Fix w8a8_int8 import error on NPU by @iforgetmyname in https://github.com/sgl-project/sglang/pull/8147
[3/n] chore: decouple AWQ implementation from vLLM dependency by @Hongbosherlock in https://github.com/sgl-project/sglang/pull/8113
[router] Refactor router and policy traits with dependency injection by @slin1237 in https://github.com/sgl-project/sglang/pull/7987
[AMD] Add triton awq_dequantize kernel to support AWQ on ROCm by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/7661
[Doc] Steps to add a new attention backend by @merrymercy in https://github.com/sgl-project/sglang/pull/8155
chore: tune mem fraction static for vlm by @mickqian in https://github.com/sgl-project/sglang/pull/6881
Support NVFP4 quantized dense models on AMD CDNA2/CDNA3 GPUs by @haohui in https://github.com/sgl-project/sglang/pull/7302
Feat: Support audio in Phi4-mm model by @byjiang1996 in https://github.com/sgl-project/sglang/pull/8048
[PD] Support non-MLA models PD different TP with DP attention by @ShangmingCai in https://github.com/sgl-project/sglang/pull/7931
[health_generate] fix: fix the /health_generate always success bug by @acelyc111 in https://github.com/sgl-project/sglang/pull/8028
[router] router metrics cleanup by @slin1237 in https://github.com/sgl-project/sglang/pull/8158
[router] allow router to have empty workers by @slin1237 in https://github.com/sgl-project/sglang/pull/8160
Add GB200 wide-EP docker by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/8157
[1/N] MoE Refactor: refactor select_experts by @ch-wan in https://github.com/sgl-project/sglang/pull/7966
chore: bump sgl-kernel v0.2.6 by @zhyncs in https://github.com/sgl-project/sglang/pull/8165
chore: upgrade sgl-kernel 0.2.6 by @zhyncs in https://github.com/sgl-project/sglang/pull/8166
Fix suffix mismatch for the metrics. by @Charles-L-Chen in https://github.com/sgl-project/sglang/pull/8168
Update README.md by @merrymercy in https://github.com/sgl-project/sglang/pull/8171
Clean up server args by @merrymercy in https://github.com/sgl-project/sglang/pull/8161
Fix LoRA buffer contamination during adapter eviction by @lifuhuang in https://github.com/sgl-project/sglang/pull/8103
Fix Dockerfile.gb200 by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/8169
[router] add ut for worker and errors by @slin1237 in https://github.com/sgl-project/sglang/pull/8170
bugfix: fix sglang crash in NVIDIA MIG container by @Garrybest in https://github.com/sgl-project/sglang/pull/8167
Support start up LoRA server without initial adapters by @lifuhuang in https://github.com/sgl-project/sglang/pull/8019
Clean warning logs for gate_proj loading in Lora by @Fridge003 in https://github.com/sgl-project/sglang/pull/8172
Fix tuning_fused_moe_triton.py by @ch-wan in https://github.com/sgl-project/sglang/pull/8175
[Feature] Simple Improve Health Check Mechanism for Production-Grade Stability by @whybeyoung in https://github.com/sgl-project/sglang/pull/8115
Add bf16 output option for dsv3_router_gemm kernel by @Fridge003 in https://github.com/sgl-project/sglang/pull/7999
Enable FlashInfer support encoder models and add head_dim padding workaround by @ccs96307 in https://github.com/sgl-project/sglang/pull/6230
Add get_hidden_dim to qwen3.py for correct lora by @logachevpa in https://github.com/sgl-project/sglang/pull/7312
feat: add h200 tp 16 kimi k2 moe config by @zhyncs in https://github.com/sgl-project/sglang/pull/8176
feat: add b200 tp 16 kimi k2 moe config by @zhyncs in https://github.com/sgl-project/sglang/pull/8178
fix moe gate dtype, fix tbo, fix fake dispatch by @Atream in https://github.com/sgl-project/sglang/pull/7825
Revert "[Feature] Simple Improve Health Check Mechanism for Production-Grade Stability" by @merrymercy in https://github.com/sgl-project/sglang/pull/8181
feat: update nccl 2.27.6 by @zhyncs in https://github.com/sgl-project/sglang/pull/8182
Feat: Support for Persimmon Model by @ppraneth in https://github.com/sgl-project/sglang/pull/7983
feat: add h200 tp 16 kimi k2 moe config by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/8183
Fix eagle3 cuda graph by @Ja1Zhou in https://github.com/sgl-project/sglang/pull/8163
fix: fix the bug of loading Internvl3 by @coco-alen in https://github.com/sgl-project/sglang/pull/8067
Fix dtype error in CI by @ispobock in https://github.com/sgl-project/sglang/pull/8197
[router] add ut for pd request, metrics and config by @slin1237 in https://github.com/sgl-project/sglang/pull/8184
[feature] enable NPU CI by @ping1jing2 in https://github.com/sgl-project/sglang/pull/7935
[fix] fix modelopt fp4 on b200 by @Alcanderian in https://github.com/sgl-project/sglang/pull/8195
chore: bump sgl-kernel v0.2.6.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/8200
Apply fused sorted token ids padding by @ispobock in https://github.com/sgl-project/sglang/pull/8193
[Refactor] simplify multimodal data processing by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8107
[router] add ut for pd router by @slin1237 in https://github.com/sgl-project/sglang/pull/8208
[router] upgade router version to 0.1.6 by @slin1237 in https://github.com/sgl-project/sglang/pull/8209
Remve router gemm output dtype conversion by @ispobock in https://github.com/sgl-project/sglang/pull/8204
chore: upgrade sgl-kernel 0.2.6.post1 by @zhyncs in https://github.com/sgl-project/sglang/pull/8202
[Feature] Add a test for Layer-wise Prefill by @jason-fxz in https://github.com/sgl-project/sglang/pull/8231
docs: update 2025 h2 roadmap by @zhyncs in https://github.com/sgl-project/sglang/pull/8237
fix: retrieve mm token by modality, raise error if none by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8221
[AMD] Remove vllm's scaled_fp8_quant and moe_sum when SGLANG_USE_AITER=1 by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/7484
fix: sgl-router remove dead code by @oldsharp in https://github.com/sgl-project/sglang/pull/8257
[fix] benchmark : routed_scaling_factor is None by @panpan0000 in https://github.com/sgl-project/sglang/pull/8059
[Benchmark] add disable-auto-run param for hicache/bench_multiturn by @rzwei in https://github.com/sgl-project/sglang/pull/7822
Preliminary Support for Qwen3XMLDetector by @yhyang201 in https://github.com/sgl-project/sglang/pull/8260
chore: bump v0.4.9.post3 by @zhyncs in https://github.com/sgl-project/sglang/pull/8265
Skip llama4 vision module loading when multimodal disabled by @ispobock in https://github.com/sgl-project/sglang/pull/8272
Fix sgl-kernel ci test by @ispobock in https://github.com/sgl-project/sglang/pull/8284
Introduce Stable LoRA ID System for Overlapped Updates and Prefix Caching by @lifuhuang in https://github.com/sgl-project/sglang/pull/8261
Hicache IO kernel refactoring by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8264
bug fix and tag by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8282
HiCache Fix by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8288
[sgl-kernel] Opt per_token_quant_fp8 with warp reduce by @yuan-luo in https://github.com/sgl-project/sglang/pull/8130
[router] add common ut infra to mock worker and app by @slin1237 in https://github.com/sgl-project/sglang/pull/8295
fix: workaround for deepgemm warmup issue by @zhyncs in https://github.com/sgl-project/sglang/pull/8302
[Performance][PD Disaggregation] optimize TokenToKVPoolAllocator by sorting free pages by @YiXR in https://github.com/sgl-project/sglang/pull/8133
Fix the issue of incorrect finish reason in final stream response chunk returned during tool call by @xianzhiT in https://github.com/sgl-project/sglang/pull/7708
fix: match chat-template for internvl3 by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8262
Fix gemma3n with hybrid swa by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8240
chore: upgrade sgl-kernel 0.2.7 by @zhyncs in https://github.com/sgl-project/sglang/pull/8304
fix: prevent crashes due to logit bias dimension mismatch by @0xymoro in https://github.com/sgl-project/sglang/pull/7685
feat(function call): complete utility method for KimiK2Detector and enhance documentation by @CatherineSue in https://github.com/sgl-project/sglang/pull/8043
Fix incomplete tool call capture issue in streaming response of DeepSeek-V3 when enable MTP by @xianzhiT in https://github.com/sgl-project/sglang/pull/7562
[AMD] Pull latest image for AMD CI by @michael-amd in https://github.com/sgl-project/sglang/pull/8070
Pin the version of petit kernel to fix the APIs by @haohui in https://github.com/sgl-project/sglang/pull/8235
[bug] fix pd completion protocol for batching support by @slin1237 in https://github.com/sgl-project/sglang/pull/8317
[router] fix pd model completion request by @slin1237 in https://github.com/sgl-project/sglang/pull/8303
fix bug when eos_ids==0 by @bzantium in https://github.com/sgl-project/sglang/pull/8315
[router] add endpoint unit test by @slin1237 in https://github.com/sgl-project/sglang/pull/8298
[code style] Clean dead triton kernel code in fused_moe and useless vllm_ops import by @BBuf in https://github.com/sgl-project/sglang/pull/8310
chore: upgrade flashinfer v0.2.9rc1 by @Swipe4057 in https://github.com/sgl-project/sglang/pull/8301
[router] add streaming unit test by @slin1237 in https://github.com/sgl-project/sglang/pull/8299
[router] add request format unit test by @slin1237 in https://github.com/sgl-project/sglang/pull/8300
HiCache Storage TP Refinement by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8307
breakdown kernel update by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8334
support idle batch for TBO by @sherry-1001 in https://github.com/sgl-project/sglang/pull/8233
[Feature] Integrate quick allreduce and select the best allreduce implementation by @lihaoyang-amd in https://github.com/sgl-project/sglang/pull/6619
DP Enhancement by @ch-wan in https://github.com/sgl-project/sglang/pull/8280
fix: Fix failed functional tests https://github.com/meta-llama/llama-stack-evals by @ynwang007 in https://github.com/sgl-project/sglang/pull/8266
[AMD] Add silu_and_mul, gelu_and_mul, gelu_tanh_and_mul, and gelu_quick kernels for AMD GPUs by @hubertlu-tw in https://github.com/sgl-project/sglang/pull/7135
[CPU] Add tutorial docs for SGL on CPU by @ZailiWang in https://github.com/sgl-project/sglang/pull/8000
chore: upgrade mooncake 0.3.5 by @ShangmingCai in https://github.com/sgl-project/sglang/pull/8341
[torch.compile bug] avoid biased_grouped_topk_impl func repeatedly triggering torch.compile in forward pass by @BBuf in https://github.com/sgl-project/sglang/pull/8353
[P/D] Support ipv6 in P/D scenario by @thefacetakt in https://github.com/sgl-project/sglang/pull/7858
Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct by @Xu-Wenqing in https://github.com/sgl-project/sglang/pull/8344
[Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector by @CatherineSue in https://github.com/sgl-project/sglang/pull/8357
Clean up server_args, triton cache manager by @merrymercy in https://github.com/sgl-project/sglang/pull/8332
fix: upgrade nccl version by @zhyncs in https://github.com/sgl-project/sglang/pull/8359
[Feat] Add reasoning parser for Qwen/Qwen3-235B-A22B-Thinking-2507 by @CatherineSue in https://github.com/sgl-project/sglang/pull/8363
fix: kimi k2 xgrammar crash by @zhyncs in https://github.com/sgl-project/sglang/pull/8367
Fix FP4 MoE accuracy from missing routed_scaling_factor by @trevor-m in https://github.com/sgl-project/sglang/pull/8333
[CI] Fix flaky threshold by @merrymercy in https://github.com/sgl-project/sglang/pull/8370
chore: bump v0.4.9.post4 by @zhyncs in https://github.com/sgl-project/sglang/pull/8305
Fix test_moe_fused_gate_combined sgl-kernel ci test by @ispobock in https://github.com/sgl-project/sglang/pull/8374
Uodate Dockerfile.gb200 to latest sglang by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/8356
chore: improve mmmu benchmark by @mickqian in https://github.com/sgl-project/sglang/pull/7000
Save peak memory in logits processor by @ch-wan in https://github.com/sgl-project/sglang/pull/8343
Extract update_weights from RL Engine to SGLang to keep simplicity and fix torch reduce by @hebiao064 in https://github.com/sgl-project/sglang/pull/8267
chore: improvements on mm_utils by @mickqian in https://github.com/sgl-project/sglang/pull/7737
vlm: optimize tensor transport by @mickqian in https://github.com/sgl-project/sglang/pull/6003
Tiny assert EPLB is used together with expert parallel by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8381
model: support intern-s1 by @RunningLeon in https://github.com/sgl-project/sglang/pull/8350
Add perf tests for LoRA by @lifuhuang in https://github.com/sgl-project/sglang/pull/8314
Remove slot usage in code to be backward-compatible with python 3.9 by @lifuhuang in https://github.com/sgl-project/sglang/pull/8396
Add docker release flow for gb200 by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/8394
HiCache, check before terminate prefetching by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/8372
Add nvfp4 scaled mm benchmark. by @HydraQYH in https://github.com/sgl-project/sglang/pull/8401
Urgent Fix: intern-s1 chat-template matching by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/8403
Tool to dump and compare internal activation tensors by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7976
Minor tool for comparison of benchmark results by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7974
Fix bench script making input data on L2 cache by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7739
[NVIDIA] Add Flashinfer MoE blockscale fp8 backend by @kaixih in https://github.com/sgl-project/sglang/pull/8036
Update Cutlass in sgl-kernel to v4.1 by @Fridge003 in https://github.com/sgl-project/sglang/pull/8392
fix: minor fix TransportProxyTensor under tp by @mickqian in https://github.com/sgl-project/sglang/pull/8382
[router] add different policies for p node and d node by @slin1237 in https://github.com/sgl-project/sglang/pull/8395
Add A800 fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct by @lambert0312 in https://github.com/sgl-project/sglang/pull/8351
fix: fix the missing metrics on non-rank0 nodes by @acelyc111 in https://github.com/sgl-project/sglang/pull/7720
[2/N] MoE Refactor: Unify weight loader and quant methods by @ch-wan in https://github.com/sgl-project/sglang/pull/8397
Use FlashInfer FP4 gemm. by @elfiegg in https://github.com/sgl-project/sglang/pull/8241
Support precomputed_embeddings for Llama 4 by @AlienKevin in https://github.com/sgl-project/sglang/pull/8156
[hotfix] fix merge conflicts in FlashInferEPMoE by @ch-wan in https://github.com/sgl-project/sglang/pull/8405
chore: update CODEOWNERS by @zhyncs in https://github.com/sgl-project/sglang/pull/8407
chore: upgrade flashinfer v0.2.9rc2 by @zhyncs in https://github.com/sgl-project/sglang/pull/8406
Support triton kernels v3.4.0 for fused_moe by @yuan-luo in https://github.com/sgl-project/sglang/pull/8258
[Bugfix] Prevent PD server crash from invalid grammar by @ShangmingCai in https://github.com/sgl-project/sglang/pull/8062
Change to use native arm runner by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/8414
Support overlapped lora updates by @lifuhuang in https://github.com/sgl-project/sglang/pull/8213
Support ue8m0 for triton quant kernel by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7603
Fix: Improve test_openai_function_calling unit test and fix reasoning_parser.py think_start_token logic by @byjiang1996 in https://github.com/sgl-project/sglang/pull/8316
bugfix: Fix multiple finish_reason chunks and tool_calls finish reason check by @CatherineSue in https://github.com/sgl-project/sglang/pull/8417
Fix test_openai_server by @CatherineSue in https://github.com/sgl-project/sglang/pull/8419
Fix docker buildx push error by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/8425
bugfix: Fix XGrammar backend to use model's EOS tokens for constrained generation by @CatherineSue in https://github.com/sgl-project/sglang/pull/8422
[router] improve router logs and request id header by @slin1237 in https://github.com/sgl-project/sglang/pull/8415
[feat] Support different attention backends for prefill and decode by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/6338
chore: bump transformer to 4.54.0 by @hebiao064 in https://github.com/sgl-project/sglang/pull/8416
[PD] Fix abort_request for PD disaggregation by @ShangmingCai in https://github.com/sgl-project/sglang/pull/8352
GLM-4.5 Model Support by @zRzRzRzRzRzRzR in https://github.com/sgl-project/sglang/pull/8224
Remove zstd compression for building Dockerfile.gb200 by @kyleliang-nv in https://github.com/sgl-project/sglang/pull/8442
doc: add bench_one_batch_server in the benchmark doc by @Qiaolin-Yu in https://github.com/sgl-project/sglang/pull/8441
GLM-4.5 Model Support Follow-up by @byjiang1996 in https://github.com/sgl-project/sglang/pull/8445
fix GLM4_MOE launch with compressed_tensor quant model by @zminglei in https://github.com/sgl-project/sglang/pull/8456
Fix per_token_group_quant_8bit when hidden_dim // group_size is not divided by 4. by @strgrb in https://github.com/sgl-project/sglang/pull/8449
Revert "[kernel] opt moe align block kernel by block/warp scan algorithm" by @BBuf in https://github.com/sgl-project/sglang/pull/8457
chore: bump v0.4.9.post5 by @zhyncs in https://github.com/sgl-project/sglang/pull/8458
fix:reorder topk experts to ensure shared expert replaces minimal score by @erictanjn in https://github.com/sgl-project/sglang/pull/8125
Update PR template by @ispobock in https://github.com/sgl-project/sglang/pull/8465
feat: throttle requests at scheduler based on --max_queued_requests by @harrisonlimh in https://github.com/sgl-project/sglang/pull/7565
fix: update dep by @zhyncs in https://github.com/sgl-project/sglang/pull/8467
[NVIDIA] Change to use num_local_experts by @kaixih in https://github.com/sgl-project/sglang/pull/8453
Fix parsing ChatCompletionMessage by @Onyad in https://github.com/sgl-project/sglang/pull/7273
[3/N] MoE Refactor: Simplify DeepEP Output by @ch-wan in https://github.com/sgl-project/sglang/pull/8421
feat: support glm4 tuning by @zhyncs in https://github.com/sgl-project/sglang/pull/8473
Fix DEEPEP BF16 compatibility for Deepseek Style model like GLM 4.5 by @hebiao064 in https://github.com/sgl-project/sglang/pull/8469
Update codeowner by @merrymercy in https://github.com/sgl-project/sglang/pull/8476
chore: add glm4 fp8 tp8 config by @zhyncs in https://github.com/sgl-project/sglang/pull/8478
chore: add glm 4.5 fp8 tp4 config by @zhyncs in https://github.com/sgl-project/sglang/pull/8480
[CI]Add genai-bench Performance Validation for PD Router by @key4ng in https://github.com/sgl-project/sglang/pull/8477
Update CODEOWNERS by @merrymercy in https://github.com/sgl-project/sglang/pull/8485
Rename the last step in pr-test.yml as pr-test-finish by @merrymercy in https://github.com/sgl-project/sglang/pull/8486
Reduce memory usage for fp4 moe by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8413
Tiny add warnings for DeepEP when it is suboptimal by @fzyzcjy in https://github.com/sgl-project/sglang/pull/8426
Support colocating requests by @fzyzcjy in https://github.com/sgl-project/sglang/pull/7973
Fix incorrect KV cache allocation for MTP models. by @lifuhuang in https://github.com/sgl-project/sglang/pull/8482
Add PVC and update resource limits in k8s config by @haitwang-cloud in https://github.com/sgl-project/sglang/pull/8489
chore: bump v0.4.9.post6 by @zhyncs in https://github.com/sgl-project/sglang/pull/8517
Always trigger pr-test by @merrymercy in https://github.com/sgl-project/sglang/pull/8527
Update README.md by @merrymercy in https://github.com/sgl-project/sglang/pull/8528
[sgl-kernel performace] fix fp8 quant kernels dispatch __nv_fp8_e4m3 bug to improve performance 10%-20% by @BBuf in https://github.com/sgl-project/sglang/pull/8499
Update cutlass_moe.py by @elfiegg in https://github.com/sgl-project/sglang/pull/8535
Fix moe align kernel test by @ispobock in https://github.com/sgl-project/sglang/pull/8531
Split the scheduler into multiple mixin classes to reduce the file size by @merrymercy in https://github.com/sgl-project/sglang/pull/8483
bring back kimi vl ci by @hebiao064 in https://github.com/sgl-project/sglang/pull/8537
fix: temporarily disable cuda-ipc for mm data tensor by @mickqian in https://github.com/sgl-project/sglang/pull/8431
Support EPLB in FusedMoE by @ch-wan in https://github.com/sgl-project/sglang/pull/8448
feat(hicache): support file backend reading directory config form env. by @hzh0425 in https://github.com/sgl-project/sglang/pull/8498
feature(pd-hicache): Prefill instances support reusing the RemoteStorage Cache via HiCache. by @hzh0425 in https://github.com/sgl-project/sglang/pull/8516
[router] allow longer time out for router e2e by @slin1237 in https://github.com/sgl-project/sglang/pull/8560
Update cutlass_moe.py by @elfiegg in https://github.com/sgl-project/sglang/pull/8545
Update CODEOWNERS by @ShangmingCai in https://github.com/sgl-project/sglang/pull/8562
[feature] [sgl-router] Add a dp-aware routing strategy by @oldsharp in https://github.com/sgl-project/sglang/pull/6869
[Hot-Fix] moe_aligned_block_size CI failed in AMD by @yuan-luo in https://github.com/sgl-project/sglang/pull/8461
[Model] Add support for Arcee Foundational Model by @adarshxs in https://github.com/sgl-project/sglang/pull/8154
Revert "Fix the input tools format and history tool_calls in OpenAI API (#6556)" by @CatherineSue in https://github.com/sgl-project/sglang/pull/8584
Add hf3fs support for hicache storage (based on #7704) by @pansicheng in https://github.com/sgl-project/sglang/pull/7280
[router] migrate router from actix to axum by @slin1237 in https://github.com/sgl-project/sglang/pull/8479
[Fix]Fix index oob in get_group_gemm_starts kernel. by @HydraQYH in https://github.com/sgl-project/sglang/pull/8564
Bump transfomers to 4.54.1 to fix Gemma cache issue. by @lifuhuang in https://github.com/sgl-project/sglang/pull/8541
Add GKE's default CUDA runtime lib location to PATH and LD_LIBRARY_PATH. by @pyc96 in https://github.com/sgl-project/sglang/pull/8544
Bug: Fix google gemma3n-mm audio input not working bug by @byjiang1996 in https://github.com/sgl-project/sglang/pull/8365
update sgl-kernel for EP: kernel part by @ch-wan in https://github.com/sgl-project/sglang/pull/8514
chore: bump sgl-kernel v0.2.8 by @zhyncs in https://github.com/sgl-project/sglang/pull/8599
[bugfix] Fix 2 minor bugs in the hicache storage layer by @yapple in https://github.com/sgl-project/sglang/pull/8404
fix incorrect increase of hit count by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/8533
Support l3 cache (mooncake store) for hiradix cache by @huangtingwei9988 in https://github.com/sgl-project/sglang/pull/7211
update sgl-kernel for EP: python part by @ch-wan in https://github.com/sgl-project/sglang/pull/8550
add SVG logo by @hnyls2002 in https://github.com/sgl-project/sglang/pull/8603
[4/N] MoE Refactor: Unified Triton Kernel for FusedMoE and EPMoE by @ch-wan in https://github.com/sgl-project/sglang/pull/8515
fix: fork should not run pypi router by @yihong0618 in https://github.com/sgl-project/sglang/pull/8604
model: support Step3V by @CatherineSue in https://github.com/sgl-project/sglang/pull/8583
[Feature] Hybrid EP and TP by @ch-wan in https://github.com/sgl-project/sglang/pull/8590
chore: bump v0.4.10 by @zhyncs in https://github.com/sgl-project/sglang/pull/8608

New Contributors

@valarLip made their first contribution in https://github.com/sgl-project/sglang/pull/7423
@ssssnow made their first contribution in https://github.com/sgl-project/sglang/pull/7236
@xianzhiT made their first contribution in https://github.com/sgl-project/sglang/pull/7277
@yilian49 made their first contribution in https://github.com/sgl-project/sglang/pull/7510
@ll819214 made their first contribution in https://github.com/sgl-project/sglang/pull/7386
@MasterJH5574 made their first contribution in https://github.com/sgl-project/sglang/pull/7543
@TomQuartz made their first contribution in https://github.com/sgl-project/sglang/pull/7584
@finetunej made their first contribution in https://github.com/sgl-project/sglang/pull/7619
@narutolhy made their first contribution in https://github.com/sgl-project/sglang/pull/7648
@SimonCqk made their first contribution in https://github.com/sgl-project/sglang/pull/7684
@ZeldaHuang made their first contribution in https://github.com/sgl-project/sglang/pull/7714
@ayrnb made their first contribution in https://github.com/sgl-project/sglang/pull/7278
@ping1jing2 made their first contribution in https://github.com/sgl-project/sglang/pull/7722
@TianyuZhang1214 made their first contribution in https://github.com/sgl-project/sglang/pull/7673
@rudeigerc made their first contribution in https://github.com/sgl-project/sglang/pull/7741
@Capronir made their first contribution in https://github.com/sgl-project/sglang/pull/7778
@yangsijia-serena made their first contribution in https://github.com/sgl-project/sglang/pull/7772
@leng-yue made their first contribution in https://github.com/sgl-project/sglang/pull/7596
@HydraQYH made their first contribution in https://github.com/sgl-project/sglang/pull/7782
@MoonBall made their first contribution in https://github.com/sgl-project/sglang/pull/7623
@nanjiangwill made their first contribution in https://github.com/sgl-project/sglang/pull/7812
@haohui made their first contribution in https://github.com/sgl-project/sglang/pull/7187
@ZhangShuaiyi made their first contribution in https://github.com/sgl-project/sglang/pull/7860
@almaslof made their first contribution in https://github.com/sgl-project/sglang/pull/7898
@kyleliang-nv made their first contribution in https://github.com/sgl-project/sglang/pull/7608
@likesen-alibaba made their first contribution in https://github.com/sgl-project/sglang/pull/6449
@Yuechguo made their first contribution in https://github.com/sgl-project/sglang/pull/7814
@hanming-lu made their first contribution in https://github.com/sgl-project/sglang/pull/7367
@ppraneth made their first contribution in https://github.com/sgl-project/sglang/pull/7862
@mqhc2020 made their first contribution in https://github.com/sgl-project/sglang/pull/7902
@ykcombat made their first contribution in https://github.com/sgl-project/sglang/pull/7653
@hzh0425 made their first contribution in https://github.com/sgl-project/sglang/pull/7982
@Yangruipis made their first contribution in https://github.com/sgl-project/sglang/pull/7887
@chen700564 made their first contribution in https://github.com/sgl-project/sglang/pull/6556
@artetaout made their first contribution in https://github.com/sgl-project/sglang/pull/8047
@kozoy made their first contribution in https://github.com/sgl-project/sglang/pull/5163
@jason-fxz made their first contribution in https://github.com/sgl-project/sglang/pull/7634
@acelyc111 made their first contribution in https://github.com/sgl-project/sglang/pull/7597
@ziqifan617 made their first contribution in https://github.com/sgl-project/sglang/pull/8102
@kzjeef made their first contribution in https://github.com/sgl-project/sglang/pull/8117
@aftersnow made their first contribution in https://github.com/sgl-project/sglang/pull/7030
@Charles-L-Chen made their first contribution in https://github.com/sgl-project/sglang/pull/8168
@Garrybest made their first contribution in https://github.com/sgl-project/sglang/pull/8167
@ccs96307 made their first contribution in https://github.com/sgl-project/sglang/pull/6230
@logachevpa made their first contribution in https://github.com/sgl-project/sglang/pull/7312
@coco-alen made their first contribution in https://github.com/sgl-project/sglang/pull/8067
@oldsharp made their first contribution in https://github.com/sgl-project/sglang/pull/8257
@rzwei made their first contribution in https://github.com/sgl-project/sglang/pull/7822
@YiXR made their first contribution in https://github.com/sgl-project/sglang/pull/8133
@0xymoro made their first contribution in https://github.com/sgl-project/sglang/pull/7685
@bzantium made their first contribution in https://github.com/sgl-project/sglang/pull/8315
@sherry-1001 made their first contribution in https://github.com/sgl-project/sglang/pull/8233
@lihaoyang-amd made their first contribution in https://github.com/sgl-project/sglang/pull/6619
@ynwang007 made their first contribution in https://github.com/sgl-project/sglang/pull/8266
@thefacetakt made their first contribution in https://github.com/sgl-project/sglang/pull/7858
@RunningLeon made their first contribution in https://github.com/sgl-project/sglang/pull/8350
@AlienKevin made their first contribution in https://github.com/sgl-project/sglang/pull/8156
@zRzRzRzRzRzRzR made their first contribution in https://github.com/sgl-project/sglang/pull/8224
@erictanjn made their first contribution in https://github.com/sgl-project/sglang/pull/8125
@harrisonlimh made their first contribution in https://github.com/sgl-project/sglang/pull/7565
@Onyad made their first contribution in https://github.com/sgl-project/sglang/pull/7273
@haitwang-cloud made their first contribution in https://github.com/sgl-project/sglang/pull/8489
@yapple made their first contribution in https://github.com/sgl-project/sglang/pull/8404
@yihong0618 made their first contribution in https://github.com/sgl-project/sglang/pull/8604

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.4.8...v0.4.10

Open source →