NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI, #3266 most downloaded on PyPI
SGLang is a fast serving framework for large language models and vision language models.
Latest stable
0.5.23
Last release 18 days ago
30 Oct 2026
Ships on a steady schedule
a new release about every 2 weeks
Some releases are documented
notes for 29 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
3 years old
160 releases · first in 2024
Drag across the ruler to compare, or tap a release to open it.
remove deprecated tile_tokens_dim by @b8zhong in https://github.com/sgl-project/sglang/pull/13186
can_run_dp_cuda_graph gather logic by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12891matrix.part in pr-test. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12920PREBUILT_EXTEND mode by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12948: for LoRA adapter syntax by @neelabhsinha in https://github.com/sgl-project/sglang/pull/12912process_prefill_chunk back to fix PP event loop by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13009(overlap, non-overlap) x (page>=1) x (spec, non-spec, spec v2) x (retract, finished) by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12224file:// scheme in load_video by @netanel-haber in https://github.com/sgl-project/sglang/pull/13076total_throughput to output-file in bench_serving.py by @Rohan138 in https://github.com/sgl-project/sglang/pull/9790Scheduler.recv_requests by @Arist12 in https://github.com/sgl-project/sglang/pull/7886**recv_req.parameters by @CharlieFRuan in https://github.com/sgl-project/sglang/pull/7906_concat_mla_absorb_q_general to replace torch.cat by @bingps in https://github.com/sgl-project/sglang/pull/12215RequestMetricsExporter utility to export request-level metrics by @scottjlee in https://github.com/sgl-project/sglang/pull/10973tile_tokens_dim by @b8zhong in https://github.com/sgl-project/sglang/pull/13186calculate_dimensions by @adarshxs in https://github.com/sgl-project/sglang/pull/13294lang by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13335--range-begin in run_suite.py by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13381run-ci label. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13486lmsys/gpt-oss-20b-bf16 to model validation check by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13557Olmo3 by @vincentzed in https://github.com/sgl-project/sglang/pull/13666send_one.py by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13768allocate_lens and enable over-allocation by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13478per_commit -> registered by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13928predict shape does not match verify output shapes by @timmy-feng in https://github.com/sgl-project/sglang/pull/12786get_load API by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13991--hooks to --forward-hooks by @hnyls2002 in https://github.com/sgl-project/sglang/pull/13994pr-gate after check-changes by @hnyls2002 in https://github.com/sgl-project/sglang/pull/14009SingletonCache support to Vision{Sdpa|Triton|Ascend}Attention by @netanel-haber in https://github.com/sgl-project/sglang/pull/13151Note truncated.
Your coding agent can read these notes before it upgrades. Set up the MCP server →
Nothing published for this version
Nothing published for this version
Nothing published for this version
Fix(security): block unsafe pickle deserialization to mitigate CVE-2025-10164 by @thelongestusernameofall in https://github.com/sgl-project/sglang/pul…
ipc_name of __getitem__ in some IO structs by @whybeyoung in https://github.com/sgl-project/sglang/pull/12053MagicMock to mock server_args in tests by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12154parse_json_schema_response by @CatherineSue in https://github.com/sgl-project/sglang/pull/12152ResponsesContext and fix error propagation in responses api by @CatherineSue in https://github.com/sgl-project/sglang/pull/12164--no-build-isolation by @ishandhanani in https://github.com/sgl-project/sglang/pull/12170--enable-beta-spec argument by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/12177w13_weight_fp8 and w2_weight_fp8 in UE8M0 requantization by @ch-wan in https://github.com/sgl-project/sglang/pull/12259page_table and result in same type by @Johnsonms in https://github.com/sgl-project/sglang/pull/12300BasePrefixCache by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12436from python.sglang.xxx by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12483concat_mla_absorb_q_kernel fails for long inputs by @bingps in https://github.com/sgl-project/sglang/pull/12453trtllm_fp8_per_tensor_scale_moe by @b8zhong in https://github.com/sgl-project/sglang/pull/12569--ignore-eos in PD case for benchmarking by @ishandhanani in https://github.com/sgl-project/sglang/pull/12597output_ids inconsistency by @hnyls2002 in https://github.com/sgl-project/sglang/pull/12628seed bench_serving to cache key, remove redundant function definition. by @cicirori in https://github.com/sgl-project/sglang/pull/12680--debug-tensor-dump-layers to list by @guoyuhong in https://github.com/sgl-project/sglang/pull/12691Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.4...v0.5.5
Nothing published for this version
Nothing published for this version
Nothing published for this version
[8/N] MoE Refactor: deprecate EPMoE by @ch-wan in https://github.com/sgl-project/sglang/pull/11211
detect_format and has_tool_markers by @CatherineSue in https://github.com/sgl-project/sglang/pull/11270--sampling-defaults and fix ChatCompletionRequest defaults by @CatherineSue in https://github.com/sgl-project/sglang/pull/11304ngram_utils -> ngram_info by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11316EPMoE by @ch-wan in https://github.com/sgl-project/sglang/pull/11211any_whitespace for xgrammar and llguidance backends. by @lulor in https://github.com/sgl-project/sglang/pull/8919global_config to environ by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11332global_server_args_dict by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11331global_server_args_dict" by @ch-wan in https://github.com/sgl-project/sglang/pull/11520global_server_args_dict by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11528tp_worker.worker by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11548sglang.srt.environ by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11547processing.rs by @CatherineSue in https://github.com/sgl-project/sglang/pull/11553generate_tool_constraints by @CatherineSue in https://github.com/sgl-project/sglang/pull/11562serve_grpc to launch_server and log id for HealthCheck by @CatherineSue in https://github.com/sgl-project/sglang/pull/11564/v1/chat/completions endpoint by @CatherineSue in https://github.com/sgl-project/sglang/pull/11588TokenizerManager utility function by @scottjlee in https://github.com/sgl-project/sglang/pull/11586pr-test-pd-router.yml by @CatherineSue in https://github.com/sgl-project/sglang/pull/11739max_tokens deprecation warning log by @CatherineSue in https://github.com/sgl-project/sglang/pull/11775python/sglang to ruff F401 checks and remove unused imports in files by @CatherineSue in https://github.com/sgl-project/sglang/pull/11685deep_gemm_wrapper out of quantization by @ch-wan in https://github.com/sgl-project/sglang/pull/11784lmms-eval to speed up multimodal CI by @b8zhong in https://github.com/sgl-project/sglang/pull/11000send_sone script by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11817retry() by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11834continue_final_message in ChatTemplateParams and add minijinja-contrib by @CatherineSue in https://github.com/sgl-project/sglang/pull/11882tp_size / moe_ep_size / moe_intermediate_size / weight_block_size_n by @kevin85421 in https://github.com/sgl-project/sglang/pull/11702v1/responses API by @CatherineSue in https://github.com/sgl-project/sglang/pull/11926alloc_req_slots by @rogeryoungh in https://github.com/sgl-project/sglang/pull/11616Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.3...v0.5.4
Nothing published for this version
Nothing published for this version
Nothing published for this version
[HiCache] Cleaning the deprecated host memory state by @xiezhq-hermann in https://github.com/sgl-project/sglang/pull/10778
init_incremental_detokenization by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10412--enable-custom-logit-processor: agree with cli arg by @thalahors in https://github.com/sgl-project/sglang/pull/10076get_load and shortest queue strategy by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10201--dataset-path in bench_one_batch_server by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10475kv_cache_quant_algo in ModelOpt checkpoints by @brayden-hai in https://github.com/sgl-project/sglang/pull/10336from sglang.python by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10493is_blackwell platform check by @b8zhong in https://github.com/sgl-project/sglang/pull/10498bench_one_batch_server by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10548ChatCompletionRequest and fix validation logic by @CatherineSue in https://github.com/sgl-project/sglang/pull/10675get_worker_urls_for_model in http/router.rs by @CatherineSue in https://github.com/sgl-project/sglang/pull/10754sglang.environ by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10771FutureMap by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10715environ into sglang.srt to avoid break SRT auto sync. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10791build_constraint in grpc router by @CatherineSue in https://github.com/sgl-project/sglang/pull/10881grpc/client.rs to grpc_client/sglang_scheduler.rs by @CatherineSue in https://github.com/sgl-project/sglang/pull/10924get_ip by @merrymercy in https://github.com/sgl-project/sglang/pull/10883transformers: the error: AttributeError: 'TransformersForCausalLM' object has no attribute 'tp_size' by @vincentzed in https://github.com/sgl-project/sglang/pull/9614JsonParser and LlamaParser by @CatherineSue in https://github.com/sgl-project/sglang/pull/11073get_pooled in process_single_choice by @CatherineSue in https://github.com/sgl-project/sglang/pull/11079--repeat in run_eval by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11101logprob_start_len = -1 by @CatherineSue in https://github.com/sgl-project/sglang/pull/11113.item() in paged allocator. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11156io_struct and base sglang io classes. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10133thinking_mode in eval by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11192skip_sample adjust by @hnyls2002 in https://github.com/sgl-project/sglang/pull/11225v1/responses to be more OpenAI-compatible. by @vincentzed in https://github.com/sgl-project/sglang/pull/9624Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.2...v0.5.3
Nothing published for this version
Nothing published for this version
[Vulnerability]feat(conn): set bootstrap server host by @jinmingyi1998 in https://github.com/sgl-project/sglang/pull/9931
response_format support for completion API by @cicirori in https://github.com/sgl-project/sglang/pull/9665topk>1 and page>1 for paged attention backends. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9784silu_and_mul_scaled_fp4_grouped_quant perf by @kaixih in https://github.com/sgl-project/sglang/pull/9556set_interal_state API by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9850current_platform bug by @BBuf in https://github.com/sgl-project/sglang/pull/9919Router arguments passing and build it in docker image by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9964get_fused_moe_impl_class function by @kaixih in https://github.com/sgl-project/sglang/pull/9764tokenizer_communicator_mixin by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10028is_alive timeout by @hnyls2002 in https://github.com/sgl-project/sglang/pull/10159Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.1...v0.5.2
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Improve LoRA Perf by Deprecating FlashInfer and Eliminating Redundant Tensor Ops by @lifuhuang in https://github.com/sgl-project/sglang/pull/8940
cuda < 12.4 by @hnyls2002 in https://github.com/sgl-project/sglang/pull/8701routed_scaling_factor is None by @hnyls2002 in https://github.com/sgl-project/sglang/pull/8709--attention-backend triton by @ch-wan in https://github.com/sgl-project/sglang/pull/8723renormalize=False in Triton kernels by @ch-wan in https://github.com/sgl-project/sglang/pull/8735once_cell with LazyLock in worker.rs and remove once_cell dependency from Cargo.toml by @htiennv in https://github.com/sgl-project/sglang/pull/8698get_col_major_tma_aligned_tensor for Blackwell deepgemm in EpMoE by @kaixih in https://github.com/sgl-project/sglang/pull/8955builtin_tools by @CatherineSue in https://github.com/sgl-project/sglang/pull/9114num_token_non_padded by @ch-wan in https://github.com/sgl-project/sglang/pull/9107sglang.srt.layers.quantization.scalar_types with sgl_kernel.scalar_type by @Hongbosherlock in https://github.com/sgl-project/sglang/pull/8951from python.sglang.srt -> from sglang.srt by @netanel-haber in https://github.com/sgl-project/sglang/pull/9268update_tensor_inplace function by @b8zhong in https://github.com/sgl-project/sglang/pull/6307CMakeLists.txt binary_dir by @EduardDurech in https://github.com/sgl-project/sglang/pull/7019--allow-auto-truncate argument in tokenizer manager. by @hnyls2002 in https://github.com/sgl-project/sglang/pull/9391required by @CatherineSue in https://github.com/sgl-project/sglang/pull/9525Full Changelog: https://github.com/sgl-project/sglang/compare/v0.4.10...v0.5.1
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
This is a regular release with many new optimizations, features, and fixes. Please checkout the following exciting roadmaps and blogs
This is a regular release with many new optimizations, features, and fixes. Please checkout the following exciting roadmaps and blogs
--model as an alias for --model-path in server_args by @CatherineSue in https://github.com/sgl-project/sglang/pull/7505docs/references/production_metrics.md by @rudeigerc in https://github.com/sgl-project/sglang/pull/7741server_args.py & Fix session control & Other cleanups by @merrymercy in https://github.com/sgl-project/sglang/pull/7748pybase64 instead of base64 by @b8zhong in https://github.com/sgl-project/sglang/pull/7724get_model_info endpoint by @Arist12 in https://github.com/sgl-project/sglang/pull/7660srt/layer/quantization by @ch-wan in https://github.com/sgl-project/sglang/pull/7989select_experts by @ch-wan in https://github.com/sgl-project/sglang/pull/7966torch.compile in forward pass by @BBuf in https://github.com/sgl-project/sglang/pull/8353num_local_experts by @kaixih in https://github.com/sgl-project/sglang/pull/8453Full Changelog: https://github.com/sgl-project/sglang/compare/v0.4.8...v0.4.10
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Re-structured the OpenAI-compatible server to support production and enterprise environments. Key improvements include:
Re-structured the OpenAI-compatible server to support production and enterprise environments. Key improvements include:
Consistent metrics and logging for better observability and debugging.
Unified error handling, request validation, and processing logic for improved reliability and maintainability.
Improved request tracking across sessions and components.
Fixed bugs in embedding requests and reasoning parsers.
This work was a collaborative effort involving engineers from academic and industry institutions. Special thanks to the Oracle Cloud team and the SGLang team and community — including @slin1237, @CatherineSue, @key4ng, @JustinTong0323, @jhinpan, @yhyang201, @woodx9 and @whybeyoung — for their invaluable contributions.
Added support for DeepSeek R1 with FP4 and MTP on NVIDIA Blackwell GPU.
Integrated FlashInfer NVFP4 MoE, supporting TP, EP, and DP.
Supported 2-stream shared expert execution.
Achieved up to 90 TPS per user at isl/osl/bs = 1k/1k/16 on B200.
Further optimization in progress. Special thanks to the FlashInfer, NVIDIA Enterprise Products, Novita AI, DataCrunch, Google Cloud, and SGLang teams — especially @Alcanderian and @pyc96 — for their critical contributions.
The sglang/srt/openai_api directory has been removed and replaced with sglang/srt/entrypoints/openai.
Update your imports to the new module path. For example:
- from sglang.srt.openai_api.protocol import Tool
+ from sglang.srt.entrypoints.openai.protocol import Tool
INFO to DEBUG for dp and add force quit for tokenizer manager by @ishandhanani in https://github.com/sgl-project/sglang/pull/7251_normalize_rid before other normalization in io_struct by @CatherineSue in https://github.com/sgl-project/sglang/pull/7363openai_api with entrypoints/openai by @CatherineSue in https://github.com/sgl-project/sglang/pull/7351TokenToKVPoolAllocator by @hnyls2002 in https://github.com/sgl-project/sglang/pull/7414BaseFormatDetector.parse_streaming_increment by @CatherineSue in https://github.com/sgl-project/sglang/pull/7479Full Changelog: https://github.com/sgl-project/sglang/compare/v0.4.7...v0.4.8
Nothing published for this version
The PD disaggregation and large-scale EP functionalities from the blog post have now been fully merged into the latest release.
The PD disaggregation and large-scale EP functionalities from the blog post have now been fully merged into the latest release.
The blog has been successfully reproduced by over six industry teams, including the TensorRT LLM team.
SGLang’s large-scale EP is now actively used by leading organizations such as Cursor, Qwen, Alimama, Alibaba Cloud, iFlytek, and more. It has been deployed and validated at large scale, running on GPU clusters with thousands of devices.
PD disaggregation and large-scale EP, in addition to supporting DeepSeek V3/R1, now also support Qwen 3 in the latest release.
Full Blackwell support for DeepSeek V3/R1, Llama 4, and Qwen 3. Further optimizations are underway.
SGLang's DeepSeek V3/R1 now achieves 190 TPS on single H200, outperforming other frameworks by over 50%.
We extend our sincere thanks to the following contributors, listed in alphabetical order: Alibaba Cloud, AMD Team, Ant Group, Baseten Team, Cursor Team, Dynamo Team, EAGLE Team, FlashInfer Team, Google Vertex AI Team, iFlytek MaaS Team, Intel Team, LinkedIn Team, Meituan Team, Microsoft Copilot Team, Mooncake Team, NVIDIA Team, Oracle Team, Qwen Team, Voltage Park Team and open source community users. Your support and collaboration are deeply appreciated!
max_completion_tokens for OpenAIChatCompletions by @CatherineSue in https://github.com/sgl-project/sglang/pull/5857calculate_num_image_tokens from qwen2_vl.py by @JustinTong0323 in https://github.com/sgl-project/sglang/pull/5783chat_template_kwargs documentation by @vincentzed in https://github.com/sgl-project/sglang/pull/5679SGLANG_ and SGL_ environment variables by @b8zhong in https://github.com/sgl-project/sglang/pull/6206--moe-dense-tp-size=1 by @ch-wan in https://github.com/sgl-project/sglang/pull/5657launch_dummy_health_check_server to start inside of running asyncio loop by @ishandhanani in https://github.com/sgl-project/sglang/pull/6330return_hidden_states for the OpenAI API by @kyle-pena-kuzco in https://github.com/sgl-project/sglang/pull/6137return_hidden_states for the OpenAI API (#6137)" by @zhyncs in https://github.com/sgl-project/sglang/pull/6440Cargo.lock, add it into .gitignore by @hnyls2002 in https://github.com/sgl-project/sglang/pull/6438required and specific function mode by @CatherineSue in https://github.com/sgl-project/sglang/pull/6550parse_streaming_increment by @CatherineSue in https://github.com/sgl-project/sglang/pull/6715n_share_experts_fusion as num_fused_shared_experts by @ch-wan in https://github.com/sgl-project/sglang/pull/6735num_fused_shared_experts as num_shared_experts when shared_experts fusion is not disabled by @ch-wan in https://github.com/sgl-project/sglang/pull/6736Full Changelog: https://github.com/sgl-project/sglang/compare/v0.4.6...v0.4.7
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
One column per month.