PyPI · #5266 most downloaded on PyPI

mlx-lm

0.32.0548K downloads/moml-explore/mlx-lm ↗New releases in your daily digest, changelog included.

LLMs with MLX and the Hugging Face Hub

Last release 3 days ago
01 Oct 2026
Ships unpredictably
gaps range from 9 days to 5 months
Some releases are documented
notes for 23 of the last 60 stable releases
1 version withdrawn
withdrawn after publishing
3 years old
97 releases · first in 2024

Release timeline

97 releases since 2024

202420252026

Releases

0.32.01 Oct 2026
Co-authored-by: michalk8 46717574+michalk8@users.noreply.github.com
Assisted-by: Codex:GPT-5.6

Co-authored-by: michalk8 46717574+michalk8@users.noreply.github.com
Open source →
0.31.322 Apr 2026
Thread local generation stream to accompany MLX v0.31.2
Highlights

Lots of bugfixes

Thread local generation stream to accompany MLX v0.31.2

What's Changed

Bump the patch version by @angeloskath in #1124

Fix batch dimension mismatch in BatchKVCache and BatchRotatingKVCache extend() by @razorback16 in #1141

Fix parallel tool call handling in server by @kernelpool in #1170

Fix MiniMax M2 parallel tool calling by @kernelpool in #1171

Fix missing tree_reduce import in models/cache.py by @siiea-ai in #1165

Apertus tie_word_embeddings fix by @BlackSamorez in #1143

Fix batch dimension mismatch in ArraysCache extend() by @techtoboggan in #1169

Fix dwq: check for actual safetensors in target_dir by @micuentadecasa in #1173

fix: handle NoneType check for think tokens in TokenizerWrapper by @yuetyeelo2855 in #1167

Fix Gemma4 tool parser: support hyphenated function names and braces in string args by @AkashKhamkar in #1150

Fix empty tool_call_end breaking Mistral tool calls by @eyupcanakman in #1151

Fix ArraysCache extend by @angeloskath in #1177

Fix Gemma 4 KV-shared layers creating unused projections by @glyphVault in #1158

Thread local generation stream by @angeloskath in #1090

New Contributors

@razorback16 made their first contribution in #1141

@siiea-ai made their first contribution in #1165

@BlackSamorez made their first contribution in #1143

@techtoboggan made their first contribution in #1169

@micuentadecasa made their first contribution in #1173

@yuetyeelo2855 made their first contribution in #1167

@AkashKhamkar made their first contribution in #1150

@glyphVault made their first contribution in #1158

Full Changelog: v0.31.2...v0.31.3

0.31.27 Apr 2026

Caching system prompt and user messages for non-trimmable caches

Highlights

Caching system prompt and user messages for non-trimmable caches
Batch generator refactoring

What's Changed

Bump the patch version by @angeloskath in #959
Presence and frequency penalties by @angeloskath in #971
Eval self.left_padding whenever it is updated in BatchRotatingKVCache by @rltakashige in #960
Late binding caused incorrect cache checkpoint by @angeloskath in #976
Move to metal agnostic device_info by @angeloskath in #979
Fix CompletionsDataset mask_prompt crash by @eyupcanakman in #967
Bump the patch version by @angeloskath in #981
Fix test after latest MLX update by @angeloskath in #996
Clear cache trainer memory by @N8python in #986
feat(server): add --allowed-origins by @nwtgck in #987
Delta net precision by @angeloskath in #997
avoid mutating input in SuScaledRoPE and YarnRoPE by @mm65x in #1003
handle missing content-length header in server by @mm65x in #1001
fall back to ast.literal_eval for malformed JSON in qwen3_coder tool parser by @mm65x in #1004
Nemotron super support by @angeloskath in #992
Supporting delay in mlx_lm benchmark by @AndreasPlt in #1010
Fix flaky test by @angeloskath in #1020
Fix missing cache advance from qwen 3.5 by @angeloskath in #1024
Refactor LRUPromptCache by @angeloskath in #1019
Fix SSM dt clamp default for Nemotron-H by @kernelpool in #1026
Inserting logits processors into BatchGenerator in batch_generate by @arthurhjorth in #1008
fix: break shared-buffer memory leak in GatedDeltaNet cache by @adurham in #1077
Fix PromptTrie.pop_prefixes() off-by-one when pruning immediate prefixes by @LxYuan0420 in #1078
Batch generation refactoring and various fixes by @angeloskath in #1072
perf: use max instead of argsort in apply_min_p sampling by @matteocelani in #1083
Add gemma 4 by @Blaizzy in #1093
Bring back max-kv-size to the batch generator by @angeloskath in #1106
Add Gemma 4 tool call parser by @nicdavidson in #1105
Fix Gemma 4 quantized per-layer projection loading by @spicyneuron in #1112
Fix output corruption in speculative decoding by @kernelpool in #1109
Gemma4 final fixes and multi-token think/tool start/end by @angeloskath in #1114
Align batch logits processor token contract by @neilmehta24 in #1115

New Contributors

@rltakashige made their first contribution in #960
@eyupcanakman made their first contribution in #967
@nwtgck made their first contribution in #987
@mm65x made their first contribution in #1003
@AndreasPlt made their first contribution in #1010
@arthurhjorth made their first contribution in #1008
@adurham made their first contribution in #1077
@LxYuan0420 made their first contribution in #1078
@matteocelani made their first contribution in #1083
@nicdavidson made their first contribution in #1105

Full Changelog: v0.31.0...v0.31.2

Open source →

0.31.07 Mar 2026withdrawn: Batched KV cache cross contamination

Fix save/load of CacheList by @angeloskath in #886

What's Changed

Fix save/load of CacheList by @angeloskath in #886
Share model by @angeloskath in #871
Fix mixed quant predicates for MLA models by @spicyneuron in #892
Add JoyAI LLM Flash by @kernelpool in #894
perplexity: add --trust-remote-code option by @ivanfioravanti in #896
server: add usage.prompt_tokens_details.cached_tokens to json response by @percontation in #849
Fix qwen3.5 casting to fp32 by @awni in #902
Fix sharded rms norm in MiniMax M2.5 by @angeloskath in #898
Bump for next version by @awni in #904
Add tie_word_embeddings modulars in mistral and qwen3 moe by @Goekdeniz-Guelmez in #889
Allow reading LFM2 models nested rope params by @ykhrustalev in #908
Improve the cache size limits by @angeloskath in #906
Make the cache limits more friendly by @angeloskath in #910
Add 'mx.clear_cache()' to piecewise prompt processing in server. by @N8python in #917
Add filter guard to ArraysCache.nbytes property by @f1yn in #918
Add tokens to eval to avoid large graphs when they are not used by @awni in #924
Clear the cache during batch generation by @awni in #926
Fix qwen3.5 sanitize by @awni in #928
step3p5: use rotating cache for sliding attention layers by @lyonsno in #949
Proposal: --prefill-step-size as cmd line argument for speed/memory usage trade-off by @Abioy in #943
fix: convert() uses incorrect defaults for quantization mode by @spicyneuron in #935
Bump minor by @angeloskath in #954
Ensure normalization does not promote to fp32 by @angeloskath in #951
Better caching in the server by @angeloskath in #911
Adds tensor parallelism for Qwen 3.5 by @angeloskath in #957

New Contributors

@spicyneuron made their first contribution in #892
@ykhrustalev made their first contribution in #908
@f1yn made their first contribution in #918
@lyonsno made their first contribution in #949
@Abioy made their first contribution in #943

Full Changelog: v0.30.7...v0.31.0

Open source →

0.30.712 Feb 2026

Fix Kimi Linear by @kernelpool in #853

What's Changed

Fix Kimi Linear by @kernelpool in #853
Bump version for next release by @awni in #865
Pythonic tool calling for LFM2 models by @viktike in #864
Fix DeepSeek V3.2 indexer and weight loading by @kernelpool in #866
Make validation set optional in training process by @Goekdeniz-Guelmez in #857
Mistral tool parser by @awni in #874
LongCat MLA by @kernelpool in #868
[MODEL] support qwen3.5 series w/o vision by @JJJYmmm in #869
Faster DSV32 generation by @kernelpool in #885
Add GLM5 by @Goekdeniz-Guelmez in #867

New Contributors

@viktike made their first contribution in #864
@JJJYmmm made their first contribution in #869

Full Changelog: v0.30.6...v0.30.7

Open source →

0.30.64 Feb 2026

Transformers v5 by @awni in #811

What's Changed

Transformers v5 by @awni in #811
Add LongCat Flash tool parser by @kernelpool in #810
Add Kimi-K2.5 by @kernelpool in #813
Bump mlx version and version by @awni in #816
Fix NemotronH config compatibility with HuggingFace format by @LuqDaMan in #820
Fix for Exception - MultiLinear.to_quantized() missing 'mode' by @inferencers in #809
Fix Kimi K2.5 tool call handling by @kernelpool in #821
Actually add cli by @awni in #823
Add LongCat Flash Lite by @kernelpool in #819
Fix mixed quant by @awni in #825
Support distributed inference in the server by @angeloskath in #741
fix cli by @solarpunkin in #827
Enable loading custom models by @awni in #830
Allow default creation of BatchRotatingKVCache instead of BatchKVCache in batch mode by @christian-lms in #834
Add Step 3.5 Flash by @kernelpool in #836
server: support chat_template_kwargs and top_logprobs by @percontation in #829
fix: handle GLM 4.7 tool call fallbacks by @jalehman in #792
Deepseek V3.2 implementation fixes by @sjug in #838
Fix Step 3.5 Flash model conversion by @kernelpool in #840
Fix batch mamba by @awni in #842
Fix sliding window mask during generation by @kernelpool in #843
DSV3 MLA by @awni in #839

New Contributors

@jalehman made their first contribution in #792

Full Changelog: v0.30.5...v0.30.6

Open source →

0.30.525 Jan 2026

import logging as it throws no logging error in place of actual error by @Maanas-Verma in #778

What's Changed

import logging as it throws no logging error in place of actual error by @Maanas-Verma in #778
server: use OpenAI compatible finish_reason by @percontation in #782
move Xielu Activation in Apertus to activations.py by @Goekdeniz-Guelmez in #772
bump transformers by @awni in #746
Update glm4_moe_lite to store KV latent in cache by @N8python in #780
Adding TeleChat3 by @Goekdeniz-Guelmez in #773
add kimi tool parser by @Evanev7 in #791
Allow qq ops with activation quantization by @awni in #749
fix: use correct variable for logprobs in batch generation by @LuqDaMan in #800
Sync random seed across ranks in distributed chat by @kernelpool in #801
Fix ArraysCache.from_state not initializing left_padding and lengths by @lpalbou in #807

New Contributors

@Maanas-Verma made their first contribution in #778
@percontation made their first contribution in #782
@LuqDaMan made their first contribution in #800
@lpalbou made their first contribution in #807

Full Changelog: v0.30.4...v0.30.5

Open source →

0.30.419 Jan 2026

Add AWQ/GPTQ weight transformation utilities by @ericcurtin in #730

What's Changed

Add AWQ/GPTQ weight transformation utilities by @ericcurtin in #730
Add IQuest Coder V1 Loop variant by @kernelpool in #716
Fix sliding window batching by @awni in #738
Fix Batch Generation: Add extract method to ArraysCache for item retrieval by @Goekdeniz-Guelmez in #740
Make MambaCache compatible with batch generation for nemotron-h by @nikhilmitrax in #690
Add a server benchmark for continuous batching by @awni in #728
Fix tools parameter in apply_chat_template call by @kernelpool in #747
Refactor tokenizer error handling to use warnings instead of exceptio… by @cubist38 in #744
Make cache list batchable by @awni in #743
Fix batch generation for IQuestLoopCoder model by @kernelpool in #748
Fix type hint and pydoc for batch_generate by @tibbes in #745
Handle empty caches during batch merge by @ivanfioravanti in #755
Update for latest mlx by @awni in #759
Use compiled Swiglu by @awni in #753
Adds support for Nemotron Super 49b v1.5 by @lazarust in #756
fix(falcon_h1): support tied embeddings and correct muP scaling by @solarpunkin in #764
Fix swiglu parameter order by @kernelpool in #767
Fix CacheList batching by @kernelpool in #769
fix: unused batch_size parameter for mlx_lm.evaluate by @AndrewTan517 in #762
Add gpt-oss sharding by @Evanev7 in #761
Fix LongCat Flash extended context support by @kernelpool in #768
Add minimax tensor sharding by @Evanev7 in #760
Shard LongCat Flash by @kernelpool in #771
Add glm4 moe lite model by @ivanfioravanti in #776

New Contributors

@ericcurtin made their first contribution in #730
@nikhilmitrax made their first contribution in #690
@tibbes made their first contribution in #745
@solarpunkin made their first contribution in #764
@AndrewTan517 made their first contribution in #762
@Evanev7 made their first contribution in #761

Full Changelog: v0.30.2...v0.30.4

Open source →

0.30.018 Dec 2025

fix: server busy-waiting during idle request polling by @zenyr in https://github.com/ml-explore/mlx-lm/pull/674

What's Changed

fix: server busy-waiting during idle request polling by @zenyr in https://github.com/ml-explore/mlx-lm/pull/674
Fixes for transformers v5 by @awni in https://github.com/ml-explore/mlx-lm/pull/684
Add mimo v2 flash by @awni in https://github.com/ml-explore/mlx-lm/pull/685
More useful error message for unsupported batching by @awni in https://github.com/ml-explore/mlx-lm/pull/687
Model parallel generation by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/676
Bump to transformer v5 by @awni in https://github.com/ml-explore/mlx-lm/pull/689
Revert return dict and wrap apply_chat_template by @awni in https://github.com/ml-explore/mlx-lm/pull/691
Bump the version by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/692

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.29.0...v0.30.0

Open source →

0.28.43 Dec 2025

version by @awni in https://github.com/ml-explore/mlx-lm/pull/559

What's Changed

version by @awni in https://github.com/ml-explore/mlx-lm/pull/559
Add Minimax-M2 by @Blaizzy in https://github.com/ml-explore/mlx-lm/pull/568
Align checkpoint loading with Jamba Mini and Large by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/555
Fix dequant + minor refactor by @awni in https://github.com/ml-explore/mlx-lm/pull/572
fix eval thinking by @awni in https://github.com/ml-explore/mlx-lm/pull/578
Fixed typo in load_adapters that broke adapter loading by @jyork03 in https://github.com/ml-explore/mlx-lm/pull/583
Fix AttributeError when loading custom draft models by @kernelpool in https://github.com/ml-explore/mlx-lm/pull/590
Add gen options and CoT removal by @awni in https://github.com/ml-explore/mlx-lm/pull/587
Add parallel_residual setting to gptneox by @spotbot2k in https://github.com/ml-explore/mlx-lm/pull/586
Fixed/improved behavior of the mask_prompt feature. by @jyork03 in https://github.com/ml-explore/mlx-lm/pull/584
add MiniMax-M2 in supported models by @sriting in https://github.com/ml-explore/mlx-lm/pull/575
Fix: Remove call to deleted method by @jyork03 in https://github.com/ml-explore/mlx-lm/pull/591
Make mlx-lm more type-checker friendly by @tnadav in https://github.com/ml-explore/mlx-lm/pull/573
Fix: JSON parse error handling: avoid referencing stream before init by @jyork03 in https://github.com/ml-explore/mlx-lm/pull/592
Adding ring mini linear by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/513
Add Kimi Linear by @Blaizzy in https://github.com/ml-explore/mlx-lm/pull/577
DWQ for very large models by @awni in https://github.com/ml-explore/mlx-lm/pull/536
Fix Byte Decoder Lookup for Esoteric Single-Characters by @N8python in https://github.com/ml-explore/mlx-lm/pull/600
Fix input_embeddings prefill bug in generate_step by @Blaizzy in https://github.com/ml-explore/mlx-lm/pull/606
ACKNOWLEDGMENTS.md House keeping by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/594
FIX: Add missing sentencepiece dependency for tokenizers by @Deekshith-Dade in https://github.com/ml-explore/mlx-lm/pull/611
switch to github actions by @awni in https://github.com/ml-explore/mlx-lm/pull/618
Fix for kimi k2 by @awni in https://github.com/ml-explore/mlx-lm/pull/593
Allow providing prompt caches in batched generation by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/602
Fix olmo3 by @awni in https://github.com/ml-explore/mlx-lm/pull/628
add support for Trinity/AfMoE model by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/640
Ministral3 by @awni in https://github.com/ml-explore/mlx-lm/pull/642
Add a prompt cache that can hold multiple prompts by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/625
Fix flaky losses test by @awni in https://github.com/ml-explore/mlx-lm/pull/643
Fix lora fusion for non affine quantization by @awni in https://github.com/ml-explore/mlx-lm/pull/647
Batching in the server by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/626
Add deepseek v32 by @awni in https://github.com/ml-explore/mlx-lm/pull/512
version bump by @awni in https://github.com/ml-explore/mlx-lm/pull/649
Fix the release action by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/650

New Contributors

@jyork03 made their first contribution in https://github.com/ml-explore/mlx-lm/pull/583
@spotbot2k made their first contribution in https://github.com/ml-explore/mlx-lm/pull/586
@sriting made their first contribution in https://github.com/ml-explore/mlx-lm/pull/575
@tnadav made their first contribution in https://github.com/ml-explore/mlx-lm/pull/573
@Deekshith-Dade made their first contribution in https://github.com/ml-explore/mlx-lm/pull/611

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.28.3...v0.28.4

Open source →

0.28.317 Oct 2025Deprecated

Removing the deprecated wandb params by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/524

What's Changed

Removing the deprecated wandb params by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/524
Memory efficient ssm by @awni in https://github.com/ml-explore/mlx-lm/pull/525
Fix lora MoEs by @awni in https://github.com/ml-explore/mlx-lm/pull/522
Fix bailing moe by @awni in https://github.com/ml-explore/mlx-lm/pull/521
GPT2 Batching Fix by @shepardxia in https://github.com/ml-explore/mlx-lm/pull/529
Remove act loss and add temp in DWQ by @awni in https://github.com/ml-explore/mlx-lm/pull/500
Cleanup and simplify model I/O by @awni in https://github.com/ml-explore/mlx-lm/pull/532
Fix: Add future annotations import to qwen3_next.py for Python 3.9 compatibility by @mzau in https://github.com/ml-explore/mlx-lm/pull/533
Add lfm2 moe by @Blaizzy in https://github.com/ml-explore/mlx-lm/pull/537
Fix example command to quantize a model using GPTQ by @felladrin in https://github.com/ml-explore/mlx-lm/pull/539
minor typing issues by @mercush in https://github.com/ml-explore/mlx-lm/pull/540
Fix cuda install by @awni in https://github.com/ml-explore/mlx-lm/pull/542
Add Qwen3-VL language model implementation by @vincentamato in https://github.com/ml-explore/mlx-lm/pull/547
Fix mask for batched SSM by @awni in https://github.com/ml-explore/mlx-lm/pull/546
Added gradient accumulation to training loop by @dotvignesh in https://github.com/ml-explore/mlx-lm/pull/511
Support data parallel eval for generation tasks by @awni in https://github.com/ml-explore/mlx-lm/pull/549
Optimize Bailing MoE by @kernelpool in https://github.com/ml-explore/mlx-lm/pull/550
Adding jamba by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/544
Add Qwen3-VL (Dense) language model implementation by @vincentamato in https://github.com/ml-explore/mlx-lm/pull/553
LLM Benchmarks by @awni in https://github.com/ml-explore/mlx-lm/pull/552
Add support for nanochat by @dnakov in https://github.com/ml-explore/mlx-lm/pull/554
version by @awni in https://github.com/ml-explore/mlx-lm/pull/558

New Contributors

@shepardxia made their first contribution in https://github.com/ml-explore/mlx-lm/pull/529
@mzau made their first contribution in https://github.com/ml-explore/mlx-lm/pull/533
@felladrin made their first contribution in https://github.com/ml-explore/mlx-lm/pull/539
@mercush made their first contribution in https://github.com/ml-explore/mlx-lm/pull/540
@dotvignesh made their first contribution in https://github.com/ml-explore/mlx-lm/pull/511

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.28.2...v0.28.3

Open source →

0.28.22 Oct 2025

fix bailing moe by @awni in https://github.com/ml-explore/mlx-lm/pull/514

What's Changed

fix bailing moe by @awni in https://github.com/ml-explore/mlx-lm/pull/514
Fix batching for models with nested cache structures by @kernelpool in https://github.com/ml-explore/mlx-lm/pull/510
Fix: Correct weight masking for zero-computation experts in LongCat Flash MoE by @kernelpool in https://github.com/ml-explore/mlx-lm/pull/508
Simplify to_lora to not hardcode model types by @awni in https://github.com/ml-explore/mlx-lm/pull/515
Add Olmo3 by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/445
Make mixed quantization affect attention in DeepSeek V3, others by @n8sh1 in https://github.com/ml-explore/mlx-lm/pull/506
Add Apriel 1.5 by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/520
feat: Refactor granitemoehybrid to support dense and non-hybrid variants by @gabe-l-hart in https://github.com/ml-explore/mlx-lm/pull/518

New Contributors

@kernelpool made their first contribution in https://github.com/ml-explore/mlx-lm/pull/510

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.28.1...v0.28.2

Open source →

0.28.127 Sept 2025

Fix quant predicate by @awni in https://github.com/ml-explore/mlx-lm/pull/485

What's Changed

Fix quant predicate by @awni in https://github.com/ml-explore/mlx-lm/pull/485
Fix passing argument model_config in utils.load() by @ariahw in https://github.com/ml-explore/mlx-lm/pull/494
Fix KV cache quantization for hybrid models by @awni in https://github.com/ml-explore/mlx-lm/pull/495
fix for LFM2 by @awni in https://github.com/ml-explore/mlx-lm/pull/493
fix loading for qwen2 VL by @awni in https://github.com/ml-explore/mlx-lm/pull/491
Enable training for qwen3 next by @awni in https://github.com/ml-explore/mlx-lm/pull/496
Add batch support for sliding window cache by @awni in https://github.com/ml-explore/mlx-lm/pull/487
qwen3 next batching by @awni in https://github.com/ml-explore/mlx-lm/pull/478
Add Falcon H1 by @Blaizzy in https://github.com/ml-explore/mlx-lm/pull/231
Fix RotatingKVCache update by @awni in https://github.com/ml-explore/mlx-lm/pull/503
Add Code World Model support by @dnakov in https://github.com/ml-explore/mlx-lm/pull/505
Use depends in pipeline parallel by @awni in https://github.com/ml-explore/mlx-lm/pull/483

New Contributors

@ariahw made their first contribution in https://github.com/ml-explore/mlx-lm/pull/494

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.28.0...v0.28.1

Open source →

0.28.017 Sept 2025

Allow fp8 by @awni in https://github.com/ml-explore/mlx-lm/pull/431

What's Changed

Allow fp8 by @awni in https://github.com/ml-explore/mlx-lm/pull/431
Avoid cache-trimming crash in server for longcat chat and baichuan_m1 by @n8sh1 in https://github.com/ml-explore/mlx-lm/pull/434
Fix hunyuan v1 dense by @awni in https://github.com/ml-explore/mlx-lm/pull/440
Changes needed to facilitate batching by @awni in https://github.com/ml-explore/mlx-lm/pull/430
remove manual conv class in mamba1 by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/436
adding Kwai-Klear/Klear-46B-A2.5B-Instruct by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/437
Add lille 130m by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/429
model: GraniteMoeHybrid by @gabe-l-hart in https://github.com/ml-explore/mlx-lm/pull/442
fix server paths by @awni in https://github.com/ml-explore/mlx-lm/pull/448
sdpa with sinks by @awni in https://github.com/ml-explore/mlx-lm/pull/418
fix(quantization): Parameterize hardcoded group_size in mixed_quant_predicate_builder by @squaredice in https://github.com/ml-explore/mlx-lm/pull/449
Adding Ling Mini by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/450
Adding Qwen3 Next by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/441
Faster ssm by @awni in https://github.com/ml-explore/mlx-lm/pull/451
Update bitnet, nemotron h to use build in relu2 from MLX by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/446
fix qwen3 next by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/453
Adding GLM by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/457
Add an introduction to the default LLM in README.md by @aopstudio in https://github.com/ml-explore/mlx-lm/pull/461
Fix TypeError: Model.__call__() got an unexpected keyword argument 'mask' for qwen2_vl, mistral3 by @neilmehta24 in https://github.com/ml-explore/mlx-lm/pull/464
Add groups to ssm kernel and update more models by @awni in https://github.com/ml-explore/mlx-lm/pull/456
Fix gemma3 window mask by @awni in https://github.com/ml-explore/mlx-lm/pull/465
Batch generation by @awni in https://github.com/ml-explore/mlx-lm/pull/443
Batch support for mamba-style models by @awni in https://github.com/ml-explore/mlx-lm/pull/468
fix: handle cache offset safely for mamba error by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/472
Adds LLaMA 4 text model implementation in MLX by @robbiemu in https://github.com/ml-explore/mlx-lm/pull/469
Allow sampler to work with batched_generate by @N8python in https://github.com/ml-explore/mlx-lm/pull/473
Adding support for mamba2 by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/392
Fix llama4 text and make trainable by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/474
Extends quantization predicate with config by @robbiemu in https://github.com/ml-explore/mlx-lm/pull/476
Gated-Delta Fused Kernel (Qwen3Next) by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/454

New Contributors

@gabe-l-hart made their first contribution in https://github.com/ml-explore/mlx-lm/pull/442
@squaredice made their first contribution in https://github.com/ml-explore/mlx-lm/pull/449
@aopstudio made their first contribution in https://github.com/ml-explore/mlx-lm/pull/461
@robbiemu made their first contribution in https://github.com/ml-explore/mlx-lm/pull/469

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.27.1...v0.28.0

Open source →

0.27.14 Sept 2025

Fix mlx_lm.perplexity seed w/ np.random.seed to ensure determinism across runs by @N8python in https://github.com/ml-explore/mlx-lm/pull/415

What's Changed

Fix mlx_lm.perplexity seed w/ np.random.seed to ensure determinism across runs by @N8python in https://github.com/ml-explore/mlx-lm/pull/415
server allow specifying seed by @n8sh1 in https://github.com/ml-explore/mlx-lm/pull/414
Adding ibm Granite MoE by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/413
had to add self.args and self.model_type into the model class for mlx-lm-lore by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/422
add Apertus from Swiss AI by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/421
Add nemotron h by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/407
Adding longcat flash by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/423
Fix Nemotron H loading error by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/426

New Contributors

@n8sh1 made their first contribution in https://github.com/ml-explore/mlx-lm/pull/414

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.27.0...v0.27.1

Open source →

0.27.029 Aug 2025

Add mlx_lm.perplexity by @N8python in https://github.com/ml-explore/mlx-lm/pull/397

What's Changed

Add mlx_lm.perplexity by @N8python in https://github.com/ml-explore/mlx-lm/pull/397
Benchmark script by @awni in https://github.com/ml-explore/mlx-lm/pull/396
Don't reload default model by @awni in https://github.com/ml-explore/mlx-lm/pull/400
only apply lm_head to the last token by @awni in https://github.com/ml-explore/mlx-lm/pull/406
Fix prompt cache corruption when generation is interrupted by @dojoteef in https://github.com/ml-explore/mlx-lm/pull/405
support mxfp4 by @awni in https://github.com/ml-explore/mlx-lm/pull/385
version by @awni in https://github.com/ml-explore/mlx-lm/pull/410

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.26.4...v0.27.0

Open source →

0.26.425 Aug 2025

Revert symmetric kl by @awni in https://github.com/ml-explore/mlx-lm/pull/359

What's Changed

Revert symmetric kl by @awni in https://github.com/ml-explore/mlx-lm/pull/359
Adding bailing_moe (ling-lite, -plus, -coder) by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/369
Add SwanLab experiment tracking support for MLX by @ShaohonChen in https://github.com/ml-explore/mlx-lm/pull/317
Fix gpt-oss lora nan by @awni in https://github.com/ml-explore/mlx-lm/pull/370
Properly tie embeddings and lm head for gemma3 by @awni in https://github.com/ml-explore/mlx-lm/pull/373
Fix distributed evaluate by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/368
Add SSE keepalive to stop client disconnects during prompt processing by @dysangel in https://github.com/ml-explore/mlx-lm/pull/362
Add LFM2-VL model implementation by @christian-lms in https://github.com/ml-explore/mlx-lm/pull/378
Adding trust_remote_code=True for training by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/383
remove comma and add muon by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/381
add into the lora to layer utils by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/382
Make KL and JS metal kernels only if metal is available by @vsabolcec in https://github.com/ml-explore/mlx-lm/pull/387
fix sampling with small top k by @awni in https://github.com/ml-explore/mlx-lm/pull/388
fix window attention mask by @awni in https://github.com/ml-explore/mlx-lm/pull/390
Add support for ByteDance Seed-OSS-36B-Instruct model by @dnakov in https://github.com/ml-explore/mlx-lm/pull/391
Add Qwen2-VL model implementation by @vincentamato in https://github.com/ml-explore/mlx-lm/pull/384

New Contributors

@ShaohonChen made their first contribution in https://github.com/ml-explore/mlx-lm/pull/317
@dysangel made their first contribution in https://github.com/ml-explore/mlx-lm/pull/362
@dnakov made their first contribution in https://github.com/ml-explore/mlx-lm/pull/391
@vincentamato made their first contribution in https://github.com/ml-explore/mlx-lm/pull/384

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.26.3...v0.26.4

Open source →

0.26.36 Aug 2025

Resolve streaming last token error and correct total token usage by @zenyr in https://github.com/ml-explore/mlx-lm/pull/342

What's Changed

Resolve streaming last token error and correct total token usage by @zenyr in https://github.com/ml-explore/mlx-lm/pull/342
Fix NameError in loglikelihood_rolling method by @snellingio in https://github.com/ml-explore/mlx-lm/pull/339
fix error on unsupported response type in server by @emmanuel-ferdman in https://github.com/ml-explore/mlx-lm/pull/344
Add --trust-remote-code cli option by @dojoteef in https://github.com/ml-explore/mlx-lm/pull/319
Add validation set for DWQ by @awni in https://github.com/ml-explore/mlx-lm/pull/343
feat: add --confirm-run-unsafe-code CLI option to allow execution of untrusted code by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/348
Allow per model quant config by @awni in https://github.com/ml-explore/mlx-lm/pull/349
Add gpt_oss model by @christian-lms in https://github.com/ml-explore/mlx-lm/pull/354
Jensen-Shannon divergence loss kernel by @vsabolcec in https://github.com/ml-explore/mlx-lm/pull/352
Route the gpt_oss to fused sdpa by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/356
Hunyuan V1 Dense model support by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/351
Add Additional Features of GPT-OSS Model : Lora, Alternating attention, MoE Support by @Shashikant86 in https://github.com/ml-explore/mlx-lm/pull/357

New Contributors

@zenyr made their first contribution in https://github.com/ml-explore/mlx-lm/pull/342
@snellingio made their first contribution in https://github.com/ml-explore/mlx-lm/pull/339
@emmanuel-ferdman made their first contribution in https://github.com/ml-explore/mlx-lm/pull/344
@dojoteef made their first contribution in https://github.com/ml-explore/mlx-lm/pull/319
@vsabolcec made their first contribution in https://github.com/ml-explore/mlx-lm/pull/352
@Shashikant86 made their first contribution in https://github.com/ml-explore/mlx-lm/pull/357

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.26.2...v0.26.3

Open source →

0.26.230 Jul 2025

chore: fix gemma3n intermediate_size config by @mzbac in https://github.com/ml-explore/mlx-lm/pull/332

What's Changed

chore: fix gemma3n intermediate_size config by @mzbac in https://github.com/ml-explore/mlx-lm/pull/332
Add GLM4.5 by @awni in https://github.com/ml-explore/mlx-lm/pull/333
Changed GLM-4 MoE support for DWQ quantization by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/336
Add system prompt to chat script by @jussikuosa in https://github.com/ml-explore/mlx-lm/pull/334
Initialize empty cache when cache=None for Gemma‑3n models by @brchristian in https://github.com/ml-explore/mlx-lm/pull/323

New Contributors

@mzbac made their first contribution in https://github.com/ml-explore/mlx-lm/pull/332
@jussikuosa made their first contribution in https://github.com/ml-explore/mlx-lm/pull/334
@brchristian made their first contribution in https://github.com/ml-explore/mlx-lm/pull/323

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.26.1...v0.26.2

Open source →

0.26.126 Jul 2025

Add dsv3 for lora by @awni in https://github.com/ml-explore/mlx-lm/pull/284

What's Changed

Add dsv3 for lora by @awni in https://github.com/ml-explore/mlx-lm/pull/284
GPTQ quantization by @awni in https://github.com/ml-explore/mlx-lm/pull/279
Fix MoE fine tuning by @awni in https://github.com/ml-explore/mlx-lm/pull/288
fix hunyuan by @awni in https://github.com/ml-explore/mlx-lm/pull/286
Allow trust_remote_code in convert.py by @christian-lms in https://github.com/ml-explore/mlx-lm/pull/289
Type Signature Fixes by @MattBeton in https://github.com/ml-explore/mlx-lm/pull/290
Fix gemma3n config load bug by @neilmehta24 in https://github.com/ml-explore/mlx-lm/pull/292
kimi k2 by @awni in https://github.com/ml-explore/mlx-lm/pull/293
Add LFM2 by @Blaizzy in https://github.com/ml-explore/mlx-lm/pull/291
feat: DWQ for Hunyuan-A13B-Instruct and trust_remote_code argument by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/303
fix: update import for huggingface model in evaluate.py by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/275
Fix ddp workers loading the same data by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/294
Fix server finish reason by @awni in https://github.com/ml-explore/mlx-lm/pull/307
Add support for SGD & Adafactor by @N8python in https://github.com/ml-explore/mlx-lm/pull/306
Allow empty prompt with input_embeddings by @will-lms in https://github.com/ml-explore/mlx-lm/pull/308
fix naive detokenizer by @awni in https://github.com/ml-explore/mlx-lm/pull/312
add exaone4 by @awni in https://github.com/ml-explore/mlx-lm/pull/310
add v1/models/repo_id by @awni in https://github.com/ml-explore/mlx-lm/pull/313
Update W&B logging crash in MLX-LM-LORA by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/316
Fix DSV3 training by @awni in https://github.com/ml-explore/mlx-lm/pull/324
Lora works with cuda backend by @awni in https://github.com/ml-explore/mlx-lm/pull/330
Adding Muon Optimizer by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/325

New Contributors

@christian-lms made their first contribution in https://github.com/ml-explore/mlx-lm/pull/289
@MattBeton made their first contribution in https://github.com/ml-explore/mlx-lm/pull/290
@N8python made their first contribution in https://github.com/ml-explore/mlx-lm/pull/306

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.26.0...v0.26.1

Open source →

0.26.08 Jul 2025

fix dynamic quant for bias by @awni in https://github.com/ml-explore/mlx-lm/pull/216

What's Changed

fix dynamic quant for bias by @awni in https://github.com/ml-explore/mlx-lm/pull/216
feat: add MiniCPM4 model structure code minicpm4.py and minicpm4 mode… by @pzc163 in https://github.com/ml-explore/mlx-lm/pull/212
Implementation of AFM in MLX by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/232
Support cuda back-end by @awni in https://github.com/ml-explore/mlx-lm/pull/241
Fx cast predicate by @awni in https://github.com/ml-explore/mlx-lm/pull/243
Pipe input_embeddings through mistral3 model_type by @will-lms in https://github.com/ml-explore/mlx-lm/pull/254
Fix IndexError in NaiveStreamingDetokenizer by @Muhtasham in https://github.com/ml-explore/mlx-lm/pull/253
Enable tool use in the server and add an example using openai client by @awni in https://github.com/ml-explore/mlx-lm/pull/217
Allow models to be pickled + test by @awni in https://github.com/ml-explore/mlx-lm/pull/261
Gemma3n text only support by @awni in https://github.com/ml-explore/mlx-lm/pull/258
Add Ernie4.5 by @johnmai-dev in https://github.com/ml-explore/mlx-lm/pull/263
Allow converting local models by @awni in https://github.com/ml-explore/mlx-lm/pull/265
Adding support for rednote-hilab/dots.llm1.inst by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/211
Add bitnet1.58 with custom metal kernel by @Blaizzy in https://github.com/ml-explore/mlx-lm/pull/219
Adding ernie4.5 moe by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/267
Add Hunyuan-A13B-Instruct MoE support by @ivanfioravanti in https://github.com/ml-explore/mlx-lm/pull/273
Parse JSON arguments when OpenAI tool calling by @crodjer in https://github.com/ml-explore/mlx-lm/pull/271
Add SmolLM3. by @Vaibhavs10 in https://github.com/ml-explore/mlx-lm/pull/272
Feat: add falcon-e support for bitnet models by @younesbelkada in https://github.com/ml-explore/mlx-lm/pull/268
Allow generation without README by @will-lms in https://github.com/ml-explore/mlx-lm/pull/278
Allow prompt and input_embeddings by @will-lms in https://github.com/ml-explore/mlx-lm/pull/266
KL loss and memory improvements for DWQ and dynamic quant by @angeloskath in https://github.com/ml-explore/mlx-lm/pull/280
remove sentencepiece by @awni in https://github.com/ml-explore/mlx-lm/pull/282
Automate PyPi upload by @awni in https://github.com/ml-explore/mlx-lm/pull/283

New Contributors

@pzc163 made their first contribution in https://github.com/ml-explore/mlx-lm/pull/212
@Muhtasham made their first contribution in https://github.com/ml-explore/mlx-lm/pull/253
@crodjer made their first contribution in https://github.com/ml-explore/mlx-lm/pull/271
@Vaibhavs10 made their first contribution in https://github.com/ml-explore/mlx-lm/pull/272
@younesbelkada made their first contribution in https://github.com/ml-explore/mlx-lm/pull/268

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.25.1...v0.26.0

Open source →

0.25.17 Jun 2025

Add input_embeddings input to generate_step, Gemma 3, Qwen 2 by @mattjcly in https://github.com/ml-explore/mlx-lm/pull/179

What's Changed

Add input_embeddings input to generate_step, Gemma 3, Qwen 2 by @mattjcly in https://github.com/ml-explore/mlx-lm/pull/179
Wandb fix by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/177
fix upload to hub usage by @IANTHEREAL in https://github.com/ml-explore/mlx-lm/pull/180
Pixtral text support, pipe input_embeddings through llama arch by @mattjcly in https://github.com/ml-explore/mlx-lm/pull/181
Fix server cache by @awni in https://github.com/ml-explore/mlx-lm/pull/183
fix WandB setting in lora_congig.yaml by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/184
fix server.py null tool content (#185) by @jpangburn in https://github.com/ml-explore/mlx-lm/pull/186
Fix URL for LoRA Data in Error Message by @rgeorgi in https://github.com/ml-explore/mlx-lm/pull/188
Faster top-p and min-p sampling by @awni in https://github.com/ml-explore/mlx-lm/pull/187
Add Evaluation Progress by @rgeorgi in https://github.com/ml-explore/mlx-lm/pull/158
DWQ updates by @awni in https://github.com/ml-explore/mlx-lm/pull/196
nits for qwen3_moe file by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/199
Update LoRA training logic to reset lora_parameters when fine-tuning … by @Goekdeniz-Guelmez in https://github.com/ml-explore/mlx-lm/pull/200
Fix chat templates by @awni in https://github.com/ml-explore/mlx-lm/pull/205
Add total params to metadata + cleanup by @awni in https://github.com/ml-explore/mlx-lm/pull/207
Dynamic quants by @awni in https://github.com/ml-explore/mlx-lm/pull/202
some nits in lora by @awni in https://github.com/ml-explore/mlx-lm/pull/208
Fix quant package by @awni in https://github.com/ml-explore/mlx-lm/pull/214

New Contributors

@mattjcly made their first contribution in https://github.com/ml-explore/mlx-lm/pull/179
@IANTHEREAL made their first contribution in https://github.com/ml-explore/mlx-lm/pull/180
@jpangburn made their first contribution in https://github.com/ml-explore/mlx-lm/pull/186
@rgeorgi made their first contribution in https://github.com/ml-explore/mlx-lm/pull/188

Full Changelog: https://github.com/ml-explore/mlx-lm/compare/v0.24.1...v0.25.1

Open source →