PyPI · #1737 most downloaded on PyPI

compressed-tensors

0.18.07.0M downloads/movllm-project/compressed-tensors ↗New releases in your daily digest, changelog included.

Library for utilization of compressed safetensors of neural network models

Last release 4 days ago
15 Sep 2026
Ships on a steady schedule
a new release about every 9 days
Most releases are documented
notes for 28 of 32 stable releases
Nothing withdrawn
no release was ever pulled
2 years old
208 releases · first in 2024

Release timeline

208 releases since 2024

202420252026

Releases

0.9.5a2025051213 May 2025pre-release
Nothing published for this version
0.9.5a2025050910 May 2025pre-release
Nothing published for this version
0.9.5a202505078 May 2025pre-release
Nothing published for this version
0.9.5a202505023 May 2025pre-release
Nothing published for this version

One column per month.

0.9.5a2025042829 Apr 2025pre-release

Nothing published for this version

0.9.5a2025042528 Apr 2025pre-release

Nothing published for this version

0.9.5a2025042425 Apr 2025pre-release

Nothing published for this version

0.9.424 Apr 2025

Remove compression_ratio calculation by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/293

What's Changed

Remove compression_ratio calculation by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/293
Build with setuptools scm by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/292
fix a few minor issues by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/294
Some fixes for AWQ by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/269
Fix upload issue when package already existed on PyPI by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/297
Update action tags by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/298
Pick up fix from nm-actions by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/299
[Compressor] Update packed compressor to support zp packing by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/296
[Decompression] Update Decompression Lifecycle by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/285
[Accelerate] allow get_execution_device to be used when initializing a model by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/303

Full Changelog: https://github.com/neuralmagic/compressed-tensors/compare/0.9.3...0.9.4

Open source →

0.9.4a2025042123 Apr 2025pre-release

Nothing published for this version

0.9.4a2025041415 Apr 2025pre-release

Nothing published for this version

0.9.4a2025041212 Apr 2025pre-release

Nothing published for this version

0.9.4a2025041011 Apr 2025pre-release

Nothing published for this version

0.9.4a202504089 Apr 2025pre-release

Nothing published for this version

0.9.32 Apr 2025

remove testmo by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/258

What's Changed

remove testmo by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/258
update tag for summary-test action by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/259
[Bugfix] Support offloaded parameters when initializing KV cache parameters by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/261
Update: CompressedLinear to decompress once by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/266
[BugFix]: AttributeError in CompressedLinear by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/273
Fix case when using weight_packed, not weight by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/278
Report test results to Report Portal by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/271
use fine-grained token for workflow by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/283
Rectify Asym Compression/Decompression Pathways by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/225
Bump CT Version by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/288

Full Changelog: https://github.com/neuralmagic/compressed-tensors/compare/0.9.2...0.9.3

Open source →

0.9.218 Feb 2025

ModelCompressor type checking import by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/220

What's Changed

ModelCompressor type checking import by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/220
Fix warning for dynamic quantization args by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/227
Depreciate get_observer by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/214
Accelerate Utilities: Throw warning when updating with different shapes by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/231
Use faster operations on packed-quantized, add tests by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/211
Update build workflow to Python 3.12 by @dbarbuzzi in https://github.com/neuralmagic/compressed-tensors/pull/248
Replace COMPRESSION_PARAM_NAMES with Abstract Property by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/249
Kylesayrs/update readme by @brian-dellabetta in https://github.com/neuralmagic/compressed-tensors/pull/252
Add: missing and unexpected keys in ModelCompressor by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/250
switch runners by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/254
Bump version for patch release by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/255

Full Changelog: https://github.com/neuralmagic/compressed-tensors/compare/0.9.1...0.9.2

Open source →

0.9.123 Jan 2025

BugFix: Shape should be a flat list by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/241

What's Changed

BugFix: Shape should be a flat list by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/241
Bump: Compressed Tensors version for release by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/244

Full Changelog: https://github.com/neuralmagic/compressed-tensors/compare/0.9.0...0.9.1

Open source →

0.9.015 Jan 2025

Replace depreciated pydantic functions by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/221

What's Changed

Replace depreciated pydantic functions by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/221
[Bugfix] Update expected shape for per token strategy by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/210
Accelerate Utilities by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/193
Update & fix Testmo actions/logic by @dbarbuzzi in https://github.com/neuralmagic/compressed-tensors/pull/230
Composability by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/219
Add 24 sparse bitmask by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/235
Fix: Disable Sparse Decompression for Dense Compressors by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/237
Update: Test for Compatibility with Transformers 4.48 by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/239
Inline 'get_release_and_version' definition by @dbarbuzzi in https://github.com/neuralmagic/compressed-tensors/pull/240
switch runners from a100 to h100 for now by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/242
bump for release by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/243

New Contributors

@dbarbuzzi made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/230

Full Changelog: https://github.com/neuralmagic/compressed-tensors/compare/0.8.1...0.9.0

Open source →

0.8.111 Dec 2024

Skip accelerate tests by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/208

What's Changed

Skip accelerate tests by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/208
Remove QuantizationScheme.default_scheme by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/202
Allow ModelCompressor.from_pretrained to load from quantization_config, not compression config by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/207
Quantization Scheme Validation by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/209
Fix uninitialized variable in quantized compressors by @markmc in https://github.com/neuralmagic/compressed-tensors/pull/205
Implement aliasable mixin and alias activation ordering by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/213
Revert "Implement aliasable mixin and alias activation ordering (#213)" by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/217
Implement aliasable mixin and alias activation ordering (python3.9 fix) by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/218
bump by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/226

New Contributors

@markmc made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/205

Full Changelog: https://github.com/neuralmagic/compressed-tensors/compare/0.8.0...0.8.1

Open source →

0.8.012 Nov 2024

[Observer Restructure]: Separate out scale/zp and observer init; separate out calibration from forward pass by @dsikka in https://github.com/neuralmag

What's Changed

[Observer Restructure]: Separate out scale/zp and observer init; separate out calibration from forward pass by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/188
Fix device allocation for MSE observer by @anmarques in https://github.com/neuralmagic/compressed-tensors/pull/190
drop 3.8 and add 3.12 to testing by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/196
Fix test which required accelerate, apply style by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/194
[Bugfix] Move observer and g_idx until after module in onloaded by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/195
Add sparsity structure enum by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/197
Observer Restructure: Remove Observers, calibration, and applying frozen steps from lifecycle by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/189
Clean up observer defaulting logic, better error message by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/200
apply style and quality by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/201
[BugFix] Fix Marlin24 Bug by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/203
Bump version to v0.8.0 by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/204

New Contributors

@anmarques made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/190

Full Changelog: https://github.com/neuralmagic/compressed-tensors/compare/0.7.1...0.8.0

Open source →

0.7.117 Oct 2024

[Observer Restructure]: Remove MemoryLess Observer; use helper function for dynamic quantization by @dsikka in https://github.com/neuralmagic/compress

What's Changed

[Observer Restructure]: Remove MemoryLess Observer; use helper function for dynamic quantization by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/187
bump up to 0.7.1 for patch release by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/192

Full Changelog: https://github.com/neuralmagic/compressed-tensors/compare/0.7.0...0.7.1

Open source →

0.7.09 Oct 2024

Make INT8 activation PRESET_SCHEMES explicit by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/158

What's Changed

Make INT8 activation PRESET_SCHEMES explicit by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/158
Write the current version into model configs by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/160
[KV-Cache] Make k_scale, v_scale as attributes of self_attn using HFCache by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/148
[Bugfix] Fix quant config parsing by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/162
Ignore Dense sparsity config by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/169
fix bug by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/170
Replace compression_config to be quantization_config for HFQuantizer support by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/164
ignore list by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/171
switch default to release and disable pushing to pypi for now by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/175
Fix missing quant_method value by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/174
Fix ModelCompressor parsing in HF Quantizer case by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/176
Calibration Code Clarity by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/168
Add: base sparsity/quantization compressors by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/165
Update compressors folder structure by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/166
Update number of groups by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/178
Bring nightly build/test back by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/179
Remove unused function by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/156
Revert "Ignore Dense sparsity config (#169)" by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/181
Workaround HF Quantizer apply_quantization_config misuse by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/180
bump up version to 0.7.0 by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/186

Full Changelog: https://github.com/neuralmagic/compressed-tensors/compare/0.6.0...0.7.0

Open source →

0.6.023 Sept 2024

Add simple GHA workflow to run tests by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/2

What's Changed

Add simple GHA workflow to run tests by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/2
Define BaseModels for Quantization by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/3
Quantization refactor by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/5
Apply quantization config implementation by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/4
decorate fake quant with torch.no_grad by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/8
fix observer bugs by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/9
[lifecycle] docstrings + ux update to work with torch.apply by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/11
Fix Device Mismatch by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/12
Serialize Config from Model by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/7
[Observers] pull shared logic into a helper function by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/13
Rename the repo to compressed-tensors by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/14
fix style post rename PR by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/25
Quantization Examples and Correctness Fixes by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/26
Fix failing GHA by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/29
Pretrained Model Reload + SparseGPT Support by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/31
[Release 0.3.0] Basic Readme and user-facing pathways by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/30
Quantization Fixes by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/35
Final details for package by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/36
bump version to 0.3.1 license an packaging updates by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/37
Dyanmic Quantization by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/15
[Release 0.3.2] Additional patches to enable compatibility with SparseML, UX changes by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/43
Update target match conditions; make public by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/44
[Lifecycle][Tests] Feature Branch by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/38
[Observers] group size + channel wise + per token by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/32
[BugFix] Update code to be compatible with py38 by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/48
[Fix] Fix the messed-up test structure by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/49
Bump the version before the release by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/50
Compressed lifecycle implementation (INT8 only) by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/33
group size speedups + fixes by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/51
Group and Channelwise Compression Support by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/52
Int4 Packed Compressor by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/47
Fix for auto device map quantization by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/54
Enable generating compressed-tensors-nightly by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/53
[BugFix][Again] Update code to be compatible with py38 by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/56
Fix per_token slowdown by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/57
[GPTQ Modifier UX] Add default scheme by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/61
fix group size min max tracking by adding tensor ids by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/60
Support for aliased scheme settings in quant config by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/40
Remove Symmetric Zero Point in Compressed Outputs by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/59
Misc Fixes by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/55
Fix for Symmetric Zero Point Reloading by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/64
Additional Symmetric ZP Fix by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/65
Make ZP int8 instead of int64 by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/67
Add a function to check if a string is a preset scheme by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/66
Rename Packed Weights by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/63
Fixed Grouped Quantization Reload by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/68
Fix incorrect loading of dtype by @eldarkurtic in https://github.com/neuralmagic/compressed-tensors/pull/70
Fix Python 3.8 Compatability by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/71
Update nightly build to run at 6pm by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/72
Update time for the runner by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/74
Fixes to enable FSDP one-shot by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/58
Update Compression Config for HfQuantizer Compatability by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/73
Remove version restriction on transformers by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/76
remove pydantic version cap by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/80
reduce appropriate dim by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/75
Marlin24 Compressor by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/77
Fix GPTQ Aliases by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/81
initial fixes for compatibility with HFQuantizer by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/79
bump version to 0.4.0 by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/83
import is_release from version.py by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/85
Add release build workflow by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/89
Assert correct device when dequantizing (like we do for quantizing) by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/90
update default symmetry to True on presets by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/92
Fp8 Quantization Support by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/62
default W4A16 alias to use group_size=128 by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/94
[compressor] Add packed int8 support by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/91
Fix Decompress kwargs by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/100
[Quant KV Cache] Implementation by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/86
Fix Transient Tests by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/101
Speed Up Packed Compression by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/103
[Fix] remove tests/quantization by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/99
Allow creating compressor when trust_remote_code=True by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/104
Update Quantization Presets by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/105
[MOE] Add a set of functionalities to support quantization of MOE models by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/46
[BugFix]Fix Name Mangling Issue in compressed_tensors.utils by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/102
Update Quantization Scheme Standards for better readability by @markurtz in https://github.com/neuralmagic/compressed-tensors/pull/106
quatization lifecycle - disable forward pass override + helper for weight quant param updates by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/111
Add FP8 Dynamic Scheme for Latest Llama3.1 Meta Models and Fix W4A8 Representation by @markurtz in https://github.com/neuralmagic/compressed-tensors/pull/114
Model Offloading Support by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/113
Fix Test to Account for Model Change by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/116
Make publish workflow manually triggerable by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/117
bump version to 0.5.0 by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/119
Fix Execution Device Helper Fn by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/120
Do not mutate config by apply_quantization_config by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/107
Rename Quant Method by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/122
Revert Config Change by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/124
Bug Fix for Calibration Setup by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/123
Fix Issue #112 by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/126
follow up, better tests by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/128
Update README.md by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/121
Adding MSE Clipping Support by @abhinavnmagic in https://github.com/neuralmagic/compressed-tensors/pull/115
Move kv cache scales from k/v_proj.output_scale to self_attn.k/v_scale by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/133
Make Accelerate Dependency Optional by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/131
Group Index Quantization Support by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/134
Move safe_permute, update layer helper arguments by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/137
Remove g_idx arg on observer by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/139
Activation Ordering by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/97
Warning Raise Bugfix by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/141
LLM compressor GHA fix by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/140
fix nightly by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/144
Naive Run Compressed Support by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/109
Run Compressed Fixes by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/147
Support quantizing only the kv cache by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/135
[Forward Call] fake quant fix by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/145
Activation Ordering Strategies by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/146
Support bool arguments for actorder by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/150
Skip writing empty g_idx to disk, fix compress_quantized_weights by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/143
Raise error if group_size is passed but wrong strategy by @kylesayrs in https://github.com/neuralmagic/compressed-tensors/pull/149
bump up main to 0.6.0 by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/154
Add workflows to build and test nightly and release by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/155
Update QuantizationScheme defaults by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/157
Add: targets and ignore to sparsity compression config by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/159

New Contributors

@dbogunowicz made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/2
@mgoin made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/36
@dsikka made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/44
@rahul-tuli made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/48
@eldarkurtic made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/70
@dhuangnm made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/89
@markurtz made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/106
@abhinavnmagic made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/115
@kylesayrs made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/134

Full Changelog: https://github.com/neuralmagic/compressed-tensors/commits/0.6.0

Open source →

0.5.08 Aug 2024

Add simple GHA workflow to run tests by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/2

What's Changed

Add simple GHA workflow to run tests by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/2
Define BaseModels for Quantization by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/3
Quantization refactor by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/5
Apply quantization config implementation by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/4
decorate fake quant with torch.no_grad by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/8
fix observer bugs by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/9
[lifecycle] docstrings + ux update to work with torch.apply by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/11
Fix Device Mismatch by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/12
Serialize Config from Model by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/7
[Observers] pull shared logic into a helper function by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/13
Rename the repo to compressed-tensors by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/14
fix style post rename PR by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/25
Quantization Examples and Correctness Fixes by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/26
Fix failing GHA by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/29
Pretrained Model Reload + SparseGPT Support by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/31
[Release 0.3.0] Basic Readme and user-facing pathways by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/30
Quantization Fixes by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/35
Final details for package by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/36
bump version to 0.3.1 license an packaging updates by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/37
Dyanmic Quantization by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/15
[Release 0.3.2] Additional patches to enable compatibility with SparseML, UX changes by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/43
Update target match conditions; make public by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/44
[Lifecycle][Tests] Feature Branch by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/38
[Observers] group size + channel wise + per token by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/32
[BugFix] Update code to be compatible with py38 by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/48
[Fix] Fix the messed-up test structure by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/49
Bump the version before the release by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/50
Compressed lifecycle implementation (INT8 only) by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/33
group size speedups + fixes by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/51
Group and Channelwise Compression Support by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/52
Int4 Packed Compressor by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/47
Fix for auto device map quantization by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/54
Enable generating compressed-tensors-nightly by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/53
[BugFix][Again] Update code to be compatible with py38 by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/56
Fix per_token slowdown by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/57
[GPTQ Modifier UX] Add default scheme by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/61
fix group size min max tracking by adding tensor ids by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/60
Support for aliased scheme settings in quant config by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/40
Remove Symmetric Zero Point in Compressed Outputs by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/59
Misc Fixes by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/55
Fix for Symmetric Zero Point Reloading by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/64
Additional Symmetric ZP Fix by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/65
Make ZP int8 instead of int64 by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/67
Add a function to check if a string is a preset scheme by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/66
Rename Packed Weights by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/63
Fixed Grouped Quantization Reload by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/68
Fix incorrect loading of dtype by @eldarkurtic in https://github.com/neuralmagic/compressed-tensors/pull/70
Fix Python 3.8 Compatability by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/71
Update nightly build to run at 6pm by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/72
Update time for the runner by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/74
Fixes to enable FSDP one-shot by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/58
Update Compression Config for HfQuantizer Compatability by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/73
Remove version restriction on transformers by @mgoin in https://github.com/neuralmagic/compressed-tensors/pull/76
remove pydantic version cap by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/80
reduce appropriate dim by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/75
Marlin24 Compressor by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/77
Fix GPTQ Aliases by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/81
initial fixes for compatibility with HFQuantizer by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/79
bump version to 0.4.0 by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/83
import is_release from version.py by @horheynm in https://github.com/neuralmagic/compressed-tensors/pull/85
Add release build workflow by @dhuangnm in https://github.com/neuralmagic/compressed-tensors/pull/89
Assert correct device when dequantizing (like we do for quantizing) by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/90
update default symmetry to True on presets by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/92
Fp8 Quantization Support by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/62
default W4A16 alias to use group_size=128 by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/94
[compressor] Add packed int8 support by @dsikka in https://github.com/neuralmagic/compressed-tensors/pull/91
Fix Decompress kwargs by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/100
[Quant KV Cache] Implementation by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/86
Fix Transient Tests by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/101
Speed Up Packed Compression by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/103
[Fix] remove tests/quantization by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/99
Allow creating compressor when trust_remote_code=True by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/104
Update Quantization Presets by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/105
[MOE] Add a set of functionalities to support quantization of MOE models by @dbogunowicz in https://github.com/neuralmagic/compressed-tensors/pull/46
[BugFix]Fix Name Mangling Issue in compressed_tensors.utils by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/102
Update Quantization Scheme Standards for better readability by @markurtz in https://github.com/neuralmagic/compressed-tensors/pull/106
quatization lifecycle - disable forward pass override + helper for weight quant param updates by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/111
Add FP8 Dynamic Scheme for Latest Llama3.1 Meta Models and Fix W4A8 Representation by @markurtz in https://github.com/neuralmagic/compressed-tensors/pull/114
Model Offloading Support by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/113
Fix Test to Account for Model Change by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/116
Make publish workflow manually triggerable by @rahul-tuli in https://github.com/neuralmagic/compressed-tensors/pull/117
bump version to 0.5.0 by @bfineran in https://github.com/neuralmagic/compressed-tensors/pull/119
[Cherry Pick] dont set quantization data on reload (#123) by @Satrat in https://github.com/neuralmagic/compressed-tensors/pull/125

New Contributors

@mgoin made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/36
@dsikka made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/44
@rahul-tuli made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/48
@eldarkurtic made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/70
@dhuangnm made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/89
@markurtz made their first contribution in https://github.com/neuralmagic/compressed-tensors/pull/106

Full Changelog: https://github.com/neuralmagic/compressed-tensors/commits/0.5.0

Open source →

0.4.021 Jun 2024

Scheme alias support in quant config

New Features:

Scheme alias support in quant config (#40)
New compressors: packed int4 (#47), Marlin 2:4 (#77)

Changes:

None

Resolved Issues:

Group-size quantization implementation addressed to ensure correctness. (#60)

Known Issues:

None

Open source →

0.3.37 May 2024

Nothing published for this version

0.3.229 Apr 2024

Nothing published for this version

0.3.125 Apr 2024

Nothing published for this version

0.3.025 Apr 2024

Nothing published for this version

Your coding agent can read these notes before it upgrades. Set up the MCP server →