NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI
PyPI
Data preparation for speech processing models training.
Latest stable
1.33.0
Last release 2 months ago
11 Sep 2026
Ships fairly regularly
a new release about every 2 months
Nearly every release is documented
notes for 53 of 58 stable releases
Nothing withdrawn
no release was ever pulled
6 years old
65 releases · first in 2020
Drag across the ruler to compare, or tap a release to open it.
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Your coding agent can read these notes before it upgrades. Set up the MCP server →
Nothing published for this version
Nothing published for this version
Make lilcom optional and fix failing unit tests on MacOS by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1555
lilcom optional and fix failing unit tests on MacOS by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1555MixedCut by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1553LHOTSE_IO_BACKEND for reading AudioSource(type='url'); update docs by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1557AudioSamples(mono_downmix=True) to handle mixed single/multi channel batches gracefully by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1563CutSet.mix(..., tag="noise"), MixTrack.{is_snr_reference,mute} bools, and MixedCut.unmix(tag=...) by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1559Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.32.2...v1.33.0
lilcom optional and fix failing unit tests on MacOS by @pzelasko in #1555MixedCut by @pzelasko in #1553LHOTSE_IO_BACKEND for reading AudioSource(type='url'); update docs by @pzelasko in #1557AudioSamples(mono_downmix=True) to handle mixed single/multi channel batches gracefully by @pzelasko in #1563CutSet.mix(..., tag="noise"), MixTrack.{is_snr_reference,mute} bools, and MixedCut.unmix(tag=...) by @pzelasko in #1559Full Changelog: v1.32.2...v1.33.0
NSF grant acknowledgment in README.md by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1539
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.32.1...v1.32.2
Full Changelog: v1.32.1...v1.32.2
Fix issue with importing Lhotse v1.32.0 on Windows.
Fix issue with importing Lhotse v1.32.0 on Windows.
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.32.0...v1.32.1
Fix issue with importing Lhotse v1.32.0 on Windows.
Full Changelog: v1.32.0...v1.32.1
Add recipe for NOTSOFAR-1 by @domklement in https://github.com/lhotse-speech/lhotse/pull/1517
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/1.31.0...v1.32.0
Full Changelog: 1.31.0...v1.32.0
(skipping 1.31.0 because I accidentally uploaded and deleted this version in the past in PyPI, and can't re-use it now)
(skipping 1.31.0 because I accidentally uploaded and deleted this version in the past in PyPI, and can't re-use it now)
text file with utterances having empty references by @KarelVesely84 in #1496supervision_intervals and supervision_masks by @yfyeung in #1492Full Changelog: v1.30.3...1.31.0
[fix] avoid import librosa repeatly by @yuekaizhang in https://github.com/lhotse-speech/lhotse/pull/1475
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.30.2...v1.30.3
Full Changelog: v1.30.2...v1.30.3
Handle edge case in .repeat() when the input is empty by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1474
Thanks for debugging the issue @monica-sekoyan
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.30.1...v1.30.2
Patch release with bug fixes to AIStore and multi-storage-client logic amongst others.
Patch release with bug fixes to AIStore and multi-storage-client logic amongst others.
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.30.0...v1.30.1
Patch release with bug fixes to AIStore and multi-storage-client logic amongst others.
Full Changelog: v1.30.0...v1.30.1
add multi-storage-client backend for file open by @jayya2 in https://github.com/lhotse-speech/lhotse/pull/1455
Learn more about multi-storage-client here.
validate_cut_set from qa.py by @t13m in https://github.com/lhotse-speech/lhotse/pull/1458features_dtype arg to collate_features function by @t13m in https://github.com/lhotse-speech/lhotse/pull/1456Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.29.0...v1.30.0
Learn more about multi-storage-client here.
validate_cut_set from qa.py by @t13m in #1458features_dtype arg to collate_features function by @t13m in #1456Full Changelog: v1.29.0...v1.30.0
Recipe for the Chinese Dysarthric Speech Database by @JinZr in https://github.com/lhotse-speech/lhotse/pull/1423
CutSet.from_huggingface_dataset() for importing HF datasets by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1433Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.28.0...v1.29.0
CutSet.from_huggingface_dataset() for importing HF datasets by @pzelasko in #1433Full Changelog: v1.28.0...v1.29.0
Implement conversion from CutSet to HuggingFace dataset by @domklement in https://github.com/lhotse-speech/lhotse/pull/1398
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.27.0...v1.28.0
[Recipe] Wenetspeech4tts by @yuekaizhang in https://github.com/lhotse-speech/lhotse/pull/1384
CutSet.prefetch() for background cuts loading during iteration by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1380Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.26.0...v1.27.0
Add EARS recipe by @Ryu1845 in https://github.com/lhotse-speech/lhotse/pull/1375
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.25.0...v1.26.0
[feature] Add .narrowband() effect (mulaw, lpc10 codecs) by @rouseabout in https://github.com/lhotse-speech/lhotse/pull/1348
.narrowband() effect (mulaw, lpc10 codecs) by @rouseabout in https://github.com/lhotse-speech/lhotse/pull/1348DynamicBucketingSampler by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1372MixedCut transforms serialization by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1370Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.24.2...v1.25.0
Add KsponSpeech recipe by @whsqkaak in https://github.com/lhotse-speech/lhotse/pull/1353
Several new APIs for manifest classes added in #1361:
cut.iter_data() which iterates over (key, manifest) pairs of all data items attached to a given cut (e.g., ("recording", Recording(...)), ("custom_features", TemporalArray(...)))is_in_memory property for all manifest types to indicate if it contains data that is held in memoryis_placeholder for non-cut manifests to indicate if a manifest is just a placeholder (has some metadata, but can't be used to load data)cut.drop_in_memory_data() which converts manifests with in-memory data to placeholders (this is useful for manifests that live longer than just dataloading to avoid blowing up CPU memory and/or slowing down the program)Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.24.1...v1.24.2
Support for reading data from AIStore using Python SDK by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1354
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.24...v1.24.1
Notably, there's a new optimization for dynamic bucketing sampler in multi-GPU training - it will choose the same (or the closest possible) bucket on
Notably, there's a new optimization for dynamic bucketing sampler in multi-GPU training - it will choose the same (or the closest possible) bucket on each DDP rank to keep the total training step times closer. The expected speedup is dependent on the model and the number of GPUs. We observed 8 and 13% speedups across two experiments compared to non-synchronized bucket selection. The new option is called sync_buckets and is enabled by default.
reverb_rir: support Cut input and in memory data by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1332Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.23...v1.24
MDCC recipe by @JinZr in https://github.com/lhotse-speech/lhotse/pull/1302
aishell recipe by @JinZr in https://github.com/lhotse-speech/lhotse/pull/1305baker. by @csukuangfj in https://github.com/lhotse-speech/lhotse/pull/1304CutSet.mix() randomness and data utilization by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1315trim_to_supervision_groups by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1322Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.22...v1.23
Extending Lhotse dataloading to text/multimodal data by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1295
As an experimental feature, we are extending the API of Lhotse samplers to enable key sampling features for non-audio data such as text. That means text (and other) data can be dynamically multiplexed and bucketed in the same way as audio data with some lightweight wrappers. Please refer to new documentation here: https://lhotse.readthedocs.io/en/latest/datasets.html#customizing-sampling-constraints
Lhotse MultiCuts:
cut = cut.with_channels([0, 1, ...]) to modify the channels they refer tocut.target_recording, audio can be read via cut.load_target_recording() and channels will be auto-selected by looking up cut.target_recording_channel_selector).Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.21...v1.22
This release patches lhotse to handle cases when libsox is not available for torchaudio. The audio backend code went through additional round of refac
This release patches lhotse to handle cases when libsox is not available for torchaudio. The audio backend code went through additional round of refactoring, and libsndfile is now preferred as a default since it showed faster audio decoding performance in our testing. Going forward, when LHOTSE_AUDIO_BACKEND is set, we will use the same backend for audio loading, audio saving, and reading audio metadata (if possible). This release also adds support for Python 3.12 and PyTorch 2.2.
AudioBackend specific save_audio and info, managing missing SoX in torchaudio, Python 3.12 / PyTorch 2.2 support, using libsndfile as preferred audio backend by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1288Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.20...v1.21
Extended the subset of lhotse that works without installing torchaudio by @pzelasko in #1253 #1255
drop_last=False always returns an equal number of mini-batches by re-distributing and/or duplicating some data by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1277DynamicBucketingSampler by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1276CutSet.mux weights in multiprocess dataloading by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1266exceeded sampler warning false positives by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1270Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.19...v1.20
Nothing published for this version
Nothing published for this version
Support for OPUS encoding in Lhotse Shar format by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1238
CutSampler.map() for transforming CutSet mini-batches by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1246normalize_loudness for MixedCuts with PaddingCuts by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1249Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.18...v1.19
MMS forced alignment backend by @flyingleafe in https://github.com/lhotse-speech/lhotse/pull/1185
CutSet.from_shar(seed="trng") and DynamicCutSampler(quadratic_duration=...) by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1199DynamicBucketingSampler + various fixes by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1210tal_csasr by @JinZr in https://github.com/lhotse-speech/lhotse/pull/1198 https://github.com/lhotse-speech/lhotse/pull/1213MixedCut.truncate: fix the case when only PaddingCuts are left by @flyingleafe in https://github.com/lhotse-speech/lhotse/pull/1157num_jobs>1 and recordings with transforms by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1196trim_to_alignments issue by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/1193deterministic_rng to more flaky tests by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1200speaker_ids by @JinZr in https://github.com/lhotse-speech/lhotse/pull/1206attacut module for Thai word tokenization (in MMS forced alignment) by @flyingleafe in https://github.com/lhotse-speech/lhotse/pull/1232Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.17...v1.18
Remove deprecated code, make minor cleanups by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1139
copy_data: copy CutSet + its data to a new location by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1130resumable_download by @flyingleafe in https://github.com/lhotse-speech/lhotse/pull/1135audio.py into smaller modules by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1144save_audio by @flyingleafe in https://github.com/lhotse-speech/lhotse/pull/1147cut_into_windows for long cuts by @flyingleafe in https://github.com/lhotse-speech/lhotse/pull/1150export_to_kaldi on resampled recordings by @sih4sing5hong5 in https://github.com/lhotse-speech/lhotse/pull/1162CutSet.describe to enable parallel statistics computation by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1168fix_manifests in all recipes by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/1128Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.16...v1.17
Add speech translation corpus MuST-C by @csukuangfj in https://github.com/lhotse-speech/lhotse/pull/1079
New:
Improvements:
CutSet.from_files constructor for random order multi-file cutsets by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1085StatelessSampler: remove TRNG, make base_seed a mandatory parameter by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1109del (cut|supervision).custom_field by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1113tolerance for matching supervisions with features in Kaldi imported data dirs by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1119Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.15...v1.16
Resumable dataset downloads by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1045
mono_downmix option for MultiCut.to_mono() by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/1052MixedCut.load_audio() by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1054resumable_download for fully downloaded files by @flyingleafe in https://github.com/lhotse-speech/lhotse/pull/1060MixedCut logic by @flyingleafe in https://github.com/lhotse-speech/lhotse/pull/1073Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.14...v1.15
Add CLIs for creating Lhotse Shar directories and computing features by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1042
TORCHAUDIO_USE_BACKEND_DISPATCHER=1)pyloudnorm by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/1016parts and num_jobs options for tedlium by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/1030transforms attribute for MixedCut by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/1035Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.13...v1.14
Lhotse Shar tutorial notebook by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1006
New
Fixes
(cut|recording).dereverb_wpe() API + more stable numpy version by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/1000trim_to_supervision_groups + tolerance for overspans by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/963move_to_memory for MixedCut and PaddingCut by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/970MixedCut.to_mono() by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/973Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.12...v1.13
Fix the use of deprecated np.float in numpy>=1.24 by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/936
reverb_rir by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/920trim_to_alignments() method by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/926trim_to_supervision_groups method by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/930Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.11...v1.12
Remove some deprecated methods by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/900
This release has three new recipes and mostly bug fixes.
CutSet.from_shar by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/907CutSet.from_shar + better error msg by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/901cuts.describe() + fix for trim_to_unsupervised_segments() by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/899Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.10...v1.11
This release has a major (experimental) feature called Lhotse Shar. It's a data format inspired by WebDataset tar files that's intended to be very fas
This release has a major (experimental) feature called Lhotse Shar. It's a data format inspired by WebDataset tar files that's intended to be very fast for sequential reading of data stored in tarfile shards. It extends the ideas of WebDataset by allowing to store multiple types of features and metadata in separate tar archives that are iterated and loaded together with cuts. It allows to extend existing data with new fields (think different feature extractors, alignments, embeddings, etc.) without triggering a hard copy as would be the case with previous sequential formats supported by Lhotse. Preliminary benchmarking indicated it is as fast as WebDataset both with local disks and cloud storage.
A tutorial notebook about Lhotse Shar is planned to be released later this year.
Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.9...v1.10
MultiCut data type: simplifies working with multi-channel data (contribution from @desh2608)
MultiCut data type: simplifies working with multi-channel data (contribution from @desh2608)MultiCut to store multi-channel recordings with shared supervision by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/822cut_id to utt_id if there is exactly one supervision per cut by @wgb14 in https://github.com/lhotse-speech/lhotse/pull/853num2words to get word timestamps for numbers by @eschmidbauer in https://github.com/lhotse-speech/lhotse/pull/849trim_to_supervisions() by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/855Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.8...v1.9
Python 3.6 is no longer supported as of Lhotse v1.8. If you need to use Python 3.6, please revert to Lhotse 1.7 and earlier.
workflows, now integrates optional third party packages that assist corpus creators in automated data curation. With release 1.8, we support OpenAI Whisper for automatic transcription and segmentation, and torchaudio Wav2Vec2/Hubert ASR bundles for forced alignment.pad_silence option to extend_by by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/816git blame history by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/820workflows), starting with OpenAI Whisper support by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/824Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.7...v1.8
add test data to bvcc by @oplatek in https://github.com/lhotse-speech/lhotse/pull/797
snip_edges=True in online_inference of Kaldi feature extractors by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/802Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.6...v1.7
Feature/fix 754 voxceleb download by @mikuchar in https://github.com/lhotse-speech/lhotse/pull/776
Cut.has(field) method to query Cuts for custom attributes by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/792Full Changelog: https://github.com/lhotse-speech/lhotse/compare/1.5...v1.6
Describe more information about cuts by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/772
world_size>1 by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/773Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.4...v1.5
Deprecate strict, proportional_sampling, and bucket_method arguments by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/756
dill if installed for pickling lambdas by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/748multi_cn recipes: aishell2, magicdata, primewords, stcmds, tal_asr, tal_csasr, thchs_30 by @shanguanma in https://github.com/lhotse-speech/lhotse/pull/738strict, proportional_sampling, and bucket_method arguments by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/756lhotse cut simple CLI by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/759Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.3...1.4
Fix plotting MixedCut audio tracks by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/723
lilcom_chunky as default in CLI by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/729h5py an optional dependency by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/741CutSet.mix() argument cuts is not a lazy manifest by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/742CutSet: more methods are lazy + two simplified common use-cases attach_tensor and load_audio by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/744Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.2...v1.3
Adding lhotse recipe to prepare eval2000 data by @GoVivace in https://github.com/lhotse-speech/lhotse/pull/679
.repeat(..., preserve_id=...) option for repeating manifests by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/699BucketingSampler is used with a lazy CutSet by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/710stop_early option. by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/716pre-commit, isort + CI checks + running it on all code by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/720Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.1...v1.2
auto_increment_epoch in IterableDatasetWrapper, strict=True default for bucketing by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/661
auto_increment_epoch in IterableDatasetWrapper, strict=True default for bucketing by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/661Recording objects in cuts custom fields by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/662nara_wpe based WPE dereverberation as data augmentation by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/663max_cuts option to DynamicBucketingSampler by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/681recording_id in MixedCut.compute_and_store_features(..., mix_eagerly=True) by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/682.cut_into_windows method to individual cuts by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/685cut.truncate, format selection in cut.save_audio, less memory use in AudioMixer by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/690compute_and_store_features by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/683Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v1.0...v1.1
Deprecate compute_and_store_recording, add save_audio by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/486
This feature allows attaching arbitrary type of data to cuts: alignments, multiple feature sets, etc.
custom_collate_field pads to the longest read array instead of using CutSet.pad() by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/482with_path_prefix to Array and TemporalArray by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/499DynamicBucketingSampler: on-the-fly bucketing with restricted memory usage by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/517DynamicCutSampler (like DynamicBucketingSampler but without bucketing) by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/579lhotse split-lazy and CutSet.split_lazy() for memory-efficient splits by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/558orjson for up to 50% JSONL reading by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/563stop_early arg for CutSet.mux by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/585fault_tolerant option by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/599WebdatasetWriter for iterative cut writing by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/602map, filter, repeat, shuffle by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/626fill_supervision method for cuts and cut sets by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/505merge_supervisions method for cuts and cut sets by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/503lhotse cut trim-to-supervision <in> <out> by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/514rng arg for CutSet.truncate() by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/557SingleCutSampler -> SimpleCutSampler by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/546audio_lens in multi channel audio collater by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/616OnTheFlyFeatures(..., return_audio=True) by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/629__repr__ of manifests with binary data by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/596lhotse.testing package when installing lhotse by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/511collate_audio after adding fault_tolerant arg by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/540trim_to_supervisions(keep_overlapping=False) for completely overlapping cuts by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/553.is_lazy property to account for new types of lazy iterators by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/576.subset(first=...) for lazy manifests without len by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/625LazyJsonlIterator and fix TimeConstraint equality by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/632storage_path=None by @pzelasko in https://github.com/lhotse-speech/lhotse/pull/637Full Changelog: https://github.com/lhotse-speech/lhotse/compare/v0.12...v1.0
_This is mostly a bug, documentation, and installation fix release._
This is mostly a bug, documentation, and installation fix release.
Lhotse will now cache up to 512 audio/feature arrays that were read from disk to speed-up some common usage patterns.
This behavior can be easily disabled with lhotse.set_caching_enabled(False).
LibriCSS and Aishell 4 recipe by @desh2608 in https://github.com/lhotse-speech/lhotse/pull/433
Reduced memory usage in CommonVoice data prep
CutSet.describe() doesn't need pandas anymore (#412)torchaudio sox_io audio reading backend and making it default (#414)CommonVoice data preparation recipe
len() attribute from CutSamplers (#392)import lhotse and CLI start-up (#403)cuts.subset() (#395 #398, thanks @janvainer).filter() into the sampler report (#391)Lhotse CutSampler classes now return mini-batch CutSets instead of a list of string cut IDs (Lhotse Dataset classes are adjusted correspondingly)
CutSampler classes now return mini-batch CutSets instead of a list of string cut IDs (Lhotse Dataset classes are adjusted correspondingly) (#345)Cut is now an abstract base class for all cut types; what was previously called Cut is now called MonoCut) (#328)lhotse obtain is now lhotse download (#329)ZipSampler for batches constructed from different cut sources (#344 #347 #363 thanks for fixes @janvainer)drop_last option and get_report() method for cut samplers (#357)find_pessimistic_batches utility to help fail fast with GPU OOM (#358)BucketingSampler (#365)BucketingSampler (#372)OnTheFlyFeatures (padding audio instead of features) (#352)ChunkedLilcomHdf5Writer (and reader) for efficient chunk reads of lilcom-compressed arrays (#334)CutSet.trim_to_supervision has new arguments for including actual acoustic context next to the supervisions (#330 #331)SupervisionSegment is now mutable (and all Lhotse manifests will remain mutable) (#333).shuffle() method for Lhotse *Set classes (#341)lhotse fix CLI (#360)lhotse install-sph2pipe for handling LDC corpora compressed with shorten (auto-registers sph2pipe so no further actions are needed) (#370)CutSet.subset(cut_ids=...) (#353)lhotse.dataset.sampling file into a directory module (#366)GigaSpeech (#283, thanks @jimbozhang)
SupervisionSegment (#304, #310, #313, thanks @desh2608)cut.resample() (#299)drop_last option in *Set.split() (#315)Multilingual Librispeech (#282 -- can be quite slow but we plan on improving the speed in further releases)
len() that returns the number of batches, and return a consistent number of batches in all distributed workers. (#213, #222, #223, #224, #255, #267, thanks @janvainer)Lhotse now implements a number of PyTorch datasets and samplers. The core features are:
Lhotse now implements a number of PyTorch datasets and samplers. The core features are:
DataLoader)max_framesCutMix transform)DistributedSampler)Example code:
from torch.utils.data import DataLoader
from lhotse.dataset import SpeechRecognitionDataset, SingleCutSampler
cuts = CutSet(...)
dset = SpeechRecognitionDataset(cuts)
sampler = SingleCutSampler(cuts, max_frames=50000)
# Dataset performs batching by itself, so we have to indicate that
# to the DataLoader with batch_size=None
dloader = DataLoader(dset, sampler=sampler, batch_size=None, num_workers=1)
for batch in dloader:
... # process data
The resampling is performed at the moment of reading the audio samples from disk. It automatically adjusts the duration/num_samples in the data manifest.
recording = recording.resample(22050)
recording_set = recording_set.resample(8000)
CutSet.subset() got first and last arguments (like Kaldi's subset_data_dir.sh) and a CLI mode (#188)CutSet.from_manifest() creates deterministic Cut IDs by default (#186)Stop using deprecated torchaudio.info
cuts_sp = cuts.perturb_speed(0.9)
lhotse.validate()) (#175)lhotse.validate(cuts)
# As simple as:
cuts = cuts.compute_and_store_features(lhotse.Fbank(), 'path/to/feats', num_jobs=20)
# Modify the above with:
cuts = cuts.compute_and_store_features(lhotse.Fbank(), 'path/to/feats', num_jobs=20, storage_type=lhotse.LilcomHdf5Storage)
CutSet mixing for noise data augmentation (#180)# Can be performed after feature extraction for dynamic feature-domain mixing!
cuts = cuts.mix(noise_cuts, snr=[10, 30], mix_prob=0.5)
map() and modify_ids() methods (#165)CutSet.subset and CutSet.filter_supervisions (#145, thanks @janvainer)
New features:
CutSet.subset and CutSet.filter_supervisions (#145, thanks @janvainer)New corpora:
Bugfixes:
Other:
One column per quarter.