Overview

Arrow based datasets

We have added support for Parquet-files, as well as Arrow's binary format. This is an opt-in feature, requiring pyarrow to be installed. Use pip install 'gluonts[pro]' or pip install 'gluonts[arrow]' to ensure the correct version is installed.

FileDataset has been reworked to support .parquet and .arrow files. Previously, it had assumed all files to use jsonlines. To continue using jsonlines ensure that the the files use one of the .json, .jsonl, .json.gz, jsonl.gz suffixes.

Depending on the dataset size and shape, Arrow can be much faster than the json variant. In more extreme cases we saw speedups of more than 100x when using arrow vs jsonlines (see #2003 for some examples).

To convert a given dataset into arrow, you can use the gluonts.dataset.arrow utility:

python -m gluonts.dataset.arrow write </path/to/dataset> my-dataset.arrow

PandasDataset

We have added support for pandas.DataFrame and pandas.Series as well. You can now directly model data given in a DataFrame using gluonts.dataset.pandas.PandasDataset. In this tutorial we describe in depth how you can use PandasDataset to speed up modelling using GluonTS.

Changelog

New Features

• #1631 - Add TimeLimitCallback to mx/trainer callbacks. (by @yx1215)
• #1780 - adding MQF2 (Multi-horizon) (by @KelvinKan)
• #1903 - Added QuarterlyBegin time feature (by @kashif)
• #1924 - Porting SimpleFeedForwardEstimator to PyTorch (by @lostella)
• #1925 - DeepAR PyTorch: make samplers configurable (by @lostella)
• #1935 - added support for pandas dataframes (by @rsnirwan)
• #1962 - Add support for beta-NLL loss (by @kashif)
• #1982 - Add Uber-TLC dataset to dataset repository. (by @Hongqing-work)
• #1990 - Add info cli. (by @jaheba)
• #1987 - Add HP tuning example with Optuna (by @npnv)
• #2000 - Add arrow-based dataset. (by @vafl, @lostella, @jaheba)
• #2002 - add ND for item_metrics (by @melopeo)
• #2006 - Added support of "long" RTS, making short RTS be "past_feat_dynamic_real" (by @zoolhasson)
• #2061 - Add DatasetWriter. (by @jaheba)
• #2074 - Add support for second frequency. (by @kashif)

Breaking Changes

• #1917 - Breaking: Fix return types of features (by @lostella)
• #1941 - Breaking: Update dependency fbprophet -> prophet (by @lostella)
• #1946 - Breaking: Split incremental quantile output into separate class (by @lostella)
• #1965 - Breaking: reorg torch package, shorten import paths (by @lostella)
• #1980 - Use pd.Period instead of pd.Timestamp. (by @jaheba)
• #1997 - Remove freq argument from Forecast. (by @kashif)
• #2011 - Remove dct_reduce. (by @jaheba)
• #2018 - Remove multiprocessing dataloader. (by @jaheba)
• #2019 - Rework FileDataset. (by @jaheba)
• #2053 - Add dataset_writer to get_dataset. (by @Hongqing-work)
• #2070 - Add jsonl.encode_json, remove serialize_data_entry. (by @jaheba)

Bug Fixes / Minor Improvements

• #1704 - Settings._let will pop element it added instead of just the last one. (by @jaheba)
• #1905 - Fix typing issues in torch estimators, update base estimators docstrings (by @lostella)
• #1909 - Fix the use of the scaling parameter in Transformer model (by @StanislasGuinel)
• #1916 - Fix AddTimeFeatures transformation for multiples of base frequencies (by @lostella)
• #1920 - Fix: use broadcast_lesser in place of comparisons in ISQF (by @vincentqb)
• #1931 - Fix dummy estimator (by @canerturkmen)
• #1933 - Fix Pytorch Lightning tutorial. (by @jaheba)
• #1938 - Fixed autograd inplace operations error in Transformed Distribution (by @shubhamkapoor)
• #1950 - Fix: Hard threshold positive distribution parameters (by @lostella)
• #1952 - Fix forecast keys (quantiles) output by TemporalFusionTransformer (by @lostella)
• #1968 - Fix: use of num_parallel_samples in deepAR (by @kashif)
• #1969 - Fix: torch DeepAR observed indicator in multivariate case (by @kashif)
• #1975 - use FieldName (by @kashif)
• #1983 - Documentation: add docstrings for torch-based models (by @lostella)
• #1986 - Fix OffsetSplitter for negative offsets (by @lostella)
• #1989 - Pin protobuf version. (by @jaheba)
• #1991 - Remove packaged pytorch-ts from gluonts.nursery.SCott (by @lostella)
• #1999 - Documentation: fix and speed up tutorials (by @lostella)
• #2004 - Refactor splitter assertion and add error message (by @RSNirwan)
• #2005 - Rework itertools, add col-to-row and row-to-col functions. (by @jaheba)
• #2008 - Re-add cache for parsing 'pd.Period'. (by @jaheba)
• #2013 - Update website template, clean up homepage and tutorials (by @lostella)
• #2014 - Expose Estimator, Predictor, Forecast in gluonts.model. (by @jaheba)
• #2015 - Fix mean in AffineTransformedDistribution (by @stailx)
• #2016 - Fix torch affine transformed distribution (by @lostella)
• #2020 - Remove unnecessary files from docs folder, update gitignore (by @lostella)
• #2021 - Update references to dev branch. (by @lostella)
• #2024 - Fix README. Use DataFramesDataset. (by @jaheba)
• #2025 - Make HP tuning tutorial more accurate (by @jaheba)
• #2028 - Re-add support for Python 3.6 (by @jaheba)
• #2029 - Add support for nan values in Rotbaum (by @zoolhasson)
• #2035 - Simplify lag values computation in torch DeepAR (by @lostella)
• #2036 - Minor improvements to the hierarchical model (by @rshyamsundar)
• #2047 - Make Quantile derive from pydantic.BaseModel. (by @jaheba)
• #2050 - Add concepts section to docs. (by @jaheba)
• #2051 - Add tutorial on DataFramesDataset (by @RSNirwan)
• #2057 - Add optional parameter time_axis to forecast_start. (by @melopeo)
• #2062 - Fix type annotations for predict_to_numpy (by @lostella)
• #2068 - Docs: simplify call to evaluator (by @lostella)

Backporting fixes:

• Cap numpy compatibility in mxnet extra requirements #2506
• Add gluonts.util.safe_extract #2606
• Fix call to extractall #2648

Backporting fixes:

• Fix r_forecast wrapper to shift start date when truncating time series (#2216 by @abdulfatir)
• Fix dtype bug in piecewise_linear and add a test (#2224 by @rshyamsundar)
• Fix bug in to_quantile_forecast (#2225 by @eugeneteoh)
• Fix gluonts.mx.trainer.Trainer in case of empty data loader (#2228 by @lostella)
• Fix feed-forward models when features are provided (#2238 by @lostella)

Full Changelog: https://github.com/awslabs/gluon-ts/compare/v0.9.8...v0.9.9

Backporting fixes:

• Fix SymbolBlock serde issue (#2187 by @lostella)

Backporting fixes:

• Fix dtype for "item_id" column in metrics dataframe (https://github.com/awslabs/gluon-ts/pull/2183 by @lostella)
• Fix recursive case for gluonts.mx.batchify.stack (https://github.com/awslabs/gluon-ts/pull/2184 by @lostella)
• Fix item_id values in ConstantValuePredictor (https://github.com/awslabs/gluon-ts/pull/2192 by @codingWhale13)

Backporting fixes:

• Fix: DistributionLoss not encodable (#2092 by @jaheba)

• Re-add support for Python 3.6 in v0.9.x. (#2032 by @jaheba)

Backporting fixes:

• Fix: use of num_parallel_samples in deepAR (#1968 by @kashif)
• Fix: torch DeepAR observed indicator in multivariate case (#1969 by @kashif)
• Fix OffsetSplitter for negative offsets (#1986 by @lostella)
• Fix mean in AffineTransformedDistribution (#2015 by @stailx)

Backporting fixes:

• Fix: Hard threshold positive distribution parameters (#1950 by @lostella)
• Fix forecast keys (quantiles) output by TemporalFusionTransformer (#1952 by @lostella)

Backporting fixes:

• Fix: use broadcast_lesser in place of comparisons in ISQF (#1920 by @vincentqb)
• Fix dummy estimator (#1931 by @canerturkmen)
• Fix Pytorch Lightning tutorial (#1933 by @jaheba)
• Fixed autograd inplace operations error in Transformed Distribution (#1938 by @shubhamkapoor)

Backporting fixes:

• Fix AddTimeFeatures transformation for multiples of base frequencies (https://github.com/awslabs/gluon-ts/pull/1916)
• Update docs requirements (https://github.com/awslabs/gluon-ts/pull/1919)

Backporting fixes:

• Added QuarterlyBegin time feature (#1903)
• Fix the use of the scaling parameter in Transformer model (#1909)

Changelog

New Features

• Add ckpt_path argument to PyTorchLightningEstimator. (#1872)
• Add TSBench (#1865)
• add SCott code to nursery (#1827)
• Add dynamic code for shell. (#1821)
• Adding torch.isqf (#1815)
• Add tsbench readme placeholder (#1808)
• Adding ISQF distribution class (#1746)
• Adding IQF to remove quantile crossing and required retraining for ne… (#1693)
• Hierarchical Forecaster: End-to-End model based on DeepVAR (#1665)
• Adding glouonts.torch.piecewise_linear (#1663)
• Add quantitle regression mode to AutoGluon-based TabularEstimator (#1611)
• add dummy estimator to trivial models (#1602)

Bug Fixes

• Add file path argument to m5 dataset generation (#1896)
• Fix negative binomial parameter map (#1893)
• Fix negative binomial sampling (#1884)
• Fixes for Monash Forecasting Repository datasets (#1879)
• Fix serde.flat type handling. (#1851)
• Fix datesplitter. (#1850)
• changed metadata creation function (#1847)
• Check equality of transformations. (#1844)
• Fix samples scaling in PyTorch DeepAR (#1836)
• Fix _version for cases when git is not installed. (#1825)
• Fixed data leakage bug in implementation of dynamic real and categorical features (#1809)
• fix for #1725, reverse breaking changes to data loader and handle all zero batches (#1779)
• Upgrade pytorch and pytorch-lightning requirements and some fixes. (#1765)
• Fix torch NOPScaler shape. (#1752)
• Convert batchify list to np array (#1732)
• Fix gluonts.json; added bdump/bdumps. (#1721)
• Fix scaling for pytorch negative binomial output (#1702)
• Fix frequency string conversion from ts format, add test (#1652)
• Fix NegativeBinomial constructor args in NegativeBinomialOutput (torch) (#1651)
• Add batch_size attribute to MQCNNEstimator and MQRNNEstimator (#1645)
• Add additional datasets from the Monash Time Series Forecasting Repository (#1632)

Breaking Changes

• Extend default quantiles for MQ* Estimators to match MSIS quantiles. (#1866)
• changed metadata creation function (#1847)
• Remove support module. (#1792)
• Set minimum Python version to 3.7. (#1791)
• Exceptions cleanup. (#1615)

Other Changes & Improvements

• Update mypy to 0.910. (#1875)
• Bump ujson from 4.3.0 to 5.1.0 in /src/gluonts/nursery/tsbench (#1869)
• Update black to v22. (#1867)
• Fix docstring typo in feature.py (#1863)
• Fix scott checks. (#1845)
• Remove requirement for @validated in from_hyperparameters. (#1826)
• Fix test collect ignore. (#1817)
• Split tests into one workflow for each framework. (#1805)
• Mark transformer as flaky. (#1801)
• Mark empirical_distribution test as flaky. (#1798)
• Use of int/float/object over np.int/float/object for dtype. (#1795)
• Rework tests. (#1786)
• Update typing_extension version. (#1785)
• Use of independent random seed. (#1767)
• Upgrade pytorch and pytorch-lightning requirements and some fixes. (#1765)
• Remove sphinx-autobuild sphinx-autorun, update sphinx version. (#1745)
• Exlude bin folders from apidoc. (#1744)
• Don't run doctest on nursery. (#1743)
• Hierarchical: Compute relative reconciliation error and add tests (#1722)
• Fixing doc build from mqcnn-iqf commit (#1699)
• Replace miniver with custom versioning code. (#1662)
• Cap numba<0.54, ipykernel<6.2.0 (#1661)
• Removed assert for cardinality and static feats (#1659)

Backporting fixes:

• loosen RTOL in test/distribution/test_flows.py to make test_flow_invertibility pass (#1604)
• Add batch_size attribute to MQCNNEstimator and MQRNNEstimator (#1645)
• Fix NegativeBinomial constructor args in NegativeBinomialOutput (torch) (#1651)
• Fix frequency string conversion from ts format, add test (adapted from #1652)

New Features

• add dummy estimator for seasonal_naive (#1598)
• Add STL-AR as one more R baseline model (#1568)
• Allow validation data for TabularEstimator. (#1562)
• QRX fixes and added functionality (#1544)
• Extend FileDataset's Parameters to load_datasets (#1538)
• Serde: Allow encoding of functions and methods. (#1519)
• Settings: Enable partial assignment (#1504)
• Settings: Support for nested args in _inject. (#1503)
• Transform.apply (#1494)
• PyTorch implementation of DeepAR (#1460)
• support Min freq for get_seasonality() method (#1459)
• add deep renewal processes for intermittent demand forecasting (#1458)
• Add transform objects for dealing with sparse time series. (#1421)
• spliced binned pareto (#1410)
• Add callbacks mechanism to Trainer class (#1168)

Bug Fixes

• Fix frequency metadata bug for lstnet datasets (#1593)
• Fix single dispatch register for py36 (#1591)
• R fixes for methods that produce point forecasts or prediction intervals directly (#1564)
• Fix computation of OWA (#1557)
• Fixed QRX bug: ".values()" to ".values" (#1552)
• QRX fixes and added functionality (#1544)
• Fix serde issue with some distribution output types, add test (#1543)
• Add item_id to r forecast predictors (#1537)
• fix ProphetPredictor serialization issue (#1535)
• Add constant dummy time features to TFT for yearly data (#1518)
• Settings: Fix partial assignment. (#1516)
• Fix anomaly detection example (#1515)
• Fix Settings._inject to check if it can provide the value. (#1501)
• Change miniver fallback version from unknown to 0.0.0. (#1457)
• Fix get_lags_for_frequency for minute data in DeepVAR (#1455)
• Fix missing import in gluonts.mx.model.GluonEstimator (#1450)
• Fix train-test split data leakage for m4_yearly and wiki-rolling_nips. (#1445)
• fix compatibility for pandas < 1.1 in time_feature/_base.py (#1437)
• fix edge case in iteration based model averaging (#1345)

Breaking Changes

• QRX fixes and added functionality (#1544)
• Transform.apply (#1494)

Other Changes & Improvements

• shallow import for gluonts.mx module (#1592)
• Mark torch distribution inference tests as flaky (#1586)
• Update REFERENCES.md (#1583)
• Delete pytorch_predictor_example.ipynb (#1574)
• Improve tests for R methods (#1567)
• Rename flake8 action step. (#1555)
• Set max_idle_transforms to the length of the dataset (#1546)
• Add datasets from forecastingdata.org (#1542)
• Train invoke with (#1530)
• Consolidate ZeroFeature from DeepState (#1522)
• Fix indentation (#1500)
• Simplify loader.py (#1495)
• adjustments to variable length functionality in batchify (#1442)
• Use miniver for version resolution. (#1434)
• Add docstrings for metrics. (#1422)
• Fixes for MXNet 1.8 (#1403)

Backporting fixes:

• Fix frequency metadata bug for lstnet datasets (#1593)
• Add batch_size attribute to MQCNNEstimator and MQRNNEstimator (#1645)
• Fix NegativeBinomial constructor args in NegativeBinomialOutput (torch) (#1651)

Backporting fixes:

• Fix serde issue with some distribution output types, add test (#1543)

Backporting fixes:

• Train invoke with (#1530)
• fix ProphetPredictor serialization issue (#1535)
• Add item_id to r forecast predictors (#1537)
• Serde: Allow encoding of functions and methods. (#1519)
• Disable tests on Windows for PRs, fix other workflows (#1525)

Backporting fixes:

• Fix Settings._inject to check if it can provide the value. (#1501)
• Fix indentation (#1500)
• Fix anomaly detection example (#1515)
• Add constant dummy time features to TFT for yearly data (#1518)

Backporting fixes:

• Fix get_lags_for_frequency for minute data in DeepVAR (#1455)

Backporting fixes:

• Fixes for MXNet 1.8 (#1403)
• Fix train-test split data leakage for m4_yearly and wiki-rolling_nips. (#1445)
• Lock the version for mxnet theme to 0.3.15 (#1451)
• Fix missing import in gluonts.mx.model.GluonEstimator (#1450)

Backporting fixes:

• fix compatibility for pandas < 1.1 in time_feature/_base.py (#1437)

GluonTS adds improved support for PyTorch-based models, new options for existing models, and general improvements to components and tooling.

Breaking changes

This release comes with a few breaking changes (but for good reasons). In particular, models trained and serialized prior to 0.7.0 may not be de-serializable using 0.7.0.

• Changes in model components and abstractions:
  • #1256 and #1206 contain significant changes to the GluonEstimator abstract class, as well as InstanceSplitter and InstanceSampler implementations. You are affected by this change only if you implemented custom models based on GluonEstimator. The change makes it easier to define (and understand, in case you're reading the code) how fixed-length instances are to be sampled from the original dataset for training or validation purposes. Furthermore, this PR breaks data transformation into more explicit "pre-processing" steps (deterministic ones, e.g. feature engineering) vs "iteration" steps (possibly random, e.g. random training instance sampling), so that a cache_data option is now available in the train method to have the pre-processed data cached to memory, and be iterated quicker, whenever it fits.
  • #1233 splits normalized/unnormalized time features from gluonts.time_features into distinct types.
  • #1223 updates the interface of ISSM types, making it easier to define custom ones e.g. by having a custom set of seasonality patterns. Related changes to DeepStateEstimator enable these customizations when defining a DeepState model.
• Changes in Trainer:
  • #1178 removes the input_names argument from the __call__ method. Now the provided data loaders are expected to produce batches containing only the fields that the network being trained consumes. This can be easily obtained by transforming the dataset with SelectFields.
• Package structure reorg:
  • #1183 puts all MXNet-dependant modules under gluonts.mx, with some exceptions (gluonts.model and gluonts.nursery). With the new structure, one is not forced to install MXNet unless they specifically require modules that depend on it.
  • #1402 makes the Evaluator class lighter, by moving the evaluation metrics to gluonts.evaluation.metrics instead of having them as static methods of the class.

New features

PyTorch support:

• PyTorchPredictor serde (#1086)
• Add equality operator for PytorchPredictor (#1190)
• Allow Pytorch predictor to be trained and loaded on different devices (#1244)
• Add distribution-based forecast types for torch, output layers, tests (#1266)
• Add more distribution output classes for PyTorch, add tests (#1272)
• Add pytorch tutorial notebook (#1289)

Distributions:

• Zero Inflated Poisson Distribution (#1130)
• GenPareto cdf and quantile functions (#1142)
• Added quantile function based on cdf bisection (#1145)
• Add AffineTransformedDistribution (#1161)

Models:

• add estimator/predictor types for autogluon tabular (#1105)
• Added thetaf method to the R predictor (#1281)
• Adding neural ode code for lotka volterra and corresponding notebook (#1023)
• Added lightgbm support for QRX/Rotbaum (#1365)
• Deepar imputation model (#1380)
• Initial commit for GMM-TPP (#1397)

Datasets & tooling:

• Implemented generate_rolling_datasets (#844)
• Add a MinMax scaler (#1134)
• introduce functional api for data generation recipes (#1153)
• include m3 dataset (#1169)
• Improvements for data generation (#1195)
• Add most forecasters as entry points. (#1351)

Backporting fixes:

• Fix train-test split data leakage for m4_yearly and wiki-rolling_nips. (#1445)
• Lock the version for mxnet theme to 0.3.15 (#1451)

Backporting fixes:

• fix s3fs ImportError for fsspec by updating the requirement depending on the python version (#1391)
• fix compatibility for pandas < 1.1 in time_feature/_base.py (#1437)

Backporting fixes:

• Added lead_time argument to PyTorchPredictor (#1316)
• Fix serialization of lead_time (#1328)
• Fix serialization of lead_time in torch predictor (#1329)

Backporting fixes:

• Use broadcast_logical_or in inflated_beta (#1226)
• Fix type error for using quantile_weights and add a proper Pytest (#1231)
• Fixed MASE in N-BEATS: removed redundant factor (#1288)
• Fixing bug where dropout was not used, also remove unused halt option (#1315)
• Fixes for Python 3.8 (#1318)

Backporting fixes:

• Fix serde for np.dtype. (#1299)

Backporting fixes:

• PyTorchPredictor serde (#1086)
• Add equality operator for PytorchPredictor (#1190)
• fix pytorch predictor serde (#1194)

Backporting fixes:

• fixes in dataset mutability (#1171)
• Added item_id to NPTS and Naive2 forecasts (#1173)

Backporting fixes:

• fix chain method of Transformation (#1156)

Backporting fixes:

• Masking edge case fix (#1137)

Changelog

New Features:

Model averaging (#823) add SampleForecast and Predictor objects for TPPs (#819) Add temperature scaling to categorical distrubution (#792) Representation module (#755) New methods for missing value imputation (#843) Add shuffling function (#873) make distributions pickleable (#889) Aggregate lag transformation (#886) SimpleFeedForward to produce DistributionForecast (#870) Added support for gzipped files. (#914) MQCNN: Support for past dynamic features and scaling (#916) Implemented model iteration averaging to reduce model variance (#901) add nan support to simple feedforward model (#933) Added timeout option for batch requests. (#931) Implemented moving average (#926) NanMixture: Distribution to model missing values (#913) Added Rotbaum (#653) DeepTPP: RNN-based temporal point processes model (#976) Added logging of scored instances for batch-transform. (#1010) Enable distribution output in seq2seq (#1008) Implemented activation regularization (#955) Adding mean absolute quantile loss to avoid the case of dividing by 0 as a possible HPO metric (#1012) Inflated Beta Distributions (#1018) Implement different dropout strategies (#963) Adding support for num_forking as MQ-CNN hp (#1022) Generalised Pareto distribution (#1031) Added use of supported quantiles in shell when QuantileForecastGenerator is used. (#1048) PyTorch Predictor (#1051) Add TFT model (#962) ConvTrans Implementation (#961) Add evaluation metrics for anomaly detection (#1065) Add piecewise linear quantile function output with fixed knots (#1074) include callback in trainer and example for warm starting (#1087) initial pytorch distribution output class (#1082) Glide (#995) specialize plot method for QuantileForecast (#1114)

Bug fixess

Fixed disabling of tqdm. (#839) Fix comparison of ParameterDict when non prefixed variables are in dict. (#859) Fixing edge case of prediction length 1. (#867) Frequency String for Pandas Timestamp (#884) fix imports (#885) Fixed invalid num_worker possibility. (#892) Corrected the formula for the stddev of the MixtureDistribution. (#900) Fix pathes in R for Windows. (#903) Scale the negative binomial's gamma (#909) Shape squeeze edge case bug fix (#911) Use of \n to split lines in batch transform. (#920) Fixing cardinality array when use_feat_static_cat = False but feat_static_cat present in dataset (#918) Fix batch-transform case, where request is empty. (#927) fix DeterministicOutput, add tests (#982) Fixing the FileDataset case with caching off for num_workers calculation (#986) Overriding early stopping for iteration-based averaging strategies (#993) Bug Fixes, Warnings, and One-Hot Encodings for Rotbaum (#980) Fixing case with only time features and yearly freq (#1002) Fixed import of Trainer. (#1005) Fixed DeepAR typing error (#1017) Fix sampling for MixtureDistribution class (#1042) MQ-CNN: Bound context_length by the max_ts_len - prediction_length (#1037) Fix gamma nans (#1061) Fix scaling for MQ-(C|R)NN when distribution outputs are used (#1070) added value in support to mixture output (#1077) fix Gamma distribution's NaN gradients for zero inputs (#1078) Fix dataset.splitter max_history argument (#1085) Fix max window (#1097) Ignore NaN values during training and throw a warning (training got stuck before) (#1104) Fix a few bugs about tensor shapes in default values for TFT implementation (#1093) Fixes awslabs/gluon-ts#1106 (#1125)

Breaking changes

Mqcnn rts (#668) Changed dataset.splitter to use DataEntry instead of TimeSeriesItem (#890) Refactoring data loading utilities (#898) Removed TimeSeriesItem. (#904) refactor imputation transformation (#907) making backtest_metrics simpler (#924) Moved get_seasonality from evaluation to time_feature. (#971) Removed mxContext from core. (#977)

Other changes and improvements

Update bug_report.md (#835) Dockerfile for R container added (#841) Added mx module. (#876) Adapted use of mx module. Applied isort. (#878) Simplified AsNumpyArray. (#879) Removed unused Transformation.estimate. (#880) Added README to shell. (#882) Docs requirements (#883) add documentation related to shuffle_buffer_length/ (#910) Default QuantileForecast.mean to p50. (#930) Addded trimming to encoded sagemaker parameters in shell package. (#917) Shell: Fix writing of output/failure file in case of error in provided hyper-parameters. (#942) Pass listify_dataset as a hyperparameter through the shell (#934) Evaluation metrics now stored in output folder (#938) Make TrainEnv path argument explicit. (#943) Removing mp worker del method. (#944) Fixed logical error in data_loder tests. (#951) Pass multiprocessing parameters through the shell (#952) Fix pandas requirement. (#967) Fix shell.train. Moved Dockerfiles to examples/dockerfiles (#946) Cleaned up unused imports. (#1007) Fix docstrings for SimpleFeedForward (#1009) Fix docstrings, enable distr_output in MQRNN (#1021) Update README.md (#1024) Update holidays version (#1033) improved and simplified aggregate lag transformation (#1028) Refactoring forecast generators and predictors for framework independence (#1052) Improved logging for batch-transform. (#1059) Reverting #1042 and adding shape assertions to the MixtureDistribution (#1058) Using pad_to_size function to remove duplicate code in pad_arrays (#1047) re-organized modules and imports (#1068) speed labels_to_ranges using numba (#1071) Fix numba warning; mask np.nan labels (#1072) added PyTorch predictor example notebook (#1053) refactor multiprocessing batcher to work with spawn method (#1080) Using zero floating point tolerance in denominator rather than checkign for exact zero equality (#1079) Fix FieldNames of Train/test splitter (#1083) Added Stateful to serde. (#1088) Added ty.checked decorator. (#1091) update links in readme (#1090) Adding test_quantiles hyperparameter to the shell to specify the quantiles for evaluation (#1096) Refactored serde into a package. (#1100) Refactored shell. (#1101) Updated pytest to v5. (#1102) cap pydantic version (#1115) add item_id to forecast from seasonal naive (#1113) reduce number of batches used in test (#1131) fix pandas usage and remove version cap (#1132)

Backporting fixes:

• remove kwargs from hybrid_forward input name inference (#846)
• Fix the quantiles used to compute MSIS (#849)
• Fix to_time_series_item in splitter (#874)
• updated Pandas deprecation (#875)
• Updated Frequency String for Pandas Timestamp (#884)
• splitting features and uses DataEntry instead of TimeSeriesItem (#890)
• fix multiprocessing in backtest function (#915)
• Fix LDS distribution for the case where sequence_length==1 (#921)
• Defaulting MSIS to NaN when it can't be calculated. (#923)

• Fix pandas version to 1.0.x.

Changelog

New features

• Dirichlet Multinomial distribution (#482)
• Datasets from the GP-Copula paper (#476)
• Marginal CDFtoGaussianTransformation (#486)
• DeepVAR model (#491)
• GP-Copula model (#497)
• Add transform objects for temporal point processes (#341)
• Added operator to allow for easier chaining of transformations. (#505)
• Gamma distribution implemented. (#502)
• Beta distribution implemented. (#512)
• Sagemaker SDK Integration (#444, #585)
• Add loc argument to distribution output classes (#540)
• Shopping holidays (#542)
• Add Poisson distribution (#532)
• N-Beats model (#553, #588, #655)
• Support slicing of distributions (#645)
• Naive2 model and OWA evaluation metric (#602)
• Add LSTNet (#596, #700, #791, #804)
• Data loading utils for M5 competition datasets (#716)
• Add MAPE to evaluator (#725)
• Add label smoothing to binned distribution (#731)
• Multiprocessing data loader. (#689, #739, #747, #759, #742)
• Add Categorical Distribution (#746)
• Added multiprocessing support for evaluation. (#741)
• Add variable length functionality to DataLoaders (#780)
• Add axis option to Scaler classes (#790)
• Add lead_time to predictors and estimators (#700)
• Add logit normal distribution (#811)

Bug fixes

• Fix instance splitter issue with short time series (#533)
• Fixed distribution sampling issues. (#526)
• Fix quantile of Binned distribution (#536)
• Fixed FileDataset SourceContext (#538)
• Fix quantile fn for transformed distribution (#544)
• Fix bug in cdf method of piecewise linear distributions (#564)
• Fixed taxi dataset cardinality (#552)
• Fix item_id field in provided datasets (#566)
• Fix Dockerfile to use Python 3.7. (#579)
• Fix DeepState trend model to work in symbolic mode (#578)
• Fix for symbol block serialization issue (#582, #591)
• Fixed LSTNet implementation (#586, )
• Fix mean_ts method of Forecast objects (#624)
• Fix r-forecast package on windows. (#626)
• Fix forecast index bug, add test (#644)
• Fix the sign method of affine transformation (#613)
• Fixing context when converting to symbol block predictor (#651)
• Fix data loader and include validation channel in test (#680)
• Fix incompatible date_range and matplotlib register in pandas v1.0 (#679)
• Fix binned distribution for mxnet 1.6 (#728)
• Remove asserts on loc and scale (#734)
• Fix default scaler in seq2seq models (#745)
• Fix pydanitc create_model usage. (#768)
• Fix feature slicing in WavenetSampler (#770)
• Fix bug with iteration over datasets (#787)
• Use forecast_start in RForecastPredictor (#798)
• Fix negative binomial's scaling (#719, #814)

Breaking changes

• Moved gp module to be part of gp_forecaster. (#572)

Other changes and improvements

• Changed FileDataset to be more easily inheritable. (#498)
• Added strategies for timezone information. (#500)
• Split up transform into its own module. (#499)
• Distribution dependent loss masking. (#534)
• Remove dataset class in favor of alias (#560)
• Clean up lifted operations, add pow operation (#571)
• Removed expand_dims when reading in time-series values. (#574)
• Updated dependency to Pandas v1.0 (#576)
• Refactored DataLoader. (#619)
• Refactored instance sampler. (#648)
• Log epochs in trainer (#676)
• Improve trainer handling of learning rate scheduling and logging (#701)
• Upgrade to mxnet 1.6 (#709)
• Moved model tests into their own folders. (#727)
• Refactor wavenet model (#743)
• Disable TQDM when running on SageMaker. (#810)

Changelog

• Fix that allows GluonTS to work with the latest pydantic v1.5 (#783)

• Fix WaveNet prediction length during training (#347)
• Relax requirements constraints (#456)
• Added aggregation functionality to MultivariateEvaluator (#459)
• Removed unused static method in DeepARNetwork (#460)
• Updated pydantic to version 1. (#465)
• Fix use of numpy.histogram. (#472)
• Fix validation error in transformed distribution (#475)
• Refined doc requiremnents; using sphinx 2. (#477)

Changelog

v0.4.1 includes:

• Added median as alias for p50 to Forecast. (#450)
• Use validation to prevent overfitting (#378)
• Fix deepstate serialization, add tests (#445)
• Fix escaping of string in serde.dump_code. (#439)
• Added multivariate grouper and tests (#432)
• Fixes to setup.py to make it work on Windows (#433)

Models

• Added Deep State model. (#229)
• Added Deep Factor model. (#271)
• Fixed bug when changing default activation function in WaveNet (#299)
• Option for DeepAR and DeepState to allow an embedding vector instead of the same value for all categorical features. (#315)
• Add option for feat_static_real in DeepAREstimator. (#324)
• Fixed DeepState samples tensor shape. (#340)
• Added support for changing dataytpe in DeepAREstimator. (#363)
• Made cardinality argument compulsory in DeepStateEstimator. (#413)
• DeepStateEstimator: Some adjustments to hyperparameter settings. (#415)

Distributions

• Include quantile method in distribution. (#314)
• Added slice_axis methods to Distribution. (#397)
• Added Dirichlet distribution. (#417)

Other new features

• Added more operators for synthetic data generation. (#286)
• Included DistributionForecast and make plot generic. (#316)

Bug fixes

• Updated lag error message. (#266)
• Fix mistake in notebook. (#269)
• Fix pandas warnings in dataset generation. (#270)
• Fix numerical issue with negative binomial distribution. (#288)
• Fixes fieldname issues. (#292)
• Fixed a wrong reshaping in DeepAR estimator. (#330)
• Small fixes to Box-Cox transformation. (#349)
• Improve BinnedDistribution. (#350)
• Small fix for binned distribution. (#352)
• Assure Learning Rate Scheduler does not increase the learning rate. (#359)
• Fix dim and copy_dim methods in SampleForecast. (#366)
• Fixed the logging of the number of parameters during training. (#386)
• Fix empty time_features issue. (#387)
• Fix batch shape in Binned Distribution (#406)
• Fix bug in multivariate Gaussian. (#407)
• Fix edge case in evaluation where prediction length is 1 and prediction target is nan. (#422)

Other changes

• Make item_id field uniform across predictors. (#268)
• Added Dockerfile. (#285)
• Pytest-timeout==1.3; removes warnings from logs. (#306)
• Flask~=1.1; removes some warnings. (#307)
• Make tensors and distributions serializable. (#312)
• Added SageMaker batch transform support. (#317)
• Manage mxnet context when deserializing predictors. (#318)
• Add missing time features for business day frequency. (#325)
• Switched to timestamp alignment from rollback to rollforward. (#328)
• Adding GPU support to the cholesky jitter and eig tests. (#342)
• Adding GP example on synthetic dataset with built-in plotting. (#343)
• Introduced ForecastGenerator to wrap mxnet output into forecast object. (#348)
• Add synthetic data generation tutorial. (#356)
• Added pd.Timestamp to serde. (#357)
• Using custom SerDe methods for deserializing params in Sagemaker. (#364)
• Fixes for serializing sets and numpy numbers in SerDe. (#368)
• Store GluonTS Version with stored model (#388)
• Dockerfile for GPU container. Fix for installing GPU version of MXNet. (#403)
• Added debug option to batch-transform. (#404)
• Use static categorical feature in benchmark_m4. (#410)
• Remove dataset.validate. (#412)
• Renamed num_eval_samples to num_samples. (#421)
• Remove mxnet requirement. (#429)

• Adapted mean predictor to use random samples. (#239)

• Added predict_item to RepresentablePredictor and adapted subclasses. (#240)

• Added fallback predictor and decorator.

• Forecasts always start at the end of the whole target.

• Fix shell to have a canonical freq key in hyperparameters.

• Made fallback process-safe. Added ConstantValuePredictor.

• GluonTSException bypass fallback.

• Black everything. (#244)

• Adding failure information to failure file. (#247)

• Added error message to top of failure file. (#248)

• fix the empty item list (#249)
  

• fix the shape error of the canonical network (#251)
  

• Fix documentation and enforce stricter doc builds (#226)
  

• Reformatted math equations for the log_prob method of the GaussianProcess class (#252)
  

• Fix yearly freq in process start field. (#253)
  

• fix issue with MultivariateGaussianOutput (#257)
  

• Fix shapes in CanonicalNetworkBase (#254)
  

• Improvements for wavenet and some utils (#262)
  

• Removed `get_granularity`. (#265)
  

• Bump pandas version and remove timestamp workarounds (#230)

• Fix num_eval_samples (#232)

• Fixed backtest test. (#235)

• Moved simple predictors to a distinct model folder. (#237)

• fix #234: Added method to fixup non json-spec compliant floats to make the resp… (#236)

Changes include:

• Serialize training metrics through the logger
• Improvements in the core package
• Minor changes in the shell.sagemaker package
• Add support for artificial datasets in the dataset repository
• Add MeanPredictor to model.testutil
• More flexible shell.serve API
• Add utilities for shell tests
• Added throughput logging for inference.

• Updated shell.

• Exclude MXNet 1.5.* from allowed requirements

• Added transformer model, tests and evaluations

• Minor improvements, changes and fixes.

• Changed shell metrics to be similar to SageMaker DeepAR.

More shell fixes

• Add proper documentation strings.
• Fix the printed script name in help output.
• Add a force-static train parameter that forces the creation of a static predictor from the expected model location.

• Fixup of shell. (#180)

• Re-added locate for Forecaster detection. (#181)

• Minor fixes.

New features

• Adding option to rescale time series instead of clipping them, in artificial dataset generation (#114)
• Include cdf method in distributions (#120, #133)
• Add support for feat_dynamic_real field in DeepAREstimator (#125)
• Add NPTS model (#148)
• Added shell package (#151, #156, #173)
• Custom seasonality for complex seasonal dataset (#155)

Other changes

• Renamed time_granularity -> freq, time_freq -> freq (#172, #173)
• Include safety check that catches infinite loops in transformations (#139)

Fixes

• Fixes to paths (#143)
• Fix default context_length in MQCNN and MQRNN estimators (#167)

• Fixed problem with the num_eval_samples argument of GluonPredictor.predict (#103)
• Fixed pydantic requirement, which was causing issues on Python 3.7 (#108)

• fixed bugs in distributions (#92)
• updated mxnet requirement (#96)
• removed pandas warnings (#97)

Issues fixed:

• Indexing issue in the RForecastPredictor (#63)
• Issues with the LDS and Binned distributions (#57 #65 #68)
• Hardcoded paths that affected Windows users (#77)
• Worked around Pandas bug affecting some versions (#79)
• Bug in the artificially generate datasets (#83)
• Circular dependency issue (#88)
• Removed FutureWarning complaints by Pandas (#88)
• Bumped version requirement for rpy2 (#89)

First public release of GluonTS