NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #1843 most downloaded on PyPI
Distributed scheduler for Dask
Last release 1 months ago
24 Aug 2026
Ships fairly regularly
a new release about every 5 weeks
Nearly every release is documented
notes for 59 of the last 60 stable releases
1 version withdrawn
withdrawn after publishing
11 years old
269 releases · first in 2015
One column per quarter.
- Add initial deprecation utilities ( dask#7810 ) James Bourbeau
Released on June 18, 2021
Replace funding page with ‘Supported By’ section on dask.org ( dask#7817 ) James Bourbeau
Add initial deprecation utilities ( dask#7810 ) James Bourbeau
Enforce dtype conservation in ufuncs that explicitly use dtype= ( dask#7808 ) Doug Davis
Add Coiled to list of paid support organizations ( dask#7811 ) Kristopher Overholt
Small tweaks to the HTML repr for Layer & HighLevelGraph ( dask#7812 ) Genevieve Buckley
Add dark mode support to HLG HTML repr ( dask#7809 ) Jacob Tomlinson
Remove compatibility entries for old distributed ( dask#7801 ) Elliott Sales de Andrade
Implementation of HTML repr for HighLevelGraph layers ( dask#7763 ) Genevieve Buckley
Update default blockwise token to avoid DataFrame column name clash ( dask#6546 ) James Bourbeau
Use dispatch concat for merge_asof ( dask#7806 ) Julia Signell
Fix upstream freq tests ( dask#7795 ) Julia Signell
Use more context managers from the standard library ( dask#7796 ) James Bourbeau
Simplify skips in parquet tests ( dask#7802 ) Elliott Sales de Andrade
Remove check for outdated bokeh ( dask#7804 ) Elliott Sales de Andrade
More test coverage uploads ( dask#7799 ) James Bourbeau
Remove ImportError catching from dask/init.py ( dask#7797 ) James Bourbeau
Allow DataFrame.join() to take a list of DataFrames to merge with ( dask#7578 ) Krishan Bhasin
Fix maximum recursion depth exception in dask.array.linspace ( dask#7667 ) Daniel Mesejo-León
Fix docs links ( dask#7794 ) Julia Signell
Initial da.select() implementation and test ( dask#7760 ) Gabriel Miretti
Layers must implement get_output_keys method ( dask#7790 ) Genevieve Buckley
Don’t include or expect freq in divisions ( dask#7785 ) Julia Signell
A HighLevelGraph abstract layer for map_overlap ( dask#7595 ) Genevieve Buckley
Always include kwarg name in drop ( dask#7784 ) Julia Signell
Only rechunk for median if needed ( dask#7782 ) Julia Signell
Add add_(prefix|suffix) to DataFrame and Series ( dask#7745 ) tsuga
Move read_hdf to Blockwise ( dask#7625 ) Richard (Rick) Zamora
Make Layer.get_output_keys officially an abstract method ( dask#7775 ) Genevieve Buckley
Non-dask-arrays and broadcasting in ravel_multi_index ( dask#7594 ) Gabe Joseph
Fix for paths ending with “/” in parquet overwrite ( dask#7773 ) Martin Durant
Fixing calling .visualize() with filename=None ( dask#7740 ) Freyam Mehta
Generate unique names for SubgraphCallable ( dask#7637 ) Bruce Merry
Pin fsspec to 2021.5.0 in CI ( dask#7771 ) James Bourbeau
Evaluate graph lazily if meta is provided in from_delayed ( dask#7769 ) Florian Jetter
Add meta support for DatetimeTZDtype ( dask#7627 ) gerrymanoim
Add dispatch label to automatic PR labeler ( dask#7701 ) James Bourbeau
Fix HDFS tests ( dask#7752 ) Julia Signell
Released on June 18, 2021
Fix deadlock in handle_missing_dep if additional replicas are available (4929) `Florian Jetter`_
Add configuration to enable/disable NVML diagnostics (4893) `Peter Andreas Entschev`_
Add scheduler log tab to performance reports (4909) `Charles Blackmon-Luca`_
Add HTML repr to scheduler_info and incorporate into client and cluster reprs (4857) `Jacob Tomlinson`_
Fix error state typo (4898) `James Bourbeau`_
Allow actor exceptions to propagate (4232) `Martin Durant`_
Remove importing apply from dask.compatibility (4913) `Elliott Sales de Andrade`_
Use more informative default name for WorkerPlugin s (4908) `James Bourbeau`_
Removed unused utility functions (4911) `James Bourbeau`_
Locally rerun successfully completed futures (4813) `ArtinSarraf`_
Forget erred tasks and fix deadlocks on worker (4784) `Florian Jetter`_
Handle HTTPClientError in websocket connector (4900) `Marcos Moyano`_
Update dask_cuda usage in SSHCluster docstring (4894) `James Bourbeau`_
Remove tests for process_time and thread_time (4895) `James Bourbeau`_
Flake8 config cleanup (4888) `Florian Jetter`_
Don't strip scheduler protocol when determining host (4883) `James Bourbeau`_
Add more documentation on memory management (4874) `crusaderky`_
Add range_query tests to NVML test suite (4879) `Charles Blackmon-Luca`_
No longer cancel result future in async process when using timeouts (4882) `Florian Jetter`_
- Remove abstract tokens from graph keys in rewrite_blockwise ( dask#7721 ) Richard (Rick) Zamora
Released on June 4, 2021
Remove abstract tokens from graph keys in rewrite_blockwise ( dask#7721 ) Richard (Rick) Zamora
Ensure correct column order in csv project_columns ( dask#7761 ) Richard (Rick) Zamora
Renamed inner loop variables to avoid duplication ( dask#7741 ) Boaz Mohar
Do not return delayed object from to_zarr ( dask#7738 ) Chris Roat
Array: correct number of outputs in apply_gufunc ( dask#7669 ) Gabe Joseph
Rewrite da.fromfunction with da.blockwise ( dask#7704 ) John A Kirkham
Rename make_meta_util to make_meta ( dask#7743 ) GALI PREM SAGAR
Repartition before shuffle if the requested partitions are less than input partitions ( dask#7715 ) Vibhu Jawa
Blockwise: handle constant key inputs ( dask#7734 ) Mads R. B. Kristensen
Added raise to apply_gufunc ( dask#7744 ) Boaz Mohar
Show failing tests summary in CI ( dask#7735 ) Genevieve Buckley
sizeof sets in Python 3.9 ( dask#7739 ) Mads R. B. Kristensen
Warn if using pandas datetimelike string in dataframe.getitem ( dask#7749 ) Julia Signell
Highlight the client.dashboard_link ( dask#7747 ) Genevieve Buckley
Easier link for subscribing to the Google calendar ( dask#7733 ) Genevieve Buckley
Automatically show graph visualization in Jupyter notebooks ( dask#7716 ) Genevieve Buckley
Add autofunction for unify_chunks in API docs ( dask#7730 ) James Bourbeau
Released on June 4, 2021
Multiple worker executors (4869) `Mads R. B. Kristensen`_
Ensure PyNVML works correctly when installed with no GPUs (4873) `Peter Andreas Entschev`_
Show more in test summary (4875) `James Bourbeau`_
Move SystemMonitor s GPU initialization back to constructor (4866) `Peter Andreas Entschev`_
Mark test_server_comms_mark_active_handlers with pytest.mark.asyncio (4876) `James Bourbeau`_
Who has has what html reprs v2 (4865) `Jacob Tomlinson`_
O(1) rebalance (4774) `crusaderky`_
Ensure repr and eq for cluster always works (4799) `Florian Jetter`_
- Pandas compatibility ( dask#7712 ) Julia Signell
Released on May 28, 2021
Pandas compatibility ( dask#7712 ) Julia Signell
Fix optimize_dataframe_getitem bug ( dask#7698 ) Richard (Rick) Zamora
Update make_meta import in docs ( dask#7713 ) Benjamin Zaitlen
Implement da.searchsorted ( dask#7696 ) Tom White
Fix format string in error message ( dask#7706 ) Jiaming Yuan
Fix read_sql_table returning wrong result for single column loads ( dask#7572 ) c-thiel
Add slack join link in support.rst ( dask#7679 ) Naty Clementi
Remove unused alphabet variable ( dask#7700 ) James Bourbeau
Fix meta creation incase of object ( dask#7586 ) GALI PREM SAGAR
Add dispatch for union_categoricals ( dask#7699 ) GALI PREM SAGAR
Consolidate array Dispatch objects ( dask#7505 ) James Bourbeau
Move DataFrame dispatch.registers to their own file ( dask#7503 ) Julia Signell
Fix delayed with dataclasses where init=False ( dask#7656 ) Julia Signell
Allow a column to be named divisions ( dask#7605 ) Julia Signell
Stack nd array with unknown chunks ( dask#7562 ) Chris Roat
Promote the 2021 Dask User Survey ( dask#7694 ) Genevieve Buckley
Fix typo in DataFrame.set_index() ( dask#7691 ) James Lamb
Cleanup array API reference links ( dask#7684 ) David Hoese
Accept axis tuple for flip to be consistent with NumPy ( dask#7675 ) Andrew Champion
Bump pre-commit hook versions ( dask#7676 ) James Bourbeau
Cleanup to_zarr docstring ( dask#7683 ) David Hoese
Fix the docstring of read_orc ( dask#7678 ) Justus Magin
Doc ipyparallel & mpi4py concurrent.futures ( dask#7665 ) John A Kirkham
Update tests to support CuPy 9 ( dask#7671 ) Peter Andreas Entschev
Fix some HighLevelGraph documentation inaccuracies ( dask#7662 ) Mads R. B. Kristensen
Fix spelling in Series getitem error message ( dask#7659 ) Maisie Marshall
Released on May 28, 2021
Drop usage of WhoHas & WhatHas from Client (4863) `jakirkham`_
Ensure adaptive scaling is properly awaited and closed (4720) `Florian Jetter`_
Fix WhoHas/ HasWhat async usage (4860) `Benjamin Zaitlen`_
Add HTML reprs for Client.who_has and Client.has_what (4853) `Jacob Tomlinson`_
Prevent accidentally starting multiple Worker s in the same process (4852) `crusaderky`_
Add system tab to performance reports (4561) `Charles Blackmon-Luca`_
Let servers close faster if there are no active handlers (4805) `Florian Jetter`_
Fix UCX scrub config logging (4850) `Peter Andreas Entschev`_
Ensure worker clients are closed (3921) `Florian Jetter`_
Fix warning for attribute error when deleting a client (4807) `Florian Jetter`_
Ensure exceptions are raised if workers are incorrectly started (4733) `Florian Jetter`_
Update handling of UCX exceptions on endpoint closing (4836) `Peter Andreas Entschev`_
Ensure busy workloads properly look up who_has (4793) `Florian Jetter`_
Check distributed.scheduler.pickle in Scheduler.run_function (4838) `James Bourbeau`_
Add performance_report to API docs (4840) `James Bourbeau`_
Use dict _workers_dv in unordered use cases (4826) `jakirkham`_
Bump pre-commit hook versions (4835) `James Bourbeau`_
Do not mindlessly spawn workers when no memory limit is set (4397) `Torsten Wörtwein`_
test_memory to use gen_cluster (4811) `crusaderky`_
Increase timeout of gen_test to 30s (4821) `Florian Jetter`_
- Remove deprecated kind kwarg to comply with pandas 1.3.0 ( dask#7653 ) Julia Signell
Released on May 14, 2021
Remove deprecated kind kwarg to comply with pandas 1.3.0 ( dask#7653 ) Julia Signell
Fix bug in DataFrame column projection ( dask#7645 ) Richard (Rick) Zamora
Merge global annotations when packing ( dask#7565 ) Mads R. B. Kristensen
Avoid inplace= in pandas set_categories ( dask#7633 ) James Bourbeau
Change the active-fusion default to False for Dask-Dataframe ( dask#7620 ) Richard (Rick) Zamora
Array: remove extraneous code from RandomState ( dask#7487 ) Gabe Joseph
Implement str.concat when others=None ( dask#7623 ) Daniel Mesejo-León
Fix dask.dataframe in sandboxed environments ( dask#7601 ) Noah D. Brenowitz
Support for cupyx.scipy.linalg ( dask#7563 ) Benjamin Zaitlen
Move timeseries and daily-stock to Blockwise ( dask#7615 ) Richard (Rick) Zamora
Fix bugs in broadcast join ( dask#7617 ) Richard (Rick) Zamora
Use Blockwise for DataFrame IO (parquet, csv, and orc) ( dask#7415 ) Richard (Rick) Zamora
Adding chunk & type information to Dask HighLevelGraph s ( dask#7309 ) Genevieve Buckley
Add pyarrow sphinx intersphinx_mapping ( dask#7612 ) Ray Bell
Remove skip on test freq ( dask#7608 ) Julia Signell
Defaults in read_parquet parameters ( dask#7567 ) Ray Bell
Remove ignore_abc_warning ( dask#7606 ) Julia Signell
Harden DataFrame merge between column-selection and index ( dask#7575 ) Richard (Rick) Zamora
Get rid of ignore_abc decorator ( dask#7604 ) Julia Signell
Remove kwarg validation for bokeh ( dask#7597 ) Julia Signell
Add loky example ( dask#7590 ) Naty Clementi
Delayed: nout when arguments become tasks ( dask#7593 ) Gabe Joseph
Update distributed version in mindep CI build ( dask#7602 ) James Bourbeau
Support all or no overlap between partition columns and real columns ( dask#7541 ) Richard (Rick) Zamora
Released on May 14, 2021
Merge global annotations on the client (4691) `Mads R. B. Kristensen`_
Add support for click 8 (4810) `James Bourbeau`_
Add HTML reprs to some scheduler classes (4795) `James Bourbeau`_
Use JupyterLab theme variables (4796) `Ian Rose`_
Allow the dashboard to run on multiple ports (4786) `Jacob Tomlinson`_
Remove release_dep from WorkerPlugin API (4791) `James Bourbeau`_
Support for UCX 1.10+ (4787) `Peter Andreas Entschev`_
Reduce complexity of test_gather_allow_worker_reconnect (4739) `Florian Jetter`_
Fix doctests in utils.py (4785) `Jacob Tomlinson`_
Ensure deps are actually logged in worker (4753) `Florian Jetter`_
Add stacklevel keyword into performance_report() to allow for selecting calling code to be displayed (4777) `Nathan Danielsen`_
Unregister worker plugin (4748) `Naty Clementi`_
Fixes some pickling issues in the Cythonized Scheduler (4768) `jakirkham`_
Improve graceful shutdown if nanny is involved (4725) `Florian Jetter`_
Update cythonization in CI (4764) `James Bourbeau`_
Use contextlib.nullcontext (4763) `James Bourbeau`_
Cython fixes for MemoryState (4761) `jakirkham`_
Fix errors in check_thread_leak (4747) `James Bourbeau`_
Handle missing key case in report_on_key (4755) `jakirkham`_
Drop temporary set variables s (4758) `jakirkham`_
- Avoid pandas level= keyword deprecation ( dask#7577 ) James Bourbeau
Released on April 23, 2021
Handle Blockwise HLG pack/unpack for concatenate=True ( dask#7455 ) Richard (Rick) Zamora
map_partitions : use tokenized info as name of the SubgraphCallable ( dask#7524 ) Mads R. B. Kristensen
Using tmp_path and tmpdir to avoid temporary files and directories hanging in the repo ( dask#7592 ) Naty Clementi
Contributing to docs (development guide) ( dask#7591 ) Naty Clementi
Add more packages to Python 3.9 CI build ( dask#7588 ) James Bourbeau
Array: Fix NEP-18 dispatching in finalize ( dask#7508 ) Gabe Joseph
Misc fixes for numpydoc ( dask#7569 ) Matthias Bussonnier
Avoid pandas level= keyword deprecation ( dask#7577 ) James Bourbeau
Map e.g. .repartition(freq="M") to .repartition(freq="MS") ( dask#7504 ) Ruben van de Geer
Remove hash seeding in parallel CI runs ( dask#7128 ) Elliott Sales de Andrade
Add defaults in parameters in to_parquet ( dask#7564 ) Ray Bell
Simplify transpose axes cleanup ( dask#7561 ) Julia Signell
Make ValueError in len(index_names) > 1 explicit it’s using fastparquet ( dask#7556 ) Ray Bell
Fix dict -column appending for pyarrow parquet engines ( dask#7527 ) Richard (Rick) Zamora
Add a documentation auto label ( dask#7560 ) Doug Davis
Add dask.delayed.Delayed to docs so it can be referenced by other sphinx docs ( dask#7559 ) Doug Davis
Fix upstream idxmaxmin for uneven split_every ( dask#7538 ) Julia Signell
Make normalize_token for pandas Series / DataFrame future proof (no direct block access) ( dask#7318 ) Joris Van den Bossche
Redesigned setitem implementation ( dask#7393 ) David Hassell
histogram , histogramdd improvements (docs; return consistencies) ( dask#7520 ) Doug Davis
Force nightly pyarrow in the upstream build ( dask#7530 ) Joris Van den Bossche
Fix Configuration Reference ( dask#7533 ) Benjamin Zaitlen
Use .to_parquet on dask.dataframe in doc string ( dask#7528 ) Ray Bell
Avoid double msgpack serialization of HLGs ( dask#7525 ) Mads R. B. Kristensen
Encourage usage of yaml.safe_load() in configuration doc ( dask#7529 ) Hristo Georgiev
Fix reshape bug. Add relevant test. Fixes #7171. ( dask#7523 ) JSKenyon
Support custom_metadata= argument in to_parquet ( dask#7359 ) Richard (Rick) Zamora
Clean some documentation warnings ( dask#7518 ) Daniel Mesejo-León
Getting rid of more docs warnings ( dask#7426 ) Julia Signell
Added product (alias of prod ) ( dask#7517 ) Freyam Mehta
Fix upstream array_ufunc tests ( dask#7494 ) Julia Signell
Escape from map_overlap to map_blocks if depth is zero ( dask#7481 ) Genevieve Buckley
Add check_type to array assert_eq ( dask#7491 ) Julia Signell
Released on April 23, 2021
Avoid active_threads changing size during iteration (4729) `James Bourbeau`_
Fix UnboundLocalError in AdaptiveCore.adapt() (4731) `Anderson Banihirwe`_
Minor formatting updates for HTTP endpoints doc (4736) `James Bourbeau`_
Unit test for metrics["memory"]=None (4727) `crusaderky`_
Enable configuration of prometheus metrics namespace (4722) `Jacob Tomlinson`_
Reintroduce weight function (4723) `James Bourbeau`_
Add ready->memory to transitions in worker (4728) `Gil Forsyth`_
Fix regressions in 4651 (4719) `crusaderky`_
Add descriptions for UCX config options (4683) `Charles Blackmon-Luca`_
Split RAM measure into dask keys/other old/other new (4651) `crusaderky`_
Fix DeprecationWarning on Python 3.9 (4717) `George Sakkis`_
ipython causes test_profile_nested_sizeof crash on windows (4713) `crusaderky`_
Add iterate_collection argument to serialize (4641) `Richard J Zamora`_
When closing Server, close all listeners (4704) `Florian Jetter`_
Fix timeout in client.restart (4690) `Matteo De Wint`_
Avoid repeatedly using the same worker on first task with quiet cluster (4638) `Doug Davis`_
Grab func for finish case only if used (4702) `jakirkham`_
Remove hostname check in test_dashboard (4706) `James Bourbeau`_
Faster tests_semaphore::test_worker_dies (4703) `Florian Jetter`_
Clean up test_dashboard (4700) `crusaderky`_
Add timing information to TaskGroup (4671) `Matthew Rocklin`_
Remove WSSConnector TLS presence check (4695) `Marcos Moyano`_
Fix typo and remove unused time.time import (4689) `Hristo Georgiev`_
Don't initialize CUDA context in monitor (4688) `Charles Blackmon-Luca`_
Add support for extra conn args for HTTP protocols (4682) `Marcos Moyano`_
Adjust timings in test_threadpoolworkers (4681) `Florian Jetter`_
Add GPU metrics to SystemMonitor (4661) `Charles Blackmon-Luca`_
Removing dumps_msgpack() and loads_msgpack() (4677) `Mads R. B. Kristensen`_
Expose worker SystemMonitor s to scheduler via RPC (4657) `Charles Blackmon-Luca`_
- Adding support for multidimensional histograms with dask.array.histogramdd ( dask#7387 ) Doug Davis
Released on April 2, 2021
Adding support for multidimensional histograms with dask.array.histogramdd ( dask#7387 ) Doug Davis
Update docs on number of threads and workers in default LocalCluster ( dask#7497 ) cameron16
Add labels automatically when certain files are touched in a PR ( dask#7506 ) Julia Signell
Extract ignore_order from kwargs ( dask#7500 ) GALI PREM SAGAR
Only provide installation instructions when distributed is missing ( dask#7498 ) Matthew Rocklin
Start adding isort ( dask#7370 ) Julia Signell
Add ignore_order parameter in dd.concat ( dask#7473 ) Daniel Mesejo-León
Use powers-of-two when displaying RAM ( dask#7484 ) crusaderky
Added License Classifier ( dask#7485 ) Tom Augspurger
Replace conda with mamba ( dask#7227 ) crusaderky
Fix typo in array docs ( dask#7478 ) James Lamb
Use concurrent.futures in local scheduler ( dask#6322 ) John A Kirkham
Released on April 2, 2021
Fix un-merged frames (4666) `Matthew Rocklin`_
Add informative error message to install uvloop (4664) `Matthew Rocklin`_
Remove incorrect comment regarding default LocalCluster creation (4660) `cameron16`_
Treat empty/missing writeable as a no-op (4659) `jakirkham`_
Avoid list mutation in pickle_loads (4653) `Matthew Rocklin`_
Ignore OSError exception when scaling down (4633) `Gerald`_
Add isort to pre-commit hooks, package resorting (4647) `Charles Blackmon-Luca`_
Use powers-of-two when displaying RAM (4649) `crusaderky`_
Support Websocket communication protocols (4396) `Marcos Moyano`_
scheduler.py / worker.py code cleanup (4626) `crusaderky`_
Update out-of-date references to config.yaml (4643) `Hristo Georgiev`_
Suppress OSError on SpecCluster shutdown (4567) `Jacob Tomlinson`_
Replace conda with mamba (4585) `crusaderky`_
Expand documentation on pure functions (4644) `James Lamb`_
- Add a dispatch for is_categorical_dtype to handle non-pandas objects ( dask#7469 ) brandon-b-miller
Released on March 26, 2021
Add a dispatch for is_categorical_dtype to handle non-pandas objects ( dask#7469 ) brandon-b-miller
Use multiprocessing.Pool in test_read_text ( dask#7472 ) John A Kirkham
Add missing meta kwarg to gufunc class ( dask#7423 ) Peter Andreas Entschev
Example for memory-mapped Dask array ( dask#7380 ) Dieter Weber
Fix NumPy upstream failures xfail pandas and fastparquet failures ( dask#7441 ) Julia Signell
Fix bug in repartition with freq ( dask#7357 ) Ruben van de Geer
Fix array_function dispatching for tril / triu ( dask#7457 ) Peter Andreas Entschev
Use concurrent.futures.Executors in a few tests ( dask#7429 ) John A Kirkham
Require NumPy >=1.16 ( dask#7383 ) crusaderky
Minor sort_values housekeeping ( dask#7462 ) Ryan Williams
Ensure natural sort order in parquet part paths ( dask#7249 ) Ryan Williams
Remove global env mutation upon running test_config.py ( dask#7464 ) Hristo Georgiev
Update NumPy intersphinx URL ( dask#7460 ) Gabe Joseph
Add rot90 ( dask#7440 ) Trevor Manz
Update docs for required package for endpoint ( dask#7454 ) Nick Vazquez
Master -> main in slice_array docstring ( dask#7453 ) Gabe Joseph
Expand dask.utils.is_arraylike docstring ( dask#7445 ) Doug Davis
Simplify BlockwiseIODeps importing ( dask#7420 ) Richard (Rick) Zamora
Update layer annotation packing method ( dask#7430 ) James Bourbeau
Drop duplicate test in test_describe_empty ( dask#7431 ) John A Kirkham
Add Series.dot method to dataframe module ( dask#7236 ) Madhu94
Added df kurtosis -method and testing ( dask#7273 ) Jan Borchmann
Avoid quadratic-time performance for HLG culling ( dask#7403 ) Bruce Merry
Temporarily skip problematic sparse test ( dask#7421 ) James Bourbeau
Update some CI workflow names ( dask#7422 ) James Bourbeau
Fix HDFS test ( dask#7418 ) Julia Signell
Make changelog subtitles match the hierarchy ( dask#7419 ) Julia Signell
Add support for normalize in value_counts ( dask#7342 ) Julia Signell
Avoid unnecessary imports for HLG Layer unpacking and materialization ( dask#7381 ) Richard (Rick) Zamora
Bincount fix slicing ( dask#7391 ) Genevieve Buckley
Add sliding_window_view ( dask#7234 ) Deepak Cherian
Fix typo in docs/source/develop.rst ( dask#7414 ) Hristo Georgiev
Switch documentation builds for PRs to readthedocs ( dask#7397 ) James Bourbeau
Adds sort_values to dask.DataFrame ( dask#7286 ) gerrymanoim
Pin sqlalchemy<1.4.0 in CI ( dask#7405 ) James Bourbeau
Comment fixes ( dask#7215 ) Ryan Williams
Dead code removal / fixes ( dask#7388 ) Ryan Williams
Use single thread for pa.Table.from_pandas calls ( dask#7347 ) Richard (Rick) Zamora
Replace 'container' with 'image' ( dask#7389 ) James Lamb
DOC hyperlink repartition ( dask#7394 ) Ray Bell
Pass delimiter to fsspec in bag.read_text ( dask#7349 ) Martin Durant
Update read_hdf default mode to "r" ( dask#7039 ) rs9w33
Embed literals in SubgraphCallable when packing Blockwise ( dask#7353 ) Mads R. B. Kristensen
Update test_hdf.py to not reuse file handlers ( dask#7044 ) rs9w33
Require additional dependencies: cloudpickle, partd, fsspec, toolz ( dask#7345 ) Julia Signell
Prepare Blockwise + IO infrastructure ( dask#7281 ) Richard (Rick) Zamora
Remove duplicated imports from test_slicing.py ( dask#7365 ) Hristo Georgiev
Add test deps for pip development ( dask#7360 ) Julia Signell
Support int slicing for non-NumPy arrays ( dask#7364 ) Peter Andreas Entschev
Automatically cancel previous CI builds ( dask#7348 ) James Bourbeau
dask.array.asarray should handle case where xarray class is in top-level namespace ( dask#7335 ) Tom White
HighLevelGraph length without materializing layers ( dask#7274 ) Gabe Joseph
Drop support for Python 3.6 ( dask#7006 ) James Bourbeau
Fix fsspec usage in create_metadata_file ( dask#7295 ) Richard (Rick) Zamora
Change default branch from master to main ( dask#7198 ) Julia Signell
Add Xarray to CI software environment ( dask#7338 ) James Bourbeau
Update repartition argument name in error text ( dask#7336 ) Eoin Shanaghy
Run upstream tests based on commit message ( dask#7329 ) James Bourbeau
Use pytest.register_assert_rewrite on util modules ( dask#7278 ) Bruce Merry
Add example on using specific chunk sizes in from_array() ( dask#7330 ) James Lamb
Move NumPy skip into test ( dask#7247 ) Julia Signell
Released on March 26, 2021
Add standalone dashboard page for GPU usage (4556) `Jacob Tomlinson`_
Handle stream is None case in TCP comm finalizer (4631) `James Bourbeau`_
Include LIST_PICKLE in NumPy array serialization (4632) `James Bourbeau`_
Rename annotation plugin in test_highlevelgraph.py (4618) `James Bourbeau`_
UCX use nbytes instead of len (4621) `Mads R. B. Kristensen`_
Skip NumPy and pandas tests if not importable (4563) `Ben Greiner`_
Remove utils.shutting_down in favor of sys.is_finalizing (4624) `James Bourbeau`_
Handle async clients when closing (4623) `Matthew Rocklin`_
Drop log from remove_key_from_stealable (4609) `jakirkham`_
Introduce events log length config option (4615) `Fabian Gebhart`_
Upstream config serialization and inheritance (4372) `Jacob Tomlinson`_
Add check to scheduler creation in SpecCluster (4605) `Jacob Tomlinson`_
Make length of events deque configurable (4604) `Fabian Gebhart`_
Add explicit fetch state to worker TaskState (4470) `Gil Forsyth`_
Update develop.rst (4603) `Florian Jetter`_
pickle_loads(): Handle empty memoryview (4595) `Mads R. B. Kristensen`_
Switch documentation builds for PRs to readthedocs (4599) `James Bourbeau`_
Track frame sizes along with frames (4593) `jakirkham`_
Add support for a list of keys when using batch_size in client.map (4592) `Sultan Orazbayev`_
If SpecCluster fails to start attempt to gracefully close out again (4590) `Jacob Tomlinson`_
Multi-lock extension (4503) `Mads R. B. Kristensen`_
Update PipInstall plugin command (4584) `James Bourbeau`_
IPython magics: remove deprecated ioloop workarounds (4530) `Min RK`_
Add GitHub actions workflow to cancel duplicate builds (4581) `James Bourbeau`_
Remove outdated macOS build badge from README (4576) `James Bourbeau`_
Dask master -> main (4569) `Julia Signell`_
Drop support for Python 3.6 (4390) `James Bourbeau`_
Add docstring for dashboard_link property (4572) `Doug Davis`_
Change default branch from master to main (4495) `Julia Signell`_
Msgpack handles extract serialize (4531) `Mads R. B. Kristensen`_
This is the first release with support for Python 3.9 and the last release with support for Python 3.6
Released on March 5, 2021
Note
This is the first release with support for Python 3.9 and the last release with support for Python 3.6
Bump minimum version of distributed ( dask#7328 ) James Bourbeau
Fix percentiles_summary with dask_cudf ( dask#7325 ) Peter Andreas Entschev
Temporarily revert recent Array.setitem updates ( dask#7326 ) James Bourbeau
Blockwise.clone ( dask#7312 ) crusaderky
NEP-35 duck array update ( dask#7321 ) James Bourbeau
Don’t allow setting .name for array ( dask#7222 ) Julia Signell
Use nearest interpolation for creating percentiles of integer input ( dask#7305 ) Kyle Barron
Test exp with CuPy arrays ( dask#7322 ) John A Kirkham
Check that computed chunks have right size and dtype ( dask#7277 ) Bruce Merry
pytest.mark.flaky ( dask#7319 ) crusaderky
Contributing docs: add note to pull the latest git tags before pip installing Dask ( dask#7308 ) Genevieve Buckley
Support for Python 3.9 ( dask#7289 ) crusaderky
Add broadcast-based merge implementation ( dask#7143 ) Richard (Rick) Zamora
Add split_every to graph_manipulation ( dask#7282 ) crusaderky
Typo in optimize docs ( dask#7306 ) Julius Busecke
dask.graph_manipulation support for xarray.Dataset ( dask#7276 ) crusaderky
Add plot width and height support for Bokeh 2.3.0 ( dask#7297 ) James Bourbeau
Add NumPy functions tri , triu_indices , triu_indices_from , tril_indices , tril_indices_from ( dask#6997 ) Illviljan
Remove “cleanup” task in DataFrame on-disk shuffle ( dask#7260 ) Sinclair Target
Use development version of distributed in CI ( dask#7279 ) James Bourbeau
Moving high level graph pack/unpack Dask ( dask#7179 ) Mads R. B. Kristensen
Improve performance of merge_percentiles ( dask#7172 ) Ashwin Srinath
DOC: add dask-sql and fugue ( dask#7129 ) Ray Bell
Example for working with categoricals and parquet ( dask#7085 ) McToel
Adds tree reduction to bincount ( dask#7183 ) Thomas J. Fan
Improve documentation of name in from_array ( dask#7264 ) Bruce Merry
Fix cumsum for empty partitions ( dask#7230 ) Julia Signell
Add map_blocks example to dask array creation docs ( dask#7221 ) Julia Signell
Fix performance issue in dask.graph_manipulation.wait_on() ( dask#7258 ) crusaderky
Replace coveralls with codecov.io ( dask#7246 ) crusaderky
Pin to a particular black rev in pre-commit ( dask#7256 ) Julia Signell
Minor typo in documentation: array-chunks.rst ( dask#7254 ) Magnus Nord
Fix bugs in Blockwise and ShuffleLayer ( dask#7213 ) Richard (Rick) Zamora
Fix parquet filtering bug for "pyarrow-dataset" with pyarrow-3.0.0 ( dask#7200 ) Richard (Rick) Zamora
graph_manipulation without NumPy ( dask#7243 ) crusaderky
Support for NEP-35 ( dask#6738 ) Peter Andreas Entschev
Avoid running unit tests during doctest CI build ( dask#7240 ) James Bourbeau
Run doctests on CI ( dask#7238 ) Julia Signell
Cleanup code quality on set arithmetics ( dask#7196 ) crusaderky
Add dask.array.delete ( dask#7125 ) Julia Signell
Unpin graphviz now that new conda-forge recipe is built ( dask#7235 ) Julia Signell
Don’t use NumPy 1.20 from conda-forge on Mac ( dask#7211 ) crusaderky
map_overlap : Don’t rechunk axes without overlap ( dask#7233 ) Deepak Cherian
Pin graphviz to avoid issue with latest conda-forge build ( dask#7232 ) Julia Signell
Use html_css_files in docs for custom CSS ( dask#7220 ) James Bourbeau
Graph manipulation: clone , bind , checkpoint , wait_on ( dask#7109 ) crusaderky
Fix handling of filter expressions in parquet pyarrow-dataset engine ( dask#7186 ) Joris Van den Bossche
Extend setitem to more closely match numpy ( dask#7033 ) David Hassell
Clean up Python 2 syntax ( dask#7195 ) crusaderky
Fix regression in Delayed._length ( dask#7194 ) crusaderky
dask_layers() tests and tweaks ( dask#7177 ) crusaderky
Properly convert HighLevelGraph in multiprocessing scheduler ( dask#7191 ) Jim Crist-Harif
Don’t fail fast in CI ( dask#7188 ) James Bourbeau
Released on March 5, 2021
Note
This is the first release with support for Python 3.9 and the last release with support for Python 3.6
tcp.write(): cast memoryview to byte itemsize (4555) `Mads R. B. Kristensen`_
Refcount the thread_state.asynchronous flag (4557) `Mads R. B. Kristensen`_
Python 3.9 (4460) `crusaderky`_
Better bokeh defaults for dashboard (4554) `Benjamin Zaitlen`_
Expose system monitor dashboard as individual plot for lab extension (4540) `Jacob Tomlinson`_
Pass on original temp dir from nanny to worker (4549) `Martin Durant`_
Serialize and split (4541) `Mads R. B. Kristensen`_
Use the new HLG pack/unpack API in Dask (4489) `Mads R. B. Kristensen`_
Handle annotations for culled tasks (4544) `Tom Augspurger`_
Make sphinx autosummary and autoclass consistent (4367) `Casey Clements`_
Move _transition* to SchedulerState (4545) `jakirkham`_
Migrate from travis to GitHub actions (4504) `crusaderky`_
Move new_task to SchedulerState (4527) `jakirkham`_
Batch more Scheduler sends (4526) `jakirkham`_
transition_memory_released and get_nbytes() optimizations (4516) `jakirkham`_
Pin black pre-commit (4533) `James Bourbeau`_
Read & write all frames in one pass (4506) `jakirkham`_
Skip stream.write call for empty frames (4507) `jakirkham`_
Prepend frame metadata header (4505) `jakirkham`_
transition_processing_memory optimizations, etc. (4487) `jakirkham`_
Attempt to get client from worker in Queue and Variable (4490) `James Bourbeau`_
Use main branch for zict (4499) `jakirkham`_
Use a callback to close TCP Comms, rather than check every time (4453) `Matthew Rocklin`_
- Deprecate HighlevelGraph.dicts in favor of .layers ( dask#7145 ) Amit Kumar
Released on February 5, 2021
Add percentile support for NEP-35 ( dask#7162 ) Peter Andreas Entschev
Added support for Float64 in column assignment ( dask#7173 ) Nils Braun
Coarsen rechunking error ( dask#7127 ) Davis Bennett
Fix upstream CI tests ( dask#6896 ) Julia Signell
Revise HighLevelGraph Mapping API ( dask#7160 ) crusaderky
Update low-level graph spec to use any hashable for keys ( dask#7163 ) James Bourbeau
Generically rebuild a collection with different keys ( dask#7142 ) crusaderky
Make easier to link issues in PRs ( dask#7130 ) Ray Bell
Add dask.array.append ( dask#7146 ) D-Stacks
Allow dask.array.ravel to accept array_like argument ( dask#7138 ) D-Stacks
Fixes link in array design doc ( dask#7152 ) Thomas J. Fan
Fix example of using blockwise for an outer product ( dask#7119 ) Bruce Merry
Deprecate HighlevelGraph.dicts in favor of .layers ( dask#7145 ) Amit Kumar
Align FastParquetEngine with pyarrow engines ( dask#7091 ) Richard (Rick) Zamora
Merge annotations ( dask#7102 ) Ian Rose
Simplify contents of parts list in read_parquet ( dask#7066 ) Richard (Rick) Zamora
check_meta( ): use class when checking DataFrame types ( dask#7099 ) Mads R. B. Kristensen
Cache several properties ( dask#7104 ) Illviljan
Fix parquet getitem optimization ( dask#7106 ) Richard (Rick) Zamora
Add cytoolz back to CI environment ( dask#7103 ) James Bourbeau
Released on February 5, 2021
Bump minimum Dask to 2021.02.0 (4486) `James Bourbeau`_
Update TaskState documentation about dependents attribute (4440) `Florian Jetter`_
DOC: Autoreformat all functions docstrings (4475) `Matthias Bussonnier`_
Use cached version of is_coroutine_function in stream handling to (4481) `Ian Rose`_
Optimize transitions (4451) `jakirkham`_
Create PULL_REQUEST_TEMPLATE.md (4476) `Ray Bell`_
DOC: typo, directives ends with 2 colons :: (4472) `Matthias Bussonnier`_
DOC: Proper numpydoc syntax for distributed/protocol/*.py (4473) `Matthias Bussonnier`_
Update pytest.skip usage in test_server_listen (4467) `James Bourbeau`_
Unify annotations (4406) `Ian Rose`_
Added worker resources from config (4456) `Tom Augspurger`_
Fix var name in worker validation func (4457) `Gil Forsyth`_
Refactor task_groups & task_prefixes (4452) `jakirkham`_
Use parent._tasks in heartbeat (4450) `jakirkham`_
Refactor SchedulerState from Scheduler (4365) `jakirkham`_
- Partially fix cumprod ( dask#7089 ) Julia Signell
Released on January 22, 2021
Partially fix cumprod ( dask#7089 ) Julia Signell
Test pandas 1.1.x / 1.2.0 releases and pandas nightly ( dask#6996 ) Joris Van den Bossche
Use assign to avoid SettingWithCopyWarning ( dask#7092 ) Julia Signell
'mode' argument passed to bokeh.output_file() ( dask#7034 ) ( dask#7075 ) patquem
Skip empty partitions when doing groupby.value_counts ( dask#7073 ) Julia Signell
Add error messages to assert_eq() ( dask#7083 ) James Lamb
Make cached properties read-only ( dask#7077 ) Illviljan
Released on January 22, 2021
Make system monitor interval configurable (4447) `Matthew Rocklin`_
Add uvloop config value (4448) `Matthew Rocklin`_
Additional optimizations to stealing (4445) `jakirkham`_
Give clusters names (4426) `Jacob Tomlinson`_
Use worker comm pool in Semaphore (4195) `Florian Jetter`_
Set runspec on all new tasks to avoid deadlocks (4432) `Florian Jetter`_
Support TaskState objects in story methods (4434) `Matthew Rocklin`_
Support missing event loop in Client.asynchronous (4436) `Matthew Rocklin`_
Don't require network to inspect tests (4433) `Matthew Rocklin`_
- Avoid deprecated yield_fixture in test_sql.py ( dask#6968 ) Richard (Rick) Zamora
Released on January 15, 2021
map_partitions with review comments ( dask#6776 ) Kumar Bharath Prabhu
Make sure that population is a real list ( dask#7027 ) Julia Signell
Propagate storage_options in read_csv ( dask#7074 ) Richard (Rick) Zamora
Remove all BlockwiseIO code ( dask#7067 ) Richard (Rick) Zamora
Fix CI ( dask#7069 ) James Bourbeau
Add option to control rechunking in reshape ( dask#6753 ) Tom Augspurger
Fix linalg.lstsq for complex inputs ( dask#7056 ) Johnnie Gray
Add compression='infer' default to read_csv ( dask#6960 ) Richard (Rick) Zamora
Revert parameter changes in svd_compressed #7003 ( dask#7004 ) Eric Czech
Skip failing s3 test ( dask#7064 ) Martin Durant
Revert BlockwiseIO ( dask#7048 ) Richard (Rick) Zamora
Add some cross-references to DataFrame.to_bag() and Series.to_bag() ( dask#7049 ) Rob Malouf
Rewrite matmul as blockwise without contraction/concatenate ( dask#7000 ) Rafal Wojdyla
Use functools.cached_property in da.shape ( dask#7023 ) Illviljan
Use meta value in series non_empty ( dask#6976 ) Julia Signell
Revert “Temporarly pin sphinx version to 3.3.1 ( dask#7002 )” ( dask#7014 ) Rafal Wojdyla
Revert python-graphviz pinning ( dask#7037 ) Julia Signell
Accidentally committed print statement ( dask#7038 ) Julia Signell
Pass dropna and observed in agg ( dask#6992 ) Julia Signell
Add index to meta after .str.split with expand ( dask#7026 ) Ruben van de Geer
CI: test pyarrow 2.0 and nightly ( dask#7030 ) Joris Van den Bossche
Temporarily pin python-graphviz in CI ( dask#7031 ) James Bourbeau
Underline section in numpydoc ( dask#7013 ) Matthias Bussonnier
Keep normal optimizations when adding custom optimizations ( dask#7016 ) Matthew Rocklin
Temporarily pin sphinx version to 3.3.1 ( dask#7002 ) Rafal Wojdyla
DOC: Misc formatting ( dask#6998 ) Matthias Bussonnier
Add inline_array option to from_array ( dask#6773 ) Tom Augspurger
Revert “Initial pass at blockwise array creation routines ( dask#6931)” (:pr:`6995 ) James Bourbeau
Set npartitions in set_index ( dask#6978 ) Julia Signell
Upstream config serialization and inheritance ( dask#6987 ) Jacob Tomlinson
Bump the minimum time in test_minimum_time ( dask#6988 ) Martin Durant
Fix pandas dtype inference for read_parquet ( dask#6985 ) Richard (Rick) Zamora
Avoid data loss in set_index with sorted=True ( dask#6980 ) Richard (Rick) Zamora
Bugfix in read_parquet for handling un-named indices with index=False ( dask#6969 ) Richard (Rick) Zamora
Use class when comparing meta data ( dask#6981 ) Mads R. B. Kristensen
Comparing string versions won’t always work ( dask#6979 ) Rafal Wojdyla
Fix dask#6925 ( dask#6982 ) sdementen
Initial pass at blockwise array creation routines ( dask#6931 ) Ian Rose
Simplify has_parallel_type() ( dask#6927 ) Mads R. B. Kristensen
Handle annotation unpacking in BlockwiseIO ( dask#6934 ) Simon Perkins
Avoid deprecated yield_fixture in test_sql.py ( dask#6968 ) Richard (Rick) Zamora
Remove bad graph logic in BlockwiseIO ( dask#6933 ) Richard (Rick) Zamora
Get config item if variable is None ( dask#6862 ) Jacob Tomlinson
Update from_pandas docstring ( dask#6957 ) Richard (Rick) Zamora
Prevent fuse_roots from clobbering annotations ( dask#6955 ) Simon Perkins
Released on January 15, 2021
Add time started to scheduler info (4425) `Jacob Tomlinson`_
Log adaptive error (4422) `Jacob Tomlinson`_
Xfail normalization tests (4411) `Jacob Tomlinson`_
Use dumps_msgpack and loads_msgpack when packing high level graphs (4409) `Mads R. B. Kristensen`_
Add nprocs auto option to dask-worker CLI (4377) `Jacob Tomlinson`_
Type annotation of _reevaluate_occupancy_worker (4398) `jakirkham`_
Type TaskGroup in active_states (4408) `jakirkham`_
Fix test_as_current_is_thread_local (4402) `jakirkham`_
Use list comprehensions to bind TaskGroup type (4401) `jakirkham`_
Make tests pass after 2028 (4403) `Bernhard M. Wiedemann`_
Fix compilation warnings, decide_worker now a C func, stealing improvements (4375) `jakirkham`_
Drop custom __eq__ from Status (4270) `jakirkham`_
test_performance_report: skip without bokeh (4388) `Bruno Pagani`_
Nanny now respects dask settings from ctx mgr (4378) `Florian Jetter`_
Better task duration estimates for outliers (4213) `selshowk`_
Dask internal inherit config (4364) `Jacob Tomlinson`_
Provide setup.py option to profile Cython code (4362) `jakirkham`_
Optimizations of *State and Task* objects and stealing (4358) `jakirkham`_
Cast SortedDict s to dict s in a few key places & other minor changes (4355) `jakirkham`_
Use task annotation priorities for user-level priorities (4354) `James Bourbeau`_
Added docs to highlevelgraph pack/unpack (4352) `Mads R. B. Kristensen`_
Optimizations in notable functions used by transitions (4351) `jakirkham`_
Silence exception when releasing futures on process shutdown (4309) `Benjamin Zaitlen`_
- Switched to CalVer for versioning scheme.
Released on December 10, 2020
Switched to CalVer for versioning scheme.
Introduced new APIs for HighLevelGraph to enable sending high-level representations of task graphs to the distributed scheduler.
Introduced new HighLevelGraph layer objects including BasicLayer , Blockwise , BlockwiseIO , ShuffleLayer , and more.
Added support for applying custom Layer -level annotations like priority , retries , etc. with the dask.annotations context manager.
Updated minimum supported version of pandas to 0.25.0 and NumPy to 1.15.1.
Support for the pyarrow.dataset API to read_parquet .
Several fixes to Dask Array’s SVD.
Make observed kwarg optional ( dask#6952 ) Julia Signell
Min supported pandas 0.25.0 numpy 1.15.1 ( dask#6895 ) Julia Signell
Make order of categoricals unambiguous ( dask#6949 ) Julia Signell
Improve “pyarrow-dataset” statistics performance for read_parquet ( dask#6918 ) Richard (Rick) Zamora
Add observed keyword to groupby ( dask#6854 ) Julia Signell
Make sure include_path_column works when there are multiple partitions per file ( dask#6911 ) Julia Signell
Fix: array.overlap and array.map_overlap block sizes are incorrect when depth is an unsigned bit type ( dask#6909 ) GFleishman
Fix syntax error in HLG docs example ( dask#6946 ) Mark
Return a Bag from sample ( dask#6941 ) Shang Wang
Add ravel_multi_index ( dask#6939 ) Illviljan
Enable parquet metadata collection in parallel ( dask#6921 ) Richard (Rick) Zamora
Avoid using _file in progressbar if it is None ( dask#6938 ) Mark Harfouche
Add Zarr to upstream CI build ( dask#6932 ) James Bourbeau
Introduce BlockwiseIO layer ( dask#6878 ) Richard (Rick) Zamora
Transmit Layer Annotations to Scheduler ( dask#6889 ) Simon Perkins
Update opportunistic caching page to remove experimental warning ( dask#6926 ) Timost
Allow pyarrow >2.0.0 ( dask#6772 ) Richard (Rick) Zamora
Support pyarrow.dataset API for read_parquet ( dask#6534 ) Richard (Rick) Zamora
Add more informative error message to da.coarsen when coarsening factors do not divide shape ( dask#6908 ) Davis Bennett
Only run the cron CI on dask/dask not forks ( dask#6905 ) Jacob Tomlinson
Add annotations to ShuffleLayers ( dask#6913 ) Matthew Rocklin
Temporarily xfail test_from_s3 ( dask#6915 ) James Bourbeau
Added dataframe skew method ( dask#6881 ) Jan Borchmann
Fix dtype in array meta ( dask#6893 ) Julia Signell
Missing name arg in helm install ... ( dask#6903 ) Ruben van de Geer
Fix: exception when reading an item with filters ( dask#6901 ) Martin Durant
Add support for cupyx sparse to dask.array.dot ( dask#6846 ) Akira Naruse
Pin array mindeps up a bit to get the tests to pass [test-mindeps] ( dask#6894 ) Julia Signell
Update/remove pandas and numpy in mindeps ( dask#6888 ) Julia Signell
Fix ArrowEngine bug in use of clear_known_categories ( dask#6887 ) Richard (Rick) Zamora
Fix documentation about task scheduler ( dask#6879 ) Zhengnan Zhao
Add human relative time formatting utility ( dask#6883 ) Jacob Tomlinson
Possible fix for 6864 set_index issue ( dask#6866 ) Richard (Rick) Zamora
BasicLayer : remove dependency arguments ( dask#6859 ) Mads R. B. Kristensen
Serialization of Blockwise ( dask#6848 ) Mads R. B. Kristensen
Address columns=[] bug ( dask#6871 ) Richard (Rick) Zamora
Avoid duplicate parquet schema communication ( dask#6841 ) Richard (Rick) Zamora
Add create_metadata_file utility for existing parquet datasets ( dask#6851 ) Richard (Rick) Zamora
Improve ordering for workloads with a common terminus ( dask#6779 ) Tom Augspurger
Stringify utilities ( dask#6852 ) Mads R. B. Kristensen
Add keyword overwrite=True to to_parquet to remove dangling files when overwriting a pyarrow Dataset . ( dask#6825 ) Greg Hayes
Removed map_tasks() and map_basic_layers() ( dask#6853 ) Mads R. B. Kristensen
Introduce QR iteration to svd_compressed ( dask#6813 ) RogerMoens
dask_distributed_pack() now takes a client argument ( dask#6850 ) Mads R. B. Kristensen
Use map_partitions instead of delayed in set_index ( dask#6837 ) Mads R. B. Kristensen
Add doc hit for as_completed().update(futures) ( dask#6817 ) manuels
Bump GHA setup-miniconda version ( dask#6847 ) Jacob Tomlinson
Remove nans when setting sorted index ( dask#6829 ) Rockwell Weiner
Fix transpose of u in SVD ( dask#6799 ) RogerMoens
Migrate to GitHub Actions ( dask#6794 ) Jacob Tomlinson
Fix sphinx currentmodule usage ( dask#6839 ) James Bourbeau
Fix minimum dependencies CI builds ( dask#6838 ) James Bourbeau
Avoid graph materialization during Blockwise culling ( dask#6815 ) Richard (Rick) Zamora
Fixed typo ( dask#6834 ) Devanshu Desai
Use HighLevelGraph.merge in collections_to_dsk ( dask#6836 ) Mads R. B. Kristensen
Respect dtype in svd compression_matrix #2849 ( dask#6802 ) RogerMoens
Add blocksize to task name ( dask#6818 ) Julia Signell
Check for all-NaN partitions ( dask#6821 ) Rockwell Weiner
Change “institutional” SQL doc section to point to main SQL doc ( dask#6823 ) Martin Durant
Fix: DataFrame.join doesn’t accept Series as other ( dask#6809 ) David Katz
Remove to_delayed operations from to_parquet ( dask#6801 ) Richard (Rick) Zamora
Layer annotation docstrings improvements ( dask#6806 ) Simon Perkins
Avro reader ( dask#6780 ) Martin Durant
Rechunk array if smallest chunk size is smaller than depth ( dask#6708 ) Julia Signell
Add Layer Annotations ( dask#6767 ) Simon Perkins
Add “view code” links to documentation ( dask#6793 ) manuels
Add optional IO-subgraph to Blockwise Layers ( dask#6715 ) Richard (Rick) Zamora
Add high level graph pack/unpack for distributed ( dask#6786 ) Mads R. B. Kristensen
Add missing methods of the Dataframe API ( dask#6789 ) Stephannie Jimenez Gacha
Add doc on managing environments ( dask#6778 ) Martin Durant
HLG: get_all_external_keys() ( dask#6774 ) Mads R. B. Kristensen
Avoid rechunking in reshape with chunksize=1 ( dask#6748 ) Tom Augspurger
Try to make categoricals work on join ( dask#6205 ) Julia Signell
Fix some minor typos and trailing whitespaces in array-slice.rst ( dask#6771 ) Magnus Nord
Bugfix for parquet metadata writes of empty dataframe partitions (pyarrow) ( dask#6741 ) Callum Noble
Document meta kwarg in map_blocks and map_overlap . ( dask#6763 ) Peter Andreas Entschev
Begin experimenting with parallel prefix scan for cumsum and cumprod ( dask#6675 ) Erik Welch
Clarify differences in boolean indexing between dask and numpy arrays ( dask#6764 ) Illviljan
Efficient serialization of shuffle layers ( dask#6760 ) James Bourbeau
Config array optimize to skip fusion and return a HLG ( dask#6751 ) Mads R. B. Kristensen
Temporarily use pyarrow<2 in CI ( dask#6759 ) James Bourbeau
Fix meta for min / max reductions ( dask#6736 ) Peter Andreas Entschev
Add 2D possibility to da.linalg.lstsq - mirroring numpy ( dask#6749 ) Pascal Bourgault
CI: Fixed bug causing flaky test failure in pivot ( dask#6752 ) Tom Augspurger
Serialization of layers ( dask#6693 ) Mads R. B. Kristensen
Add attrs property to Series/Dataframe ( dask#6742 ) Illviljan
Removed Mutable Default Argument ( dask#6747 ) Mads R. B. Kristensen
Adjust parquet ArrowEngine to allow more easy subclass for writing ( dask#6505 ) Joris Van den Bossche
Add ShuffleStage HLG Layer ( dask#6650 ) Richard (Rick) Zamora
Handle literal in meta_from_array ( dask#6731 ) Peter Andreas Entschev
Do balanced rechunking even if chunks are the same ( dask#6735 ) Chris Roat
Fix docstring DataFrame.set_index ( dask#6739 ) Gil Forsyth
Ensure HighLevelGraph layers always contain Layer instances ( dask#6716 ) James Bourbeau
Map on HighLevelGraph Layers ( dask#6689 ) Mads R. B. Kristensen
Update overlap *_like function calls and CuPy tests ( dask#6728 ) Peter Andreas Entschev
Fixes for svd with array_function ( dask#6727 ) Peter Andreas Entschev
Added doctest extension for documentation ( dask#6397 ) Jim Circadian
Minor fix to #5628 using @pentschev’s suggestion ( dask#6724 ) John A Kirkham
Change type of Dask array when meta type changes ( dask#5628 ) Matthew Rocklin
Add az ( dask#6719 ) Ray Bell
HLG: get_dependencies() of single keys ( dask#6699 ) Mads R. B. Kristensen
Revert “Revert “Use HighLevelGraph layers everywhere in collections ( dask#6510 )” ( dask#6697 )” ( dask#6707 ) Tom Augspurger
Allow *_like array creation functions to respect input array type ( dask#6680 ) Genevieve Buckley
Update dask-sphinx-theme version ( dask#6700 ) Gil Forsyth
Released on December 10, 2020
Switched to CalVer for versioning scheme.
The scheduler can now receives Dask HighLevelGraph s instead of raw dictionary task graphs. This allows for a much more efficient communication of task graphs from the client to the scheduler.
Added support for using custom Layer-level annotations like priority, retries, etc. with the dask.annotations context manager.
Updated minimum supported version of Dask to 2020.12.0.
Added many type annotations and updates to allow for gradually Cythonizing the scheduler.
Some common optimizations across transitions (4348) `jakirkham`_
Drop serialize extension (4344) `jakirkham`_
Log duplicate workers in scheduler (4338) `Matthew Rocklin`_
Annotation of some comm related methods in the Scheduler (4341) `jakirkham`_
Optimize assert in validate_waiting (4342) `jakirkham`_
Optimize decide_worker (4332) `jakirkham`_
Store occupancy in _reevaluate_occupancy_worker (4337) `jakirkham`_
Handle WorkerState memory_limit of None (4335) `jakirkham`_
Use bint to annotate boolean attributes (4334) `jakirkham`_
Optionally use offload executor in worker (4307) `Matthew Rocklin`_
Optimize send_task_to_worker (4331) `jakirkham`_
Optimize valid_workers (4329) `jakirkham`_
Store occupancy in transition_waiting_processing (4330) `jakirkham`_
Optimize get_comm_cost (4328) `jakirkham`_
Use .pop(...) to remove key (4327) `jakirkham`_
Use operator.attrgetter on WorkerState.address (4324) `jakirkham`_
Annotate Task* objects for Cythonization (4302) `jakirkham`_
Ensure retire_workers always return a dict (4323) `jakirkham`_
Some Cython fixes for WorkerState (4321) `jakirkham`_
Optimize WorkerState.__eq__ (4320) `jakirkham`_
Swap order of TaskGroup and TaskPrefix (4319) `jakirkham`_
Check traceback object can be unpickled (4299) `jakirkham`_
Move TaskGroup & TaskPrefix before TaskState (4318) `jakirkham`_
Remove empty test_highgraph.py file (4313) `James Bourbeau`_
Ensure that retire_workers returns a dict (4315) `Matthew Rocklin`_
Annotate WorkerState for Cythonization (4294) `jakirkham`_
Close comm on low-level errors (4239) `jochen-ott-by`_
Coerce new TaskState.nbytes value to int (4311) `jakirkham`_
Remove offload try/except for thread_name_prefix keyword (4308) `James Bourbeau`_
Fix pip install issue on CI (4310) `jakirkham`_
Transmit Layer annotations to scheduler (4279) `Simon Perkins`_
Ignores any compiled files generated by Cython (4301) `jakirkham`_
Protect against missing key in get_metrics (4300) `Matthew Rocklin`_
Provide option to build Distributed with Cython (4292) `jakirkham`_
Set WorkerState.processing w/dict in clean (4295) `jakirkham`_
Annotate ClientState for Cythonization (4290) `jakirkham`_
Annotate check_idle_saturated for Cythonization (4289) `jakirkham`_
Avoid flicker in TaskStream with "Scheduler is empty" message (4284) `Matthew Rocklin`_
Make gather_dep robust to missing tasks (4285) `Matthew Rocklin`_
Annotate extract_serialize (for Cythonization) (4283) `jakirkham`_
Move nbytes from Worker's state to TaskState (4274) `Gil Forsyth`_
Drop extra type check in _extract_serialize (4281) `jakirkham`_
Move Status to top-level import (4280) `Matthew Rocklin`_
Add __hash__ and __eq__ for TaskState (4278) `jakirkham`_
Add __hash__ and __eq__ for ClientState (4276) `jakirkham`_
Collect report's client_key``s in a ``list (4275) `jakirkham`_
Precompute hash for WorkerState (4271) `jakirkham`_
Use Status Enum in remove_worker (4269) `jakirkham`_
Add aggregated topic logs and log_event method (4230) `James Bourbeau`_
Find the set of workers instead of their frequency (4267) `jakirkham`_
Use set.update to include other comms (4268) `jakirkham`_
Support string timeouts in sync (4266) `James Bourbeau`_
Use dask.utils.stringify() instead of distributed.utils.tokey() (4255) `Mads R. B. Kristensen`_
Use .items() to walk through keys and values (4261) `jakirkham`_
Simplify frame length packing in TCP write (4257) `jakirkham`_
Comm/tcp listener: do not pass comm with failed handshake to comm_handler (4240) `jochen-ott-by`_
Fuse steps in extract_serialize (4254) `jakirkham`_
Drop test_sklearn (4253) `jakirkham`_
Document task priority tie breaking (4252) `James Bourbeau`_
__dask_distributed_pack__(): client argument (4248) `Mads R. B. Kristensen`_
Configurable timeouts for worker_client and get_client (4146) `GeethanjaliEswaran`_
Add dask/distributed versions to performance_report (4249) `Matthew Rocklin`_
Update miniconda GitHub action (4250) `James Bourbeau`_
UCX closing ignore error (4236) `Mads R. B. Kristensen`_
Redirect to dask-worker cli documentation (4247) `Timost`_
Upload file worker plugin (4238) `Ian Rose`_
Create dependency TaskState as needed in gather_dep (4241) `Gil Forsyth`_
Instantiate plugin if needed in register_worker_plugin (4198) `Julia Signell`_
Allow actors to call actors on the same worker (4225) `Martin Durant`_
Special case profile thread in leaked thread check (4229) `James Bourbeau`_
Use intersection() on a set instead of dict_keys in update_graph (4227) `Mads R. B. Kristensen`_
Communicate HighLevelGraphs directly to the Scheduler (4140) `Mads R. B. Kristensen`_
Add get_task_metadata context manager (4216) `James Bourbeau`_
Task state logs and data fix (4206) `Gil Forsyth`_
Send active task durations from worker to scheduler (4192) `James Bourbeau`_
Fix state check in test_close_gracefully (4203) `Gil Forsyth`_
Avoid materializing layers in Client.compute() (4196) `Mads R. B. Kristensen`_
Add TaskState metadata (4191) `James Bourbeau`_
Fix regression in task stealing for already released keys (4182) `Florian Jetter`_
Fix _graph_to_futures bug for futures-based dependencies (4178) `Richard J Zamora`_
High level graph dumps/loads support (4174) `Mads R. B. Kristensen`_
Implement pass HighLevelGraphs through _graph_to_futures (4139) `Mads R. B. Kristensen`_
Support async preload click commands (4170) `James Bourbeau`_
dask-worker cli memory limit option doc fix (4172) `marwan116`_
Add TaskState to worker.py (4107) `Gil Forsyth`_
Increase robustness of Semaphore.release (4151) `Lucas Rademaker`_
Skip batched comm test win / tornado5 (4166) `Tom Augspurger`_
Set Zict buffer target to maxsize when memory_target_fraction is False (4156) `Krishan Bhasin`_
Add PipInstall WorkerPlugin (3216) `Matthew Rocklin`_
Log KilledWorker events in the scheduler (4157) `Matthew Rocklin`_
Fix test_gpu_metrics failure (4154) `jakirkham`_
Pin pytest-asyncio version (4212) James Bourbeau_
Pin pytest-asyncio version (4212) `James Bourbeau`_
Replace AsyncProcess exit handler by weakref.finalize (4184) `Peter Andreas Entschev`_
Remove hard coded connect handshake timeouts (4176) `Florian Jetter`_
- Allow rechunk to evenly split into N chunks ( dask#6420 ) Scott Sievert
Allow rechunk to evenly split into N chunks ( dask#6420 ) Scott Sievert
Support SubgraphCallable in str_graph() (4148) `Mads R. B. Kristensen`_
Handle exceptions in BatchedSend (4135) `Tom Augspurger`_
Fix for missing : in autosummary docs (4143) `Gil Forsyth`_
Limit GPU metrics to visible devices only (3810) `Jacob Tomlinson`_
- _repr_html_ : color sides darker instead of drawing all the lines ( dask#6683 ) Julia Signell
repr_html : color sides darker instead of drawing all the lines ( dask#6683 ) Julia Signell
Removes warning from nanstd and nanvar ( dask#6667 ) Thomas J. Fan
Get shape of output from original array - map_overlap ( dask#6682 ) Julia Signell
Replace np.searchsorted with bisect in indexing ( dask#6669 ) Joachim B Haga
Make sure subprocesses have a consistent hash for bag groupby ( dask#6660 ) Itamar Turner-Trauring
Revert “Use HighLevelGraph layers everywhere in collections ( dask#6510 )” ( dask#6697 ) Tom Augspurger
Use pandas.testing ( dask#6687 ) John A Kirkham
Improve 128-bit floating-point skip in tests ( dask#6676 ) Elliott Sales de Andrade
Allow setting dataframe items using a bool dataframe ( dask#6608 ) Julia Signell
Fix typo ( dask#6692 ) garanews
Fix a few typos ( dask#6678 ) Pav A
Use pandas.testing (4138) `jakirkham`_
Fix a few typos (4131) `Pav A`_
Return right away in Cluster.close if cluster is already closed (4116) `Tom Rochette`_
Update async doc with example on .compute() vs client.compute() (4137) `Benjamin Zaitlen`_
Correctly tear down LoopRunner in Client (4112) `Sergey Kozlov`_
Simplify Client._graph_to_futures() (4127) `Mads R. B. Kristensen`_
Cleanup new exception traceback (4125) `Krishan Bhasin`_
Stop writing config files by default (4123) `Matthew Rocklin`_
- Partially reverted changes to Array indexing that produces large changes. This restores the behavior from Dask 2.25.0 and earlier, with a warning wh
Partially reverted changes to Array indexing that produces large changes. This restores the behavior from Dask 2.25.0 and earlier, with a warning when large chunks are produced. A configuration option is provided to avoid creating the large chunks, see Efficiency . ( dask#6665 ) Tom Augspurger
Add meta to to_dask_array ( dask#6651 ) Kyle Nicholson
Fix dask#6631 and dask#6611 ( dask#6632 ) Rafal Wojdyla
Infer object in array reductions ( dask#6629 ) Daniel Saxton
Adding v_based flag for svd_flip ( dask#6658 ) Eric Czech
Fix flakey array mean ( dask#6656 ) Sam Grayson
Removed dsk equality check from SubgraphCallable.eq ( dask#6666 ) Mads R. B. Kristensen
Use HighLevelGraph layers everywhere in collections ( dask#6510 ) Mads R. B. Kristensen
Adds hash dunder method to SubgraphCallable for caching purposes ( dask#6424 ) Andrew Fulton
Stop writing commented out config files by default ( dask#6647 ) Matthew Rocklin
Add support for collect list aggregation via agg API ( dask#6655 ) Madhur Tandon
Slightly better error message ( dask#6657 ) Julia Signell
Fix SSL connection_args for progressbar connect (4122) `jennalc`_
- Preserve dtype in svd ( dask#6643 ) Eric Czech
Preserve dtype in svd ( dask#6643 ) Eric Czech
store() : create a single HLG layer ( dask#6601 ) Mads R. B. Kristensen
Add pre-commit CI build ( dask#6645 ) James Bourbeau
Update .pre-commit-config to latest black. ( dask#6641 ) Julia Signell
Update super usage to remove Python 2 compatibility ( dask#6630 ) Poruri Sai Rahul
Remove u string prefixes ( dask#6633 ) Poruri Sai Rahul
Improve error message for to_sql ( dask#6638 ) Julia Signell
Use empty list as categories ( dask#6626 ) Julia Signell
Add autofunction to array api docs for more ufuncs ( dask#6644 ) James Bourbeau
Add a number of missing ufuncs to dask.array docs ( dask#6642 ) Ralf Gommers
Add HelmCluster docs ( dask#6290 ) Jacob Tomlinson
Fix registering a worker plugin with name arg (4105) `Nick Evans`_
Support different remote_python paths on cluster nodes (4085) `Abdulelah Bin Mahfoodh`_
Allow RuntimeError s when closing global clients (4115) `Matthew Rocklin`_
Match pre-commit in dask (4049) `Julia Signell`_
Update super usage (4110) `Poruri Sai Rahul`_
- Backend-aware dtype inference for single-chunk svd ( dask#6623 ) Eric Czech
Backend-aware dtype inference for single-chunk svd ( dask#6623 ) Eric Czech
Make array.reduction docstring match for dtype ( dask#6624 ) Martin Durant
Set lower bound on compression level for svd_compressed using rows and cols ( dask#6622 ) Eric Czech
Improve SVD consistency and small array handling ( dask#6616 ) Eric Czech
Add svd_flip #6599 ( dask#6613 ) Eric Czech
Handle sequences containing dask Arrays ( dask#6595 ) Gabe Joseph
Avoid large chunks from getitem with lists ( dask#6514 ) Tom Augspurger
Eagerly slice numpy arrays in from_array ( dask#6605 ) Deepak Cherian
Restore ability to pickle dask arrays ( dask#6594 ) Noah D. Brenowitz
Add SVD support for short-and-fat arrays ( dask#6591 ) Eric Czech
Add simple chunk type registry and defer as appropriate to upcast types ( dask#6393 ) Jon Thielen
Align coarsen chunks by default ( dask#6580 ) Deepak Cherian
Fixup reshape on unknown dimensions and other testing fixes ( dask#6578 ) Ryan Williams
Add validation and fixes for HighLevelGraph dependencies ( dask#6588 ) Mads R. B. Kristensen
Fix linting issue ( dask#6598 ) Tom Augspurger
Skip bokeh version 2.0.0 ( dask#6572 ) John A Kirkham
Added bytes/row calculation when using meta ( dask#6585 ) McToel
Handle min_count in Series.sum / prod ( dask#6618 ) Daniel Saxton
Update DataFrame.set_index docstring ( dask#6549 ) Timost
Always compute 0 and 1 quantiles during quantile calculations ( dask#6564 ) Erik Welch
Fix wrong path when reading empty csv file ( dask#6573 ) Abdulelah Bin Mahfoodh
Doc: Troubleshooting dashboard 404 ( dask#6215 ) Kilian Lieret
Fixup extraConfig example ( dask#6625 ) Tom Augspurger
Update supported Python versions ( dask#6609 ) Julia Signell
Document dask/daskhub helm chart ( dask#6560 ) Tom Augspurger
Add logging for adaptive start and stop (4101) `Matthew Rocklin`_
Don't close a nannied worker if it hasn't yet started (4093) `Matthew Rocklin`_
Respect timeouts when closing clients synchronously (4096) `Matthew Rocklin`_
Log when downloading a preload script (4094) `Matthew Rocklin`_
dask-worker --nprocs accepts negative values (4089) `Dror Speiser`_
Support zero-worker clients (4090) `Matthew Rocklin`_
Exclude fire-and-forget client from metrics (4078) `Tom Augspurger`_
Drop Serialized.deserialize() method (4073) `jakirkham`_
Add timeout= keyword to Client.wait_for_workers method (4087) `Matthew Rocklin`_
- Compare key hashes in subs() ( dask#6559 ) Mads R. B. Kristensen
Compare key hashes in subs() ( dask#6559 ) Mads R. B. Kristensen
Rerun with latest black release ( dask#6568 ) James Bourbeau
License update ( dask#6554 ) Tom Augspurger
Add gs read_parquet example ( dask#6548 ) Ray Bell
Remove version from documentation page names ( dask#6558 ) James Bourbeau
Update kubernetes-helm.rst ( dask#6523 ) David Sheldon
Stop 2020 survey ( dask#6547 ) Tom Augspurger
Update for black (4081) `Tom Augspurger`_
Provide informative error when connecting an older version of Dask (4076) `Matthew Rocklin`_
Simplify pack_frames (4068) `jakirkham`_
Simplify frame_split_size (4067) `jakirkham`_
Use list.insert to add prelude up front (4066) `jakirkham`_
Graph helper text (4064) `Julia Signell`_
Graph dashboard: Reset container data if task number is too large (4056) `Florian Jetter`_
Ensure semaphore picks correct IOLoop for threadpool workers (4060) `Florian Jetter`_
Add cluster log method (4051) `Jacob Tomlinson`_
Cleanup more exception tracebacks (4054) `Krishan Bhasin`_
Improve documentation of scheduler.locks options (4062) `Florian Jetter`_
- Fix setting random seed in tests. ( dask#6518 ) Elliott Sales de Andrade
Fix setting random seed in tests. ( dask#6518 ) Elliott Sales de Andrade
Support meta in apply gufunc ( dask#6521 ) joshreback
Replace cupy.sparse with cupyx.scipy.sparse ( dask#6530 ) John A Kirkham
Bump up tolerance for rolling tests ( dask#6502 ) Julia Signell
Implement DatFrame.len ( dask#6515 ) Tom Augspurger
Infer arrow schema in to_parquet (for ArrowEngine`) ( dask#6490 ) Richard (Rick) Zamora
Fix parquet test when no pyarrow ( dask#6524 ) Martin Durant
Remove problematic filter arguments in ArrowEngine ( dask#6527 ) Richard (Rick) Zamora
Avoid schema validation by default in ArrowEngine ( dask#6536 ) Richard (Rick) Zamora
Use unpack_collections in make_blockwise_graph ( dask#6517 ) Thomas J. Fan
Move key_split() from optimization.py to utils.py ( dask#6529 ) Mads R. B. Kristensen
Make tests run on moto server ( dask#6528 ) Martin Durant
Move toolbar to above and fix y axis (#4043) `Julia Signell`_
Make behavior clearer for how to get worker dashboard (#4047) `Julia Signell`_
Worker dashboard clean up (#4046) `Julia Signell`_
Add a default argument to the datasets and a possibility to override datasets (#4052) `Nils Braun`_
Discover HTTP endpoints (#3744) `Martin Durant`_
- Reduce np.zeros , ones , and full array size with broadcasting ( dask#6491 ) Matthias Bussonnier
Reduce np.zeros , ones , and full array size with broadcasting ( dask#6491 ) Matthias Bussonnier
Add missing meta= for trim in map_overlap ( dask#6494 ) Peter Andreas Entschev
Bag repartition partition size ( dask#6371 ) joshreback
Scalar.dask_layers() to return self._name instead of self.key ( dask#6507 ) Mads R. B. Kristensen
Update dependencies correctly in fuse_root optimization ( dask#6508 ) Mads R. B. Kristensen
Adds items to dataframe ( dask#6503 ) Thomas J. Fan
Include compression in write_table call ( dask#6499 ) Julia Signell
Fixed warning in nonempty_series ( dask#6485 ) Tom Augspurger
Intelligently determine partitions based on type of first arg ( dask#6479 ) Matthew Rocklin
Fix pyarrow mkdirs ( dask#6475 ) Julia Signell
Fix duplicate parquet output in to_parquet ( dask#6451 ) michaelnarodovitch
Fix documentation da.histogram ( dask#6439 ) Roberto Panai
Add agg nunique example ( dask#6404 ) Ray Bell
Fixed a few typos in the SQL docs ( dask#6489 ) Mike McCarty
Docs for SQLing ( dask#6453 ) Martin Durant
Tidy up exception traceback in TCP Comms (4042) `Krishan Bhasin`_
Angle on the x-axis labels (4030) `Mathieu Dugré`_
Always set RMM's strides in the header (4039) `jakirkham`_
Fix documentation upload_file (4038) `Roberto Panai`_
Update UCX tests for new handshake step (4036) `jakirkham`_
Add test for informative errors in serialization cases (4029) `Matthew Rocklin`_
Add compression, pickle protocol to comm contexts (4019) `Matthew Rocklin`_
Make GPU plots robust to not having GPUs (4008) `Matthew Rocklin`_
Update PendingDeprecationWarning with correct version number (4025) `Matthias Bussonnier`_
Install PyTorch on CI (4017) `jakirkham`_
Try getting cluster dashboard_link before asking scheduler (4018) `Matthew Rocklin`_
Ignore writeable frames with builtin array (4016) `jakirkham`_
Just extend frames2 by frames (4015) `jakirkham`_
Serialize builtin array (4013) `jakirkham`_
Use cuDF's assert_eq (4014) `jakirkham`_
Clear function cache whenever we upload a new file (3993) `Jack Xiaosong Xu`_
Emit warning when assign/comparing string with Status Enum (3875) `Matthias Bussonnier`_
Track mutable frames (4004) `jakirkham`_
Improve bytes and bytearray serialization (4009) `jakirkham`_
Fix memory histogram values in dashboard (4006) `Willi Rath`_
- Compatibility for NumPy dtype deprecation ( dask#6430 ) Tom Augspurger
Compatibility for NumPy dtype deprecation ( dask#6430 ) Tom Augspurger
Implement sizeof for some bytes -like objects ( dask#6457 ) John A Kirkham
HTTP error for new fsspec ( dask#6446 ) Martin Durant
When RecursionError is raised, return uuid from tokenize function ( dask#6437 ) Julia Signell
Install deps of upstream-dev packages ( dask#6431 ) Tom Augspurger
Use updated link in setup.cfg ( dask#6426 ) Zhengnan Zhao
Add single quotes around column names if strings ( dask#6471 ) Gil Forsyth
Refactor ArrowEngine for better read_parquet performance ( dask#6346 ) Richard (Rick) Zamora
Add tolist dispatch ( dask#6444 ) GALI PREM SAGAR
Compatibility with pandas 1.1.0rc0 ( dask#6429 ) Tom Augspurger
Multi value pivot table ( dask#6428 ) joshreback
Duplicate argument definitions in to_csv docstring ( dask#6411 ) Jun Han (Johnson) Ooi
Add utility to docs to convert YAML config to env vars and back ( dask#6472 ) Jacob Tomlinson
Fix parameter server rendering ( dask#6466 ) Scott Sievert
Fixes broken links ( dask#6403 ) Jim Circadian
Complete parameter server implementation in docs ( dask#6449 ) Scott Sievert
Fix typo ( dask#6436 ) Jack Xiaosong Xu
Only call frame_split_size when there are frames (3996) `jakirkham`_
Fix failing test_bandwidth (3999) `jakirkham`_
Handle sum of memory percentage when memory_limit is 0 (3984) `Julia Signell`_
Drop msgpack pre-0.5.2 compat code (3977) `jakirkham`_
Revert to localhost for local IP if no network available (3991) `Matthew Rocklin`_
Add missing backtick in inline directive. (3988) `Matthias Bussonnier`_
Warn when threads_per_worker is set to zero (3986) `Julia Signell`_
Use memoryview in unpack_frames (3980) `jakirkham`_
Iterate over list of comms (3959) `Matthew Rocklin`_
Streamline pack_frames/unpack_frames frames (3973) `jakirkham`_
Always attempt to create dask-worker-space folder and continue if it exists (3972) `Jendrik Jördening`_
Use merge_frames with host memory only (3971) `jakirkham`_
Simplify pack_frames_prelude (3961) `jakirkham`_
Use continuation prompt for proper example parsing (3966) `Matthias Bussonnier`_
Ensure writable frames (3967) `jakirkham`_
- Correct error message in array.routines.gradient() ( dask#6417 ) johnomotani
Correct error message in array.routines.gradient() ( dask#6417 ) johnomotani
Fix blockwise concatenate for array with some dimension=1 ( dask#6342 ) Matthias Bussonnier
Fix bag.take example ( dask#6418 ) Roberto Panai
Groups values in optimization pass should only be graph and keys – not an optimization + keys ( dask#6409 ) Benjamin Zaitlen
Call custom optimizations once, with kwargs provided ( dask#6382 ) Clark Zinzow
Include pickle5 for testing on Python 3.7 ( dask#6379 ) John A Kirkham
Correct typo in error message ( dask#6422 ) Tom McTiernan
Use pytest.warns to check for UserWarning ( dask#6378 ) Richard (Rick) Zamora
Parse bytes_per_chunk keyword from string ( dask#6370 ) Matthew Rocklin
Numpydoc formatting ( dask#6421 ) Matthias Bussonnier
Unpin numpydoc following 1.1 release ( dask#6407 ) Gil Forsyth
Numpydoc formatting ( dask#6402 ) Matthias Bussonnier
Add instructions for using conda when installing code for development ( dask#6399 ) Ray Bell
Update visualize docstrings ( dask#6383 ) Zhengnan Zhao
Fix data replication error (3963) `Andrew Fulton`_
Treat falsey local directory as None (3964) `Tom Augspurger`_
Unpin numpydoc now that 1.1 is released (3957) `Gil Forsyth`_
Error hard when Dask has mismatched versions or lz4 installed (3936) `Matthew Rocklin`_
Skip coercing to bytes in merge_frames (3960) `jakirkham`_
UCX: reuse endpoints in order to fix NVLINK issue (3953) `Mads R. B. Kristensen`_
Optionally use pickle5 (3849) `jakirkham`_
Update time per task chart with filtering and pie (3933) `Benjamin Zaitlen`_
UCX: explicit shutdown message (3950) `Mads R. B. Kristensen`_
Avoid too aggressive retry of connections (3944) `Matthias Bussonnier`_
Parse timeouts in Client.sync (3952) `Matthew Rocklin`_
Synchronize on non-trivial CUDA frame transmission (3949) `jakirkham`_
Serialize memoryview with shape and format (3947) `jakirkham`_
Move scheduler_comm into Cluster.__init__ (3945) `Matthew Rocklin`_
- Register sizeof for numpy zero-strided arrays ( dask#6343 ) Matthias Bussonnier
Register sizeof for numpy zero-strided arrays ( dask#6343 ) Matthias Bussonnier
Use concatenate_lookup in concatenate ( dask#6339 ) John A Kirkham
Fix rechunking of arrays with some zero-length dimensions ( dask#6335 ) Matthias Bussonnier
Dispatch iloc` calls to getitem ( dask#6355 ) Gil Forsyth
Handle unnamed pandas RangeIndex in fastparquet engine ( dask#6350 ) Richard (Rick) Zamora
Preserve index when writing partitioned parquet datasets with pyarrow ( dask#6282 ) Richard (Rick) Zamora
Use ignore_index for pandas’ group_split_dispatch ( dask#6251 ) Richard (Rick) Zamora
Add doc describing argument ( dask#6318 ) asmith26
Link issue on using async with executor_submit (3939) `jakirkham`_
Make dashboard server listens on all IPs by default even when interface is set explicitly (3941) `Loïc Estève`_
Update logic for worker removal in check ttl (3927) `Benjamin Zaitlen`_
Close a created cluster quietly (3935) `Matthew Rocklin`_
Ensure Worker.run* handles kwargs correctly (3937) `jakirkham`_
Restore Scheduler.time_started for Dask Gateway (3934) `Tom Augspurger`_
Fix exception handling in _wait_until_connected (3912) `Alexander Clausen`_
Make local directory if it does not exist (3928) `Matthew Rocklin`_
Install vanilla status route if bokeh dependency is not satisfied (3844) `joshreback`_
Make Worker.delete_data sync (3922) `Peter Andreas Entschev`_
Fix ensure_bytes import location (3919) `jakirkham`_
Fix race condition in repeated calls to cluster.adapt() (3915) `Jacob Tomlinson`_
- Cast chunk sizes to python int dtype ( dask#6326 ) Gil Forsyth
Cast chunk sizes to python int dtype ( dask#6326 ) Gil Forsyth
Add shape=None to *_like() array creation functions ( dask#6064 ) Anderson Banihirwe
Update expected error msg for protocol difference in fsspec ( dask#6331 ) Gil Forsyth
Fix for floats < 1 in parse_bytes ( dask#6311 ) Gil Forsyth
Fix exception causes all over the codebase ( dask#6308 ) Ram Rachum
Fix duplicated tests ( dask#6303 ) James Lamb
Remove unused testing function ( dask#6304 ) James Lamb
Add high-level CSV Subgraph ( dask#6262 ) Gil Forsyth
Fix ValueError when merging an index-only 1-partition dataframe ( dask#6309 ) Krishan Bhasin
Make index.map clear divisions. ( dask#6285 ) Julia Signell
Add link to 2020 survey ( dask#6328 ) Tom Augspurger
Update bag.rst ( dask#6317 ) Ben Shaver
Notify worker plugins when a task is released (3817) `Nick Evans`_
Update heartbeat checks in scheduler (3896) `Benjamin Zaitlen`_
Make encryption default if Security is given arguments (3887) `Matthew Rocklin`_
Show cpu_fraction on hover for dashboard workers circle plot. (3906) `Loïc Estève`_
Prune virtual client on variable deletion (3910) `Marco Neumann`_
Fix total aggregated metrics in dashboard (3897) `Loïc Estève`_
Support Bokeh 2.1 (3904) `Matthew Rocklin`_
Update related-work.rst (3889) `DomHudson`_
Skip test_pid_file in older versions of Python (3888) `Matthew Rocklin`_
Replace stream= with comm= in handlers (3860) `Julien Jerphanion`_
Check hosts for None value in SSH cluster. (3883) `Matthias Bussonnier`_
Allow dictionaries in security= keywords (3874) `Matthew Rocklin`_
Use pickle protocol 5 with NumPy object arrays (3871) `jakirkham`_
Cast any frame to uint8 (same type as bytes) (3870) `jakirkham`_
Use Enum for worker, scheduler and nanny status. (3853) `Matthias Bussonnier`_
Drop legacy buffer_interface assignment (3869) `jakirkham`_
Drop old frame splitting in NumPy serialization (3868) `jakirkham`_
Drop no longer needed local import pickle (3865) `jakirkham`_
Fix typo in feed's log message (3867) `jakirkham`_
Tidy pickle (3866) `jakirkham`_
Handle empty times in task stream (3862) `Benjamin Zaitlen`_
Change asyncssh objects to sphinx references (3861) `Jacob Tomlinson`_
Improve SSHCluster docstring for connect_options (3859) `Jacob Tomlinson`_
Validate address parameter in client constructor (3842) `joshreback`_
Use SpecCluster name in worker names (3855) `Loïc Estève`_
Allow async add_worker and remove_worker plugin methods (3847) `James Bourbeau`_
- Cast slicing index to dask array if same shape as original ( dask#6273 ) Julia Signell
Cast slicing index to dask array if same shape as original ( dask#6273 ) Julia Signell
Fix stack error message ( dask#6268 ) Stephanie Gott
full & full_like : error on non-scalar fill_value ( dask#6129 ) Huite
Support for multiple arrays in map_overlap ( dask#6165 ) Eric Czech
Pad resample divisions so that edges are counted ( dask#6255 ) Julia Signell
Random sampling of k elements from a dask bag #4799 ( dask#6239 ) Antonio Ercole De Luca
Add dropna , sort , and ascending to sort_values ( dask#5880 ) Julia Signell
Generalize from_dask_array ( dask#6263 ) GALI PREM SAGAR
Add derived docstring for SeriesGroupby.nunique ( dask#6284 ) Julia Signell
Remove NotImplementedError in resample with rule ( dask#6274 ) Abdulelah Bin Mahfoodh
Add dd.to_sql ( dask#6038 ) Ryan Williams
Update remote data section ( dask#6258 ) Ray Bell
Merge frames in deserialize_bytes (3639) `John Kirkham`_
Allow SSHCluster to take a list of connect_options (3854) `Jacob Tomlinson`_
Add favicon to performance report (3852) `Jacob Tomlinson`_
Add dashboard plots for the amount of time spent per key and for transfer/serialization (3792) `Benjamin Zaitlen`_
Fix variable name in journey of a task documentation (3840) `Matthias Bussonnier`_
Fix typo in journey of a task doc (3838) `James Bourbeau`_
Register dask_cudf serializers (3832) `John Kirkham`_
Fix key check in rebalance missing keys (3834) `Jacob Tomlinson`_
Allow collection of partial profile information in case of exceptions (3773) `Florian Jetter`_
- Small improvements to da.pad ( dask#6213 ) Mark Boer
Small improvements to da.pad ( dask#6213 ) Mark Boer
Return tuple if multiple outputs in dask.array.apply_gufunc , add test to check for tuple ( dask#6207 ) Kai Mühlbauer
Support stack with unknown chunksizes ( dask#6195 ) swapna
Random Choice on Bags ( dask#6208 ) Antonio Ercole De Luca
Raise warning delayed.visualise() ( dask#6216 ) Amol Umbarkar
Ensure other pickle arguments work ( dask#6229 ) John A Kirkham
Overhaul fuse() config ( dask#6198 ) crusaderky
Update dask.order.order to consider “next” nodes using both FIFO and LIFO ( dask#5872 ) Erik Welch
Use 0 as fill_value for more agg methods ( dask#6245 ) Julia Signell
Generalize rearrange_by_column_tasks and add DataFrame.shuffle ( dask#6066 ) Richard (Rick) Zamora
Xfail test_rolling_numba_engine for newer numba and older pandas ( dask#6236 ) James Bourbeau
Generalize fix_overlap ( dask#6240 ) GALI PREM SAGAR
Fix DataFrame.shape with no columns ( dask#6237 ) noreentry
Avoid shuffle when setting a presorted index with overlapping divisions ( dask#6226 ) Krishan Bhasin
Adjust the Parquet engine classes to allow more easily subclassing ( dask#6211 ) Marius van Niekerk
Fix dd.merge_asof with left_on='col' & right_index=True ( dask#6192 ) noreentry
Disable warning for concat ( dask#6210 ) Tung Dang
Move AUTO_BLOCKSIZE out of read_csv signature ( dask#6214 ) Jim Crist-Harif
.loc indexing with callable ( dask#6185 ) Endre Mark Borza
Avoid apply in _compute_sum_of_squares for groupby std agg ( dask#6186 ) Richard (Rick) Zamora
Minor correction to test_parquet ( dask#6190 ) Brian Larsen
Adhering to the passed pat for delimeter join and fix error message ( dask#6194 ) GALI PREM SAGAR
Skip test_to_parquet_with_get if no parquet libs available ( dask#6188 ) Scott Sanderson
Added documentation for distributed.Event class ( dask#6231 ) Nils Braun
Doc write to remote ( dask#6124 ) Ray Bell
Record the time since the last run task on the scheduler (3830) `Matthew Rocklin`_
Set colour of nbytes pane based on thresholds (3805) `Krishan Bhasin`_
Include total number of tasks in the performance report (3822) `Abdulelah Bin Mahfoodh`_
Allow to pass in task key strings in the worker restrictions (3826) `Nils Braun`_
Control de/ser offload (3793) `Martin Durant`_
Parse timeout parameters in Variable/Event/Lock to support text timeouts (3825) `Nils Braun`_
Don't send empty dependencies (3423) `Jakub Beránek`_
Add distributed Dask Event that mimics threading.Event (3821) `Nils Braun`_
Enhance VersionMismatchWarning messages (3786) `Abdulelah Bin Mahfoodh`_
Support Pickle's protocol 5 (3784) `jakirkham`_
Replace utils.ignoring with contextlib.suppress (3819) `Nils Braun`_
Make re-creating conda environments from the CI output easier (3816) `Lucas Rademaker`_
Add prometheus metrics for semaphore (3757) `Lucas Rademaker`_
Fix worker plugin called with superseded transition (3812) `Nick Evans`_
Add retries to server listen (3801) `Jacob Tomlinson`_
Remove commented out lines from scheduler.py (3803) `James Bourbeau`_
Fix RuntimeWarning for never awaited coroutine when using distributed.Semaphore (3713) `Florian Jetter`_
Fix profile thread leakage during test teardown on some platforms (3795) `Florian Jetter`_
Await self before handling comms (3788) `Matthew Rocklin`_
Fix typo in Cluster docstring (3787) `Scott Sanderson`_
- Fix array general-reduction name ( dask#6176 ) Nick Evans
Fix array general-reduction name ( dask#6176 ) Nick Evans
Replace dim with shape in unravel_index ( dask#6155 ) Julia Signell
Moment: handle all elements being masked ( dask#5339 ) Gabe Joseph
Remove Redundant string concatenations in dask code-base ( dask#6137 ) GALI PREM SAGAR
Upstream compat ( dask#6159 ) Tom Augspurger
Ensure sizeof of dict and sequences returns an integer ( dask#6179 ) James Bourbeau
Estimate python collection sizes with random sampling ( dask#6154 ) Florian Jetter
Update test upstream ( dask#6146 ) Tom Augspurger
Skip test for mindeps build ( dask#6144 ) Tom Augspurger
Switch default multiprocessing context to “spawn” ( dask#4003 ) Itamar Turner-Trauring
Update manifest to include dask-schema ( dask#6140 ) Benjamin Zaitlen
Harden inconsistent-schema handling in pyarrow-based read_parquet ( dask#6160 ) Richard (Rick) Zamora
Add compute kwargs to methods that write data to disk ( dask#6056 ) Krishan Bhasin
Fix issue where unique returns an index like result from backends ( dask#6153 ) GALI PREM SAGAR
Fix internal error in map_partitions with collections ( dask#6103 ) Tom Augspurger
Add phase of computation to index TOC ( dask#6157 ) Benjamin Zaitlen
Remove unused imports in scheduling script ( dask#6138 ) James Lamb
Fix indent ( dask#6147 ) Martin Durant
Add Tom’s log config example ( dask#6143 ) Martin Durant
Client.get_dataset to always create Futures attached to itself (3729) `crusaderky`_
Remove dev-requirements since it is unused (3782) `Julia Signell`_
Use bokeh column for /system instead of custom css (3781) `Julia Signell`_
Attempt to fix test_preload_remote_module on windows (3775) `James Bourbeau`_
Fix broadcast for TLS comms (3766) `Florian Jetter`_
Don't validate http preloads locally (3768) `Rami Chowdhury`_
Allow range of ports to be specified for Workers (3704) `James Bourbeau`_
Add UCX support for RDMACM (3759) `Peter Andreas Entschev`_
Support web addresses in preload (3755) `Matthew Rocklin`_
Connect to dashboard when address provided (3758) Tom Augspurger_
Connect to dashboard when address provided (3758) `Tom Augspurger`_
Move test_gpu_metrics test (3721) `Tom Augspurger`_
Nanny closing worker on KeyboardInterrupt (3747) `Mads R. B. Kristensen`_
Replace OrderedDict with dict in scheduler (3740) `Matthew Rocklin`_
Fix exception handling typo (3751) `Jonas Haag`_
Ensure BokehTornado uses prefix (3746) James Bourbeau_
Ensure BokehTornado uses prefix (3746) `James Bourbeau`_
Warn if cluster closes before starting (3735) `Matthew Rocklin`_
Memoryview serialisation (3743) `Martin Durant`_
Allows logging config under distributed key (2952) `Dillon Niederhut`_
- Update dask.array.from_array to warn when passed a Dask collection ( dask#6122 ) James Bourbeau
Update dask.array.from_array to warn when passed a Dask collection ( dask#6122 ) James Bourbeau
Un-numpy like behaviour in dask.array.pad ( dask#6042 ) Mark Boer
Add support for repeats=0 in da.repeat ( dask#6080 ) James Bourbeau
Fix yaml layout for schema ( dask#6132 ) Benjamin Zaitlen
Configuration Reference ( dask#6069 ) Benjamin Zaitlen
Add configuration option to turn off task fusion ( dask#6087 ) Matthew Rocklin
Skip pyarrow on windows ( dask#6094 ) Tom Augspurger
Set limit to maximum length of fused key ( dask#6057 ) Lucas Rademaker
Add test against #6062 ( dask#6072 ) Martin Durant
Bump checkout action to v2 ( dask#6065 ) James Bourbeau
Generalize categorical calls to support cudf Categorical ( dask#6113 ) GALI PREM SAGAR
Avoid reading _metadata on every worker ( dask#6017 ) Richard (Rick) Zamora
Use group_split_dispatch and ignore_index in apply_concat_apply ( dask#6119 ) Richard (Rick) Zamora
Handle new (dtype) pandas metadata with pyarrow ( dask#6090 ) Richard (Rick) Zamora
Skip test_partition_on_cats_pyarrow if pyarrow is not installed ( dask#6112 ) James Bourbeau
Update DataFrame len to handle columns with the same name ( dask#6111 ) James Bourbeau
ArrowEngine bug fixes and test coverage ( dask#6047 ) Richard (Rick) Zamora
Added mode ( dask#5958 ) Adam Lewis
Update “helm install” for helm 3 usage ( dask#6130 ) JulianWgs
Extend preload documentation ( dask#6077 ) Matthew Rocklin
Fixed small typo in DataFrame map_partitions() docstring ( dask#6115 ) Eugene Huang
Fix typo: “double” should be times, not plus ( dask#6091 ) David Chudzicki
Fix first line of array.random.* docs ( dask#6063 ) Martin Durant
Add section about Semaphore in distributed ( dask#6053 ) Florian Jetter
Reinstate support for legacy @gen_cluster functions (3738) `crusaderky`_
Relax NumPy requirement in UCX (3731) `jakirkham`_
Add Configuration Schema (3696) `Matthew Rocklin`_
Reuse CI scripts for local installation process (3698) `crusaderky`_
Use PeriodicCallback class from tornado (3725) `James Bourbeau`_
Add remote_python option in ssh cmd (3709) `Abdulelah Bin Mahfoodh`_
Configurable polling interval for cluster widget (3723) `Julia Signell`_
Fix copy-paste in docs (3728) `Julia Signell`_
Replace gen.coroutine with async-await in tests (3706) `crusaderky`_
Fix flaky test_oversubscribing_leases (3726) `Florian Jetter`_
Add batch_size to Client.map (3650) `Tom Augspurger`_
Adjust semaphore test timeouts (3720) `Florian Jetter`_
Dask-serialize dicts longer than five elements (3689) `Richard J Zamora`_
Force threads_per_worker (3715) `crusaderky`_
Idempotent semaphore acquire with retries (3690) `Florian Jetter`_
Always use readinto in TCP (3711) `jakirkham`_
Avoid DeprecationWarning from pandas (3712) `Tom Augspurger`_
Allow modification of distributed.comm.retry at runtime (3705) `Florian Jetter`_
Do not log an error on unset variable delete (3652) `Jonathan J. Helmus`_
Add remote_python keyword to the new SSHCluster (3701) `Abdulelah Bin Mahfoodh`_
Replace Example with Examples in docstrings (3697) `Matthew Rocklin`_
Add Cluster __enter__ and __exit__ methods (3699) `Matthew Rocklin`_
Fix propagating inherit config in SSHCluster for non-bash shells (3688) `Abdulelah Bin Mahfoodh`_
Add Client.wait_to_workers to Client autosummary table (3692) `James Bourbeau`_
Replace Bokeh Server with Tornado HTTPServer (3658) `Matthew Rocklin`_
Fix dask-ssh after removing local-directory from dask_scheduler cli (3684) `Abdulelah Bin Mahfoodh`_
Support preload modules in Nanny (3678) `Matthew Rocklin`_
Refactor semaphore internals: make _get_lease synchronous (3679) `Lucas Rademaker`_
Don't make task graphs too big (3671) `Martin Durant`_
Pass through connection/listen_args as splatted keywords (3674) `Matthew Rocklin`_
Run preload at import, start, and teardown (3673) `Matthew Rocklin`_
Use relative URL in scheduler dashboard (3676) `Nicholas Smith`_
Expose Security object as public API (3675) `Matthew Rocklin`_
Add zoom tools to profile plots (3672) `James Bourbeau`_
Update Scheduler.rebalance return value when data is missing (3670) `James Bourbeau`_
- Added np.iscomplexobj implementation ( dask#6045 ) Tom Augspurger
Added np.iscomplexobj implementation ( dask#6045 ) Tom Augspurger
Update test_rearrange_disk_cleanup_with_exception to pass without cloudpickle installed ( dask#6052 ) James Bourbeau
Fixed flaky test-rearrange ( dask#5977 ) Tom Augspurger
Use _meta_nonempty for dtype casting in stack_partitions ( dask#6061 ) mlondschien
Fix bugs in _metadata creation and filtering in parquet ArrowEngine ( dask#6023 ) Richard (Rick) Zamora
DOC: Add name caveats ( dask#6040 ) Tom Augspurger
Enable more UCX tests (3667) `jakirkham`_
Remove openssl 1.1.1d pin for Travis (3668) `Jonathan J. Helmus`_
More documentation for Semaphore (3664) `Florian Jetter`_
Get CUDA context to finalize Numba DeviceNDArray (3666) `jakirkham`_
Add Resources option to get_task_stream and call output_file (3653) `Prasun Anand`_
Add Semaphore extension (3573) `Lucas Rademaker`_
Replace ncores with nthreads in work stealing tests (3615) `James Bourbeau`_
Clean up some test warnings (3662) `Matthew Rocklin`_
Write "why killed" docs (3596) `Martin Durant`_
Update Python version checking (3660) `James Bourbeau`_
Add newlines to ensure code formatting for retire_workers (3661) `Rami Chowdhury`_
Clean up performance report test (3655) `Matthew Rocklin`_
Avoid diagnostics time in performance report (3654) `Matthew Rocklin`_
Introduce config for default task duration (3642) `Gabriel Sailer`_
UCX simplify receiving frames in comm (3651) `jakirkham`_
Bump checkout GitHub action to v2 (3649) `James Bourbeau`_
Handle exception in faulthandler (3646) `Jacob Tomlinson`_
Add prometheus metric for suspicious tasks (3550) `Gabriel Sailer`_
Remove local-directory keyword (3620) `Prasun Anand`_
Don't create output Futures in Client when there are mixed Client Futures (3643) `James Bourbeau`_
Add link to contributing.md (3621) `Prasun Anand`_
Update bokeh dependency in CI builds (3637) `James Bourbeau`_
- Support dtype and other keyword arguments in da.random ( dask#6030 ) Matthew Rocklin
Support dtype and other keyword arguments in da.random ( dask#6030 ) Matthew Rocklin
Register support for cupy sparse hstack / vstack ( dask#5735 ) Corey J. Nolet
Force self.name to str in dask.array ( dask#6002 ) Chuanzhu Xu
Set rename_fused_keys to None by default in bag.optimize ( dask#6000 ) Lucas Rademaker
Copy dict in to_graphviz to prevent overwriting ( dask#5996 ) JulianWgs
Stricter pandas xfail ( dask#6024 ) Tom Augspurger
Fix CI failures ( dask#6013 ) James Bourbeau
Update toolz to 0.8.2 and use tlz ( dask#5997 ) Ryan Grout
Move Windows CI builds to GitHub Actions ( dask#5862 ) James Bourbeau
Improve path-related exceptions in read_hdf ( dask#6032 ) psimaj
Fix dtype handling in dd.concat ( dask#6006 ) mlondschien
Handle cudf’s leftsemi and leftanti joins ( dask#6025 ) Richard J Zamora
Remove unused npartitions variable in dd.from_pandas ( dask#6019 ) Daniel Saxton
Added shuffle to DataFrame.random_split ( dask#5980 ) petiop
Fix indentation in scheduler-overview docs ( dask#6022 ) Matthew Rocklin
Update task graphs in optimize docs ( dask#5928 ) Julia Signell
Optionally get rid of intermediary boxes in visualize, and add more labels ( dask#5976 ) Julia Signell
UCX synchronize default stream only on CUDA frames (3638) `Peter Andreas Entschev`_
Add as_completed.clear method (3617) `Matthew Rocklin`_
Drop unused line from pack_frames_prelude (3634) `John Kirkham`_
Add logging message when closing idle dask scheduler (3632) `Matthew Rocklin`_
Include frame lengths of CUDA objects in header["lengths"] (3631) `John Kirkham`_
Ensure Client connection pool semaphore attaches to the Client event loop (3546) `James Bourbeau`_
Remove dead stealing code (3619) `Florian Jetter`_
Check nbytes and types before reading data (3628) `John Kirkham`_
Ensure that we don't steal blacklisted fast tasks (3591) `Florian Jetter`_
Support async Listener.stop functions (3613) `Matthew Rocklin`_
Add str/repr methods to as_completed (3618) `Matthew Rocklin`_
Add backoff to comm connect attempts. (3496) `Matthias Urlichs`_
Make Listeners awaitable (3611) `Matthew Rocklin`_
Increase number of visible mantissas in dashboard plots (3585) `Scott Sievert`_
Pin openssl to 1.1.1d for Travis (3602) `Jacob Tomlinson`_
Replace tornado.queues with asyncio.queues (3607) `James Bourbeau`_
Remove dill from CI environments (3608) `Loïc Estève`_
Fix linting errors (3604) `James Bourbeau`_
Synchronize default CUDA stream before UCX send/recv (3598) `Peter Andreas Entschev`_
Add configuration for Adaptive arguments (3509) `Gabriel Sailer`_
Change Adaptive docs to reference adaptive_target (3597) `Julia Signell`_
Optionally compress on a frame-by-frame basis (3586) `Matthew Rocklin`_
Add Python version to version check (3567) `James Bourbeau`_
Import tlz (3579) `John Kirkham`_
Pin numpydoc to avoid double escaped * (3530) `Gil Forsyth`_
Avoid performance_report crashing when a worker dies mid-compute (3575) `Krishan Bhasin`_
Pin bokeh in CI builds (3570) `James Bourbeau`_
Disable fast fail on GitHub Actions Windows CI (3569) `James Bourbeau`_
Fix typo in Client.shutdown docstring (3562) `John Kirkham`_
Add local_directory option to dask-ssh (3554) `Abdulelah Bin Mahfoodh`_
- Improve reuse of temporaries with numpy ( dask#5933 ) Bruce Merry
Improve reuse of temporaries with numpy ( dask#5933 ) Bruce Merry
Make map_blocks with block_info produce a Blockwise ( dask#5896 ) Bruce Merry
Optimize make_blockwise_graph ( dask#5940 ) Bruce Merry
Fix axes ordering in da.tensordot ( dask#5975 ) Gil Forsyth
Adds empty mode to array.pad ( dask#5931 ) Thomas J. Fan
Remove toolz.memoize dependency in dask.utils ( dask#5978 ) Ryan Grout
Close pool leaking subprocess ( dask#5979 ) Tom Augspurger
Pin numpydoc to 0.8.0 (fix double autoescape) ( dask#5961 ) Gil Forsyth
Register deterministic tokenization for range objects ( dask#5947 ) James Bourbeau
Unpin msgpack in CI ( dask#5930 ) JAmes Bourbeau
Ensure dot results are placed in unique files. ( dask#5937 ) Elliott Sales de Andrade
Add remaining optional dependencies to Travis 3.8 CI build environment ( dask#5920 ) James Bourbeau
Skip parquet getitem optimization for some keys ( dask#5917 ) Tom Augspurger
Add ignore_index argument to rearrange_by_column code path ( dask#5973 ) Richard J Zamora
Add DataFrame and Series memory_usage_per_partition methods ( dask#5971 ) James Bourbeau
xfail test_describe when using Pandas 0.24.2 ( dask#5948 ) James Bourbeau
Implement dask.dataframe.to_numeric ( dask#5929 ) Julia Signell
Add new error message content when columns are in a different order ( dask#5927 ) Julia Signell
Use shallow copy for assign operations when possible ( dask#5740 ) Richard J Zamora
Changed above to below in dask.array.triu docs ( dask#5984 ) Henrik Andersson
Array slicing: fix typo in slice_with_int_dask_array error message ( dask#5981 ) Gabe Joseph
Grammar and formatting updates to docstrings ( dask#5963 ) James Lamb
Update develop doc with conda option ( dask#5939 ) Ray Bell
Update title of DataFrame extension docs ( dask#5954 ) James Bourbeau
Fixed typos in documentation ( dask#5962 ) James Lamb
Add original class or module as a kwarg on bind* methods ( dask#5946 ) Julia Signell
Add collect list example ( dask#5938 ) Ray Bell
Update optimization doc for python 3 ( dask#5926 ) Julia Signell
Update TaskGroup remove logic (3557) `James Bourbeau`_
Fix-up CuPy sparse serialization (3556) `John Kirkham`_
API docs for LocalCluster and SpecCluster (3548) `Tom Augspurger`_
Serialize sparse arrays (3545) `John Kirkham`_
Allow tasks with restrictions to be stolen (3069) `Stan Seibert`_
Use UCX default configuration instead of raising (3544) `Peter Andreas Entschev`_
Support using other serializers with register_generic (3536) `John Kirkham`_
DOC: update to async await (3543) `Tom Augspurger`_
Use pytest.raises in test_ucx_config.py (3541) `John Kirkham`_
Fix/more ucx config options (3539) `Benjamin Zaitlen`_
Update heartbeat CommClosedError error handling (3529) `James Bourbeau`_
Use makedirs when constructing local_directory (3538) `John Kirkham`_
Mark None as MessagePack serializable (3537) `John Kirkham`_
Mark bool as MessagePack serializable (3535) `John Kirkham`_
Use 'temporary-directory' from dask.config for Nanny's directory (3531) `John Kirkham`_
Add try-except around getting source code in performance report (3505) `Matthew Rocklin`_
Fix typo in docstring (3528) `Davis Bennett`_
Make work stealing callback time configurable (3523) `Lucas Rademaker`_
RMM/UCX Config Flags (3515) `Benjamin Zaitlen`_
Revise develop-docs: conda env example (3406) `Darren Weber`_
Remove import ucp from the top of ucx.py (3510) `Peter Andreas Entschev`_
Rename logs to get_logs (3473) `Jacob Tomlinson`_
Stop keep alives when worker reconnecting to the scheduler (3493) `Jacob Tomlinson`_
- Deprecate doc_wraps ( dask#5912 ) Tom Augspurger
Cache result of Array.shape ( dask#5916 ) Bruce Merry
Improve accuracy of estimate_graph_size for rechunk ( dask#5907 ) Bruce Merry
Skip rechunk steps that do not alter chunking ( dask#5909 ) Bruce Merry
Support dtype and other kwargs in coarsen ( dask#5903 ) Matthew Rocklin
Push chunk override from map_blocks into blockwise ( dask#5895 ) Bruce Merry
Avoid using rewrite_blockwise for a singleton ( dask#5890 ) Bruce Merry
Optimize slices_from_chunks ( dask#5891 ) Bruce Merry
Avoid unnecessary getitem in block() when chunks have correct dimensionality ( dask#5884 ) Thomas Robitaille
Add include_path option for dask.bag.read_text ( dask#5836 ) Yifan Gu
Fixes ValueError in delayed execution of bagged NumPy array ( dask#5828 ) Surya Avala
CI: Pin msgpack ( dask#5923 ) Tom Augspurger
Rename test_inner to test_outer ( dask#5922 ) Shiva Raisinghani
quote should quote dicts too ( dask#5905 ) Bruce Merry
Register a normalizer for literal ( dask#5898 ) Bruce Merry
Improve layer name synthesis for non-HLGs ( dask#5888 ) Bruce Merry
Replace flake8 pre-commit-hook with upstream ( dask#5892 ) Julia Signell
Call pip as a module to avoid warnings ( dask#5861 ) Cyril Shcherbin
Close ThreadPool at exit ( dask#5852 ) Tom Augspurger
Remove dask.dataframe import in tokenization code ( dask#5855 ) James Bourbeau
Require pandas>=0.23 ( dask#5883 ) Tom Augspurger
Remove lambda from dataframe aggregation ( dask#5901 ) Matthew Rocklin
Fix exception chaining in dataframe/init.py ( dask#5882 ) Ram Rachum
Add support for reductions on empty dataframes ( dask#5804 ) Shiva Raisinghani
Expose sort= argument for groupby ( dask#5801 ) Richard J Zamora
Add df.empty property ( dask#5711 ) rockwellw
Use parquet read speed-ups from fastparquet.api.paths_to_cats . ( dask#5821 ) Igor Gotlibovych
Deprecate doc_wraps ( dask#5912 ) Tom Augspurger
Update array internal design docs for HighLevelGraph era ( dask#5889 ) Bruce Merry
Move over dashboard connection docs ( dask#5877 ) Matthew Rocklin
Move prometheus docs from distributed.dask.org ( dask#5876 ) Matthew Rocklin
Removing duplicated DO block at the end ( dask#5878 ) K.-Michael Aye
map_blocks see also ( dask#5874 ) Tom Augspurger
More derived from ( dask#5871 ) Julia Signell
Fix typo ( dask#5866 ) Yetunde Dada
Fix typo in cloud.rst ( dask#5860 ) Andrew Thomas
Add note pointing to code of conduct and diversity statement ( dask#5844 ) Matthew Rocklin
Add dask serialization of CUDA objects (3482) `John Kirkham`_
Suppress cuML ImportError (3499) `John Kirkham`_
Msgpack 1.0 compatibility (3494) `James Bourbeau`_
Register cuML serializers (3485) `John Kirkham`_
Check exact equality for worker state (3483) `Brett Naul`_
Serialize 1-D, contiguous, uint8 CUDA frames (3475) `John Kirkham`_
Update NumPy array serialization to handle non-contiguous slices (3474) `James Bourbeau`_
Propose fix for collection based resources docs (3480) `Chris Roat`_
Remove --verbose flag from CI runs (3484) `Matthew Rocklin`_
Do not duplicate messages in scheduler report (3477) `Jakub Beránek`_
Register Dask cuDF serializers (3478) `John Kirkham`_
Add support for Python 3.8 (3249) `James Bourbeau`_
Add last seen column to worker table and highlight errant workers (3468) `kaelgreco`_
Change default value of local_directory from empty string to None (3441) `condoratberlin`_
Clear old docs (3458) `Matthew Rocklin`_
Change default multiprocessing behavior to spawn (3461) `Matthew Rocklin`_
Split dashboard host on additional slashes to handle inproc (3466) `Jacob Tomlinson`_
Update locality.rst (3470) `Dustin Tindall`_
Minor gen.Return cleanup (3469) `James Bourbeau`_
Update comparison logic for worker state (3321) `rockwellw`_
Update minimum tblib version to 1.6.0 (3451) `James Bourbeau`_
Add total row to workers plot in dashboard (3464) `Julia Signell`_
Workaround RecursionError on profile data (3455) `Tom Augspurger`_
Include code and summary in performance report (3462) `Matthew Rocklin`_
Skip test_open_close_many_workers on Python 3.6 (3459) `Matthew Rocklin`_
Support serializing/deserializing rmm.DeviceBuffer s (3442) `John Kirkham`_
Always add new TaskGroup to TaskPrefix (3322) `James Bourbeau`_
Rerun black on the code base (3444) `John Kirkham`_
Ensure __causes__ s of exceptions raised on workers are serialized (3430) `Alex Adamson`_
Adjust numba.cuda import and add check (3446) `John Kirkham`_
Fix name of Numba serialization test (3447) `John Kirkham`_
Checks for command parameters in ssh2 (3078) `Peter Andreas Entschev`_
Update worker_kwargs description in LocalCluster constructor (3438) `James Bourbeau`_
Ensure scheduler updates task and worker states after successful worker data deletion (3401) `James Bourbeau`_
Avoid loop= keyword in asyncio coordination primitives (3437) `Matthew Rocklin`_
Call pip as a module to avoid warnings (3436) `Cyril Shcherbin`_
Add documentation of parameters in coordination primitives (3434) `Søren Fuglede Jørgensen`_
Replace tornado.locks with asyncio for Events/Locks/Conditions/Semaphore (3397) `Matthew Rocklin`_
Remove object from class hierarchy (3432) `Anderson Banihirwe`_
Add dashboard_link property to Client (3429) `Jacob Tomlinson`_
Allow memory monitor to evict data more aggressively (3424) `fjetter`_
Make _get_ip return an IP address when defaulting (3418) `Pierre Glaser`_
Support version checking with older versions of Dask (3390) `Igor Gotlibovych`_
Add Mac OS build to CI (3358) `James Bourbeau`_
- Compatibility with pandas 1.0’s API breaking changes and deprecations ( dask#5792 ) Tom Augspurger
Support for pandas 1.0’s new BooleanDtype and StringDtype ( dask#5815 ) Tom Augspurger
Compatibility with pandas 1.0’s API breaking changes and deprecations ( dask#5792 ) Tom Augspurger
Fixed non-deterministic tokenization of some extension-array backed pandas objects ( dask#5813 ) Tom Augspurger
Fixed handling of dataclass class objects in collections ( dask#5812 ) Matteo De Wint
Fixed resampling with tz-aware dates when one of the endpoints fell in a non-existent time ( dask#5807 ) dfonnegra
Delay initial Zarr dataset creation until the computation occurs ( dask#5797 ) Chris Roat
Use parquet dataset statistics in more cases with the pyarrow engine ( dask#5799 ) Richard J Zamora
Fixed exception in groupby.std() when some of the keys were large integers ( dask#5737 ) H. Thomson Comer
Fixed ZeroDivisionError in dashboard when no workers were present (3407) `James Bourbeau`_
Respect the dashboard-prefix when redirecting from the root (3387) `Chrysostomos Nanakos`_
Allow enabling / disabling work-stealing after the cluster has started (3410) `John Kirkham`_
Support *args and **kwargs in offload (3392) `Matthew Rocklin`_
Add lifecycle hooks to SchedulerPlugin (3391) `Matthew Rocklin`_
Raise RuntimeError if no running loop (3385) James Bourbeau_
Raise RuntimeError if no running loop (3385) `James Bourbeau`_
Fix get_running_loop import (3383) `James Bourbeau`_
Get JavaScript document location instead of window and handle proxied url (3382) `Jacob Tomlinson`_
- Unify chunks in broadcast_arrays ( dask#5765 ) Matthew Rocklin
Unify chunks in broadcast_arrays ( dask#5765 ) Matthew Rocklin
xfail CSV encoding tests ( dask#5791 ) Tom Augspurger
Update order to handle empty dask graph ( dask#5789 ) James Bourbeau
Redo dask.order.order ( dask#5646 ) Erik Welch
Add transparent compression for on-disk shuffle with partd ( dask#5786 ) Christian Wesp
Fix repr for empty dataframes ( dask#5781 ) Shiva Raisinghani
Pandas 1.0.0RC0 compat ( dask#5784 ) Tom Augspurger
Remove buggy assertions ( dask#5783 ) Tom Augspurger
Pandas 1.0 compat ( dask#5782 ) Tom Augspurger
Fix bug in pyarrow-based read_parquet on partitioned datasets ( dask#5777 ) Richard J Zamora
Compat for pandas 1.0 ( dask#5779 ) Tom Augspurger
Fix groupby/mean error with with categorical index ( dask#5776 ) Richard J Zamora
Support empty partitions when performing cumulative aggregation ( dask#5730 ) Matthew Rocklin
set_index accepts single-item unnested list ( dask#5760 ) Wes Roach
Fixed partitioning in set index for ordered Categorical ( dask#5715 ) Tom Augspurger
Note additional use case for normalize_token.register ( dask#5766 ) Thomas A Caswell
Update bag repartition docstring ( dask#5772 ) Timost
Small typos ( dask#5771 ) Maarten Breddels
Fix typo in Task Expectations docs ( dask#5767 ) James Bourbeau
Add docs section on task expectations to graph page ( dask#5764 ) Devin Petersohn
Move Windows CI to GitHub Actions (3373) `Jacob Tomlinson`_
Add client join and leave hooks (3371) `Jacob Tomlinson`_
Add cluster map dashboard (3361) `Jacob Tomlinson`_
Close connection comm on retry (3365) `James Bourbeau`_
Fix scheduler state in case of worker name collision (3366) `byjott`_
Add --worker-class option to dask-worker CLI (3364) `James Bourbeau`_
Remove locale check that fails on OS X (3360) `Jacob Tomlinson`_
Rework version checking (2627) `Matthew Rocklin`_
Add websocket scheduler plugin (3335) `Jacob Tomlinson`_
Return task in dask-worker on_signal function (3354) `James Bourbeau`_
Fix failures on mixed integer/string worker names (3352) `Benedikt Reinartz`_
Avoid calling nbytes multiple times when sending data (3349) `Markus Mohrhard`_
Avoid setting event loop policy if within IPython kernel and no running event loop (3336) `Mana Borwornpadungkitti`_
Relax intermittent failing test_profile_server (3346) `Matthew Rocklin`_
- Support Array.view with dtype=None ( dask#5736 ) Anderson Banihirwe
Support Array.view with dtype=None ( dask#5736 ) Anderson Banihirwe
Add dask.array.nanmedian ( dask#5684 ) Deepak Cherian
xfail test_temporary_directory on Python 3.8 ( dask#5734 ) James Bourbeau
Add support for Python 3.8 ( dask#5603 ) James Bourbeau
Use id to dedupe constants in rewrite_blockwise ( dask#5696 ) Jim Crist
Raise error when converting a dask dataframe scalar to a boolean ( dask#5743 ) James Bourbeau
Ensure dataframe groupby-variance is greater than zero ( dask#5728 ) Matthew Rocklin
Fix DataFrame.iter ( dask#5719 ) Tom Augspurger
Support Parquet filters in disjunctive normal form, like PyArrow ( dask#5656 ) Matteo De Wint
Auto-detect categorical columns in ArrowEngine-based read_parquet ( dask#5690 ) Richard J Zamora
Skip parquet getitem optimization tests if no engine found ( dask#5697 ) James Bourbeau
Fix independent optimization of parquet-getitem ( dask#5613 ) Tom Augspurger
Update helm config doc ( dask#5750 ) Ray Bell
Link to examples.dask.org in several places ( dask#5733 ) Tom Augspurger
Add missing “ in performance report example ( dask#5724 ) James Bourbeau
Resolve several documentation build warnings ( dask#5685 ) James Bourbeau
add info on performance_report ( dask#5713 ) Benjamin Zaitlen
Add more docs disclaimers ( dask#5710 ) Julia Signell
Fix simple typo: wihout -> without ( dask#5708 ) Tim Gates
Update numpydoc dependency ( dask#5694 ) James Bourbeau
Add lock around dumps_function cache (3337) `Matthew Rocklin`_
Add setuptools to dependencies (3320) `James Bourbeau`_
Use TaskPrefix.name in Graph layout (3328) `Matthew Rocklin`_
Add missing " in performance report example (3329) `John Kirkham`_
Add performance report docs and color definitions to docs (3325) `Benjamin Zaitlen`_
Switch startstops to dicts and add worker name to transfer (3319) `Jacob Tomlinson`_
Add plugin entry point for out-of-tree comms library (3305) `Patrick Sodré`_
All scheduler task states in prometheus (3307) `fjetter`_
Use worker name in logs (3309) `Stephan Erb`_
Add TaskGroup and TaskPrefix scheduler state (3262) `Matthew Rocklin`_
Update latencies with heartbeats (3310) `fjetter`_
Update inlining Futures in task graph in Client._graph_to_futures (3303) `James Bourbeau`_
Use hostname as default IP address rather than localhost (3308) `Matthew Rocklin`_
Clean up flaky test_nanny_throttle (3295) `Tom Augspurger`_
Add lock to scheduler for sensitive operations (3259) `Matthew Rocklin`_
Log address for each of the Scheduler listerners (3306) `Matthew Rocklin`_
Make ConnectionPool.close asynchronous (3304) `Matthew Rocklin`_
- Fix da.std to work with NumPy arrays ( dask#5681 ) James Bourbeau
Fix da.std to work with NumPy arrays ( dask#5681 ) James Bourbeau
Register sizeof functions for Numba and RMM ( dask#5668 ) John A Kirkham
Update meeting time ( dask#5682 ) Tom Augspurger
Modify dd.DataFrame.drop to use shallow copy ( dask#5675 ) Richard J Zamora
Fix bug in _get_md_row_groups ( dask#5673 ) Richard J Zamora
Close sqlalchemy engine after querying DB ( dask#5629 ) Krishan Bhasin
Allow dd.map_partitions to not enforce meta ( dask#5660 ) Matthew Rocklin
Generalize concat_unindexed_dataframes to support cudf-backend ( dask#5659 ) Richard J Zamora
Add dataframe resample methods ( dask#5636 ) Benjamin Zaitlen
Compute length of dataframe as length of first column ( dask#5635 ) Matthew Rocklin
Doc fixup ( dask#5665 ) James Bourbeau
Update doc build instructions ( dask#5640 ) James Bourbeau
Fix ADL link ( dask#5639 ) Ray Bell
Add documentation build ( dask#5617 ) James Bourbeau
Add dask-spec CLI tool (3090) `Matthew Rocklin`_
Connectionpool: don't hand out closed connections (3301) `byjott`_
Retry operations on network issues (3294) `byjott`_
Skip Security.temporary() tests if cryptography not installed (3302) `James Bourbeau`_
Support multiple listeners in the scheduler (3288) `Matthew Rocklin`_
Updates RMM comment to the correct release (3299) `John Kirkham`_
Add title to performance_report (3298) `Matthew Rocklin`_
Forgot to fix slow test (3297) `Benjamin Zaitlen`_
Update SSHCluster docstring parameters (3296) `James Bourbeau`_
worker.close() awaits batched_stream.close() (3291) `Mads R. B. Kristensen`_
Fix asynchronous listener in UCX (3292) `Benjamin Zaitlen`_
Avoid repeatedly adding deps to already in memory stack (3293) `James Bourbeau`_
xfail ucx empty object typed dataframe (3279) `Benjamin Zaitlen`_
Fix distributed.wait documentation (3289) `Tom Rochette`_
Move Python 3 syntax tests into main tests (3281) `Matthew Rocklin`_
xfail test_workspace_concurrency for Python 3.6 (3283) `Matthew Rocklin`_
Add performance_report context manager for static report generation (3282) `Matthew Rocklin`_
Update function serialization caches with custom LRU class (3260) `James Bourbeau`_
Make Listener.start asynchronous (3278) `Matthew Rocklin`_
Remove dask-submit and dask-remote (3280) `Matthew Rocklin`_
Worker profile server (3274) `Matthew Rocklin`_
Improve bandwidth workers plot (3273) `Matthew Rocklin`_
Make profile coroutines consistent between Scheduler and Worker (3277) `Matthew Rocklin`_
Enable saving profile information from server threads (3271) `Matthew Rocklin`_
Remove memory use plot (3269) `Matthew Rocklin`_
Add offload size to configuration (3270) `Matthew Rocklin`_
Fix layout scaling on profile plots (3268) `Jacob Tomlinson`_
Set x_range in CPU plot based on the number of threads (3266) `Matthew Rocklin`_
Use base-2 values for byte-valued axes in dashboard (3267) `Matthew Rocklin`_
Robust gather in case of connection failures (3246) `fjetter`_
Use DeviceBuffer from newer RMM releases (3261) `John Kirkham`_
Fix dev requirements for pytest (3264) `Elliott Sales de Andrade`_
Add validate options to configuration (3258) `Matthew Rocklin`_
- Use auto rechunking in da.rechunk if no value given ( dask#5605 ) Matthew Rocklin
Use auto rechunking in da.rechunk if no value given ( dask#5605 ) Matthew Rocklin
Add simple action to activate GH actions ( dask#5619 ) James Bourbeau
Fix “file_path_0” bug in aggregate_row_groups ( dask#5627 ) Richard J Zamora
Add chunksize argument to read_parquet ( dask#5607 ) Richard J Zamora
Change test_repartition_npartitions to support arch64 architecture ( dask#5620 ) ossdev07
Categories lost after groupby + agg ( dask#5423 ) Oliver Hofkens
Fixed relative path issue with parquet metadata file ( dask#5608 ) Nuno Gomes Silva
Enable gpu-backed covariance/correlation in dataframes ( dask#5597 ) Richard J Zamora
Fix institutional faq and unknown doc warnings ( dask#5616 ) James Bourbeau
Add doc for some utils ( dask#5609 ) Tom Augspurger
Removes html_extra_path ( dask#5614 ) James Bourbeau
Fixed See Also referencence ( dask#5612 ) Tom Augspurger
Fix hanging worker when the scheduler leaves (3250) `Tom Augspurger`_
Fix NumPy writeable serialization bug (3253) `James Bourbeau`_
Skip numba.cuda tests if CUDA is not available (3255) `Peter Andreas Entschev`_
Add new dashboard plot for memory use by key (3243) `Matthew Rocklin`_
Fix array.shape() -> array.shape (3247) `Jed Brown`_
Fixed typos in pubsub.py (3244) `He Jia`_
Fixed cupy array going out of scope (3240) `Mads R. B. Kristensen`_
Remove gen.coroutine usage in scheduler (3242) `Jim Crist-Harif`_
Use inspect.isawaitable where relevant (3241) `Jim Crist-Harif`_
- Implement complete dask.array.tile function ( dask#5574 ) Bouwe Andela
Implement complete dask.array.tile function ( dask#5574 ) Bouwe Andela
Add median along an axis with automatic rechunking ( dask#5575 ) Matthew Rocklin
Allow da.asarray to chunk inputs ( dask#5586 ) Matthew Rocklin
Use key_split in Bag name ( dask#5571 ) Matthew Rocklin
Switch Doctests to Py3.7 ( dask#5573 ) Ryan Nazareth
Relax get_colors test to adapt to new Bokeh release ( dask#5576 ) Matthew Rocklin
Add dask.blockwise.fuse_roots optimization ( dask#5451 ) Matthew Rocklin
Add sizeof implementation for small dicts ( dask#5578 ) Matthew Rocklin
Update fsspec, gcsfs, s3fs ( dask#5588 ) Tom Augspurger
Add dropna argument to groupby ( dask#5579 ) Richard J Zamora
Revert “Remove import of dask_cudf, which is now a part of cudf ( dask#5568 )” ( dask#5590 ) Matthew Rocklin
Add best practice for dask.compute function ( dask#5583 ) Matthew Rocklin
Create FUNDING.yml ( dask#5587 ) Gina Helfrich
Add screencast for coordination primitives ( dask#5593 ) Matthew Rocklin
Move funding to .github repo ( dask#5589 ) Tom Augspurger
Update calendar link ( dask#5569 ) Tom Augspurger
Add UCX config values (3135) `Matthew Rocklin`_
Relax test_MultiWorker (3210) `Matthew Rocklin`_
Avoid ucp.init at import time (3211) `Matthew Rocklin`_
Clean up rpc to avoid intermittent test failure (3215) `Matthew Rocklin`_
Respect protocol if given to Scheduler (3212) `Matthew Rocklin`_
Use legend_field= keyword in bokeh plots (3218) `Matthew Rocklin`_
Cache psutil.Process object in Nanny (3207) `Matthew Rocklin`_
Replace gen.sleep with asyncio.sleep (3208) `Matthew Rocklin`_
Avoid offloading serialization for small messages (3224) `Matthew Rocklin`_
Add desired_workers metric (3221) `Gabriel Sailer`_
Fail fast when importing distributed.comm.ucx (3228) `Matthew Rocklin`_
Add module name to Future repr (3231) `Matthew Rocklin`_
Add name to Pub/Sub repr (3235) `Matthew Rocklin`_
Import CPU_COUNT from dask.system (3199) `James Bourbeau`_
Efficiently serialize zero strided NumPy arrays (3180) `James Bourbeau`_
Cache function deserialization in workers (3234) `Matthew Rocklin`_
Respect ordering of futures in futures_of (3236) `Matthew Rocklin`_
Bump dask dependency to 2.7.0 (3237) `James Bourbeau`_
Avoid setting inf x_range (3229) `rockwellw`_
Clear task stream based on recent behavior (3200) `Matthew Rocklin`_
Use the percentage field for profile plots (3238) `Matthew Rocklin`_
- Changed deprecated bokeh-port to dashboard-address ( dask#5507 ) darindf
This release drops support for Python 3.5
Reuse code for assert_eq util method ( dask#5496 ) Vijayant
Update da.array to always return a dask array ( dask#5510 ) James Bourbeau
Skip transpose on trivial inputs ( dask#5523 ) Ryan Abernathey
Avoid NumPy scalar string representation in tokenize ( dask#5527 ) James Bourbeau
Remove unnecessary tiledb shape constraint ( dask#5545 ) Norman Barker
Removes bytes from sparse array HTML repr ( dask#5556 ) James Bourbeau
Drop Python 3.5 ( dask#5528 ) James Bourbeau
Update the use of fixtures in distributed tests ( dask#5497 ) Matthew Rocklin
Changed deprecated bokeh-port to dashboard-address ( dask#5507 ) darindf
Avoid updating with identical dicts in ensure_dict ( dask#5501 ) James Bourbeau
Test Upstream ( dask#5516 ) Tom Augspurger
Accelerate reverse_dict ( dask#5479 ) Ryan Grout
Update test_imports.sh ( dask#5534 ) James Bourbeau
Support cgroups limits on cpu count in multiprocess and threaded schedulers ( dask#5499 ) Albert DeFusco
Update minimum pyarrow version on CI ( dask#5562 ) James Bourbeau
Make cloudpickle optional ( dask#5511 ) crusaderky
Add an example of index_col usage ( dask#3072 ) Bruno Bonfils
Explicitly use iloc for row indexing ( dask#5500 ) Krishan Bhasin
Accept dask arrays on columns assignemnt ( dask#5224 ) Henrique Ribeiro -
Implement unique and value_counts for SeriesGroupBy ( dask#5358 ) Scott Sievert
Add sizeof definition for pyarrow tables and columns ( dask#5522 ) Richard J Zamora
Enable row-group task partitioning in pyarrow-based read_parquet ( dask#5508 ) Richard J Zamora
Removes npartitions=’auto’ from dd.merge docstring ( dask#5531 ) James Bourbeau
Apply enforce error message shows non-overlapping columns. ( dask#5530 ) Tom Augspurger
Optimize meta_nonempty for repetitive dtypes ( dask#5553 ) Petio Petrov
Remove import of dask_cudf, which is now a part of cudf ( dask#5568 ) Mads R. B. Kristensen
Make capitalization more consistent in FAQ docs ( dask#5512 ) Matthew Rocklin
Add CONTRIBUTING.md ( dask#5513 ) Jacob Tomlinson
Document optional dependencies ( dask#5456 ) Prithvi MK
Update helm chart docs to reflect new chart repo ( dask#5539 ) Jacob Tomlinson
Add Resampler to API docs ( dask#5551 ) James Bourbeau
Fix typo in read_sql_table ( dask#5554 ) Eric Dill
Add adaptive deployments screencast [skip ci] ( dask#5566 ) Matthew Rocklin
This release drops support for Python 3.5
Adds badges to README.rst [skip ci] (3152) `James Bourbeau`_
Don't overwrite self.address if it is present (3153) `Gil Forsyth`_
Remove outdated references to debug scheduler and worker bokeh pages. (3160) `darindf`_
Update CONTRIBUTING.md (3159) `Jacob Tomlinson`_
Add Prometheus metric for a worker's executing tasks count (3163) `darindf`_
Update Prometheus documentation (3165) `darindf`_
Fix Numba serialization when strides is None (3166) `Peter Andreas Entschev`_
Await cluster in Adaptive.recommendations (3168) `Simon Boothroyd`_
Support automatic TLS (3164) `Jim Crist`_
Avoid swamping high-memory workers with data requests (3071) `Tom Augspurger`_
Update UCX variables to use sockcm by default (3177) `Peter Andreas Entschev`_
Get protocol in Nanny/Worker from scheduler address (3175) `Peter Andreas Entschev`_
Add worker and tasks state for Prometheus data collection (3174) `darindf`_
Use async def functions for offload to/from_frames (3171) `Mads R. B. Kristensen`_
Subprocesses inherit the global dask config (3192) `Mads R. B. Kristensen`_
XFail test_open_close_many_workers (3194) `Matthew Rocklin`_
Drop Python 3.5 (3179) `James Bourbeau`_
UCX: avoid double init after fork (3178) `Mads R. B. Kristensen`_
Silence warning when importing while offline (3203) `James A. Bednar`_
Adds docs to Client methods for resources, actors, and traverse (2851) `IPetrik`_
Add test for concurrent scatter operations (2244) `Matthew Rocklin`_
Expand async docs (2293) `Dave Hirschfeld`_
Add PatchedDeviceArray to drop stride attribute for cupy<7.0 (3198) `Richard J Zamora`_
- Call ensure_dict on graphs before entering toolz.merge ( dask#5486 ) Matthew Rocklin
Call ensure_dict on graphs before entering toolz.merge ( dask#5486 ) Matthew Rocklin
Consolidating hash dispatch functions ( dask#5476 ) Richard J Zamora
Support Python 3.5 in Parquet code ( dask#5491 ) Benjamin Zaitlen
Avoid identity check in warn_dtype_mismatch ( dask#5489 ) Tom Augspurger
Enable unused groupby tests ( dask#3480 ) Jörg Dietrich
Remove old parquet and bcolz dataframe optimizations ( dask#5484 ) Matthew Rocklin
Add getitem optimization for read_parquet ( dask#5453 ) Tom Augspurger
Use _constructor_sliced method to determine Series type ( dask#5480 ) Richard J Zamora
Fix map(series) for unsorted base series index ( dask#5459 ) Justin Waugh
Fix KeyError with Groupby label ( dask#5467 ) Ryan Nazareth
Use Zoom meeting instead of appear.in ( dask#5494 ) Matthew Rocklin
Added curated list of resources ( dask#5460 ) Javad
Update SSH docs to include SSHCluster ( dask#5482 ) Matthew Rocklin
Update “Why Dask?” page ( dask#5473 ) Matthew Rocklin
Fix typos in docstrings ( dask#5469 ) garanews
Refactor dashboard module (3138) `Jacob Tomlinson`_
Use setuptools.find_packages in setup.py (3150) `Matthew Rocklin`_
Move death timeout logic up to Node.start (3115) `Matthew Rocklin`_
Only include metric in WorkerTable if it is a scalar (3140) `Matthew Rocklin`_
Add Nanny(config={...}) keyword (3134) `Matthew Rocklin`_
Xfail test_worksapce_concurrency on Python 3.6 (3132) `Matthew Rocklin`_
Extend Worker plugin API with transition method (2994) `matthieubulte`_
Raise exception if the user passes in unused keywords to Client (3117) `Jonathan De Troye`_
Move new SSHCluster to top level (3128) `Matthew Rocklin`_
Bump dask dependency (3124) `Jim Crist`_
- Correct chunk size logic for asymmetric overlaps ( dask#5449 ) Ben Jeffery
Correct chunk size logic for asymmetric overlaps ( dask#5449 ) Ben Jeffery
Make da.unify_chunks public API ( dask#5443 ) Matthew Rocklin
Fix dask.dataframe.fillna handling of Scalar object ( dask#5463 ) Zhenqing Li
Remove boxes in Spark comparison page ( dask#5445 ) Matthew Rocklin
Add latest presentations ( dask#5446 ) Javad
Update cloud documentation ( dask#5444 ) Matthew Rocklin
Make dask-worker close quietly when given sigint signal (3116) `Matthew Rocklin`_
Replace use of tornado.gen with asyncio in dask-worker (3114) `Matthew Rocklin`_
UCX: allocate CUDA arrays using RMM and Numba (3109) `Mads R. B. Kristensen`_
Support calling cluster.scale as async method (3110) `Jim Crist`_
Identify lost workers in SpecCluster based on address not name (3088) `James Bourbeau`_
Add Client.shutdown method (3106) `Matthew Rocklin`_
Collect worker-worker and type bandwidth information (3094) `Matthew Rocklin`_
Send noise over the wire to keep dask-ssh connection alive (3105) `Gil Forsyth`_
Retry scheduler connect multiple times (3104) `Jacob Tomlinson`_
Add favicon of logo to the dashboard (3095) `James Bourbeau`_
Remove utils.py functions for their dask/utils.py equivalents (3042) `Matthew Rocklin`_
Lower default bokeh log level (3087) `Philipp Rudiger`_
Check if self.cluster.scheduler is a local scheduler (3099) `Jacob Tomlinson`_
Support clusters that don't have .security or ._close methods (3100) Matthew Rocklin_
Support clusters that don't have .security or ._close methods (3100) `Matthew Rocklin`_
- Add sentinel no_default to get_dependencies task ( dask#5420 ) James Bourbeau
Add sentinel no_default to get_dependencies task ( dask#5420 ) James Bourbeau
Update fsspec version ( dask#5415 ) Matthew Rocklin
Remove PY2 checks ( dask#5400 ) Jim Crist
Add option to not check meta in dd.from_delayed ( dask#5436 ) Christopher J. Wright
Fix test_timeseries_nulls_in_schema failures with pyarrow master ( dask#5421 ) Richard J Zamora
Reduce read_metadata output size in pyarrow/parquet ( dask#5391 ) Richard J Zamora
Test numeric edge case for repartition with npartitions. ( dask#5433 ) amerkel2
Unxfail pandas-datareader test ( dask#5430 ) Tom Augspurger
Add DataFrame.pop implementation ( dask#5422 ) Matthew Rocklin
Enable merge/set_index for cudf-based dataframes with cupy values ( dask#5322 ) Richard J Zamora
drop_duplicates support for positional subset parameter ( dask#5410 ) Wes Roach
Add screencasts to array, bag, dataframe, delayed, futures and setup ( dask#5429 ) ( dask#5424 ) Matthew Rocklin
Fix delimeter parsing documentation ( dask#5428 ) Mahmut Bulut
Update overview image ( dask#5404 ) James Bourbeau
Use the new UCX Python bindings (3059) `Mads R. B. Kristensen`_
Fix worker preload config (3027) `byjott`_
Fix widget with spec that generates multiple workers (3067) `Loïc Estève`_
Make Client.get_versions async friendly (3064) `Jacob Tomlinson`_
Add configuration option for longer error tracebacks (3086) `Daniel Farrell`_
Have Client get Security from passed Cluster (3079) `Matthew Rocklin`_
Respect Cluster.dashboard_link in Client._repr_html_ if it exists (3077) `Matthew Rocklin`_
Add monitoring with dask cluster docs (3072) `Arpit Solanki`_
Protocol of cupy and numba handles serialization exclusively (3047) `Mads R. B. Kristensen`_
Allow specification of worker type in SSHCLuster (3061) `Jacob Tomlinson`_
Use Cluster.scheduler_info for workers= value in repr (3058) `Matthew Rocklin`_
Allow SpecCluster to scale by memory and cores (3057) `Matthew Rocklin`_
Allow full script in preload inputs (3052) `Matthew Rocklin`_
Check multiple cgroups dirs, ceil fractional cpus (3056) `Jim Crist`_
Add blurb about disabling work stealing (3055) `Chris White`_
- Remove deprecated code ( dask#5401 ) Jim Crist
Adds explicit h5py.File mode ( dask#5390 ) James Bourbeau
Provides method to compute unknown array chunks sizes ( dask#5312 ) Scott Sievert
Ignore runtime warning in Array compute_meta ( dask#5356 ) estebanag
Add _meta to Array.dask_postpersist ( dask#5353 ) Benoit Bovy
Fixup da.asarray and da.asanyarray for datetime64 dtype and xarray objects ( dask#5334 ) Stephan Hoyer
Add shape implementation ( dask#5293 ) Tom Augspurger
Add chunktype to array text repr ( dask#5289 ) James Bourbeau
Array.random.choice: handle array-like non-arrays ( dask#5283 ) Gabe Joseph
Remove deprecated code ( dask#5401 ) Jim Crist
Fix funcname when vectorized func has no name ( dask#5399 ) James Bourbeau
Truncate funcname to avoid long key names ( dask#5383 ) Matthew Rocklin
Add support for numpy.vectorize in funcname ( dask#5396 ) James Bourbeau
Fixed HDFS upstream test ( dask#5395 ) Tom Augspurger
Support numbers and None in parse_bytes / timedelta ( dask#5384 ) Matthew Rocklin
Fix tokenizing of subindexes on memmapped numpy arrays ( dask#5351 ) Henry Pinkard
Upstream fixups ( dask#5300 ) Tom Augspurger
Allow pandas to cast type of statistics ( dask#5402 ) Richard J Zamora
Preserve index dtype after applying dd.pivot_table ( dask#5385 ) therhaag
Implement explode for Series and DataFrame ( dask#5381 ) Arpit Solanki
set_index on categorical fails with less categories than partitions ( dask#5354 ) Oliver Hofkens
Support output to a single CSV file ( dask#5304 ) Hongjiu Zhang
Add groupby().transform() ( dask#5327 ) Oliver Hofkens
Adding filter kwarg to pyarrow dataset call ( dask#5348 ) Richard J Zamora
Implement and check compression defaults for parquet ( dask#5335 ) Sarah Bird
Pass sqlalchemy params to delayed objects ( dask#5332 ) Arpit Solanki
Fixing schema handling in arrow-parquet ( dask#5307 ) Richard J Zamora
Add support for DF and Series groupby().idxmin/max() ( dask#5273 ) Oliver Hofkens
Add correlation calculation and add test ( dask#5296 ) Benjamin Zaitlen
Numpy docstring standard has moved ( dask#5405 ) Wes Roach
Reference correct NumPy array name ( dask#5403 ) Wes Roach
Minor edits to Array chunk documentation ( dask#5372 ) Scott Sievert
Add methods to API docs ( dask#5387 ) Tom Augspurger
Add namespacing to configuration example ( dask#5374 ) Matthew Rocklin
Add get_task_stream and profile to the diagnostics page ( dask#5375 ) Matthew Rocklin
Add best practice to load data with Dask ( dask#5369 ) Matthew Rocklin
Update institutional-faq.rst ( dask#5345 ) DomHudson
Add threads and processes note to the best practices ( dask#5340 ) Matthew Rocklin
Update cuDF links ( dask#5328 ) James Bourbeau
Fixed small typo with parentheses placement ( dask#5311 ) Eugene Huang
Update link in reshape docstring ( dask#5297 ) James Bourbeau
Remove six (3045) `Matthew Rocklin`_
Add missing test data to sdist tarball (3050) `Elliott Sales de Andrade`_
Use mock from unittest standard library (3049) `Elliott Sales de Andrade`_
Use cgroups resource limits to determine default threads and memory (3039) `Jim Crist`_
Move task deserialization to immediately before task execution (3015) `James Bourbeau`_
Drop joblib shim module in distributed (3040) `John Kirkham`_
Redirect configuration doc page (3038) `Matthew Rocklin`_
Support --name 0 and --nprocs keywords in dask-worker cli (3037) `Matthew Rocklin`_
Remove lost workers from SpecCluster.workers (2990) `Guillaume Eynard-Bontemps`_
Clean up test_local.py::test_defaults (3017) `Matthew Rocklin`_
Replace print statement in Queue.__init__ with debug message (3035) `Mikhail Akimov`_
Set the x_range limit of the Meory utilization plot to memory-limit (3034) `Matthew Rocklin`_
Rely on cudf codebase for cudf serialization (2998) `Benjamin Zaitlen`_
Add fallback html repr for Cluster (3023) `Jim Crist`_
Add support for zstandard compression to comms (2970) `Abael He`_
Avoid collision when using os.environ in dashboard_link (3021) `Matthew Rocklin`_
Fix ConnectionPool limit handling (3005) `byjott`_
Support Spec jobs that generate multiple workers (3013) `Matthew Rocklin`_
Tweak Logs styling (3012) `Jim Crist`_
Better name for cudf deserialization function name (3008) `Benjamin Zaitlen`_
Make spec.ProcessInterface a valid no-op worker (3004) `Matthew Rocklin`_
Return dictionaries from new_worker_spec rather than name/worker pairs (3000) `Matthew Rocklin`_
Fix minor typo in documentation (3002) `Mohammad Noor`_
Permit more keyword options when scaling with cores and memory (2997) `Matthew Rocklin`_
Add cuda_ipc to UCX environment for NVLink (2996) `Benjamin Zaitlen`_
Add threads= and memory= to Cluster and Client reprs (2995) `Matthew Rocklin`_
Fix PyNVML initialization (2993) `Richard J Zamora`_
Skip exceptions in startup information (2991) Jacob Tomlinson_
Skip exceptions in startup information (2991) `Jacob Tomlinson`_
Add support for separate external address for SpecCluster scheduler (2963) Jacob Tomlinson_
Add support for separate external address for SpecCluster scheduler (2963) `Jacob Tomlinson`_
Defer cudf serialization/deserialization to that library (2881) `Benjamin Zaitlen`_
Workaround for hanging test now calls ucp.fin() (2967) `Mads R. B. Kristensen`_
Remove unnecessary bullet point (2972) `Pav A`_
Directly import progress from diagnostics.progressbar (2975) `Matthew Rocklin`_
Handle buffer protocol objects in ensure_bytes (2969) `Tom Augspurger`_
Fix documentatation syntax and tree (2981) `Pav A`_
Improve get_ip_interface error message when interface does not exist (2964) `Loïc Estève`_
Add cores= and memory= keywords to scale (2974) `Matthew Rocklin`_
Make workers robust to bad custom metrics (2984) `Matthew Rocklin`_
- Raise exception when from_array is given a dask array ( dask#5280 ) David Hoese
Raise exception when from_array is given a dask array ( dask#5280 ) David Hoese
Avoid adjusting gufunc’s meta dtype twice ( dask#5274 ) Peter Andreas Entschev
Add meta= keyword to map_blocks and add test with sparse ( dask#5269 ) Matthew Rocklin
Add rollaxis and moveaxis ( dask#4822 ) Tobias de Jong
Always increment old chunk index ( dask#5256 ) James Bourbeau
Shuffle dask array ( dask#3901 ) Tom Augspurger
Fix ordering when indexing a dask array with a bool dask array ( dask#5151 ) James Bourbeau
Add workaround for memory leaks in bag generators ( dask#5208 ) Marco Neumann
Set strict xfail option ( dask#5220 ) James Bourbeau
test-upstream ( dask#5267 ) Tom Augspurger
Fixed HDFS CI failure ( dask#5234 ) Tom Augspurger
Error nicely if no file size inferred ( dask#5231 ) Jim Crist
A few changes to config.set ( dask#5226 ) Jim Crist
Fixup black string normalization ( dask#5227 ) Jim Crist
Pin NumPy in windows tests ( dask#5228 ) Jim Crist
Ensure parquet tests are skipped if fastparquet and pyarrow not installed ( dask#5217 ) James Bourbeau
Add fsspec to readthedocs ( dask#5207 ) Matthew Rocklin
Bump NumPy and Pandas to 1.17 and 0.25 in CI test ( dask#5179 ) John A Kirkham
Fix DataFrame.query docstring (incorrect numexpr API) ( dask#5271 ) Doug Davis
Parquet metadata-handling improvements ( dask#5218 ) Richard J Zamora
Improve messaging around sorted parquet columns for index ( dask#5265 ) Martin Durant
Add rearrange_by_divisions and set_index support for cudf ( dask#5205 ) Richard J Zamora
Fix groupby.std() with integer colum names ( dask#5096 ) Nicolas Hug
Add Series.iter ( dask#5071 ) Blane
Generalize hash_pandas_object to work for non-pandas backends ( dask#5184 ) GALI PREM SAGAR
Add rolling cov ( dask#5154 ) Ivars Geidans
Add columns argument in drop function ( dask#5223 ) Henrique Ribeiro
Update institutional FAQ doc ( dask#5277 ) Matthew Rocklin
Add draft of institutional FAQ ( dask#5214 ) Matthew Rocklin
Make boxes for dask-spark page ( dask#5249 ) Martin Durant
Add motivation for shuffle docs ( dask#5213 ) Matthew Rocklin
Fix links and API entries for best-practices ( dask#5246 ) Martin Durant
Remove “bytes” (internal data ingestion) doc page ( dask#5242 ) Martin Durant
Redirect from our local distributed page to distributed.dask.org ( dask#5248 ) Matthew Rocklin
Cleanup API page ( dask#5247 ) Matthew Rocklin
Remove excess endlines from install docs ( dask#5243 ) Matthew Rocklin
Remove item list in phases of computation doc ( dask#5245 ) Martin Durant
Remove custom graphs from the TOC sidebar ( dask#5241 ) Matthew Rocklin
Remove experimental status of custom collections ( dask#5236 ) James Bourbeau
Adds table of contents to Why Dask? ( dask#5244 ) James Bourbeau
Moves bag overview to top-level bag page ( dask#5240 ) James Bourbeau
Remove use-cases in favor of stories.dask.org ( dask#5238 ) Matthew Rocklin
Removes redundant TOC information in index.rst ( dask#5235 ) James Bourbeau
Elevate dashboard in distributed diagnostics documentation ( dask#5239 ) Martin Durant
Updates “add” layer in HLG docs example ( dask#5237 ) James Bourbeau
Update GUFunc documentation ( dask#5232 ) Matthew Rocklin
Except all exceptions when checking pynvml (2961) `Matthew Rocklin`_
Pass serialization down through small base collections (2948) `Peter Andreas Entschev`_
Use pytest.warning(Warning) rather than Exception (2958) `Matthew Rocklin`_
Allow server_kwargs to override defaults in dashboard (2955) `Bruce Merry`_
Update utils_perf.py (2954) `Shayan Amani`_
Normalize names with str in retire_workers (2949) `Matthew Rocklin`_
Update client.py (2951) `Shayan Amani`_
Add GPUCurrentLoad dashboard plots (2944) `Matthew Rocklin`_
Pass GPU diagnostics from worker to scheduler (2932) `Matthew Rocklin`_
Import from collections.abc (2938) `Jim Crist`_
Fixes Worker docstring formatting (2939) `James Bourbeau`_
Redirect setup docs to docs.dask.org (2936) `Matthew Rocklin`_
Wrap offload in gen.coroutine (2934) `Matthew Rocklin`_
Change TCP.close to a coroutine to avoid task pending warning (2930) `Matthew Rocklin`_
Fixup black string normalization (2929) `Jim Crist`_
Move core functionality from SpecCluster to Cluster (2913) `Matthew Rocklin`_
Add aenter/aexit protocols to ProcessInterface (2927) `Matthew Rocklin`_
Add real-time CPU utilization plot to dashboard (2922) `Matthew Rocklin`_
Always kill processes in clean tests, even if we don't check (2924) `Matthew Rocklin`_
Add timeouts to processes in SSH tests (2925) `Matthew Rocklin`_
Add documentation around spec.ProcessInterface (2923) `Matthew Rocklin`_
Cleanup async warnings in tests (2920) `Matthew Rocklin`_
Give 404 when requesting nonexistent tasks or workers (2921) `Martin Durant`_
Raise informative warning when rescheduling an unknown task (2916) `James Bourbeau`_
Fix docstring (2917) `Martin Durant`_
Add keep-alive message between worker and scheduler (2907) `Matthew Rocklin`_
Rewrite Adaptive/SpecCluster to support slowly arriving workers (2904) `Matthew Rocklin`_
Call heartbeat rather than reconnect on disconnection (2906) `Matthew Rocklin`_
- Use da.from_array(…, asarray=False) if input follows NEP-18 ( dask#5074 ) Matthew Rocklin
Use da.from_array(…, asarray=False) if input follows NEP-18 ( dask#5074 ) Matthew Rocklin
Add missing attributes to from_array documentation ( dask#5108 ) Peter Andreas Entschev
Fix meta computation for some reduction functions ( dask#5035 ) Peter Andreas Entschev
Raise informative error in to_zarr if unknown chunks ( dask#5148 ) James Bourbeau
Remove invalid pad tests ( dask#5122 ) Tom Augspurger
Ignore NumPy warnings in compute_meta ( dask#5103 ) Peter Andreas Entschev
Fix kurtosis calc for single dimension input array ( dask#5177 ) @andrethrill
Support Numpy 1.17 in tests ( dask#5192 ) Matthew Rocklin
Supply pool to bag test to resolve intermittent failure ( dask#5172 ) Tom Augspurger
Base dask on fsspec ( dask#5064 ) ( dask#5121 ) Martin Durant
Various upstream compatibility fixes ( dask#5056 ) Tom Augspurger
Make distributed tests optional again. ( dask#5128 ) Elliott Sales de Andrade
Fix HDFS in dask ( dask#5130 ) Martin Durant
Ignore some more invalid value warnings. ( dask#5140 ) Elliott Sales de Andrade
Fix pd.MultiIndex size estimate ( dask#5066 ) Brett Naul
Generalizing has_known_categories ( dask#5090 ) GALI PREM SAGAR
Refactor Parquet engine ( dask#4995 ) Richard J Zamora
Add divide method to series and dataframe ( dask#5094 ) msbrown47
fix flaky partd test ( dask#5111 ) Tom Augspurger
Adjust is_dataframe_like to adjust for value_counts change ( dask#5143 ) Tom Augspurger
Generalize rolling windows to support non-Pandas dataframes ( dask#5149 ) Nick Becker
Avoid unnecessary aggregation in pivot_table ( dask#5173 ) Daniel Saxton
Add column names to apply_and_enforce error message ( dask#5180 ) Matthew Rocklin
Add schema keyword argument to to_parquet ( dask#5150 ) Sarah Bird
Remove recursion error in accessors ( dask#5182 ) Jim Crist
Allow fastparquet to handle gather_statistics=False for file lists ( dask#5157 ) Richard J Zamora
Adds NumFOCUS badge to the README ( dask#5086 ) James Bourbeau
Update developer docs [ci skip] ( dask#5093 ) Jim Crist
Document DataFrame.set_index computataion behavior Natalya Rapstine
Use pip install . instead of calling setup.py ( dask#5139 ) Matthias Bussonier
Close user survey ( dask#5147 ) Tom Augspurger
Fix Google Calendar meeting link ( dask#5155 ) Loïc Estève
Add docker image customization example ( dask#5171 ) James Bourbeau
Update remote-data-services after fsspec ( dask#5170 ) Martin Durant
Fix typo in spark.rst ( dask#5164 ) Xavier Holt
Update setup/python docs for async/await API ( dask#5163 ) Matthew Rocklin
Update Local Storage HPC documentation ( dask#5165 ) Matthew Rocklin
Respect security configuration in LocalCluster (2822) `Russ Bubley`_
Add Nanny to worker docs (2826) `Christian Hudon`_
Don't make False add-keys report to scheduler (2421) `tjb900`_
Include type name in SpecCluster repr (2834) `Jacob Tomlinson`_
Extend prometheus metrics endpoint (2833) `Gabriel Sailer`_
Add alternative SSHCluster implementation (2827) `Matthew Rocklin`_
Dont reuse closed worker in get_worker (2841) `Pierre Glaser`_
SpecCluster: move init logic into start (2850) `Jacob Tomlinson`_
Document distributed.Reschedule in API docs (2860) `James Bourbeau`_
Add fsspec to installation of test builds (2859) `Martin Durant`_
Make await/start more consistent across Scheduler/Worker/Nanny (2831) `Matthew Rocklin`_
Add cleanup fixture for asyncio tests (2866) `Matthew Rocklin`_
Use only remote connection to scheduler in Adaptive (2865) `Matthew Rocklin`_
Add Server.finished async function (2864) `Matthew Rocklin`_
Align text and remove bullets in Client HTML repr (2867) `Matthew Rocklin`_
Test dask-scheduler --idle-timeout flag (2862) `Matthew Rocklin`_
Remove Client.upload_environment (2877) `Jim Crist`_
Replace gen.coroutine with async/await in core (2871) `Matthew Rocklin`_
Forcefully kill all processes before each test (2882) `Matthew Rocklin`_
Cleanup Security class and configuration (2873) `Jim Crist`_
Remove unused variable in SpecCluster scale down (2870) `Jacob Tomlinson`_
Add SpecCluster ProcessInterface (2874) `Jacob Tomlinson`_
Add Log(str) and Logs(dict) classes for nice HTML reprs (2875) `Jacob Tomlinson`_
Pass Client._asynchronous to Cluster._asynchronous (2890) `Matthew Rocklin`_
Add default logs method to Spec Cluster (2889) `Matthew Rocklin`_
Add processes keyword back into clean (2891) `Matthew Rocklin`_
Update black (2901) `Matthew Rocklin`_
Move Worker.local_dir attribute to Worker.local_directory (2900) `Matthew Rocklin`_
Link from TapTools to worker info pages in dashboard (2894) `Matthew Rocklin`_
Avoid exception in Client._ensure_connected if closed (2893) `Matthew Rocklin`_
Convert Pythonic kwargs to CLI Keywords for SSHCluster (2898) `Matthew Rocklin`_
Use kwargs in CLI (2899) `Matthew Rocklin`_
Name SSHClusters by providing name= keyword to SpecCluster (2903) `Matthew Rocklin`_
Request feed of worker information from Scheduler to SpecCluster (2902) `Matthew Rocklin`_
Clear out compatibillity file (2896) `Matthew Rocklin`_
Remove future imports (2897) `Matthew Rocklin`_
Use click's show_default=True in relevant places (2838) `Christian Hudon`_
Close workers more gracefully (2905) `Matthew Rocklin`_
Close workers gracefully with --lifetime keywords (2892) `Matthew Rocklin`_
Add closing <li> tags to Client._repr_html_ (2911) `Matthew Rocklin`_
Add endline spacing in Logs._repr_html_ (2912) `Matthew Rocklin`_
- Add recompute= keyword to svd_compressed for lower-memory use ( dask#5041 ) Matthew Rocklin
Add recompute= keyword to svd_compressed for lower-memory use ( dask#5041 ) Matthew Rocklin
Change array_function implementation for backwards compatibility ( dask#5043 ) Ralf Gommers
Added dtype and shape kwargs to apply_along_axis ( dask#3742 ) Davis Bennett
Fix reduction with empty tuple axis ( dask#5025 ) Peter Andreas Entschev
Drop size 0 arrays in stack ( dask#4978 ) John A Kirkham
Removes index keyword from pandas to_parquet call ( dask#5075 ) James Bourbeau
Fixes upstream dev CI build installation ( dask#5072 ) James Bourbeau
Ensure scalar arrays are not rendered to SVG ( dask#5058 ) Willi Rath
Environment creation overhaul ( dask#5038 ) Tom Augspurger
s3fs, moto compatibility ( dask#5033 ) Tom Augspurger
pytest 5.0 compat ( dask#5027 ) Tom Augspurger
Fix compute_meta recursion in blockwise ( dask#5048 ) Peter Andreas Entschev
Remove hard dependency on pandas in get_dummies ( dask#5057 ) GALI PREM SAGAR
Check dtypes unchanged when using DataFrame.assign ( dask#5047 ) asmith26
Fix cumulative functions on tables with more than 1 partition ( dask#5034 ) tshatrov
Handle non-divisible sizes in repartition ( dask#5013 ) George Sakkis
Handles timestamp and preserve_index changes in pyarrow ( dask#5018 ) Richard J Zamora
Fix undefined meta for str.split(expand=False) ( dask#5022 ) Brett Naul
Removed checks used for debugging merge_asof ( dask#5011 ) Cody Johnson
Don’t use type when getting accessor in dataframes ( dask#4992 ) Matthew Rocklin
Add melt as a method of Dask DataFrame ( dask#4984 ) Dustin Tindall
Adds path-like support to to_hdf ( dask#5003 ) James Bourbeau
Point to latest K8s setup article in JupyterHub docs ( dask#5065 ) Sean McKenna
Changes vizualize to visualize ( dask#5061 ) David Brochart
Fix from_sequence typo in delayed best practices ( dask#5045 ) James Bourbeau
Add user survey link to docs ( dask#5026 ) James Bourbeau
Fixes typo in optimization docs ( dask#5015 ) James Bourbeau
Update community meeting information ( dask#5006 ) Tom Augspurger
Fix typo that prevented error message (2825) `Russ Bubley`_
Remove dask-mpi (2824) `Matthew Rocklin`_
Updates to use update_graph in task journey docs (2821) `James Bourbeau`_
Fix Client repr with memory_info=None (2816) `Matthew Rocklin`_
Fix case where key, rather than TaskState, could end up in ts.waiting_on (2819) `tjb900`_
Use Keyword-only arguments (2814) `Matthew Rocklin`_
Relax check for worker references in cluster context manager (2813) `Matthew Rocklin`_
Add HTTPS support for the dashboard (2812) `Jim Crist`_
Use dask.utils.format_bytes (2810) `Tom Augspurger`_
We neglected to include python_requires= in our setup.py file, resulting in confusion for Python 2 users who erroneously get packages for 2.0.0. This
We neglected to include python_requires= in our setup.py file, resulting in confusion for Python 2 users who erroneously get packages for 2.0.0. This is fixed in 2.0.1 and we have removed the 2.0.0 files from PyPI.
Add python_requires entry to setup.py (2807) `Matthew Rocklin`_
Correctly manage tasks beyond deque limit in TaskStream plot (2797) `Matthew Rocklin`_
Fix diagnostics page for memory_limit=None (2770) `Brett Naul`_
This is a small bugfix release due to a config change upstream.
This is a small bugfix release due to a config change upstream.
Use config accessor method for "scheduler-address" (2676) `James Bourbeau`_
Avoid deprecation warnings (2653) Matthew Rocklin_
Add Type Attribute to TaskState (2657) `Matthew Rocklin`_
Add waiting task count to progress title bar (2663) `James Bourbeau`_
DOC: Clean up reference to cluster object (2664) `K.-Michael Aye`_
Allow scheduler to politely close workers as part of shutdown (2651) `Matthew Rocklin`_
Check direct_to_workers before using get_worker in Client (2656) `Matthew Rocklin`_
Fixed comment regarding keeping existing level if less verbose (2655) `Brett Randall`_
Add idle timeout to scheduler (2652) `Matthew Rocklin`_
Avoid deprecation warnings (2653) `Matthew Rocklin`_
Use an LRU cache for deserialized functions (2623) `Matthew Rocklin`_
Rename Worker._close to Worker.close (2650) `Matthew Rocklin`_
Add Comm closed bookkeeping (2648) `Matthew Rocklin`_
Explain LocalCluster behavior in Client docstring (2647) `Matthew Rocklin`_
Add last worker into KilledWorker exception to help debug (2610) `@plbertrand`_
Set working worker class for dask-ssh (2646) `Martin Durant`_
Add as_completed methods to docs (2642) `Jim Crist`_
Add timeout to Client._reconnect (2639) `Jim Crist`_
Limit test_spill_by_default memory, reenable it (2633) `Peter Andreas Entschev`_
Use proper address in worker -> nanny comms (2640) `Jim Crist`_
Fix deserialization of bytes chunks larger than 64MB (2637) `Peter Andreas Entschev`_
Adaptive: recommend close workers when any are idle (2330) Michael Delgado_
Adaptive: recommend close workers when any are idle (2330) `Michael Delgado`_
Increase GC thresholds (2624) `Matthew Rocklin`_
Add interface= keyword to LocalCluster (2629) `Matthew Rocklin`_
Add worker_class argument to LocalCluster (2625) `Matthew Rocklin`_
Remove Python 2.7 from testing matrix (2631) `Matthew Rocklin`_
Add number of trials to diskutils test (2630) `Matthew Rocklin`_
Fix parameter name in LocalCluster docstring (2626) `Loïc Estève`_
Integrate stacktrace for low-level profiling (2575) `Peter Andreas Entschev`_
Apply Black to standardize code styling (2614) `Matthew Rocklin`_
added missing whitespace to start_worker cmd (2613) `condoratberlin`_
Updated logging module doc links from docs.python.org/2 to docs.python.org/3. (2635) `Brett Randall`_
Add basic health endpoints to scheduler and worker bokeh. (2607) amerkel2_
Add basic health endpoints to scheduler and worker bokeh. (2607) `amerkel2`_
Improved description accuracy of --memory-limit option. (2601) `Brett Randall`_
Check self.dependencies when looking at dependent tasks in memory (2606) `deepthirajagopalan7`_
Add RabbitMQ SchedulerPlugin example (2604) `Matt Nicolls`_
add resources to scheduler update_graph plugin (2603) `Matt Nicolls`_
Use ensure_bytes in serialize_error (2588) `Matthew Rocklin`_
Specify data storage explicitly from Worker constructor (2600) `Matthew Rocklin`_
Change bokeh port keywords to dashboard_address (2589) `Matthew Rocklin`_
.detach_() pytorch tensor to serialize data as numpy array. (2586) `Muammar El Khatib`_
Add warning if creating scratch directories takes a long time (2561) `Matthew Rocklin`_
Fix typo in pub-sub doc. (2599) `Loïc Estève`_
Allow return_when='FIRST_COMPLETED' in wait (2598) `Nikos Tsaousis`_
Forward kwargs through Nanny to Worker (2596) `Brian Chu`_
Use ensure_dict instead of dict (2594) `James Bourbeau`_
Specify protocol in LocalCluster (2489) `Matthew Rocklin`_
Fix LocalCluster to not overallocate memory when overcommitting threads per worker (2541) George Sakkis_
Fix LocalCluster to not overallocate memory when overcommitting threads per worker (2541) `George Sakkis`_
Make closing resilient to lacking an address (2542) `Matthew Rocklin`_
fix typo in comment (2546) `Brett Jurman`_
Fix double init of prometheus metrics (2544) `Marco Neumann`_
Skip test_duplicate_clients without bokeh. (2553) `Elliott Sales de Andrade`_
Add blocked_handlers to servers (2556) `Chris White`_
Always yield Server.handle_comm coroutine (2559) `Tom Augspurger`_
Use yaml.safe_load (2566) `Matthew Rocklin`_
Fetch executables from build root. (2551) `Elliott Sales de Andrade`_
Fix Torando 6 test failures (2570) `Matthew Rocklin`_
Fix test_sync_closed_loop (2572) `Matthew Rocklin`_
Add deprecation warning to dask_mpi.py (2522) Julia Kent_
Update style to fix recent flake8 update (2500) (2509) `Matthew Rocklin`_
Fix typo in gen_cluster log message (2503) `Loïc Estève`_
Allow KeyError when closing event loop (2498) `Matthew Rocklin`_
Avoid thread testing for TCP ThreadPoolExecutor (2510) `Matthew Rocklin`_
Find Futures inside SubgraphCallable (2505) `Jim Crist`_
Avoid AttributeError when closing and sending a message (2514) `Matthew Rocklin`_
Add deprecation warning to dask_mpi.py (2522) `Julia Kent`_
Relax statistical profiling test (2527) `Matthew Rocklin`_
Support alternative --remote-dask-worker SSHCluster() and dask-ssh CLI (2526) `Adam Beberg`_
Iterate over full list of plugins in transition (2518) `Matthew Rocklin`_
Create Prometheus Endpoint (2499) `Adam Beberg`_
Use pytest.importorskip for prometheus test (2533) `Matthew Rocklin`_
MAINT skip prometheus test when no installed (2534) `Olivier Grisel`_
Fix intermittent testing failures (2535) `Matthew Rocklin`_
Avoid using nprocs keyword in dask-ssh if set to one (2531) `Matthew Rocklin`_
Bump minimum Tornado version to 5.0
Your coding agent can read these notes before it upgrades. Set up the MCP server →