polars

🚀 Performance improvements

• improve group_tuples of high cardinality data ~10% (#7938)

✨ Enhancements

• Add seed argument to rank for random (#7913)
• Support Numpy ufunc with more than one expression (#7924)

🐞 Bug fixes

• Fix lazy encode schema (#7912)
• respect skip_nulls in apply for temporal types (#7908)

🛠️ Other improvements

• Rename argument f to function in reduce docstring (#7925)
• improve docstrings for numeric/math operator-equivalent methods (#7942)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @alexander-beedie, @alonme, @ankane, @dependabot, @dependabot[bot], @lorentzenchr, @rben01, @ritchie46 and @zundertj

🚀 Performance improvements

• use streaming instead of partitioned groupby (#7907)
• don't auto-stream groupby (#7906)
• rechunk before aggs (#7903)
• don't re-allocate groups in sorted to_dummies (#7897)
• fix hashing regression (#7833)
• rechunk dataframe before unique computation (#7814)
• improve hash quality (#7813)
• always take sorted fast path group_tuples (#7787)

✨ Enhancements

• auto-infer detecting time-zone-awareness of fmt argument in strptime; deprecate tz_aware argument (#7886)
• Add Series.pow() (#7898)
• deal with null values in cut/qcut (#7878)
• allow list/tuple lit values (#7879)
• Support writing dynamic/live formula columns via write_excel (#7871)
• support datetime/date subclasses (e.g. FreezeGun) (#7819)
• support mode for floats and categoricals (#7827)
• allow Series init with Unknown dtype to proceed as if dtype is None, to allow inference (#7830)
• support sort by 'struct' type (#7822)
• add to_repr methods to DataFrame and Series (#7802)
• thousand separators in shape of repr DataFrame (#7775)
• Improve automatic output dtype setting for map_dict. (#7797)
• new utility from_repr function that reconstructs a DataFrame from its table repr (#7781)
• deprecate default value of aggregation_function being 'first' in pivot. In a future version, it will default to None (#7784)

🐞 Bug fixes

• fix lit agg (#7904)
• disable ooc groupby (#7901)
• Use check_exact for temporal types in assert_series_equal (#7896)
• fix abs logical type (#7895)
• fix boolean min/max output type and null handling (#7894)
• Cast compound types to their simple string representation on export to Excel (#7887)
• ensure _repr_html_ escapes column names in addition to data/body elements (#7877)
• validate groupby_dynamic inputs (#7876)
• correct for chunks in arg_where (#7873)
• fix nested logical/physical list (#7872)
• fix arbitrary nested logical types (#7869)
• Relax type hints for when/then (#7857)
• don't use fxhash in sink_sorted fast path (#7849)
• parquet stats & all kernel (#7846)
• Add missing type hint for is_between (#7835)
• fill null list (#7836)
• fix explode list[null] (#7832)
• fix unicode lower/uppercase (#7826)
• raise error on invalid series concat strategy (#7823)
• don't use naive name in partitioned agg (#7810)
• Ensure CsvReader always respects the n_rows parameter (#7789)

🛠️ Other improvements

• Fix read_csv docstring formatting (#7875)
• update concat docstring for how parameter (#7834)
• don't run hash stability test on arm64 (#7825)
• Improve pl.when documentation (#7793)
• add description of ddof (#7811)
• Rename venv folder to .venv (#7790)
• add a make requirements option to install/refresh dependencies without having to recreate the venv (#7792)
• fixup stacklevels (#7796)
• Drop ruff target version (#7791)

Thank you to all our contributors for making this release possible! @LdRoW, @MarcoGorelli, @Newtoniano, @advoet, @alexander-beedie, @duskmoon314, @foxcroftjn, @ghuls, @jonashaag, @ritchie46, @stinodego and @zundertj

🐞 Bug fixes

• ensure k is lower than height (#7779)
• raise error on invalid categorical cast (#7686)
• raise error on attempt to set invalid Datetime or Duration dtype timeunit (#7768)

🛠️ Other improvements

• Add "typos" as spell checking lint (#7759)
• fix typos (#7756)

Thank you to all our contributors for making this release possible! @alexander-beedie, @ghuls, @ritchie46 and @universalmind303

🚀 Performance improvements

• change top_k algorithm (#7718)
• runtime SIMD target detection for min/max/sum and impl SIMD mean ~2-5x (#7702)
• implement top-k optimization (#7678)
• ooc-sort dump in thread local if IO-thread is full. (#7668)
• use perfect hash table for ooc partitioning (#7653)

✨ Enhancements

• add dt.datetime, dt.date, dt.time (#7735)
• new "row_totals" parameter for write_excel that adds a row-wise total column using structured references (#7751)
• More ergonomic args for min/max (#7742)
• More ergonomic args for concat_list (#7745)
• add Series.hist (#7727)
• add qcut (#7724)
• add maintain_order option to Series.cut (#7723)
• create series with only none list with specific dtype (#7722)
• add maintain_order in arr.unique (#7721)
• DataFrame.top_k/ LazyFrame.top_k (#7720)
• clearer error message when replace_time_zone encounters ambiguous or non-existent datetimes (#7685)
• include set_fmt_float value in Config load/save state (#7696)
• raise on descending date_range arguments (#7671)
• include add operator-equivalent expression (#7667)
• add expression method equivalents for existing math/logical operators (#7660)
• add is_leap_year to temporal expressions (#7618)
• full out-of core support for streaming groupby (#7630)
• clearer error message when creating duration string without integer (#7648)
• allow scan_csv to take a list of column names in a new_columns param (#7642)
• out-of-core groupby/unique of groupby on integer keys (#7604)
• allow set and/or frozenset as input to is_in expressions (#7613)

🐞 Bug fixes

• make zip_with_same_type obligatory (#7761)
• fix melt projection pushdown node (#7752)
• fix predicate pushdown for 'unique' first/last (#7749)
• fix null propagation (#7748)
• fix init from pandas Series that has no dtype and is empty (or contains only null values) (#7716)
• avoid ambiguous time error when passing python Datetime to DataFrame constructor (#7711)
• Fix infering CSV schema when skip_rows_after_heade… (#7701)
• fix race condition in null handling of window fast… (#7695)
• address Series init regression from list of np.arange objects (#7692)
• improve error message if unavailable lazy module is queried for __version__ attribute (#7680)
• fix reversed non-existant file error msg (#7657) (#7673)
• respect time zone in groupby_rolling with negative offset (#7664)
• fix empty case str.replace (#7662)
• allow for list of datetimes with timezone(timedelta!=0) in Series constructor (#7645)
• respect time zone in rolling_* functions (#7643)
• fix schema of decimal type reads (#7652)
• detect deltalake version in show_versions (#7622)
• respect time zone in offset_by (#7626)
• fix boolean Series init with integer 1/0 values (#7619)
• respect time zone in dt.round (#7611)

🛠️ Other improvements

• Display full argument names in __repr__ for Datetime a… (#7736)
• add Expr.pipe API docs link (#7734)
• Add sort_by example taking one row per group (#7712)
• Clean up a few type hints/imports (#7687)
• Move wrap_x utils to utils module (#7672)
• Reduce number of polars.internals imports (#7628)
• Remove duplicate column from Expr.sort example (#7684)
• Move expr parsing to utils (#7661)
• Eliminate function re-exports through internals (#7650)
• Move last functionality out of internals (#7649)
• More internals cleanup (#7638)
• Update lockfile (#7637)
• fix and improve type hints and function names (#7609)
• remove additional logic from scan delta (#7605)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @alexander-beedie, @borchero, @chitralverma, @didriksg, @ghuls, @jakob-keller, @minimav, @ritchie46, @stinodego, @universalmind303 and @zundertj

🚀 Performance improvements

• optimize string kernels, (elide redundant allocs) (#7602)
• even faster polars module import (~15%) (#7584)
• optimize str_replace for same length replacements ~2x (#7580)
• reinstate fast module import and optimise DataFrame init by implementing dynamic singledispatch registration (#7559)
• improve perf or str.replace_n and add n argument ~10x (#7575)
• speedup replace_literal_all of single byte replacements ~15x. (#7565)
• set sorted flags (#7558)
• extend ultrafast constant-value frame init to temporal types (over 1,000x speedup) (#7527)

✨ Enhancements

• slightly more space-efficient table output (use ellipsis char, not three periods) (#7599)
• implement decimal -> dtype cast (#7600)
• use head on pyarrow datasets (#7570)
• overwrite streaming chunk size (#7543)

🐞 Bug fixes

• remove index columns in pandas to_sql() (#7596)
• add decimal chunk_lengths (#7589)
• fix ooc sort. the fast path was invalid (#7588)
• Fix regression throwing AmbiguousTimeError in groupby_dynamic (#7454)
• activate dtype-duration for polars-ops (#7582)
• distinct project whole schema if not a subset (#7581)
• reinstate fast module import and optimise DataFrame init by implementing dynamic singledispatch registration (#7559)
• sql window functions (#7458)
• respect time zone in upsample (#7563)
• fix rolling windows for windows that shrink from lhs (#7556)
• remove pyarrow from construction and dispatch to rust (#7551)
• fix negative indexing for head/tail (#7554)
• Remove BatchedCsvReader from public API (#7546)
• fix logical/list getitem (#7545)
• pushdown key in merge sorted projection pd (#7542)
• don't upcast column to string in 'is_in' operation (#7538)

🛠️ Other improvements

• Move more code out of internals (#7597)
• add a performance hint about use of lru_cache to the apply docstrings (#7593)
• Avoid pli in type hints (part 2) (#7587)
• Avoid pli in type hints (part 1) (#7586)
• Move core objects to top level (#7576)
• Bump ruff (#7567)
• Rename namespace Array -> List in docs (#7541)
• Move fmt tests to test_fmt (#7555)
• Rename sep arg to separator (#7533)
• Minor Series cleanup (#7531)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @Vincenthays, @alexander-beedie, @ritchie46, @stinodego, @universalmind303 and @vincev

🚀 Performance improvements

• use atoi in favor of lexical in strptime -25% (#7501)
• [csv] faster utf8 validation ~20% (#7500)
• [csv] SIMD accelerate SplitFields -40% (#7498)
• (csv) don't use memchr for splitfields -~0.15% (#7494)
• csv-file use fast-float for csv float parsing (#7492)

✨ Enhancements

• literal support for binary (#7519)

🐞 Bug fixes

• fix(rust, python) respect time zone in date_range (#7503)
• transparently integrate externally-registered Excel formats (#7520)
• use physical types in sort-by args (#7518)
• keep series name in arithmetic (#7513)
• Initialize with Decimal dtype (#7511)
• fix projection pushdown of asof_joins (#7487)

🛠️ Other improvements

• update show_versions with xlsxwriter (and add as optional dependency) (#7507)
• Use new LazyFrame init in docs (#7508)
• Bump some linting versions (#7505)

Thank you to all our contributors for making this release possible! @CloseChoice, @MarcoGorelli, @alexander-beedie, @ecashin, @ritchie46 and @stinodego

🚀 Performance improvements

• speed up comparison of sorted arrays ~3.85x. (#7478)
• improve performance for datetime parsing with %Z (#7369)

✨ Enhancements

• slice pushdown in LazyFrame.unique (#7470)
• streaming LazyFrame.unique (#7466)
• automatically infer iso8601-like dates (#7457)
• push down temporal predicates to pyarrow scanner (#7421)
• slice pushdown in scan_arrow_ds (#7449)
• convert decimal 256 to 128 on entry (#7448)
• provide option to set individual row_heights on Excel export (#7447)
• optimise Excel export when all data in a multi-column conditional format is contiguous (#7427)
• dynamically change chunk_size in streaming `explo… (#7415)
• support setting multiple conditional formats on the same Excel table column/range (#7411)
• add unary +,-,! to sql (#7399)
• disallow converting key values to null in map_dict due … (#7393)
• use IO backed reader when low_memory=True. (#7394)
• The big error revamp (#7362)
• parse year-month-day as Datetime in slow-path (#7373)
• support applying one conditional format to multiple columns on Excel export (allows for heatmaps) (#7379)
• Proper superclass for Decimal (#7384)
• tweak default Date and Time format strings for Excel export (#7380)
• make melt streamable (#7364)
• don't rechunk before writing to csv (#7365)

🐞 Bug fixes

• handle an unusual edge-case introspecting dataclass type hints (#7476)
• raise error on categorical by arguments if not fro… (#7464)
• fix and test df.corr (#7463)
• make DataFrame rendering compatible with quarto and pandoc (#7455)
• sql floor & ceil (#7456)
• fix DataFrame table rendering issue in some Jupyter environments (#7450)
• allow for hourly date_range to cross DST (#7430)
• respect lexical/physical in multi-column categoric… (#7417)
• fix null_dtype slice (#7414)
• sort_by logical types (#7412)
• parse single-digit months and dates when code would have gone down fastpath (#7391)
• creating empty struct series with some unit fields (#7383)
• minor Excel export improvements/fixes (#7363)

🛠️ Other improvements

• Rename read_x functions arg file to source (#7460)
• Refactor utils module (#7435)
• Rename functions that clash with builtins (#7424)
• Showcase new ergonomic syntax in README (#7419)
• Rename Decimal prec to precision (#7401)
• Remove _base_type util (#7410)
• Rename first arg of from_x to data (#7407)
• use exc as variable name for all captured exceptions (#7403)
• Remove redundant schema keyword description from `pl.… (#7400)
• Rename cfg module to config (#7385)
• Add test for for groupby referencing the same column twice (#7340)
• Split up datatypes module (#7357)
• Clean up type checking lints (#7358)

Thank you to all our contributors for making this release possible! @Hofer-Julian, @MarcoGorelli, @SauravMaheshkar, @aldanor, @alexander-beedie, @cjackal, @ghuls, @josh, @juba, @nrebena, @rben01, @ritchie46, @stinodego and @universalmind303

🚀 Performance improvements

• optimize str.replace_all (#7353)
• optimize str.replace ~2x improvement (#7347)
• ensure utf8 apply preallocates memory (#7345)

✨ Enhancements

• make LazyFrame.explode streamable. (#7341)
• allow import of dtype groups from the top-level to improve discovery (#7339)

🐞 Bug fixes

• make decimal types opt-in (#7348)
• fix chunk_sizes in threading apply (#7351)
• don't panic when writing NullArray values to python row tuple (#7346)

🛠️ Other improvements

• add write_excel API docs link (#7338)

Thank you to all our contributors for making this release possible! @alexander-beedie, @ritchie46 and @s-banach

🏆 Highlights

• Excel export support via new write_excel IO method (#7251)
• out of core sort on multiple columns (#7244)

🚀 Performance improvements

• improve batched csv readers perf and memory perf (#7329)
• use inlined strings for field and schema (#7272)
• reuse groups in binary expressions (#7202)

✨ Enhancements

• support creation of sparklines when exporting Excel tables (#7333)
• support sqlalchemy/pandas backed write_database (#7322)
• add adbc database reader and writer (DataFrame.write_database) (#7318)
• make expr.apply streamable in selection context (#7316)
• More ergonomic unnest args (#7310)
• initial working version of Decimal Series (#7220)
• Support explicit Binary dtype in constructor (#7305)
• implement serde for literal datetime and series (#7301)
• improve error message if mmap fails in ipc (#7300)
• add multi-threaded apply (#7277)
• add support for serializing categoricals to json (#7276)
• Add Expr.arg_true (#7056)
• don't require pyarrow for initialising Series with Python datetimes (#7273)
• Excel export support via new write_excel IO method (#7251)
• deprecate describe_(optimized)_plan in favor of explain (#7264)
• enable min-max skipping for binary in parquet, enable min-max skipping for is_in exprs (#7169)
• out of core sort on multiple columns (#7244)
• support nulls_last for multi-column sort (#7242)
• allow optimizations flags in describe_plan (#7233)
• implement row encoding for boolean and binary (#7218)
• allow passing utc=True when parsing time-zone-naive date strings (#7203)
• Add **named_exprs input for struct (#7208)
• add sql "ARRAY_AGG" (#7204)

🐞 Bug fixes

• fix offset in threading apply (#7330)
• fix projection pushdown on join with unused join key (#7326)
• raise error on time -> datetime cast (#7325)
• raise error if output of 'apply' cannot be determined (#7317)
• make pl.struct mappable (#7299)
• err on duplicate with_column names (#7296)
• don't panic on str.parse_int (#7072)
• improve concat_list with empty list error message (#7236)
• fix groupby_dynamic's binning when index_column is time-zone-aware (#7278)
• fix preservation of microseconds when converting Python datetime (#7271)
• fix us precision of datetime to anyvalue conversion (#7268)
• no panic on empty cross join (#7266)
• raise error on ambiguous filter predicates (#7265)
• handle concat_list with first lit value (#7235)
• respect schema in DataFrame initialisation for time-zone-aware datetime (#7240)
• ensure every type is properly normalised (for groupby_dynamic and groupby_rolling) (#7238)
• add test of median function in lazy mode (#7224)
• dont lose precision in pl.date_range due to floating point arithmetic (#7229)
• Conversion of negative timedelta to polars duration (#7209)
• ensure parametric testing cols=int definition respects allowed_dtypes (#7213)

🛠️ Other improvements

• Fix read/write_database tests (#7327)
• Rename scan_ds to scan_pyarrow_dataset (#7320)
• don't run tests that write to disk by default (#7321)
• rename read_sql to read_database (#7315)
• Address git2 vulnerability (#7309)
• Correctly deprecate DataFrame.pearson_corr (#7307)
• Skip write_excel doctests (#7306)
• Run pytest-xdist with worksteal (#7304)
• Rename pearson_corr & spearman_rank_corr (#7014)
• refactor(python) Split io module per type (#7295)
• Move _html module to dataframe module (#7256)
• Enable strict for ruff TCH lints (#7234)
• better select on map_dict dtype (#7217)
• add warning of mmap to ipc docstring (#7216)
• exit non-zero on fix from ruff (#7215)
• ensure that DataFrame and LazyFrame init params don't diverge (#7214)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @aldanor, @alexander-beedie, @coinflip112, @csko, @dependabot, @dependabot[bot], @ghuls, @josemasar, @josh, @mslapek, @nrebena, @ozgrakkurt, @papparapa, @ptiza, @rben01, @ritchie46, @sorhawell, @stinodego, @universalmind303, @xyning and @zundertj

🚀 Performance improvements

• improve perf of multi-args exprs in groupby context (#7186)
• optimize sequence_to_pydf (#7044)
• improve single argument elementwise expression pe… (#7180)

✨ Enhancements

• show column name if read_csv errors (#7177)
• support direct LazyFrame init (same params as DataFrame) (#7122)
• add a base_type method to DataType (#7166)
• add explode for binary (#7159)
• add binary apply (#7160)
• Allow pl.Int32 Series as output in eager repeat. (#7152)
• improve error message when read_csv fails (#7150)
• Improve usability of Null type. (#7136)
• add glob support to scan_ndjson (#7143)
• add Expr.pipe (#7134)
• streaming: scale chunk_size on table width (#7119)
• additional read functions (#7102)
• More ergonomic explode args (#7115)

🐞 Bug fixes

• fix(rust, python); make list function 'map' and refactor multi-arg ex… (#7185)
• Fix Series.argsort (#7183)
• validate trees before inserting streaming node (#7179)
• Raise ValueError for getitem when column indexes are out… (#7167)
• fix list take logical types (#7163)
• fix null cmp fast paths (#7157)
• fix df division dispatch (#7155)
• don't panic un unsupported arithmetic type (#7154)
• don't let a cast unset agg_state and keep logical … (#7151)
• expose sort expressions to stack-optimizer (#7148)
• improve error message when read_csv fails (#7150)
• make cast unknown a no-op (#7147)
• fix panic on cum_prod (#7141)
• respect f32 schema in deep expressions (#7146)
• fix return type of _unpack_schema to prevent potential TypeError (#7128)
• fix docstring in set_tbl_cols() (#7121)
• fix deadlock in scan_csv()->sink_parquet() (#7118)
• subtracting Series from date has wrong sign (#7107)
• fix scan_ipc receiving storage_options (#7085)
• nested sql exprs (#7112)

🛠️ Other improvements

• ensure binary branches are executed in parall… (#7193)
• Deprecate pl.get_dummies (#7055)
• Add Series.cut, deprecate pl.cut (#7058)
• examples functional programing (#7135)
• fix docstring in set_tbl_cols() (#7121)
• Build versioned API reference (#7114)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @Trippy3, @alexander-beedie, @foxcroftjn, @ghuls, @iamsmkr, @jakob-keller, @josh, @mslapek, @papparapa, @ritchie46, @romanovacca, @stinodego, @universalmind303 and @zundertj

🚀 Performance improvements

• optimize arr.sum for list array with inner nulls (#7053)
• optimize arr.min/arr.max (#7050)
• optimize arr.mean (#7048)
• optimize arr.sum (#7047)
• optimize 'arg_where' (#7039)
• More efficient handling of *args/**kwargs (#7026)

✨ Enhancements

• allow for simple creation of n-row empty frame/series via clear (#7095)
• Make polars not copy data when importing from arrow (#7084)
• More ergonomic drop args (#7063)
• More ergonomic partition_by args (#7065)
• More ergonomic exclude args (#7082)
• allow inline expressions in asof_join (#7088)
• add 'use_statistics' option to parquet readers (#7087)

🐞 Bug fixes

• allow map_dict on categorical dtype (#7097)
• fix logical types in arr.get (#7094)
• allow fill_null in eager if type now known (#7092)
• do projection just before concat to ensure same sizes (#7089)
• fix 'filter' in groupby context when expression is… (#7041)
• fix type hint of 'when->then->otherwise' (#7040)
• accept more types in from_records (#7033)

🛠️ Other improvements

• Rename pivot aggregate_fn to aggregate_function (#7059)
• Add TYPE_CHECKING lints (#7070)
• Deprecate more non-keyword arguments (#7030)
• Rename kwarg reverse to descending (#6914)
• Rename args f/func to function (#7032)
• let read_csv take Sequence as columns, remove several type: ignore (#7028)
• add example for arr.count_match() (#7029)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @alexander-beedie, @coinflip112, @datapythonista, @jakob-keller, @moritzwilksch, @ritchie46, @stinodego, @universalmind303 and @zundertj

🚀 Performance improvements

• add arr.count_match expression and optimize arr.sum for List<Boolean> (#7023)
• optimize selection_to_pyexpr_list (#7020)
• avoid unnecessary function calls in LazyFrame.with_columns() (#7019)
• remove O^2 behavior in melt (#7003)
• Improve performance of expr_to_lit_or_expr for arguments of type Expr by ~80% (#6967)
• improve vec_hash perf for boolean and utf8 (#6963)
• don't pack utf8 columns in grouptuples ~5-15% (#6959)
• don't pack integer keys in determining ~8-18% group tuples. (#6956)
• use fxhash for all integers (#6954)

✨ Enhancements

• add arr.count_match expression and optimize arr.sum for List<Boolean> (#7023)
• add sort for struct dtype (#7021)
• More ergonomic coalesce args (#6989)
• raise informative error if invalid datetime_format passed to write_csv (#7005)
• Improve Series & Numpy arithmetic (#6983)
• More ergonomic agg args (#6982)
• rename parse_dates => try_parse_dates (#6987)
• remove packaging and/or distutils dependency with a minimal version parser utility (#6972)
• More ergonomic over args (#6986)
• add upper_bound and lower_bound methods to Series (#6990)
• More ergonomic col args (#6996)
• More ergonomic sort args (#6896)
• Make groupby agg shortcuts available in lazy (#6944)
• add map_dict method for Series (#6946)

🐞 Bug fixes

• reflect time zone conversion in lazy dataframe schema (#7022)
• ensure set_sorted never panics (#7013)
• fix struct append 0 sliced (#7012)
• fix dtype of diff for uint8 (#7010)
• fix coalesce supertype (#7000)
• if given, respect dtype time zone when instantiating pl.lit value (#6999)
• fix fill_null for categoricals (#6998)
• dtype of pow function (#6985)
• fix is_duplicated for utf8 dtype (#6997)
• Remove check for path to be non-directory if use_pyarrow (#6994)
• if given, respect dtype timeunit when instantiating pl.lit value (#6991)
• Add packaging to runtime dependencies (#6962)
• fix temporal logical types in pivot (#6957)
• typo in mean unit test - changed median -> mean (#6960)
• ensure literals are expanded in streaming (#6952)
• str.contains strict=False took no effect (#6950)

🛠️ Other improvements

• date-time unit tests refactor (#7002)
• test lit series arithmetic order (#7015)
• More test restructure (#6961)
• Properly deprecate .struct.to_frame (#6958)
• Properly deprecate GroupBy.agg_list (#6943)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @MatveyF, @alexander-beedie, @jakob-keller, @mslapek, @ozgrakkurt, @papparapa, @ritchie46, @sorhawell, @stinodego, @xhochy and @zundertj

✨ Enhancements

• add is_duplicated/is_unique for struct dtype (#6940)
• add is_between method for Series (#6933)
• supported nested fixedsizebinary conversion (#6923)
• raise error on invalid aggregation expressions (#6921)
• provide better errors when failing to read CSV data from buffers that have advanced their read position (#6920)
• truncate file path on error msg (#6917)
• Parse JSON data in Utf8 to polars dtype (#6885)
• More ergonomic groupby args (#6872)

🐞 Bug fixes

• object to_dict (#6931)
• respect maintain_order in groupby.apply (#6926)
• add special fast path for elementwise expression o… (#6924)
• fix anonymous list builder (#6916)
• reject multithreading on excessive ',\n' fields (#6906)
• fix regression with date => object typing in to_pandas method (#6902)
• dispatch suffix to asof_join by (#6899)
• improve recursive casting of nested data (#6897)
• don't fast explode on null introducing take (#6890)
• prevent external modules found on PYTHONPATH from bleeding into polars venv (#6888)
• prevent conflation of unit.io tests directory with python io module (#6889)

🛠️ Other improvements

• Bump ruff version (#6936)
• add more nested construction tests (#6912)
• Update Cargo.lock (#6893)
• unify constructor logic when initialising from a sequence of dicts (#6887)
• prevent conflation of unit.io tests directory with python io module (#6889)
• refactor datelike as temporal, and support Time dtype in Series.to_numpy (#6881)
• Consistently parse column name inputs (#6879)
• Use Self type more consistently (#6882)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @adamgreg, @alexander-beedie, @josh, @jvdd, @ritchie46 and @stinodego

🚀 Performance improvements

• speedup quantile/median ~2x (#6861)
• remove unneeded series allocations in groupby aggs (#6855)

✨ Enhancements

• restore dataframe class (#6869)
• add include_index option on init from pandas frames (#6847)
• properly implement null array (#6817)
• avoid panic error in strftime with invalid format (#6810)

🐞 Bug fixes

• fix crash in write_csv when mixed tz-naive and tz-aware datetimes are present (#6828)
• accept more types in groupby.agg (#6709)
• Fix pl.from_dataframe() as pyarrow.interchange was not i… (#6844)
• fix schema of functions: (#6845)
• stabilize integer operation to minimal required dtype (#6841)
• use explicit type-arg for PythonDataType (#6481)
• fix numpy/datetime regression (#6835)
• implement to_list for null dtype (#6834)
• Raise ValueError on passing multiple expressions Numpy ufunc (#6821)
• respect schema in ndjson (#6819)

🛠️ Other improvements

• Fail tests on warning (#6868)
• further improve struct expr docstrings (#6852)
• Deprecate non-keyword args for some functions (#6851)
• un-skip passing test (#6854)
• parenthesise col type signature to improve hint interaction with PyCharm (#6850)
• Deprecate positional join args (#6826)
• Rename argsort/argsort_by to arg_sort/arg_sort_by (#6829)
• Update dprint config excludes (#6822)
• Fix some broken noqa comments (#6823)
• Run mypy as part of the lint workflow (#6820)
• various minor docstring rendering fixes (#6818)
• fix lazy groupby docstring/rendering (#6816)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @alexander-beedie, @ghuls, @josh, @kngwyu, @oysols, @ritchie46, @stinodego and @zundertj

🚀 Performance improvements

• remove PySequence downcast (#6803)
• optimize arg_min/arg_max (#6799)

✨ Enhancements

• boolean Series broadcast comparison (eq/neq) against scalar True/False (#6797)

🐞 Bug fixes

• ensure join frame types are consistent (#6798)
• enable empty DataFrame (and Series) init from List of Structs/Lists schema/dtype (#6795)

Thank you to all our contributors for making this release possible! @alexander-beedie, @igmriegel and @ritchie46

✨ Enhancements

• add update method to ldf/df (#6787)
• accept separator for pivot and to_dummies (#6780)
• feat(rust, python) rename 'tz' to 'time_zone' in convert_time_zone and replace_time_zone (#6784)
• Allow other expressions for default arg in map_dict (#6781)
• minor ergonomic affordance; allow pl.concat from generator expression (#6779)
• rename with_time_zone to convert_time_zone and cast_time_zone to replace_time_zone (#6768)
• Add map_dict expression. (#5899)
• support timezone in csv writer (#6722)
• default to 1d interval in date_range (#6771)
• parse timezone from Datetime (#6766)
• Add option to use PyArrow backed-extension arrays when … (#6756)
• formally support duration division (#6758)
• add argmin/max for utf8 data (#6746)
• Improve numpy support: conversion of numpy arrays with … (#6738)
• Improved assert equal messages (#6737)
• Support an ignore_nulls param for EWM calculations. (#5749) (#6742)
• scan_ds predicate pushdown for string cmp (#6734)
• don't require pyarrow for utf8 -> numpy conversion (#6733)
• More ergonomic with_columns args (#6686)
• feat(python):Add return_as_string arg to DF.glimpse; default=False (#6678)
• better-characterise NotFound exceptions (#6670)
• disallow with_time_zone from/to tz-naive (#6659)
• More ergonomic select args (#6667)
• let cast_time_zone work on tz-naive and deprecate tz-localize (#6649)
• improved exceptions on attempt to use invalid schema/dtypes (#6653)

🐞 Bug fixes

• always rechunk if n_chunks > n_rows (#6786)
• fix ndjson empty array parsing (#6785)
• make some list expressions aware of groupby context (#6776)
• use explicit drop function node (#6769)
• don't set sorted flag if we reverse sort the left … (#6772)
• handle edge-case with string-literal replacement when the replace value looks like a capture group (#6765)
• respect skip_rows in glob parsing csv (#6754)
• Improve error message in DataFrame constructor (#6715)
• arrow map dtype conversion (#6732)
• respect 'None' in from_dicts (#6726)
• dedicated rename implementation. (#6688)
• return correct display/repr names for NaN-related expressions (#6721)
• strftime with time zone directive (#6673)
• typing for Series methods that can return None (#6690)
• ensure that iter_rows always returns all values from all chunks/batches in accelerated codepath (#6708)
• Support numpy ufunc when expression not first arg (#6675)
• Raise ValueError on adding float to Series of dtype date (#6677)
• remove uses of rayon global thread pool (#6682)
• true-divide output type (#6665)
• improve behaviour of dict-expansion (scalars) when mixed with numpy arrays (#6663)
• Preserve Expr name in is_between (#6661)
• Tiny improvement of Field repr (#6640)

🛠️ Other improvements

• Update mypy to version 1.0.0 (#6744)
• integrate ignore_nulls into EWM parametric tests (#6751)
• redirect tz_localize (#6749)
• Reorganize benchmark test folder (#6695)
• Split long test modules (namespaces) (#6668)
• Use pytest marker for slow tests (#6642)
• unify nan_to_null and nan_to_none parameter names, expose to DataFrame init, add test coverage (#6637)
• update extend_constant docs/typing (and test coverage) (#6646)

Thank you to all our contributors for making this release possible! @AnatolyBuga, @MarcoGorelli, @MatveyF, @alexander-beedie, @ghuls, @jgmartin, @phaile2, @plaflamme, @ritchie46, @sorhawell, @stinodego, @yuntai and @zundertj

🚀 Performance improvements

• improve dynamic groupby performance on sorted keys (#6599)

✨ Enhancements

• implement fill_null for list data (#6635)
• expression functions should be nullable (#6629)
• Implement unary plus operation on exprs and series (#6517)
• add streamable udfs (#6614)
• is_first for struct dtype (#6595)
• Added from_str_radix method to StringNameSpace that allows to parse strings from any base to i32 (#6570)
• Implement DataFrame Interchange Protocol through pyarrow (#6581)
• improve predicate pushdown (#6579)
• raise error on invalid binary cmp (#6564)

🐞 Bug fixes

• make string_repr private (#6636)
• treat literal values consistently in select context, improve related typing (#6628)
• Fix _repr_html_ double-height rows (#5645) (#6534)
• fix(rust, python) cast to and from fixed offsets (#6602)
• raise error on string numeric arithmetic (#6601)
• don't convert "ns"-precision temporal types via pyarrow (#6592)
• partially assert sortedness in groupby dynamic (#6593)
• fix(rust, python); raise oob if negative index given to take (#6590)
• fix predicate pushdown key check (#6577)
• fix schema of apply with many inputs on empty df (#6571)
• let lhs determine struct order in supertype (#6572)
• ensure consistent handling of 1D numpy arrays with respect to other sequences (#6569)
• fix(rust, python) validate utc, fmt, and tz-aware in strptime (#6550)
• add strptime to filter boundary (#6560)

🛠️ Other improvements

• make string_repr private (#6636)
• add example of using is_between with string bounds, and extend test coverage for the same (#6627)
• provide additional examples for diff methods (#6630)
• Consistent handling of env vars (#6626)
• make structify behaviour experimental, while also extending it to aliased expressions (#6615)
• Disallow clippy borrow deref ref (#6605)
• Update ruff version and some settings (#6588)
• Add release flow info to contributing guide (#6480)
• Use assert_series_equal instead of s.series_equal(...) (#6582)
• cleanup last vestiges of experimental kwargs setting (#6568)
• Use assert_frame_equal instead of assert df.frame_equal(...) (#6553)
• Update to PyO3 to 0.18.0 (#6531)

Thank you to all our contributors for making this release possible! @2-5, @MarcoGorelli, @abalkin, @alexander-beedie, @cojmeister, @dependabot, @dependabot[bot], @jjerphan, @plaflamme, @ritchie46 and @stinodego

🏆 Highlights

• Formalize list aggregation difference between groupbys, selection and window functions (#6487)
• automagically upconvert with_columns kwarg expressions with multiple output names to struct; extend **named_kwargs support to select (#6497)

⚠️ Breaking changes

• error on string <-> date cmp (#6498)
• Formalize list aggregation difference between groupbys, selection and window functions (#6487)
• show where error messages originated (#6482)
• Remove deprecated paths from Series.__getitem__ (#6048)
• change behaviour of named rows (#6302)
• Remove deprecated read/write_json arguments (#5990)
• make schema, schema_overrides, and orient consistent on all user-facing interfaces (#6387)
• Groupby iteration now returns tuples of (name, data) (#6350)
• Remove Groupby.pivot (#6016)
• Remove deprecated argument aliases (#5993)
• Change Series.shuffle default behaviour (#5991)
• Change Expr.is_between default behaviour (#5985)
• Restrict certain function parameters to be keyword-only (#6464)

✨ Enhancements

• let cast_time_zone accept None (#6539)
• automagically upconvert with_columns kwarg expressions with multiple output names to struct; extend **named_kwargs support to select (#6497)
• add some missing type annotation in series dispatch methods (#6523)
• better errors in get_ptr and a probability on a boolean… (#6522)
• add utc parameter to strptime (#6496)
• add meta 'has_multiple_outputs', 'is_regex_projec… (#6500)
• error on string <-> date cmp (#6498)
• ~30% faster iter_rows(named=True) and to_dicts(), if pyarrow available (#6493)
• show where error messages originated (#6482)
• Remove deprecated paths from Series.__getitem__ (#6048)
• change behaviour of named rows (#6302)
• Remove deprecated read/write_json arguments (#5990)
• Groupby iteration now returns tuples of (name, data) (#6350)
• Remove Groupby.pivot (#6016)
• Remove deprecated argument aliases (#5993)
• Change Series.shuffle default behaviour (#5991)
• Change Expr.is_between default behaviour (#5985)
• Restrict certain function parameters to be keyword-only (#6464)

🐞 Bug fixes

• implement ser/de for BinaryChunked (#6543)
• on frame-init from generator, initial chunk_size cannot be smaller than infer_schema_length (#6541)
• raise if tz_localize called on UTC-aware (#6526)
• make concat_list group aware (#6527)
• error on invalid expanding expression (#6521)
• create from dicts directly as struct categorical (#6520)
• fix oob in arr.get by expressions (#6519)
• fix cse schema (#6518)
• panic when max_len -1 is reached (#6494)
• Formalize list aggregation difference between groupbys, selection and window functions (#6487)
• fix(rust, python) validate tz in with_time_zone (#6417)

🛠️ Other improvements

• Remove verify_series_and_expr_api util (#6524)
• Disable some tests for Windows (#6532)
• Remove unnecessary brackets in doc examples (#6332)
• Enable some tests for Windows (#6511)
• Fix test issue with tmp directory (#6508)
• Fix some deprecation warnings (#6495)
• added all missing examples for temporal expressions (#6488)
• Utilize pytest-xdist for faster unittests (#6483)
• test(python) I/O test improvements (#6475)
• make schema, schema_overrides, and orient consistent on all user-facing interfaces (#6387)
• improved error message from Expr on incorrect usage in boolean context (#6473)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @alexander-beedie, @gab23r, @papparapa, @ritchie46, @romanovacca, @stinodego and @zundertj

✨ Enhancements

• More precise pipe type annotation (#6457)

🐞 Bug fixes

• use consistent floor division for floats/ints (#6460)
• split semi/anti join optimization (#6459)

🛠️ Other improvements

• Specify deltalake minimum version (#6363)
• deprecate iterrows in favour of iter_rows, add new @redirect class decorator (#6461)
• Improve IO test structure (#6453)

Thank you to all our contributors for making this release possible! @alexander-beedie, @josh, @ritchie46 and @stinodego

✨ Enhancements

• allow expr in str.contains (#6443)
• Deprecate with_column (#6128)
• expose efficient iterator over DataFrame slices (#6414)
• add float formatting option (#6432)
• 10% speedup for to_dicts method (#6415)
• add datetime/duration dtype selector groups covering the different timeunits (#6425)
• allow internal api to get pointer to values buffer (#6385)
• infer ISO8601 datetimes (#6357)
• minor improvement to auto-detection of ambiguous data orientation (#6376)
• allow expressions as arguments in str.ends_with (#6361)
• Make groupby rolling/dynamic iterable (#6372)
• accept expr in str.starts_with (#6355)
• Move explode to namespaces (#6351)
• Rename Series.struct.to_frame to .struct.unnest (#6352)
• auto-detect %+ as tz-aware (#6434)

🐞 Bug fixes

• fix projection pushdown on double semi join (#6440)
• ensure column-exclusion works with the new dtype groups, and improve some related typing (#6442)
• ensure from_dicts and DataFrame init from list of dicts behave consistently, update/improve related docstrings (#6431)
• cumulative_eval ensure output dtype is respected (#6435)
• allow from pandas null structs (#6430)
• fixed interaction of schema_overrides with frame-init from list of dicts (#6424)
• only use float simd on specific alignment (#6427)
• no early escape when window is equal to len in rolling_float (#6408)
• is_between typing with time in start and end (#6393)
• dont incorrectly infer Zulu time (#6378)
• raise error on invalid sort_by argument (#6382)
• take offset into account with str.explode (#6384)
• Return empty batch for pl.read_csv_batched().next_… (#6381)
• ensure pyarrow.compute module is loaded (#6353)
• implement ser/de for StructChunked (#6359)
• series of empty structs (#6347)

🛠️ Other improvements

• add explicit note about use of Config as a context manager (#6439)
• ensure from_dicts and DataFrame init from list of dicts behave consistently, update/improve related docstrings (#6431)
• Fix docstring of series.interpolate (#6399)
• Remove duplicate test (#6390)
• deprecate columns param for DataFrame init; transitioning to schema (#6366)
• Add docs and tests to Expr.flatten (#6370)
• Example of filtering partitioned delta tables (#6365)
• Uppercase project URL refs (#6362)

Thank you to all our contributors for making this release possible! @ChayimFriedman2, @MarcoGorelli, @alexander-beedie, @c-peters, @flowlight0, @gab23r, @gam-phon, @ghuls, @jgmartin, @josh, @ritchie46, @romanovacca, @stinodego, @universalmind303 and @zundertj

🚀 Performance improvements

• Improve rechunk check (#6268)
• reuse allocated scratches in ipc writer (#6287)
• use dedicated writer thread for sink_parquet (#6285)

✨ Enhancements

• add strict parameter to decoding expressions (#6342)
• allow unordered struct creating from anyvalues (#6321)
• allow pass_name in aggregation apply (#6318)
• parse abbrev month name (#6314)
• Add warning for new behaviour of named rows (#6300)
• add dt.combine for combining date and time components (#6121)
• improvements to dtype-based column selection (#6295)
• add sink_ipc (#6286)
• additional schema_overrides param for more ergonomic DataFrame init (#6230)

🐞 Bug fixes

• don't cast nulls before trying normal cast (#6339)
• properly dispatch categorical string comparison (#6336)
• expand all nested wildcards in functions (#6334)
• fix groupby rolling by_key if groups are empty (#6333)
• Fix some type hints and bugs for groupby (#6329)
• Reject None input for head/tail (#6326)
• parse abbrev month name (#6314)
• default to pyarrow for writing parquet (#6313)
• disallow alias in inline join expressions (#6312)
• block proj-pd and pred-pd on swapping rename (#6303)
• convert nested dictionary with i64 keys (#6299)
• fix(python) Print instantiated dtypes in glimpse (#6298)
• infer y-m-d datetime even if single element (#6297)
• fix panic dynamic_groupby on empty dataframe (#6294)
• implement missing DataFrame __floordiv__ op (#6280)
• Allow low and high in date_range to be str (#6275)
• allow integer-compatible row indexes that are not strictly typed as int (#6266)
• Parse negative dates with polars parser (#6256)

🛠️ Other improvements

• run cse optimization only if joins and caches… (#6337)
• Fix wrong description for variable_name argument in melt (#6331)
• Fix random groupby test failure (#6327)
• fixup test names, adjust test_struct (#6317)
• simplify _from_pandas constructor (#6310)
• Ignore hash doctests (#6304)
• Fix docstring formatting for truncate (#6291)
• Move package metadata to pyproject.toml (#6271)
• Move io tests to the same folder (#6277)
• Enable Dependabot (#5036)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @alexander-beedie, @c-peters, @dependabot, @dependabot[bot], @ghuls, @n8henrie, @ritchie46, @stinodego and @universalmind303

✨ Enhancements

• ensure ooc sort works ooc with all-constant values (#6235)
• The 1 billion row sort (#6156)
• optionally treat missing UTF8 values as the empty string at CSV parse-time (#6203)
• check file target is not an existing directory (#6187)
• support -ve indexing for DataFrame head and tail methods (#6173)
• Implement DataFrame.unique(keep="none") (#6169)
• support use of explicit Struct dtypes on DataFrame/Series init (#6145)

🐞 Bug fixes

• Add list inner dtype when printing Series (#6233)
• strptime now respects pl.Datetime's time_unit (#6231)
• fix when then otherwise with arity and aggregation… (#6224)
• collect now uses the storage_options given to scan_parquet (#6223)
• set_sorted keep schema (#6222)
• pass name to value counts in aggregation (#6221)
• don't set fast_explode on list of structs (#6220)
• address a frame init/construction error, and expose infer_schema_length to frame init (#6210)
• explode of empty nullable list (#6190)
• fix oob arr.take (#6189)
• Make with_columns in with_columns_kwargs mode compatible with more data types (#6126)
• Update docstring with_columns to reflect a new dataframe is being returned (#6122)
• fix empty streaming joins (#6149)
• fix streaming joins where the join order has been … (#6143)
• write tz-aware datetimes to csv (#6135)
• add null behavior for oob indices (#6133)

🛠️ Other improvements

• Create DataFrame from schema (#6225)
• don't set aggregated flag on null propagated aggregation. (#6191)
• undo cargo.toml change (#6219)
• Improve drop_nulls docstrings (#6127)
• Clarify docstrings for closed argument (#6198)
• minor docs and typing updates (plus additional test coverage for related areas) (#6182)
• explain n_field_strategy (#6158)

Thank you to all our contributors for making this release possible! @MarceColl, @MarcoGorelli, @alexander-beedie, @gab23r, @ghuls, @jvanbuel, @n8henrie, @rben01, @ritchie46, @ropoctl, @sorhawell, @stinodego, @winding-lines and @zundertj

🚀 Performance improvements

• first check rev-map on categorical equality check (#6085)

✨ Enhancements

• add arr.take expression (#6116)
• allow extend_constant to work with date literals (#6114)
• allow nested categorical cast (#6113)
• add a rounded_corners modifier to pl.Config.set_tbl_formatting (#6108)
• huge speedup of scalar-to-array expansion on frame init from dict (#6111)
• extend existing fast range->Series init to lists of ranges in a Series (#6099)
• additional (opt-in) options for assert_frame_equal (#6096)
• add search_sorted for arrays and utf8 dtype (#6083)

🐞 Bug fixes

• ensure multi-line type hints are parenthesised (#6100)
• fix invalid dtype in chunked array after struct cast (#6093)
• don't run cse cache_states if no projections found (#6087)
• Support all datatypes in glimpse and align with head/tail (#6091)
• Update read_csv error message (#6082)
• propogate nulls in binary arithmetic/aggregation (#6076)

🛠️ Other improvements

• Fix docstring with_context (#6118)
• Use Dataframe.item internally and in tests (#6109)
• Assert deprecation warning on check_column_names (#6110)
• enable unused import autofix via ruff (#6102)

Thank you to all our contributors for making this release possible! @alexander-beedie, @gitkwr, @huitseeker, @ritchie46, @stinodego and @zundertj

✨ Enhancements

• Improve iterating over GroupBy (#6051)
• much faster lazy type-checks (#6064)
• support array-expansion of scalars on frame init from dict (#6034)
• improve error message when writing nested data to… (#6040)

🐞 Bug fixes

• bound complex type from 3.8 to 3.11 (#6071)
• deal with unnest schema expansion in projection pd (#6063)
• correct output dtype for cummin/cumsum/cummax (#6062)
• block streaming on literal series/range (#6058)
• improve handling of dict-type "columns" param on frame-init (#6045)
• Fix typing for DataFrame.select (#6047)
• ndjson struct inference (#6049)
• fix stringcache. latest refactor introduced a hashing error (#6056)
• allow mixed field order and availability in apply that r… (#6041)
• deal with empty structs (#6039)
• fix aggregation that filters out all data (#6036)
• fix diff overflow (#6033)
• keep column names in is_null/is_not_null (#6032)
• keep name when sorting categorical in lexial order (#6029)
• tweaked property/accessor behaviour (#6021)
• properly set null anyvalue if categorical is neste… (#6025)
• Fix from_epoch function signature (#6024)
• Validate estimated_size parameter (#6018)

🛠️ Other improvements

• suggest forward fill in cumsum/cummax (#6061)
• Fix SIM105 issues. (#6042)
• Remove trailing spaces in glimpse output (#6037)
• Remove unnecessary noqa's (#6035)
• Fix flake8-pytest-style errors in tests. (#6031)
• update read_sql and row docstrings (#6028)
• Enable the isort-style import autofix via ruff (#6020)
• Update py-polars/Cargo.lock (#6013)
• Refactor pivot tests (#6012)
• Use ruff instead of isort, flake8 and pyupgrade (#5916)
• Properly deprecate groupby.pivot (#6000)

Thank you to all our contributors for making this release possible! @alexander-beedie, @ghuls, @ritchie46, @stinodego and @universalmind303

🚀 Performance improvements

• ensure set_at_idx is O(1) (#5977)

✨ Enhancements

• allow eq,ne,lt etc (#5995)
• Improve Expr.is_between API (#5981)
• large speedup for df.iterrows (~200-400%) (#5979)
• updated default table format from "UTF8_FULL" to "UTF8_FULL_CONDENSED" (#5967)
• Access rows as namedtuples (#5966)
• Improve assert_frame_equal messages (#5962)

🐞 Bug fixes

• make weekday tz-aware (#5989)
• fix categorical in struct anyvalue issue (#5987)
• fix invalid boolean simplification (#5976)
• allow empty sort on any dtype (#5975)
• properly deal with categoricals in streaming queries (#5974)

Thank you to all our contributors for making this release possible! @alexander-beedie, @ritchie46 and @stinodego

🚀 Performance improvements

• improve reducing window function performance ~33% (#5878)

✨ Enhancements

• str.strip with multiple chars (#5929)
• add iterrrows (#5945)
• read decimal as f64 (#5938)
• improve query plan scan formatting (#5937)
• allow all null cast (#5933)
• allow objects in struct types (#5925)
• handle Series init from python sequence of numpy arrays (#5918)
• merge sorted dataframes (#5817)
• impl hex and base64 for binary (#5892)
• Add datatype hierarchy (#5901)
• Add .item() on DataFrame and Series (#5893)
• make get_any_value fallible (#5877)
• Add string representation for data types (#5861)
• directly push all operator result into sink, prev… (#5856)

🐞 Bug fixes

• don't panic on ignored context (#5958)
• don't allow named expression in arr.eval (#5957)
• error on invalid dtype (#5956)
• fix panic in join expressions (#5954)
• block ordered predicates before explode (#5951)
• adhere to schema in arr.eval of empty list (#5947)
• fix from_dict schema_inference=0 (#5948)
• fix arrow nested null conversion (#5946)
• allow None in arr.slice length (#5934)
• fix time to duration cast (#5932)
• error on addition with datetime/time (#5931)
• don't create categoricals in streaming (#5926)
• object filter should keep single chunk (#5913)
• csv, read escaped "" as missing (#5912)
• fix pivot of signed integers (#5909)
• don't allow duplicate columns in read_csv arg (#5908)
• fix latest oob in streaming convertion (#5902)
• adapt k to len in topk (#5888)
• fix lazy swapping rename (#5884)
• fix window function with nullable values; regression due… (#5874)
• improve equality consistency between types (#5873)
• evaluate whole branch expression to determine if r… (#5864)
• fix top_k on empty (#5865)
• fix slice in streaming (#5854)
• Fix type hint for IO *_options arguments (#5852)

🛠️ Other improvements

• Fix docs for sink_parquet (#5952)
• Fix misspelling in LazyFrame docstring (#5917)
• add bin, series.is_sorted and merge_sorted (#5914)

Thank you to all our contributors for making this release possible! @AnatolyBuga, @alexander-beedie, @cannero, @chitralverma, @dannyvankooten, @johngunerli, @ozgrakkurt, @ritchie46, @stinodego, @winding-lines and @zundertj

🚀 Performance improvements

• impove performance reducing window functions with numeric output ~-14% (#5841)

✨ Enhancements

• allow more pyarrow literals (#5842)
• add sink_parquet (#5480)
• release GIL when writing (#5830)
• Support parsing more float string representations. (#5824)
• implement mean aggregation for duration (#5807)
• implement sensible boolean aggregates (#5806)

🐞 Bug fixes

• correct invalid type in struct anyvalue access (#5844)
• don't set fast_explode if null values in list (#5838)
• duration formatting (#5837)
• respect fetch in union (#5836)
• keep f32 dtype in fill_null by int (#5834)
• fix(python): fix delta issues (#5802)
• err on epoch on time dtype (#5831)
• fix panic in hmean (#5808)
• asof join by logical groups (#5805)

🛠️ Other improvements

• lazily import connectorx (#5835)

Thank you to all our contributors for making this release possible! @chitralverma, @ghuls and @ritchie46

🐞 Bug fixes

• fix struct dataset (#5798)
• fix parquet regression upstream in arrow2 (#5797)

🛠️ Other improvements

• remove unused cmake-rs patch (#5794)

Thank you to all our contributors for making this release possible! @OneRaynyDay, @messense, @ritchie46 and @universalmind303

🚀 Performance improvements

• set_sorted flag when creating from literal (#5728)
• use sorted fast path in streaming groupby (#5727)

✨ Enhancements

• push down predicates to pyarrow datasets (#5780)
• Support for reading delta lake tables (#5761)
• Add DataFrame.glimpse() (#5622)
• allow expression as quantile input (#5751)
• accept expression in str.extract_all (#5742)
• tz-aware strptime (#5736)
• lazy diagonal concat. (#5647)
• to_struct add upper_bound (#5714)

🐞 Bug fixes

• fix(rust, python) Summation on empty series evaluates to Some(0) (#5773)
• empty concat utf8 (#5768)
• projection pushdown with union and asof join (#5763)
• check null values in asof_join + groupby (#5756)
• fix generic streaming groupby on logical types (#5752)
• fix date_range on expressions (#5750)
• fix dtypes in join_asof_by (#5746)
• fix group order in binary aggregation (#5744)
• implement min/max aggregation for utf8 in groupby (#5737)
• fix all_null/sorted into_groups panic (#5733)
• address several edge-cases found when asserting NaN equality (#5732)
• asof join 'by', 'forward' combination (#5720)

🛠️ Other improvements

• add DataFrame.pearson_corr to reference (#5772)
• Parse fixed timezone offsets without pytz (#5769)
• chore(rust,python) Change allow_streaming to streaming (#5747)
• Remove pyarrow nightlies requirement. (#5719)
• fix incorrect accepted type in df.write_csv (#5715)

Thank you to all our contributors for making this release possible! @AnatolyBuga, @MarcoGorelli, @alexander-beedie, @andrewpollack, @braaannigan, @chitralverma, @ghuls, @ritchie46, @sa- and @zundertj

🚀 Performance improvements

• ensure fast_explode propagates (#5676)

✨ Enhancements

• Series.get_chunks (#5701)
• inversely scale chunk_size with thread count in s… (#5699)
• add streaming minmax (#5693)
• Support large page sizes on aarch64 linux builds (#5694)
• improve dynamic inference of anyvalues and structs (#5690)
• support is_in for boolean dtype (#5682)
• add notebook html repr for Series (#5653)

🐞 Bug fixes

• fix pivot on floating point indexes (#5704)
• fix arange with column/literal input (#5703)
• fix double projection that leads to uneven union d… (#5700)
• Fix Series -> Expr dispatch for @property methods (#5689)
• fix asof join schema (#5686)
• fix owned arithmetic schema (#5685)
• take glob into account in scan_csv 'with_schema_mo… (#5683)
• fix boolean schema in agg_max/min (#5678)
• fix boolean arg-max if all equal (#5680)
• respect python objects read method even if filename is f… (#5677)
• Fix DataFrame.n_chunks return type (#5650)

🛠️ Other improvements

• Parametrize test_parquet_datetime (#5696)
• Function and lazy function doctrings (#5657)
• Fix formatting (#5658)

Thank you to all our contributors for making this release possible! @alexander-beedie, @ankane, @braaannigan, @ghais, @ghuls, @jjerphan, @pickfire, @ritchie46, @stinodego and @zundertj

⚠️ Breaking changes

• Update Expr.sample signature and change random seeding (#4648)
• rollup breaking changes (#5602)
• iso weekday (#5598)
• Change null_equal default to True for Series.series_equal (#5051)
• rollup breaking changes (#5602)

🚀 Performance improvements

• fix quadratic time complexity of groupby in stream… (#5614)
• Improve performance of indexing operations on Series. (#5610)
• Aggregate projection pushdown (#5556)

✨ Enhancements

• add a cache to strptime (#5628)
• add nearest interpolation strategy (#5626)
• Update Expr.sample signature and change random seeding (#4648)
• Change null_equal default to True for Series.series_equal (#5051)
• make cast recursive (#5596)
• add arg_min/arg_max for series of dtype boolean (#5592)

🐞 Bug fixes

• early error on duplicate names in streaming groupby (#5638)
• fix streaming groupby aggregate types (#5636)
• convert panic to err in concat_list (#5637)
• fix dot diagram of single nodes (#5624)
• fix dynamic struct inference (#5619)
• tz-aware filtering (#5603)
• keep dtype when eval on empty list (#5597)
• fix ternary with list output on empty frame (#5595)
• fix tz-awareness of truncate (#5591)
• check chunks before doing chunked_id join optimiza… (#5589)
• invert cast_time_zone conversion (#5587)
• asof join ensure join column is not dropped when '… (#5585)

🛠️ Other improvements

• Remaining docstring examples for frame and lazyframe (#5630)
• use xxhash3 for string types (#5617)
• only trigger build.rs file if that file itself has cha… (#5618)
• iso weekday (#5598)
• Merge release workflows (#5564)
• Fix broken lint workflow (#5584)

Thank you to all our contributors for making this release possible! @Kuhlwein, @braaannigan, @ghuls, @matteosantama, @ritchie46 and @stinodego

🚀 Performance improvements

• improve streaming primitve groupby (#5575)
• vectorize integer vec-hash by using very simple, … (#5572)

✨ Enhancements

• prefer streaming groupby if partitionable (#5580)

🐞 Bug fixes

• fix ub due to invalid dtype on splitting dfs (#5579)

🛠️ Other improvements

• Remove old Python changelog file (#5577)
• namespace registration docs update (#5565)
• Improve contributing guide (#5558)

Thank you to all our contributors for making this release possible! @alexander-beedie, @ghuls, @ritchie46 and @stinodego

🚀 Performance improvements

• specialized utf8 groupby in streaming (#5535)

✨ Enhancements

• add dataframe.pearson_corr (#5533)
• support namespace registration (#5531)
• make map_alias fallible (#5532)
• pl.min & pl.max accept wildcard similar to pl.sum (#5511)
• additional support for using timedelta with duration-type arguments (#5487)

🐞 Bug fixes

• fix(rust, python); fix projection pushdown in asof joins (#5542)
• streaming hstack allow duplicates (#5538)
• fix streaming empty join panic (#5534)
• fix duplicate caches in cse and prevent quadratic … (#5528)
• allow appending categoricals that are all null (#5526)
• tz-aware strftime (#5525)
• make 'truncate' tz-aware (#5522)
• fix coalesce expreession expansion (#5521)
• fix nested aggregatin in when then and window expr… (#5520)
• fix sort_by expression if groups already aggregated (#5518)
• fix bug in batched parquet reader that dropped dfs… (#5506)
• preserve Series name when exporting to pandas (#5498)
• Refactor is_between (#5491)
• fix bugs in skew and kurtosis (#5484)

🛠️ Other improvements

• support tabbed panels in sphinx, add namespace docs (#5540)
• Update dev dependencies (#5517)

Thank you to all our contributors for making this release possible! @alexander-beedie, @braaannigan, @ghuls, @ritchie46, @sorhawell and @zundertj

✨ Enhancements

• additional autocomplete affordances for IPython users (#5477)
• make streaming work with multiple sinks in a sing… (#5474)
• add streaming slice operation (#5466)
• run partial streaming queries (#5464)
• streaming left joins (#5456)
• file statistics so we only (try to) keep smallest table in memory (#5454)
• streaming inner joins. (#5400)

🐞 Bug fixes

• compute correct offset for streaming join on multi… (#5479)
• return error on invalid sortby expression (#5478)
• use json for expr pickle (#5476)
• improved namespace/accessor behaviour (resolves VSCode autocomplete issue) (#5469)
• further improved lazy loading (#5459)
• fix for categorical inserts from row-oriented data (#5462)
• use of fill_null with temporal literals (#5440)

🛠️ Other improvements

• don't panic if part of query cannot run strea… (#5458)
• add build_info() to the API doc (#5442)
• Improved structure for DataFrame and LazyFrame API docs, misc design improvements (#5433)

Thank you to all our contributors for making this release possible! @alexander-beedie, @dannyvankooten, @ritchie46, @s1ck, @slonik-az, @stinodego and @universalmind303

✨ Enhancements

• build_info() provides detailed information how polars was built (#5423)
• add missing width property to LazyFrame (#5431)
• enhanced Series.dot method and related interop (#5428)
• allow regex and wildcard in groupby (#5425)
• support DataFrame init from generators (#5424)
• support Series init from generator (#5411)

🐞 Bug fixes

• fix freeze/stall when writing more than 2^31 string values to parquet (#5366)
• properly handle json with unclosed strings (#5427)
• fix null poisoning in rank operation (#5417)
• correct expr::diff dtype for temporal columns (#5416)
• fix cse for nested caches (#5412)
• don't set sorted flag in argsort (#5410)

🛠️ Other improvements

• Fix dependencies on memory allocator (#5426)
• Better docstring for keep_name (#5378) (#5421)

Thank you to all our contributors for making this release possible! @CalOmnie, @alexander-beedie, @ghuls, @ritchie46, @slonik-az, @stinodego and @universalmind303

✨ Enhancements

• 30x speedup initialising Series from python range object (#5397)
• r-associative support for commutative DataFrame operators (#5394)
• pl.from_epoch function (#5330)
• Streaming joins architecture and Cross join implementation. (#5339)
• enable frame init from sequence of pandas series, and improve lazy typechecks (handle subclasses) (#5383)
• add support for am/pm notation in parse_dates read_csv (#5373)
• add reduce/cumreduce expression as an easier fold (#5364)

🐞 Bug fixes

• explicit nan comparison in min/max agg (#5403)
• lazy proxy module does not require global registration (#5390)
• Correct CSV row indexing (#5385)

🛠️ Other improvements

• Docstrings for frame, lazyframe and time series (#5398)
• add integrated support for copying API examples, and auto-parallelise docs build (#5393)
• improve rendering of API docs type signatures, mark PivotOps as deprecated, misc tidy-ups (#5388)
• Expression docstrings (#5377)
• minor navbar improvements; adds discord and twitter links, fixes github icon (#5379)
• improve structure of sphinx-generated API docs (#5376)
• Add with_time_zone to reference guide (#5369)

Thank you to all our contributors for making this release possible! @YuRiTan, @alexander-beedie, @braaannigan, @owrior, @ritchie46 and @zundertj

✨ Enhancements

• shrink_type expression (#5351)
• don't raise error but print a warning if mp fork method… (#5342)
• tz_localize expression (#5340)
• accept expr in arr.get (#5337)
• Implement forward strategy in groupby join_asof (#5335)

🐞 Bug fixes

• unnest only pushdown column if there are projections (#5360)
• block is_null predicate in asof join (#5358)
• ensure that no-projection is seen as select all in… (#5356)
• resolve duplicated column names in pivot (#5349)
• remove unused branch in getitem (#5348)
• nested dicts / list generation (#5336)
• fix serde of expression (pickle) (#5333)
• handle old-style module loaders such that we can still lazy load them (#5331)
• explicit output type in apply (#5328)

🛠️ Other improvements

• remove multiprocessing check, and leave it to the user (#5347)
• Update dev, lint and docs dependencies (#5338)
• lazy module proxy (obviate attribute access guards for missing modules) (#5320)

Thank you to all our contributors for making this release possible! @AlecZorab, @alexander-beedie, @ghuls and @ritchie46

🐞 Bug fixes

• fix explicit list + sort aggregation in groupby co… (#5317)
• fix sort-merge dispatch of utf8 (#5315)
• close multi-threading pool in df creation (#5309)
• fix and check all uninstalled imports in ci (#5304)

🛠️ Other improvements

• Add "import polars.testing" to testing docstrings (#5316) (#5318)
• streamline lazy imports (#5302)
• Catch deprecation warnings in unit tests (#5306)
• fix and check all uninstalled imports in ci (#5304)

Thank you to all our contributors for making this release possible! @alexander-beedie, @ghuls, @ritchie46, @thatlittleboy, @universalmind303 and @zundertj

🐞 Bug fixes

• include slice in sort fast path (#5247)
• don't use zoneinfo globally (#5246)

Thank you to all our contributors for making this release possible! @ritchie46

✨ Enhancements

• make date_range timezone aware (#5234)
• infer timezone and improve display (#5232)
• allow Config to be used as a context manager, and update some docs (#5223)
• allow polars Config options to be serialised/shared, and more easily unset (#5219)

🐞 Bug fixes

• determine supertype of datetimes with timezones an… (#5240)
• fix groupby dynamic truncate for > days resolution (#5235)
• ensure that polars_type_to_constructor works with tz-aware Datetime dtypes (#5239)
• set timezone on groupby_dynamic boundaries (#5233)
• accept tuple[bool, bool] instead of Sequence[bool] for Expr.is_between (#5094)
• fix incorrect duration dtype (#5226)
• set string cache if lazy schema contains categorical (#5225)
• fix pipeline dtypes (#5224)

🛠️ Other improvements

• update lazyframe lazygroupby apply docstring (#5238)
• Consistent naming for Python release workflow (#5229)

Thank you to all our contributors for making this release possible! @YuRiTan, @alexander-beedie, @cjermain, @matteosantama, @ritchie46 and @stinodego

🚀 Performance improvements

• improve pivot performance by using faster series… (#5172)
• improve streaming performance (~15%) (#5170)
• don't block projection pushdown on unnest (#5123)

✨ Enhancements

• batched csv reader (#5212)
• accept expressions in arr.slice (#5191)
• is_sorted aggregation fast path for Utf8Chunked (#5184)
• support DataFrame init with Datetime dtypes that specify a timezone (#5174)
• frame-level n_unique() that can count unique rows or col/expr subsets (#5165)
• hybrid streaming query engine (#5139)
• return Datetime/Duration with appropriate timeunit when inferring from pytype (#5127)
• add binary dtype (#5122)

🐞 Bug fixes

• fix asof_join schema (#5213)
• fix single thread loop if schema lenght is off by 1 (#5210)
• improve numeric stability of rolling_variance (#5207)
• fix apply function over object dtype (#5206)
• fix overflow in partitioned groupby mean of int32/… (#5204)
• don't allow categorical append that is not under s… (#5195)
• include offset in arr.get (#5193)
• DataFrame.fill_null include unsigned integers (#5192)
• error on fill_nan on non float dtype (#5185)
• infer missing columns in from_dicts (#5183)
• fix rolling_float in case closure returns None (#5180)
• Implement missing extract conversion for Time datatype (#5161)
• implement missing conversion to python time object (#5152)
• Rendering long docstring lines. (#5150)
• add missing _NUMPY_AVAILABLE check in Series.__getitem__ (#5126)
• wrong operator mapped for LtEq (#5120)

🛠️ Other improvements

• skip failing test until #5177 is resolved (#5205)
• ensure streaming groupby take slice into account (#5178)
• remove aggregate pushdown optimization (#5173)
• Add support for ruff python linter. (#5151)
• improve typing; many list types are better defined as Sequence (#5164)
• Get rid of unnecessary check in SplitLines iterator (#5141)

Thank you to all our contributors for making this release possible! @alexander-beedie, @dannyvankooten, @ghuls, @ritchie46 and @sorhawell

🚀 Performance improvements

• take advantage of sorted join for frame alignment (#5106)

✨ Enhancements

• improve function expansion (#5110)
• add struct arithmetics (#5107)
• add cumfold/cumsum expression (#5103)
• error on invalid asof join inputs (#5100)

🐞 Bug fixes

• unique include null (#5112)
• don't recurse assign uniuns as it SO > 5k files (#5098)
• block projection pushdown on unnest (#5093)
• projection_node always do projection locally if no… (#5090)

🛠️ Other improvements

• deprecate name argument in drop (#5099)
• improve py-polars/Makefile (#5089)

Thank you to all our contributors for making this release possible! @alexander-beedie, @owrior, @ritchie46 and @slonik-az

🚀 Performance improvements

• more conservative JIT sort settings (#5080)

Thank you to all our contributors for making this release possible! @mcrumiller, @ritchie46 and @zundertj