fugue

• Create a hybrid engine of DuckDB and Dask
• Save Spark-like partitioned parquet files for all engines
• Enable DaskExecutionEngine to transform dataframes with nested columns
• A smarter way to determine default npartitions in Dask
• Support even partitioning on Dask
• Add handling of nested ArrayType on Spark
• Change to plugin approach to avoid explicit import
• Fixed Click version issue
• Added version caps for antlr4-python3-runtime and duckdb as they both released new versions with breaking changes.

• Make Fugue exceptions short and useful
• Ibis integration (experimental)
• Get rid of simple assignment (not used at all)
• Improve DuckDB engine to use a real DuckDB ExecutionEngine
• YIELD LOCAL DATAFRAME

• Add an option to transform to turn off native dataframe output
• Add callback parameter to transform and out_transform
• Support DuckDB
• Create fsql_ignore_case for convenience, make this an option in notebook setup
• Make Fugue SQL error more informative about case issue
• Enable pandas default SQL engine (QPD) to take lower case SQL

• Change pickle to cloudpickle for Flask RPC Server
• Add license to package

• Parsed arbitrary object into execution engine
• Made Fugue SQL accept +, ~, - in schema expression
• Fixed transform bug for Fugue DataFrames
• Fixed a very rare bug of annotation parsing

• Added Select, Aggregate, Filter, Assign interfaces
• Made compatible with Windows OS, added github actions to test on windows
• Let SparkExecutionEngine accept empty pandas dataframes
• Move to codecov
• Let Fugue SQL take input dataframes with name such as a.b

• Dask repartitioning improvement
• Separate Dask IO to use its own APIs
• Improved Dask print function by adding back head
• Made assert_or_throw lazy
• Improved notebook setup handling for jupyter lab

• HOTFIX avro support

• Added built in avro support
• Fixed dask print bug

• Fixed multi take issue for dask
• Fixed pandas, dask print slow

• Added Codacy and Slack channel badges, fixed pylint
• Created transform and out_transform functions
• Added partition syntax sugar
• Fixed FugueSQL CONNECT bug

• Fugueless 1 2 3 4 5
• Notebook experience and extension 1 2
• NativeExecutionEngine: switched to use QPD for SQL
• Spark pandas udf: migrate to applyInPandas and mapInPandas
• SparkExecutionEngine take bug
• Fugue SQL: PRINT ROWS n -> PRINT n ROWS|ROW
• Refactor yield
• Fixed Jinja templating issue
• Change _parse_presort_exp from a private function to public
• Failure to delete execution temp directory is annoying was changed to info

• Limit and Limit by Partition
• README code is working now
• Limit was renamed to take and added to SQL interface
• RPC for Callbacks to collect information from workers in real time
• Changes in handling input dataframe determinism. This fixes a bug related to thread locks with Spark DataFrames because of a deepcopy.

fixed code snippets and remove emojis (#133)

• README code snippets are fixed and emojis removed

• limit and partition by limit are implemented

• sample function
• Make csv infer schema consistent cross engine
• Make loading file more consistent cross engine

• Support **kwargs in interfaceless extensions, see this
• Support Iterable[pd.DataFrame] as output type, see this
• Alter column types
• RENAME in Fugue SQL
• CONNECT different SQL service in Fugue SQL
• Fixed Spark EVEN REPARTITION issue

• Add hook to print/show, see this.

• Fixed import issue with OutputTransformer
• Added fillna as a built-in transform, including SQL implementation