NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
Go modules · #2760 by repository stars
Last release 3 years ago
no release in 18 months
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 46 of 46 stable releases
Nothing withdrawn
no release was ever pulled
4 years old
46 releases · first in 2022
One column per quarter.
Enables users to selectively or completely disable Aqueduct's artifact snapshotting functionality. When defining a workflow, users can now decide to d
Released on June 6, 2023.
Adds support for on-demand Kubernetes clusters on Google Cloud; you can now select the Google Cloud option when asking Aqueduct to create a Kubernetes
Released on May 31, 2023.
Improves consistency of icon sizing on resources page.
Released on May 24, 2023.
flow_id and name into a single argument when retrieving workflowsAdds ability to use a parameter in save operators to dynamically name saves.
Released on May 17, 2023.
react-hook-forms.Improves error handling when the user attempts to take an action without connecting to the Aqueduct server.
Released on May 10, 2023.
The recommended SDK API for accessing resources is now client.resource , with client.integration slated to deprecated in a future release.
aqueduct.llm_op API and the aqueduct-llm package. Aqueductllm_op API supports both ad hoc execution, as pictured below, as well asSeries. See ourfrom aqueduct import Client, llm_op
client = Client() # initialize Aqueduct client so we can check if the engine name below is valid
vicuna = llm_op('vicuna_7b', engine='my_k8s_engine')
vicuna('What is the best LLM?')client.resource, with client.integration slated touse argument for LLM for decorators by @cw75 in #1234Improves the describe method on integration objects in the SDK; if there was an error connecting or validating the integration, the error will be show
Released on April 25, 2023.
describe method on integration objects in the SDK; if therenumpy.Extends the Aqueduct-Databricks integration to optionally support long-standing instance pools. If specified, Aqueduct will automatically use nodes in
Released on April 18, 2023.
AwsCredentialsSecretName by @cw75 in #1214Released on April 18, 2023.
Enables subdirectory support when configuring the AWS S3 integration. When a subdirectory is specified, everything outside of that directory will be i
Released on April 11, 2023.
[Beta] Aqueduct now has support for loading DataFrames and images from the local filesystem as parameters: ```python import aqueduct as aq client = aq
Released on April 5, 2023.
import aqueduct as aq client = aq.Client()
data = client.create_param( "df", "/path/to/my/df.csv", use_local=True, as_type=aq.TableArtifact, format='csv', )
### Enhancements
* Improves the artifact storage management process. The integrations page now
shows which system is being used for artifact storage, and if a migration
between artifact stores fails, the error will be surfaced on the
integration details page until the user triggers a new artifact migration.
* Adds metadata to read and write operators' sidesheets. Both sidesheets now
show which system is being used for the IO operation, and the save operator
now shows to what location and (if relevant) in what format the data is
being saved.
### Bugfixes
* Fixes typos in MongoDB connection dialog.
* Fixes bug where workflow status at the top of the workflow details page would
not update in sync with other parts of the page.
* Fixes a bug where stopping the Aqueduct server immediately after starting it
could potentially trigger and then kill a workflow run that would then
stay in a pending state permanently.Extends Aqueduct's parameter support to allow parameters to be None *if* there is a default value for the parameter in the function definition.
Released on March 29, 2023.
None if
there is a default value for the parameter in the function definition.$1, $2, etc. as parameter
placeholders rather than the previous { param } syntax. This allows a
paramter to be reused multiple times within the same query and is similar
to how traditional RDBMS interfaces work.[Beta] Aqueduct now has support for on-demand Kubernetes cluster creation and management on AWS. From the Aqueduct UI, you can connect Aqueduct to you
Released on March 22, 2023.
outputs argument to the @op decorator
or the .set_name() function) will immediately flag and prevent conflicts
in artifact names. Automatically named artifacts will error if multiple
artifacts with the same name are included in a single publish_flow call.Enables registering a workflow without immediately triggering a run of that workflow. When calling publish_flow the run_now parameter can be set to Fa
Released on March 14, 2023.
publish_flow the run_now parameter can be set to
False, which will tell the Aqueduct server to wait until the next
scheduled (or triggered) run. In the interim, only the DAG's structure will
be shown without any execution metadata.kubeconfig is provided when
connecting Aqueduct to Kubernetes.~ to refer to the home directory when specifying a path to
AWS credentials or a kubeconfig.Users can now run Aqueduct workflows on Spark clusters on AWS EMR. With Apache Livy as an interface, Aqueduct can submit your code to your Spark clust
Released on March 7, 2023.
Opens links to docs and feedback in new tabs rather than in the existing tab on the Aqueduct UI.
Released on February 28, 2023.
Updates workflow and data table views to show overview of all executed checks rather than just one.
Released on February 22, 2023.
Adds support for receiving Aqueduct notifications via email or in Slack workspaces. You can configure notification settings for your Aqueduct installa
Released on February 14, 2023.
@op
def fn(param):
return param
res = fn(1).get()
>>> 1 # Creates a parameter named `param` for you automatically, with a default value of 1.
res = fn(2).get()
>>> 2 # Updates `param` to have a default value of 2.
global_config could not be changed to set
Aqueduct as the compute engine.Allows customizing artifact names from the SDK in one of two ways. ```python
Released on February 7, 2023.
# Method 1: Use the decorator
@op(outputs=['sklearn model', 'churn predictions'])
def train_and_predict_churn(features):
# ...
return model, predictions
# Method 2: Use .set_name()
@op
def train_model(features):
# ...
return model
# ...
model = train_model(features)
model.set_name('churn model')
Upgrades workflow layout rendering tool to use the elkjs library.
Released on January 23, 2023
For workflows that are triggered at the end of other workflows, we now allow changing the triggering workflow from the UI's workflow settings dialog.
Released on January 17, 2023
As of this release, Aqueduct has usage tracking. Usage tracking is fully anonymized and captures API routes, performance data, and error rates without
Released on Januay 10, 2023.
Allows user to set compute engine in the operator's decorator.
Released on December 20, 2022.
Fixes a bug where the Aqueduct installation script fails if the user doesn't have conda installed.
Released on December 14, 2022.
The original table.save() syntax still works but will be deprecated in a future release. `python db.save(table, 'my_new_table', update_mode='replace')…
Released on December 13, 2022.
save API; now, to save an artifact, users can write the
following. The original table.save() syntax still works but will be
deprecated in a future release.db.save(table, 'my_new_table', update_mode='replace')
Enables operators running on Kubernetes to access GPUs and set RAM and CPU requirements. Note that using a GPU requires your Kubernetes cluster to alr
Released on November 29, 2022.
@op(resources={'num_cpus': 2, 'memory': '5Gb', 'gpu_resource_name': 'nvidia.com/gpu'})
def my_operator_with_many_resources():
return 1
aq.global_config({'engine': 'my_k8s_integration'})
Extends internal integration test framework to support automated testing against third-party compute engines.
Released on November 14, 2022.
Surfaces errors with parameter validation in workflow status summary.
Released on November 7, 2022.
requirements.txt installation and surfaces
them eagerly; previously, these errors were ignored.great_expectations when
it's being used; the library import is quite slow, so doing it on every
operator was wasteful.engine parameter to global_config, allowing users to specify a
default compute engine; engine is also now an optional parameter to
publish_flow:
engine argument to publish_flow is specified, it will override
the global_config. Otherwise, the engine set in global_config will
be used.engine argument to publish_flow or global_config is
set, the workflow will be executed on the default Aqueduct execution
engine.Hides search bar on data viewing page when there are no artifacts.
Released on October 31, 2022.
*args) in Aqueduct functions.Adds support for Tensorflow Keras models to type system.
Released on October 25, 2022.
Updates the UI to provide a simplified, more responsive layout and surface more information about workflow execution.
Released on October 18, 2022.
kubeconfig file.num_outputs argument to the @op decorator.import aqueduct as aq
@aq.op(num_outputs=3)
def multi_output:
return 1, 2, 3
a, b, c = multi_output()
publish_flow to accept both a single artifact or a list of multiple
artifacts in the artifacts parameter.publish_flow to FlowConfig object.pydantic and typing-extensions;
older versions caused inexplicable and confusing bugs.stdout or stderr logs were truncated on the UI.Released on September 26, 2022.
Released on September 26, 2022.
Released on September 20, 2022.
Released on September 20, 2022.
save operators instead of load operators to
avoid confusion.describe methods to all non-tabular artifact types.Released on September 12, 2022.
Released on September 12, 2022.
None.Adds AWS Athena integration. You can now execute SQL queries against AWS Athena using the Aqueduct integration API. (Since Athena is a query service,
Released on September 6, 2022.
aqueduct.global_config({"lazy": True}).@op
def my_op(input):
# ... modify your data...
return output
result = my_op.lazy(input) # This will not execute immediately.
result.get() # This will force execution of `my_op`.
None! :tada:
Adds support for running workflows on Kubernetes. You can now register a Kubernetes integration from the UI by providing the cluster's kubeconfig file
Released on August 25, 2022.
config argument in the SDK's publish_flow API.None! :tada:
If you did a fresh installation of Aqueduct v0.0.10, you may have run into a bug that says our schema migrator did not run successfully. To fix this,
Released on August 23, 2022.
aqueduct clear and pip3 install --upgrade aqueduct-ml.
You can then start the server via aqueduct start and everything should work again.Adds support for non-tabular data types; operators can now return any Python-serializable object. Under the hood, Aqueduct has special optimization fo
Released on August 22, 2022.
get API, which
surfaces potential errors earlier during workflow construction.Removes deprecated CLI commands, aqueduct server and aqueduct ui.
Released on August 15, 2022.
aqueduct server and aqueduct ui.__str__ method to SDK TableArtifact class to support
pretty-printing.Uses pip freeze to detect and capture local Python requirements when an explicit set of requirements is not specified during function creation.
Released on August 8, 2022.
pip freeze to detect and capture local Python requirements when an
explicit set of requirements is not specified during function creation.aqueduct start and passes the local
API key as a query parameter to automatically log in.--expose, detects and populates the public IP
address of the current machine in CLI output on aqueduct start."csv", "json", etc.aqueduct.to_operator, an existing
function can be converted into an Aqueduct operator.--verbose flag to the CLI so users can see log output in
terminal if desired.Removes need to provide API key and server address when running client and server on same machine.
Released on August 1, 2022.
Prints error message as part of preview execution stack trace, not above it.
Released on July 25, 2022.
head function was unused.Deprecates showing pyplot image in notebook on workflow creation; instead, provides link to UI.
Released on July 14, 2022.
describe method to every relational integration.pyplot image in notebook on workflow creation; instead, provides link to UI.None! :tada:
Deprecates use of ipynbname in the SDK, which prevented the SDK from running in some notebook environments.
Released on July 7, 2022.
head on TableArtifacts.View what tables are present in an integration by clicking on the integration in the UI.
Released on June 21, 2022.
/data page on the UI.aqueduct.get_apikey().Allows users to start both the backend server and UI with aqueduct start
Released on June 9, 2022.
aqueduct startnpm installed by serving the UI from the same server as the backendaqueduct versionlocalhost by default, removing requirement for firewall permissionsfn.local(args)typing library for Python3.7Initial release of the Aqueduct open-source project.
Released on May 26, 2022.
Initial release of the Aqueduct open-source project.
Your coding agent can read these notes before it upgrades. Set up the MCP server →