NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #5533 most downloaded on PyPI
Metaflow: More AI and ML, Less Engineering
Last release 1 months ago
02 Sep 2026
Ships fairly regularly
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
7 years old
267 releases · first in 2019
fix: kubernetes kill no-pods messaging by @saikonen in https://github.com/Netflix/metaflow/pull/2072
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.23...2.12.24
Increase failure/success workflow history for argo cron workflows by @savingoyal in https://github.com/Netflix/metaflow/pull/2049
@conda by @savingoyal in https://github.com/Netflix/metaflow/pull/2050_enrich_object deployer method to set instance attributes by @talsperre in https://github.com/Netflix/metaflow/pull/2052Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.22...2.12.23
One column per quarter.
This release fixes a bug introduced with the Argo Workflows Daemon for run heartbeats. The bug affected some workflows using the @project flow decorat
This release fixes a bug introduced with the Argo Workflows Daemon for run heartbeats. The bug affected some workflows using the @project flow decorator, where in some occasions a run could have been registered with incomplete/incorrect project information.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.21...2.12.22
Modify getting Metaflow version by @romain-intel in https://github.com/Netflix/metaflow/pull/2009
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.20...2.12.21
check file in runner by @madhur-ob in https://github.com/Netflix/metaflow/pull/2017
control_task_id attribute by @valayDave in https://github.com/Netflix/metaflow/pull/2024Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.19...2.12.20
add local root for @pypi by @savingoyal in https://github.com/Netflix/metaflow/pull/2004
@pypi by @savingoyal in https://github.com/Netflix/metaflow/pull/2004Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.18...2.12.19
fix: argo daemon with conda environment by @saikonen in https://github.com/Netflix/metaflow/pull/1999
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.17...2.12.18
Fix bootstrap script command by @ob-uk in https://github.com/Netflix/metaflow/pull/1994
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.16...2.12.17
fix: deal with empty string and None values in s3 endpoint_url by @saikonen in https://github.com/Netflix/metaflow/pull/1991
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.15...2.12.16
add snowpark by @madhur-ob in https://github.com/Netflix/metaflow/pull/1986
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.14...2.12.15
feature: add executable attribute to batch/kubernetes decorators by @saikonen in https://github.com/Netflix/metaflow/pull/1976
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.13...2.12.14
enable heart beat daemon and error tracking exit hook by default by @savingoyal in https://github.com/Netflix/metaflow/pull/1953
__getattr__ access by @mutongx in https://github.com/Netflix/metaflow/pull/1971Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.12...2.12.13
fix env var by @iamsgarg-ob in https://github.com/Netflix/metaflow/pull/1959
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.11...2.12.12
Sanitize path names when creating cache directories in FileCache by @romain-intel in https://github.com/Netflix/metaflow/pull/1931
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.10...2.12.11
This patch release introduces support for emitting run heartbeats for the entire duration of a run's lifecycle on Argo Workflows. Metaflow's UI relies
This patch release introduces support for emitting run heartbeats for the entire duration of a run's lifecycle on Argo Workflows. Metaflow's UI relies on a complex state machine to ascertain the correct state of any run, step, task, and attempt. For runs executed on Workflow Orchestrators like AWS Step Functions, Argo Workflows, and Airflow, by design - there is no execution-wide supervisor process that can monitor and keep track of the state of the execution for Metaflow. We instead rely on a task-local process to exfiltrate this state to Metaflow - which aids in effortlessly scaling Metaflow to millions of tasks. However, if no task is scheduled for a while, Metaflow's UI might temporarily show the run as failed (red) before correcting it (green) when the task is scheduled, which can be confusing.
With the latest release, executions on Argo Workflows can kick off a daemon process that is alive for the entire duration of the execution. This daemon process will ensure that a liveness signal is emitted reliably throughout the entire duration of the execution. You can enable this by simply adding the --enable-heartbeat-daemon flag -
python flow.py argo-workflows create --enable-heartbeat-daemon
The next release will enable this functionality as default. If this turns out to be useful for you, or if you have any feedback, ping us at chat.metaflow.org!
[card] bug fix in error card rendering by @valayDave in https://github.com/Netflix/metaflow/pull/1923
merge_artifacts in UBF joins by @valayDave in https://github.com/Netflix/metaflow/pull/1928merge_artifacts by @valayDave in https://github.com/Netflix/metaflow/pull/1929Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.8...2.12.9
pin micromamba version by @savingoyal in https://github.com/Netflix/metaflow/pull/1920
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.7...2.12.8
This release fixes an issue where values passed to flow parameters from an Argo event would unintentionally escape HTML characters. The value in the p
This release fixes an issue where values passed to flow parameters from an Argo event would unintentionally escape HTML characters. The value in the payload should now be passed as-is to the flow parameter.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.6...2.12.7
This release fixes an issue where a join step of a static split would fail on Argo Workflows in rare cases where the length of step names exceeded a t
This release fixes an issue where a join step of a static split would fail on Argo Workflows in rare cases where the length of step names exceeded a threshold.
Fixes an issue where values from an Argo Event payload did not correctly map to flow Parameters if the parameter name contained dashes.
The errors for @pypi should now be more clear in cases where it is unable to successfully resolve an environment due to not finding a suitable package.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.5...2.12.6
add timestamps to conda debug logs by @savingoyal in https://github.com/Netflix/metaflow/pull/1889
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.4...2.12.5
Fix escape hatch to make __getattr__ behavior more standard by @romain-intel in https://github.com/Netflix/metaflow/pull/1883
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.3...2.12.4
Fix default azure client provider setting by @oavdeev in https://github.com/Netflix/metaflow/pull/1877
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.2...2.12.3
Fix issue with setting metadata when using the runner by @romain-intel in https://github.com/Netflix/metaflow/pull/1875
Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.1...2.12.2
This release adds the ability to configure default decorators that will be applied to all steps. This is achieved by setting the decospecs as a value
This release adds the ability to configure default decorators that will be applied to all steps. This is achieved by setting the decospecs as a value (space separated) for METAFLOW_DECOSPECS either as an environment variable or in a config.json
The following example would add retry and kubernetes decorators with a custom memory value to all steps:
export METAFLOW_DECOSPECS="kubernetes:memory=4096 retry"
Defining a decorator with the --with keyword will override the defaults configured. Same applies for explicitly adding a decorator in the flow file.
@projectThis release fixes an issue where argo-workflows delete did not correctly remove possible sensors associated with the workflow if the workflow used the @project decorator.
@project is used by @gabriel-rp in https://github.com/Netflix/metaflow/pull/1871Full Changelog: https://github.com/Netflix/metaflow/compare/2.12.0...2.12.1
This release introduces a new Runner API that makes it simple to run flows inside Notebooks, or as part of Python code.
This release introduces a new Runner API that makes it simple to run flows inside Notebooks, or as part of Python code.
Read the blog post on the feature, or dive straight into the documentation to start using it.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.16...2.12.0
This release adds support for using GCP Secret manager to supply secret values for steps environment.
This release adds support for using GCP Secret manager to supply secret values for steps environment.
In order to enable the secret manager, you should specify the type by setting METAFLOW_DEFAULT_SECRETS_BACKEND_TYPE to gcp-secret-manager or specifying it in the decorator
@secrets(sources=[{"type": "gcp-secret-manager", "id": "some-secret-key"}])
METAFLOW_GCP_SECRET_MANAGER_PREFIX can be set in order to not have to write full secret locations.
This release also adds support for Azure Key Vault as a secrets backend. Specify az-key-vault as the secret backend type to use this.
Same as with the other secret managers, we provide a prefix config to avoid having to repeat common parts in the secret keys. Configure this by setting METAFLOW_AZURE_KEY_VAULT_PREFIX
Note: Currently only Secret object types are supported when using Azure Key Vault.
@parallel for KubernetesThis release adds support for @parallel when flows are run --with kubernetes
Example:
@step
def start(self):
self.next(self.parallel_step, num_parallel=3)
@kubernetes(cpu=1, memory=512)
@parallel
@step
def parallel_step(self):
...
It is now possible to configure the default timeout for the @timeout decorator. This can be done by setting METAFLOW_DEFAULT_RUNTIME_LIMIT in the environment, or in a config.json
Fixes an issue where tasks that were cloned from a previous run by resume would not show up as completed on the Metaflow UI due to missing metadata
There was an issue accessing the index of a foreach task via the client. With this release it is possible to do the following
from metaflow import Task
task = Task("ForeachFlow/123/foreach_step/task-00000000")
task.index
@parallel on Kubernetes] support for Jobsets by @valayDave in https://github.com/Netflix/metaflow/pull/1804Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.15...2.11.16
When running a task with the @retry decorator, previously we were able to only view the logs of the latest attempt of the task. With this release it i
When running a task with the @retry decorator, previously we were able to only view the logs of the latest attempt of the task.
With this release it is now possible to target a specific attempt with the --attempt option for the logs command
python example.py logs 123/retry_step/456 --attempt 2
This release introduces a new command for scrubbing log contents of tasks in case they contain sensitive information that needs to be redacted.
Simplest use case is scrubbing the latest task logs. By default both stdout and stderr are scrubbed
python example.py logs scrub 123/example/456
There are also options to target only a specific log stream
python example.py logs scrub 123/example/456 --stderr
python example.py logs scrub 123/example/456 --stdout
when using the@retry decorator, tasks can have multiple attempts with separate logs that require scrubbing. By default only the latest attempt is scrubbed. There are options to make scrubbing multiple attempts easier
# scrub specific attempt
python example.py logs scrub 123/retry_step/456 --attempt 1
# scrub all attempts
python example.py logs scrub 123/retry_step/456 --all
# scrub specified attempt and all prior to it (this would scrub attempts 0,1,2,3)
python example.py logs scrub 123/retry_step/456 --all --attempt 3
The command also accepts only specifying a step for scrubbing. This is useful for steps with multiple tasks, like a foreach split.
python example.py logs scrub 123/foreach_step
all the above options also apply when targeting a step for scrubbing.
Note: Log scrubbing for running tasks is not recommended, and is actively protected against. There can be occasions where a task has failed in such a way that it still counts as not completed. For such a case you can supply the --include-not-done option to try and scrub it as well.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.14...2.11.15
Increase Azure Blobstore connection_timeout. by @shrinandj in https://github.com/Netflix/metaflow/pull/1827
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.13...2.11.14
This release introduces configuration options for setting default values for cpu / memory / disk when running on Kubernetes. These can be set either w
This release introduces configuration options for setting default values for cpu / memory / disk when running on Kubernetes. These can be set either with environment variables
METAFLOW_KUBERNETES_CPU=
METAFLOW_KUBERNETES_MEMORY=
METAFLOW_KUBERNETES_DISK=
or in a Metaflow profile
{
"KUBERNETES_CPU": "",
"KUBERNETES_MEMORY": "",
"KUBERNETES_DISK": "",
}
These values will be overruled by specifying a value through the @kubernetes or @resources decorators.
This release changes the way task ids are generated on Argo Workflows in order to solve an issue where extremely wide foreach splits could not execute correctly due to hard limits on input parameters size on Argo Workflows.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.12...2.11.13
Fix: JSON Reference Path Error in AWS Step Functions Distributed Map by @nidhinnru in https://github.com/Netflix/metaflow/pull/1822
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.11...2.11.12
Add support for getting EC2 metadata through IMDSv2. by @trhodeos in https://github.com/Netflix/metaflow/pull/1808
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.10...2.11.11
This release fixes an issue where partial or empty argo event payloads would incorrectly overwrite the default values for the parameters of a triggere
This release fixes an issue where partial or empty argo event payloads would incorrectly overwrite the default values for the parameters of a triggered flow.
For example a flow with
@trigger(events=["params_event"])
class DefaultParamEventFlow(FlowSpec):
param_a = Parameter(
name="param_a",
default="default value A",
)
param_b = Parameter(
name="param_b",
default="default value B",
)
will now correctly have the default values for its parameters when triggered by
from metaflow.integrations import ArgoEvent
ArgoEvent('params_event').publish()
or a default value for param_b and the supplied value for param_a when triggered by
ArgoEvent('params_event').publish({"param_a": "custom-value"})
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.9...2.11.10
check for falsy values instead of just None by @madhur-ob in https://github.com/Netflix/metaflow/pull/1786
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.8...2.11.9
fix: support specifying subdirectory with PyPI vcs sources. by @saikonen in https://github.com/Netflix/metaflow/pull/1783
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.7...2.11.8
fix access to name and kwargs by @madhur-ob in https://github.com/Netflix/metaflow/pull/1773
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.6...2.11.7
Generate stubs properly for extensions with lazy loaded modules by @romain-intel in https://github.com/Netflix/metaflow/pull/1765
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.5...2.11.6
fix: allow dot files with --package-suffixes by @saikonen in https://github.com/Netflix/metaflow/pull/1742
--package-suffixes by @saikonen in https://github.com/Netflix/metaflow/pull/1742Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.4...2.11.5
fix: update list of reserved parameter names by @saikonen in https://github.com/Netflix/metaflow/pull/1736
Card.get_data more efficient by @valayDave in https://github.com/Netflix/metaflow/pull/1750Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.3...2.11.4
remove 3.5 and 3.6 from classifiers for stubs by @madhur-ob in https://github.com/Netflix/metaflow/pull/1727
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.2...2.11.3
fix github action for publishing stubs by @madhur-ob in https://github.com/Netflix/metaflow/pull/1707
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.1...2.11.2
You can now install metaflow[stubs] to install an additional metaflow-stubs package that provides type hints for Metaflow decorators (and other object
stubs packageYou can now install metaflow[stubs] to install an additional metaflow-stubs package that provides type hints for Metaflow decorators (and other objects). Some hints were previously included inline but this package expands their use.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.0...2.11.1
## What's Changed docs coming! Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.11...2.11.0
docs coming!
Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.11...2.11.0
[realtime-card] pre-release bug Fixes. by @valayDave in https://github.com/Netflix/metaflow/pull/1686
Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.10...2.10.11
This release adds support for git repositories as dependencies for the @pypi decorator.
@pypi dependenciesThis release adds support for git repositories as dependencies for the @pypi decorator.
You can pin the version of the dependency with "@branch", "@commit", "@tag" or the head of the main branch with "".
Under the hood @pypi will resolve the dependency to a commit at the moment of creating the environment.
Currently conda and pypi environments are created in advance before deploying on remote platforms, this includes gathering all the dependencies and bundling them up. The dependencies for pypi have been limited to binaries only in order to support cross-platform deployments.
For the git sources, we build a wheel from the source and check whether it is compatible with the target platform. This limits using sources where the output wheel is a platform specific binary, for example if building on an ARM platform but trying to deploy on x86.
@pypi by @savingoyal in https://github.com/Netflix/metaflow/pull/1681Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.9...2.10.10
This release adds support for mounting existing EFS volumes with @batch
@batchThis release adds support for mounting existing EFS volumes with @batch
You can specify one or multiple volumes to be mounted with the batch decorators efs_volumes= attribute. The volumes will be mounted under /mnt/volume-name by default, but you can also specify a custom mount point separated by a semicolon with the volume.
Examples
@batch(efs_volumes="fs-001")
@step
...
@batch(efs_volumes=["fs-002", "fs-003"])
@step
...
@batch(efs_volumes="fs-003:/mnt/custom-mountpoint"])
@step
@batchThis release also adds support for custom mount points for host volumes with @batch.
Example
@batch(host_volumes="/home:/mnt/host-homedir")
@step
@conda and @pypi changes@pypi can now handle dependencies with special characters in versions
Fixes a @conda bug where environments were not appending to PATH during runtime if the decorator was applied implicitly to the step. This bug affected dependencies that install a binary, causing them to not be found during runtime due to the environment missing from the PATH.
Previously the lifecycle methods task_pre_step and task_decorate were being called consecutively on a per-decorator basis. This release changes the lifecycle so that task_pre_step is called on all decorators before any task_decorate is called.
This release improves the way that datastore dependencies are checked, in order to avoid an issue where previously run metadata could be published to the metadata service, but the execution fails to start due to missing dependencies for pushing data to the datastore.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.8...2.10.9
raw strings to fix deprecation warnings by @russellbrooks in https://github.com/Netflix/metaflow/pull/1646
Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.7...2.10.8
This release fixes support for pip environment variables that specify a custom location for the config file (PIP_CONFIG_FILE or PIP_CONFIG).
This release fixes support for pip environment variables that specify a custom location for the config file (PIP_CONFIG_FILE or PIP_CONFIG).
The release also adds support for defining a custom index-url through the pip supported environment variable PIP_INDEX_URL
Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.6...2.10.7
The pypi decorator had a bug that caused it to be treated as disabled unless specifically passing disabled=False as an attribute to it. This release f
pypi decoratorThe pypi decorator had a bug that caused it to be treated as disabled unless specifically passing disabled=False as an attribute to it.
This release fixes the default case so that pypi environments activate correctly.
This release adds a METAFLOW_DEBUG_TRACING environment variable to toggle more verbose output for tracing related issues.
By default any errors related to missing tracing dependencies are now silenced completely, in order to not affect platforms that might want tracing environment variables present for all deployments, whether they have the required dependencies or not.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.5...2.10.6
fix nulls in pandas columns on default cards by @amerberg in https://github.com/Netflix/metaflow/pull/1606
Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.4...2.10.5
With this release it is possible to gather telemetry data using an opentelemetry endpoint.
With this release it is possible to gather telemetry data using an opentelemetry endpoint.
Specifying an endpoint in one of the environment variables
METAFLOW_OTEL_ENDPOINTMETAFLOW_ZIPKIN_ENDPOINTwill enable the corresponding tracing provider.
Some additional dependencies are required for the tracing functionality in the execution environment. These can be installed in the base Docker image, or supplied through a conda environment. The relevant packages are
opentelemetry-sdk, opentelemetry-api, opentelemetry-instrumentation, opentelemetry-instrumentation-requests
and depending on your endpoint, either opentelemetry-exporter-otlp or opentelemetry-exporter-zipkin
pypi decoratorThe pypi decorator now supports using a custom index in the users Pip configuration under global.index-url.
This enables using private indices, even ones that require authentication.
For example the following would set up one authenticated and two extra non-authenticated indices for package resolution
pip config set global.index-url "https://user:token@example.com"
pip config set global.extra-index-url "https://extra.example.com https://extra2.example.com"
resources decoratorIt is now possible to specify the ephemeral storage size for Kubernetes jobs when using the resources decorator with the disk= attribute.
argo-workflows status commandAdds a command for easily checking the current status of a workflow on Argo workflows.
python flow.py argo-workflows status [run-id]
There was an issue where relying solely on the Kubernetes apiserver for generating random pod names was resulting in significant collisions with sufficiently large number of executions.
This release adds more randomness to the pod names besides what is generated by Kubernetes.
resources decorator in combination with step functionsThis release fixes an issue where deploying flows on AWS Step Functions was failing in the following cases
@resources(shared_memory=) with any value@resources and @batch(use_tmpfs=True)Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.3...2.10.4
Safely parse pandas.DataFrame indexes for default card by @amerberg in https://github.com/Netflix/metaflow/pull/1574
pandas.DataFrame indexes for default card by @amerberg in https://github.com/Netflix/metaflow/pull/1574ArgoEvent.publish by @savingoyal in https://github.com/Netflix/metaflow/pull/1587Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.2...2.10.3
New configuration option to use same headers as metadata service for argo events webhook calls by @oavdeev in https://github.com/Netflix/metaflow/pull
New configuration option to use same headers as metadata service for argo events webhook calls by @oavdeev in https://github.com/Netflix/metaflow/pull/1560 . Default behavior is the same as before.
Metaflow CLI now supports list-workflow-templates command to list deployed argo workflows by @saikonen in https://github.com/Netflix/metaflow/pull/1577
Full Changelog: https://github.com/Netflix/metaflow/compare/2.10.0...2.10.2
Coming soon! Full Changelog: https://github.com/Netflix/metaflow/compare/2.9.15...2.10.0
Coming soon!
Full Changelog: https://github.com/Netflix/metaflow/compare/2.9.15...2.10.0
We now check for processes in the order in which they complete not in the order in which they are launched. This also increases the likelihood of fail
We now check for processes in the order in which they complete not in the order in which they are launched. This also increases the likelihood of failing fast.
Deadlocks and errors could occur when using the environment escape mechanism in two cases: (a) GC would occur at an inopportune moment or (b) subprocesses were involved. Both issues were fixed.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.9.14...2.9.15
This release fixes an issue with merging broken log lines.
This release fixes an issue with merging broken log lines.
LD_LIBRARY_PATH with Conda environmentsIn a Conda environment, it is sometimes necessary to set LD_LIBRARY_PATH to first include the Conda's environment libraries before anything else. Prior to this release, this used to cause issues with the escape hatch.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.9.13...2.9.14
The recent annotations feature introduced an issue where project, flow_name or user annotations are not being populated for Kubernetes. This release r
The recent annotations feature introduced an issue where project, flow_name or user annotations are not being populated for Kubernetes. This release reverts the changes.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.9.12...2.9.13
The annotations feature introduced in this release has an issue where project, flow_name or user annotations are not being populated for Kubernetes. T
The annotations feature introduced in this release has an issue where project, flow_name or user annotations are not being populated for Kubernetes. This has been reverted in the next release.
This release enables users to add custom annotations to the Kubernetes resources that Flows create. The annotations can be configured much in the same way as custom labels
export METAFLOW_KUBERNETES_ANNOTATIONS="first=A,second=B"
@kubernetes(annotations={"first": "A", "second": "B"})
executable by @romain-intel in https://github.com/Netflix/metaflow/pull/1454Full Changelog: https://github.com/Netflix/metaflow/compare/2.9.11...2.9.12
This release reverts a validation fix introduced in 2.9.10, which prevented executions of Metaflow tasks on AWS Batch
This release reverts a validation fix introduced in 2.9.10, which prevented executions of Metaflow tasks on AWS Batch
Full Changelog: https://github.com/Netflix/metaflow/compare/2.9.10...2.9.11
With this release, Metaflow users can get events on PagerDuty when their workflows succeed or fail on Argo Workflows. Setting up the notifications is
With this release, Metaflow users can get events on PagerDuty when their workflows succeed or fail on Argo Workflows. Setting up the notifications is similar to the existing Slack notifications support
python flow.py argo-workflows create --notify-on-error --notify-on-success --notify-pager-duty-integration-key <pager-duty-integration-key>
METAFLOW_ARGO_WORKFLOWS_CREATE_NOTIFY_PAGER_DUTY_INTEGRATION_KEY=<pager-duty-integration-key>
Full Changelog: https://github.com/Netflix/metaflow/compare/2.9.9...2.9.10
This release fixes a bug with the @conda bootstrapping process. There was an issue with the ServerSideEncryption support, that affected some of the S3
@conda with some S3 providersThis release fixes a bug with the @conda bootstrapping process. There was an issue with the ServerSideEncryption support, that affected some of the S3 operations when using S3 providers that do not implement the encryption headers (for example MinIO).
Affected operations were all that handle multiple files at once:
get_many / get_all / get_recursive / put_many / info_many
which are used as part of bootstrapping a @conda environment when executing remotely.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.9.8...2.9.9
Your coding agent can read these notes before it upgrades. Set up the MCP server →