NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI
PyPI, #5231 most downloaded on PyPI
This SDK enables a set of First Party (Google owned) pipeline components that allow users to take their experience from Vertex AI SDK and other Google Cloud services and create a corresponding pipeline using KFP or Managed Pipelines.
Latest stable
2.22.0
Last release 1 years ago
10 Nov 2025
Ships fairly regularly
a new release about every 2 months
Nearly every release is documented
notes for 58 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
6 years old
105 releases · first in 2021
Drag across the ruler to compare, or tap a release to open it.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Fix for dataproc batch components pipeline failure.
Add autotuning_config and cohort parameters to Dataproc v1 batch components.
Remove deprecated AutoML XGBoost trainer.
Bump image for Structured Data pipelines.
Enforce mutual exclusivity between network and psc_interface_config in custom job spec.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Add psc_interface_config to v1 GCPC custom job components/utils
Bump image for Structured Data pipelines.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Your coding agent can read these notes before it upgrades. Set up the MCP server →
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Explicitly set default python version to 3.9 on Starry Net dsl components to avoid future breaking changes.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Bump image for Structured Data pipelines.
Set tensorboard_experiment_name to random uuid when uploading plots in Starry Net pipeline.
Upgrade default Tensorflow version when setting test set and retrieving training artifacts to random uuid when uploading plots in Starry Net pipeline.
Bump Starry Net dataprep, trainer and predictor image versions.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Add max_wait_duration to v1 GCPC custom job components/utils
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Python 3.9-compatible dependencies and selected upstream images retain accepted vulnerabilities; prefer a newer Python version where possible. See art…
Kubeflow Pipelines 2.18 makes pipelines easier to inspect and author, adds data-transfer and retention tools, and strengthens multi-user security and operational resilience.
BaseModel inputs and outputs, and custom task cache keys. DRA requires a compatible backend, Kubernetes configuration, and resource driver. #14243, #13651, #13995, #13762Follow your normal deployment upgrade process and the upgrade guide. Standard managed installations need no manual recurring-run migration or coordinated API-server/controller cutover. Review these changes before rollout:
Recreate. Legacy V1 cache entries with unknown ownership are no longer reused by default, so affected tasks may run again.kfp, kfp-pipeline-spec, kfp-server-api, and kfp-kubernetes packages. See the SDK release.Documentation is published: SDK reference and guides, Kubernetes SDK reference, and the maintained 2.18 upgrade guide.
Full changelog: 2.17.3 → 2.18.0
Thank you to everyone who contributed code, reviews, testing, and documentation to 2.18.
<details> <summary>Release contributors and first-time contributors</summary>
@alyssacgoins, @Aman5613, @ariel-view, @Ayush-kathil, @bhuvan-somisetty, @cbartram, @copybara-service[bot], @Daniel-1600, @danish9039, @demarna1, @dobbydobap, @droctothorpe, @false200, @hbelmiro, @herikwebb, @jeffspahr, @JerT33, @juliusvonkohout, @kaikaila, @lifo9, @ManvithPanyam, @MaxwellM34, @modichika, @MonicaZhang1, @mprahl, @naruto-lgtm, @Neilblaze, @prakharsingh-74, @Priyanshu-u07, @Raakshass, @rishabhsai, @rohIta-k, @sathish-thangaraj, @sduvvuri1603, @Sharkyii, @shivv23, @thc1006, @tolgatuncoglu, @ugiordan, @VaniHaripriya, @vikas-saxena02, @vizansh, @wallyworld, @zazulam.
</details>
Deployment manifests: 2.18.0 manifests.
Source archives: ZIP · tar.gz. GitHub also provides these in the Assets panel below.
Remove default prediction column names in v1.model_evaluation.regression_component component to fix pipeline errors when using bigquery data source.
Add reservation_affinition support in v1.create_custom_training_job_from_component .
Deprecate preview.custom_job module.
Fix default location in v1.create_custom_training_job_from_component .
Update Docker image.
Bump supported KFP versions to kfp>=2.6.0,<2.11.0 .
Support Python versions 3.12 and 3.13.
Bump image for Structured Data pipelines.
Add strategy to v1 GCPC custom job components/utils
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
chore: upgrading go to 1.25.7 as it has many security fixes by @nsingla in https://github.com/kubeflow/pipelines/pull/12905
sdk-component-yaml by @puwun in https://github.com/kubeflow/pipelines/pull/12755MANAGED_PIPELINES_UPLOAD_TAGS into LoadSamples by @hbelmiro in https://github.com/kubeflow/pipelines/pull/13158Eventually for informer cache sync in pipeline version tests (#13312) by @hbelmiro in https://github.com/kubeflow/pipelines/pull/13253run.py by @puwun in https://github.com/kubeflow/pipelines/pull/12776proxy parameter to e2e-test workflow deployment and test steps - Fixes #13294 by @hbelmiro in https://github.com/kubeflow/pipelines/pull/13295Full Changelog: https://github.com/kubeflow/pipelines/compare/2.16.1...2.17.0
MANAGED_PIPELINES_UPLOAD_TAGS into LoadSamples (#13158)Eventually for informer cache sync in pipeline version tests (#13312) (#13253)run.py (#12776)proxy parameter to e2e-test workflow deployment and test steps - Fixes #13294 (#13295)sdk-component-yaml (#12755)Fix Gemini batch prediction support to v1.model_evaluation.autosxs_pipeline after output schema change.
Modify the code to support latest bp result format
Update the StarryNet package metadata.
Use instance.target_field_name format for text-bison models only, use target_field_name for gemini models.
Pass model name to eval_runner to process batch prediction's output as per the output schema of model used.
Use LLM Model Evaluation image version v0.7
Update AutoSxS and RLHF image tags
fix(backend): ensure log artifact index handling by @droctothorpe in
Full Changelog: https://github.com/kubeflow/pipelines/compare/2.16.0...2.16.1
Fix to model batch explanation component for Structured Data pipelines; image bump.
Add dynamic support for boot_disk_type, boot_disk_size in preview.custom_job.utils.create_custom_training_job_from_component .
Remove preflight validations temporarily.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
fix(frontend): Patch Artifact Storage Key XSS Vulnerability. Fixes #12670 by @JerT33 in https://github.com/kubeflow/pipelines/pull/12671
path property setter to kfp.dsl.types.artifact_types.Model class. Fixes #11728 by @jbojar in https://github.com/kubeflow/pipelines/pull/12551just command runner for developer convenience by @sameerdattav in https://github.com/kubeflow/pipelines/pull/12634add_node_selector_constraint docstring by @hbelmiro in https://github.com/kubeflow/pipelines/pull/12741github.run_id for artifact path by @hbelmiro in https://github.com/kubeflow/pipelines/pull/12773Full Changelog: https://github.com/kubeflow/pipelines/compare/2.15.0...2.16.0
path property setter to kfp.dsl.types.artifact_types.Model class. Fixes #11728 (#12551)add_node_selector_constraint docstring (#12741)github.run_id for artifact path (#12773)Updated the Starry Net pipeline's template gallery description, and added dataprep_nan_threshold and dataprep_zero_threshold args to the Starry Net pipeline.
Fix bug in Starry Net's upload decomposition plot step due to protobuf upgrade, by pinning protobuf library to 3.20.*.
Bump Starry Net image tags.
In the Starry-Net pipeline, enforce that TF Record generation always runs before test set generation to speed up pipelines runs.
Add support for running tasks on a PersistentResource (see CustomJobSpec ) via persistent_resource_id parameter on v1.custom_job.CustomTrainingJobOp and v1.custom_job.create_custom_training_job_from_component
Bump image for Structured Data pipelines.
Add check that component in preview.custom_job.utils.create_custom_training_job_from_component doesn't have any parameters that share names with any custom job fields
Add dynamic machine spec support for preview.custom_job.utils.create_custom_training_job_from_component .
Add preflight validations for LLM text generation pipeline.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
fix(deps): Bump Go versions to address CVE CVE-2025-47907 by @mprahl in https://github.com/kubeflow/pipelines/pull/12267
This release of KFP introduces several notable changes that users should consider prior to upgrading. Comprehensive upgrade and documentation notes will follow shortly. In the interim, please note the following key modifications:
For a complete overview of new features and associated changes, please consult the official release notes.
actions/checkout@v5 consistently by @hbelmiro in https://github.com/kubeflow/pipelines/pull/12121network and psc_interface_config in custom job spec from GCPC SDK by @copybara-service[bot] in https://github.com/kubeflow/pipelines/pull/12415Full Changelog: https://github.com/kubeflow/pipelines/compare/2.14.4...2.15.0
network and psc_interface_config in custom job spec from GCPC SDKactions/checkout@v5 consistently (#12121)Add Gemini batch prediction support to v1.model_evaluation.autosxs_pipeline .
Add Starry Net forecasting pipeline to preview.starry_net.starry_net_pipeline
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Add staging and temp location parameters to prophet trainer component.
Add input parameter autorater_prompt_parameters to _implementation.llm.online_evaluation_pairwise component.
Mitigate bug in v1.model_evaluation.autosxs_pipeline where batch prediction would fail the first time it is run in a project by retrying.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
feat(docs): Guide to report security vulnerabilities by @andreyvelich in https://github.com/kubeflow/pipelines/pull/12044
This release is a version alignment release. This release succeeds the KFP 2.5.0 release. In this 2.14.0 release, KFP backend is aligned in major and minor versions (X.Y) with all the KFP python packages. Please read more about this and the KFP versioning policy here.
--cache_disabled and update test to cover pipelines with outputs by @hbelmiro in https://github.com/kubeflow/pipelines/pull/12001ESC shortcut for closing SidePanel. Fixes #11873 by @EnyMan in https://github.com/kubeflow/pipelines/pull/11874Full Changelog: https://github.com/kubeflow/pipelines/compare/2.5.0...2.14.0
ESC shortcut for closing SidePanel. Fixes #11873 (#11874) (c3d05eb)--cache_disabled and update test to cover pipelines with outputs (#12001) (f240685)Use larger base reward model when tuning text-bison@001 , chat-bison@001 and t5-xxl with the preview.llm.rlhf_pipeline .
Move preview.model_evaluation.autosxs_pipeline to v1.model_evaluation.autosxs_pipeline .
Remove default prediction column names in v1.model_evaluation.classification_component component to fix pipeline errors when using bigquery data source.
Move _implementation.model_evaluation.ModelImportEvaluationOp component to preview namespace preview.model_evaluation.ModelImportEvaluationOp .
Drop support for Python 3.7 since it has reached end-of-life.
Expand number of regions supported by preview.llm.rlhf_pipeline .
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Fix model name preprocess error, pass correct model to ModelImportEvaluationOp component in v1.model_evaluation.evaluation_llm_text_generation_pipeline and v1.model_evaluation.evaluation_llm_classification_pipeline .
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Add support for text-bison@002 to preview.llm.rlhf_pipeline .
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Fix preview.model_evaluation.autosxs_pipeline documentation to show autorater_prompt_parameters as required.
Introduce placeholders: SERVICE_ACCOUNT_PLACEHOLDER , NETWORK_PLACEHOLDER , PERSISTENT_RESOURCE_ID_PLACEHOLDER and ENCRYPTION_SPEC_KMS_KEY_NAME_PLACEHOLDER
Use PERSISTENT_RESOURCE_ID_PLACEHOLDER as the default value of persistent_resource_id for CustomTrainingJobOp and create_custom_training_job_op_from_component . With this change, custom job created without explicitly setting persistent_resource_id will inherit job level persistent_resource_id , if Persistent Resource is set as job level runtime.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Log TensorBoard metrics from the preview.llm.rlhf_pipeline in real time.
Add task_type parameter to preview.llm.rlaif_pipeline .
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Fix bug in preview.llm.rlhf_pipeline that caused wrong output artifact to be used for inference after training.
Fix issue where AutoSxS was not propagating location to all sub-components.
Add CMEK support to preview.llm.infer_pipeline .
Use eval_dataset for train-time evalutation when training a reward model. Requires eval_dataset to contain the same fields as the preference dataset .
Update the documentation of GetModel .
Add CMEK support to preview.model_evaluation.autosxs_pipeline .
Updated component and pipeline inputs/outputs to support creating ModelEvaluations for ModelRegistry models in the AutoSxS pipeline.
Add DRZ-at-rest to preview.llm.rlhf_pipeline .
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Fix the missing output of pipeline remote runner. AutoMLImageTrainingJobRunOp now passes the model artifacts correctly to downstream components.
Fix the metadata of Model Evaluation resource when row based metrics is disabled in preview.model_evaluation.evaluation_llm_text_generation_pipeline .
Support Jinja2>=3.1.2,<4 .
Support custom AutoSxS tasks.
Bump supported KFP versions to kfp>=2.6.0,<=2.7.0 .
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Add CMEK support to preview.llm.rlhf_pipeline when tuning in us-central1 with GPUs.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Use large_model_reference for model_reference_name when uploading models from preview.llm.rlhf_pipeline instead of hardcoding value as text-bison@001 .
Disable caching when resolving model display names for RLHF-tuned models so a unique name is generated on each preview.llm.rlhf_pipeline run.
Upload the tuned adapter to Model Registry instead of model checkpoint from preview.llm.rlhf_pipeline .
Fix the naming of AutoSxS's question answering task. "question_answer" -> "question_answering".
Add Vertex model get component ( v1.model.ModelGetOp ).
Migrate to Protobuf 4 ( protobuf>=4.21.1,<5 ). Require kfp>=2.6.0 .
Support setting version aliases in ( v1.model.ModelUploadOp ).
Only run preview.llm.bulk_inference pipeline after RLHF tuning for third-party models when eval_dataset is provided.
Update LLM Evaluation Pipelines to use text-bison@002 model by default.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Add preview.llm.rlaif_pipeline that tunes large-language models from AI feedback.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Release AutoSxS pipeline to preview.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Fix v1.automl.training_job.AutoMLImageTrainingJobRunOp ModuleNotFoundError .
Append tune-type to existing labels when uploading models tuned by preview.llm.rlhf_pipeline instead of overriding them.
Use llama-2-7b for the base reward model when tuning llama-2-13b with the preview.llm.rlhf_pipeline
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
Bump supported KFP versions to kfp>=2.0.0b10,<=2.4.0
Add LLM Eval pipeline parameter for customizing eval dataset reference ground truth field
Create new eval dataset preprocessor for formatting eval dataset in tuning dataset format.
Support customizing eval dataset format in Eval LLM Text Generation Pipeline ( preview.model_evaluation.evaluation_llm_text_generation_pipeline ) and LLM Text Classification Pipeline ( preview.model_evaluation.evaluation_llm_classification_pipeline ). Include new LLM Eval Preprocessor component in both pipelines.
Fix the output parameter output_dir of preview.automl.vision.DataConverterJobOp .
Fix batch prediction model parameters payload sanitization error .
Add ability to perform inference with chat datasets to preview.llm.infer_pipeline .
Add ability to tune chat models with preview.llm.rlhf_pipeline .
Group preview.llm.rlhf_pipeline components for better readability.
Add environment variable support to GCPC's create_custom_training_job_from_component (both v1 and preview namespaces)
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
fix(backend): upgrade go version to 1.22.12 to fix CVE-2024-45336 by @dandawg in https://github.com/kubeflow/pipelines/pull/11631
comment body by @lociko in https://github.com/kubeflow/pipelines/pull/11772collect-logs.sh path by @hbelmiro in https://github.com/kubeflow/pipelines/pull/11775kubectl get pods to logs when waiting for pods to get ready by @hbelmiro in https://github.com/kubeflow/pipelines/pull/11779set_container_image works with dynamic images by @sradc in https://github.com/kubeflow/pipelines/pull/11795Full Changelog: https://github.com/kubeflow/pipelines/compare/2.4.1...2.5.0
comment body (#11772) (95c3f2c)Upload tensorboard metrics from preview.llm.rlhf_pipeline if a tensorboard_resource_id is provided at runtime.
Support incremental_train_base_model , parent_model , is_default_version , model_version_aliases , model_version_description in AutoMLImageTrainingJobRunOp .
Add preview.automl.vision and DataConverterJobOp .
Set display names for preview.llm pipelines.
Add sliced evaluation metrics support for custom and unstructured AutoML models in evaluation pipeline and evaluation pipeline with feature attribution.
Support service_account in ModelBatchPredictOp .
Release DataflowFlexTemplateJobOp to GA namespace ( v1.dataflow.DataflowFlexTemplateJobOp ).
Make model_checkpoint optional for preview.llm.infer_pipeline . If not provided, the base model associated with the large_model_reference will be used.
Bump apache_beam[gcp] version in GCPC container image from <2.34.0 to ==2.50.0 for compatibility with google-cloud-aiplatform , which depends on shapely<3.0.0dev . Note: upgrades to google-cloud-pipeline-components >=2.5.0 and later may require using a Dataflow worker image with apache_beam==2.50.0 .
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Add support for customizing model_parameters (maxOutputTokens, topK, topP, and temperature) in LLM eval text generation and LLM eval text classification pipelines.
fix(backend): Replaced hardcoded ServiceAccount with default config by @hbelmiro in #11578
Full Changelog: https://github.com/kubeflow/pipelines/compare/2.4.0...2.4.1
Disable caching for LLM pipeline tasks that store temporary artifacts.
Fix the mismatched arguments in 2.4.0 for the Feature Transform Engine component.
Apply latest GCPC image vulnerability resolutions (base OS and software updates).
replaced deprecated image repos with registry.k8s.io by @HumairAK in https://github.com/kubeflow/pipelines/pull/11152
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here
Install python SDK (python 3.9 above) by running:
python3 -m pip install kfp kfp-server-api --upgrade
See the Change Log
Full Changelog: https://github.com/kubeflow/pipelines/compare/2.3.0...2.4.0
Add support for running tasks on a PersistentResource (see CustomJobSpec ) via persistent_resource_id parameter on preview.custom_job.CustomTrainingJobOp and preview.custom_job.create_custom_training_job_from_component
Fix use of encryption_spec_key_name in v1.custom_job.CustomTrainingJobOp and v1.custom_job.create_custom_training_job_from_component
Add feature_selection_pipeline to preview.automl.tabular.
Bump supported KFP versions to kfp>=2.0.0b10,<=2.2.0
Add time_series_dense_encoder_forecasting_pipeline , learn_to_learn_forecasting_pipeline , sequence_to_sequence_forecasting_pipeline , and temporal_fusion_transformer_forecasting_pipeline to preview.automl.forecasting .
Add support for customizing evaluation display name on v1 and preview model_evaluation pipelines.
Include model version ID in v1.model.upload_model.ModelUploadOp 's VertexModel output (key: model ). The URI and metadata resourceName field in the outputted VertexModel now have @<model_version_id> appended, corresponding to the model that was just created. Downstream components DeleteModel and UndeployModel will respect the model version if provided.
Bump KFP SDK upper bound to 2.3.0
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Make LLM pipelines compatible with KFP SDK 2.1.3
Require KFP SDK <=2.1.3
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
docs(backend): Remove deprecated v2 compatibility mode docs by @droctothorpe in https://github.com/kubeflow/pipelines/pull/10956
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here
Install python SDK (python 3.7 above) by running:
python3 -m pip install kfp kfp-server-api --upgrade
See the Change Log
autorater_prompt_parameters to online_evaluation_pairwise component by @copybara-service in https://github.com/kubeflow/pipelines/pull/10759kfp-runtime-tests to run on master branch by @hbelmiro in https://github.com/kubeflow/pipelines/pull/11158Full Changelog: https://github.com/kubeflow/pipelines/compare/2.2.0...2.3.0
autorater_prompt_parameters to online_evaluation_pairwise component (cf7450b)Add preview.llm.infer_pipeline and preview.llm.rlhf_pipeline
Add automl_tabular_tabnet_trainer and automl_tabular_wide_and_deep_trainer to preview.automl.tabular and v1.automl.tabular
Minor feature additions to AutoML components
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here
Install python SDK (python 3.7 above) by running:
python3 -m pip install kfp kfp-server-api --upgrade
See the Change Log
preview.llm.rlhf_pipeline in real time by @copybara-service in https://github.com/kubeflow/pipelines/pull/10595preview.llm pipelines by @copybara-service in https://github.com/kubeflow/pipelines/pull/10616preview.llm.rlhf_pipeline runs if no tensorboard_id is provided by @copybara-service in https://github.com/kubeflow/pipelines/pull/10626text and chat variants of bison@001 with the preview.llm.rlhf_pipeline by @copybara-service in https://github.com/kubeflow/pipelines/pull/10663t5-xxl with the preview.llm.rlhf_pipeline by @copybara-service in https://github.com/kubeflow/pipelines/pull/10665kfp-kubernetes 1.2.0 by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10692kfp-kubernetes release instructions public by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10693preview.llm.rlhf_pipeline by @copybara-service in https://github.com/kubeflow/pipelines/pull/10710Full Changelog: https://github.com/kubeflow/pipelines/compare/2.1.0...2.2.0
preview.llm.rlhf_pipeline (22a98d9)preview.llm.rlhf_pipeline in real time (3d8069b)t5-xxl with the preview.llm.rlhf_pipeline (ff7f660)text and chat variants of bison@001 with the preview.llm.rlhf_pipeline (ac39931)preview.llm.rlhf_pipeline runs if no tensorboard_id is provided (ff0d0a7)Add preview.model_evaluation.evaluation_llm_classification_pipeline.evaluation_llm_classification_pipeline
Change AutoML Vision Error Analysis pipeline names (`v1.model_evaluation.vision_model_error_analysis_pipeline' and 'v1.model_evaluation.evaluated_annotation_pipeline')
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Add preview.model_evaluation.FeatureAttributionGraphComponentOp pipeline
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
chore(components): update container image of endpoint batch predict component for vulnerability patch by @copybara-service in https://github.com/kubef…
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here
Install python SDK (python 3.7 above) by running:
python3 -m pip install kfp kfp-server-api --upgrade
See the Change Log
preview.llm pipelines by @copybara-service in https://github.com/kubeflow/pipelines/pull/10295num_microbatches to _implementation.llm training components by @copybara-service in https://github.com/kubeflow/pipelines/pull/10248llama-2-7b for the base reward model when tuning llama-2-13 with the preview.llm.rlhf_pipeline by @copybara-service in https://github.com/kubeflow/pipelines/pull/10249kfp-pipeline-spec from source in kfp sdk tests by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10300large_model_reference as model_reference_name when uploading models from preview.llm.rlhf_pipeline instead of hardcoding value as text-bison@001 by @copybara-service in https://github.com/kubeflow/pipelines/pull/10321kfp-kubernetes execution tests by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10304preview.llm.rlhf_pipeline run instead of reusing cached result by @copybara-service in https://github.com/kubeflow/pipelines/pull/10322preview.llm.rlhf_pipeline by @copybara-service in https://github.com/kubeflow/pipelines/pull/10323dsl.OutputPath read logic #localexecution by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10334json_escape placeholder util by @copybara-service in https://github.com/kubeflow/pipelines/pull/10351DockerRunner logs #localexecution by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10354None default parameter #localexecution by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10339kfp-pipeline-spec by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10305kfp and kfp-kubernetes by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10307kfp-kubernetes docs versions and release scripts by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10388kfp-kubernetes docs build error by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10389preview.llm.bulk_inference after tuning third-party models with RLHF by @copybara-service in https://github.com/kubeflow/pipelines/pull/10425text-bison@002 model by default by @copybara-service in https://github.com/kubeflow/pipelines/pull/10428dsl.importer #localexecution by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10431dsl.OneOf with multiple consumers cannot be compiled by @connor-mccarthy in https://github.com/kubeflow/pipelines/pull/10452_implementation.llm components by @copybara-service in https://github.com/kubeflow/pipelines/pull/10474preview.llm.rlhf_pipeline by @copybara-service in https://github.com/kubeflow/pipelines/pull/10456preview.llm pipelines by @copybara-service in https://github.com/kubeflow/pipelines/pull/10536preview.llm.infer_pipeline by @copybara-service in https://github.com/kubeflow/pipelines/pull/10519create_custom_training_job_from_component docs rendering by @copybara-service in https://github.com/kubeflow/pipelines/pull/10541preview.llm.rlhf_pipeline by @copybara-service in https://github.com/kubeflow/pipelines/pull/10543preview.llm.rlhf_pipeline by @copybara-service in https://github.com/kubeflow/pipelines/pull/10542Full Changelog: https://github.com/kubeflow/pipelines/compare/2.0.5...2.1.0
num_microbatches to _implementation.llm training components (685634d)preview.llm.rlhf_pipeline (3dbf3cf)preview.llm.infer_pipeline (b7ea6e7)preview.llm.rlhf_pipeline (361c16f)preview.llm pipelines (9007fb0)text-bison@002 model by default (83cb88f)dsl.OutputPath read logic #localexecution (#10334) (654bbde)preview.llm.bulk_inference after tuning third-party models with RLHF (b9e08de)preview.llm.rlhf_pipeline run instead of reusing cached result (075d58f)preview.llm.rlhf_pipeline (2e2ba9e)large_model_reference as model_reference_name when uploading models from preview.llm.rlhf_pipeline instead of hardcoding value as text-bison@001 (f51a930)llama-2-7b for the base reward model when tuning llama-2-13 with the preview.llm.rlhf_pipeline (227eab1)dsl.OneOf with multiple consumers cannot be compiled (#10452) (21c5ffe)DockerRunner logs (#10354) (86b7e23)tune-type label when uploading models tuned by preview.llm.rlhf_pipeline (708b8bd)preview.llm.rlhf_pipeline (f67cbfa)preview.llm.infer_pipeline (d8f2c14)create_custom_training_job_from_component (91f50da)preview.llm.rlhf_pipeline components for more readability (c23b720)preview.llm.rlhf_pipeline components for more readability (bcd5922)preview.llm.rlhf_pipeline components for more readability (a927984).after() referencing task in ParallelFor group (#10257) (11f60d8)_implementation.llm.chat_dataset_preprocessor (99fd201)model_checkpoint optional for preview.llm.infer_pipeline (e8fb699)DataflowFlexTemplateJobOp to GA namespace (now v1.dataflow.DataflowFlexTemplateJobOp) (faba922)dsl.OneOf (#10067) (2d3171c)dsl.importer argument is provided by loop variable (#10116) (73d51c8)persistent_resource_id to preview GCPC custom job components/utils (fc1f12b)v1 custom_job and gcp_launcher container code to preview (abf05f4)create_templated_custom_job for Templated Custom Job Launcher (e307545)PipelineTaskFinalStatus in tasks that use .ignore_upstream_failure() (#10010) (e99f270)dsl.Collected (#10069) (fcdff29)Add AutoML tabular and forecasting components to preview namespace
Fix bug where parent_model parameter of ModelUploadOp ignored
Fix circular import bug for model evaluation components
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here
Install python SDK (python 3.7 above) by running:
python3 -m pip install kfp kfp-server-api --upgrade
See the Change Log
kfp.client (#7458)dsl.PipelineTaskFinalStatus type (#9082) (7890227)TASK_FINAL_STATUS parameter type (#9080) (c01288d)data-filter-split feature back to the ImageTrainingJOb component which was removed previously (4a4a968)kfp-pipeline-spec package (#8896) (643f421)kfp-kubernetes library (#8950) (0c11cce)kfp-kubernetes (#8982) (c8cb5b5)dsl.ParallelFor (#8631) (b575950)Google Cloud Pipeline Components v2 is generally available!
Use v1 for GA offerings
Create preview namespace for pre-GA offerings (previously experimental )
Remove experimental namespace
Migrate many components to the v1 GA namespace
Migrate components to the preview namespace
preview.model_evaluation.ModelEvaluationFeatureAttributionOp
preview.model_evaluation.DetectModelBiasOp
preview.model_evaluation.DetectDataBiasOp
preview.dataflow.DataflowFlexTemplateJobOp
Add many new components:
v1.dataflow.DataflowFlexTemplateJobOp
v1.model.evaluation.vision_model_error_analysis_pipeline
v1.model.evaluation.evaluated_annotation_pipeline
v1.model.evaluation.evaluation_automl_tabular_feature_attribution_pipeline
v1.model.evaluation.evaluation_automl_tabular_pipeline
v1.model.evaluation.evaluation_automl_unstructure_data_pipeline
v1.model.evaluation.evaluation_feature_attribution_pipeline
Make GCPC artifacts usable in user-defined KFP SDK Python components ( Containerized Python Components recommended)
Change runtime base image to marketplace.gcr.io/google/ubuntu2004
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Depend on KFP SDK v2 (GCPC v2 is not compatible with KFP v1)
Set google-api-core<1.34.0 to avoid 900s timeout
Remove google-cloud-notebooks and google-cloud-storage dependencies
Assorted minor component interface changes
Assorted bug fixes
Change force_direct_runner flag to force_direct_runner_mode in experimental evaluation components to allow users to choose the runner of the evaluation pipeline
Support upload model with pipeline job id in UploadModel GCPC component
Change default value of prediction_score_column for AutoML Forecasting & Regression components to prediction.value
Change dataflow_disk_size parameter to dataflow_disk_size_gb in all model evaluation components
Remove aiplatform.CustomContainerTrainingJobRunOp and aiplatform.CustomPythonPackageTrainingJobRunOp components
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Fix experimental evaluation component runtime bugs
Add model evaluation pipelines:
v1.model.evaluation.vision_model_error_analysis_pipeline
v1.model.evaluation.evaluated_annotation_pipeline
v1.model.evaluation.evaluation_automl_tabular_feature_attribution_pipeline
v1.model.evaluation.evaluation_automl_tabular_pipeline
v1.model.evaluation.evaluation_automl_unstructure_data_pipeline
v1.model.evaluation.evaluation_feature_attribution_pipeline
Make GCPC artifacts usable in user-defined KFP SDK Python Components and add documentation
Change force_direct_runner flag to force_direct_runner_mode in experimental evaluation components to allow users to choose the runner of the evaluation pipeline
Add experimental AutoML Forecasting Seq2Seq and Temporal Fusion Transformer pipelines
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
GCPC v2 reference documentation improvements
Change GCPC base image to marketplace.gcr.io/google/ubuntu2004
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Fix dataset components
Fix payload sanitation bug in google_cloud_pipeline_components.v1.batch_predict_job.ModelBatchPredictOp
Assorted experimental component bug fixes (note: experimental namespace will be removed in a future pre-release)
Support sparse layer masking feature selection for experimental.automl.tabular classification/regression components
Support sparse layer masking feature selection for experimental.automl.tabular classification/regression components
Fixes for GCPC v2 reference documentation
Fix experimental.dataflow.DataflowFlexTemplateJobOp component
Remove unused SDK dependency on google-cloud-notebooks and google-cloud-storage
Upgrade Notes with notices and breaking changes
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here.
Install kfp-server-api package (python 3.7 above) by running:
python3 -m pip install kfp-server-api==2.0.0-beta.2 --upgrade
Refer to:
NOTE, kfp python SDK is NOT included and released separately.
client.run_pipeline with pipeline_id and version_id gives error (#9191) (91277b1)display_name and description in @dsl.pipeline decorator (#9153) (91abbea)Add experimental.dataflow.DataflowFlexTemplateJobOp component
Remove aiplatform.CustomContainerTrainingJobRunOp and aiplatform.CustomPythonPackageTrainingJobRunOp components
Migrate other aiplatform.automl_training_job , aiplatform.ModelUndeployOp , aiplatform.EndpointDeleteOp , and aiplatform.ModelDeleteOp components to the v1 namespace
Deduplicate component definitions between experimental and v1 namespaces
Upgrade Notes with notices and breaking changes
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here.
Install kfp-server-api package (python 3.7 above) by running:
python3 -m pip install kfp-server-api==2.0.0-beta.1 --upgrade
Refer to:
NOTE, kfp python SDK is NOT included and released separately.
data-filter-split feature back to the ImageTrainingJOb component which was removed previously (4a4a968)kfp-pipeline-spec package (#8896) (643f421)kfp-kubernetes library (#8950) (0c11cce)kfp-kubernetes (#8982) (c8cb5b5).dsl attribute on kfp module object (#9048) (0cbcebc)Change base image to ubuntu OS
Set google-api-core<1.34.0 to avoid 900s timeout
Upgrade Notes with notices and breaking changes
To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here.
Install kfp-server-api package (python 3.7 above) by running:
python3 -m pip install kfp-server-api==2.0.0-beta.0 --upgrade
Refer to:
NOTE, kfp python SDK is NOT included and released separately.
dsl.ParallelFor (#8631) (b575950)latest label. (#8448) (7138072)kfp dsl compile to compile components (#8371) (4cc0e80)kfp component build only produces empty requirements.txt (#8372) (54922b3)Release of GCPC v2 beta
Supports KFP v2 beta
Experimental components that already in v1 folder are removed
Experimental components that are not fully tested (e.g. AutoML, Model Evaluation) are excluded for now, will be added in future releases
Even though the GCPC package's version is v2, the components under v1 folder have no interface change, so the those components' version remain as v1, decoupled from package version.
Nothing published for this version
Apply latest GCPC image vulnerability resolutions (base OS and software updates)
Patch 5de4d78: unpin google-api-core version
Patch cb7d9a8: Update import_model_evaluation so models with 100+ labels will not import confusion matrices at every threshold
Add data-filter-split feature back to the ImageTrainingJob component
Set google-api-core<1.34.0 to avoid 900s timeout
Change base image to ubuntu OS
Set google-api-core<1.34.0 to avoid 900s timeout
Fix AutoML Table pipeline failing on importing model evaluation metrics
Fix default value issue in bigquery query API
Nothing published for this version
Cherrypick e358dee2f8d5c01580438ee54988f01fc3f16a7c and snap a new release
- Fix images for BQML components
Cherrypick d1f1ee9f2bbd09df7ea6ab51b21f07ba5f86c871 and snap a new release
Fix aiplatform & v1 batch predict job to work with KFP v2
Fix aiplatform & v1 batch predict job to work with KFP v2
Release Structured Data team's updated components and pipelines
Support a HyperparameterTuningJobWithMetrics type to take execution_metrics path
Release Structured Data team's updated components and pipelines
Fix aiplatform serialization
Release Structured Data team's updated components and pipelines
Add components for natural language: training TFHub model and preprocessing component for batch prediction
Fix aiplatform & v1 batch predict job to work with KFP v2
Fix aiplatform & v1 batch predict job to work with KFP v2
Fix serialization for aiplatform components
Update Dataproc doc links
Update tags in Structured Data team's forecasting pipelines
Propagate vertex system labels to the downstream resources
Propagate vertex system labels to the downstream resources
Release Structured Data team's updated components and pipelines
Fix Dataproc component doc to indicate that batch_id is optional
Simplify create_custom_training_job_op_from_component
Fix list and dict types for converted aiplatform components
Support uploading for model versions for ModelUploadOp
Support uploading for model versions for ModelUploadOp
Add text classification data processing component and training component
Propagates vertex system labels to the downstream resources for batch prediction job
Add DataprocBatch resource to gcp_resources output parameter
Add DataprocBatch resource to gcp_resources output parameter
Support serving default in bq export model job op
Temporary fix for artifact types
Temporary fix for artifact types
Sync GCPC staging to prod to include AutoML model comparison and prophet pipelines
Update documentation for Eval components
Update HP tuning sample notebook
Improve folder structure for evaluation components
Model Evaluation, rename EvaluationDataSplitterOp to TargetFieldDataRemoverOp, rename ground_truth_column to target_field, rename class_names to class_labels, and remove key_columns input
Add model input to vertex ai model evaluation component
Bigquery: Update public doc for evaluate model per customer feedback
Bigquery: Update public doc for evaluate model per customer feedback
Add Infra Validation remote runner
Add notification v1 doc to the v1 page
AutoML: Sync GCPC staging to prod to include bug fix for built-in algorithms
Convert all v1 components into individual launchers and remote runners
Add notification v1 doc
Convert all v1 components into individual launchers and remote runners
Update AutoML Tables components to have latest SDK features
Add support for staging Dataflow options (sdk_location and extra_package)
AutoML: Sync GCPC staging to prod to include recent API changes
AutoML: Sync GCPC staging to prod to include recent API changes
TensorBoard: Make some input parameters optional to provide better user experience
TensorBoard: Make some input parameters optional to provide better user experience
One column per quarter.