kfp

Bug Fixes

• sdk: fix(sdk): Align the versions on kfp-kubernetes install (#12283)

Bug Fixes

• backend/sdk: Fix REST API outputs for pipeline versions with invalid platform specs (#12183) (0cafb3d)
• sdk: Set spec.description when compiling to Kubernetes manifests (#12132) (5ffddad)

Features

• backend: Add a mutating webhook for the PipelineVersion kind (#11782) (c9be64d)
• backend: Add the ability to set a proxy for accessing external resources (#11771) (6e3548f)
• backend: Add types for KFP Kubernete Native API (#11672) (0d9a7b0)
• backend: Create a validating webhook for the PipelineVersion kind (#11774) (2efcde5)
• backend: implement logs as artifacts (#11762) (cd3e747)
• backend: implement logs as artifacts + CI updates (#11809) (464ca39)
• backend/sdk: Add input parameterization for various k8s resources (#11770) (fd1b48b)
• proto: Add TTL fields to KFP IR yaml proto (#11758) (c5aba41)
• sdk: add upload pipeline and upload pipeline version from pipeline function (#11804) (1ad4f60)

Bug Fixes

• backend: Fix run submissions with OwnerReferencesPermissionEnforcement on (#11821) (69ba50b)
• backend: fixed Dockerfile (#11841) (d38418e)
• backend: Include missing go.mod for cacheserver/viewercontroller images (#11776) (715ed40)
• components: Set tensorboard_experiment_name to random uuid when uploading plots in Starry Net pipeline (cc56d04)
• deployment: Update kustomize manifests to use new label and patch syntax (#11733) (230c1b8)
• deps: widen urllib3 upper bound to <3.0.0 (#11819) (866ff35)
• docs: Remove Podman as backend README pre-req (#11824) (88cff55)
• docs: Use the latest driver and launcher images in the dev environment (#11820) (92e4921)
• local: warn about oci:// not supported too (#11794) (564522c)
• metadata-writer: use mlmd_store.get_context_types() instead of workaround (#11753) (35041ef)
• sdk: Add SDK support for setting resource limits on older KFP versions (#11839) (f9d487c)
• sdk: allow google-cloud-storage < 4 (#11735) (bd4fc5c)
• sdk: avoid conflicting component names in DAG when reusing pipelines (#11071) (d1b15ef)
• tests: free up space in some test runners (#11818) (478ca08)
• minio fsgroup for popular clusters (#11734) (8d0ae53)

Other Pull Requests

• Fix Integration tests (0359551)
• add list or dict support for add toleration json (fb18235)
• add backend support for toleration lists. (90909fc)
• switch selenium image to ghcr (7529bbe)
• add missing release note updates to sdk main branch (#11842) (611d582)
• fix component retry test (#11836) (598826e)
• chore: add cleanup resources to sdk execution tests (#11823) (eee4986)
• update driver & launcher image handling (#11533) (38a4653)
• chore: add image builds for default branch (#11800) (eacb586)
• fix setup env for kfp k8s lib tests (#11798) (f10c7bf)
• Handle optional pipeline inputs in the driver (#11788) (bb7a108)
• Fix recurring run output when always using latest (#11790) (048f283)
• increase stale action timers (#11792) (ade8a2d)
• Fix PSS restricted warnings (#11751) (01999b8)
• fix(CI) Github action is vulnerable to code execution via comment body (#11772) (95c3f2c)
• Fix Istio sidecar injection by moving from annotations to labels (#11750) (df4e9c2)

Features

• Add support for dsl.PIPELINE_TASK_EXECUTOR_OUTPUT_PATH_PLACEHOLDER and dsl.PIPELINE_TASK_EXECUTOR_INPUT_PLACEHOLDER #10240
• Add support for local component execution using local.init(), DockerRunner, and SubprocessRunner

Breaking changes

Deprecations

Bug fixes and other changes

• Support .after() referencing task in a dsl.ParallelFor group #10257
• Support Python 3.12 #10271

Documentation updates

⚠ BREAKING CHANGES

• sdk: stop auto-populating metrics as dag output (#11362)
• components: Deprecate preview.custom_job module
• sdk: Pin kfp-pipeline-spec==0.4.0, kfp-server-api>=2.1.0,<2.4.0 (#11192)

Features

• sdk/backend: Add support for placeholders in resource limits (#11501) (7c931ae)
• Introduce cache_key for cache key customization (#11434) (50b367f)
• api: add PipelineConfig to api to re-implement pipeline-level config (#11333) (c2f5649)
• backend: add configurable S3 path style support (#11246) (85fdd73)
• backend: Add Parallelism Limit to ParallelFor tasks. Fixes #8718 (#10798) (b7d8c97)
• backend: implement subdag output resolution (#11196) (c5b787a)
• backend: Remove PipelineSpec Template storage from ObjStore responsibilies. Fixes #10509 (#10790) (374b18b)
• cli: expose existing_token client property (#11400) (35793be)
• component: Created Snowflake data unload component (#11349) (22e7780)
• component: execute in a virtual env (#11326) (df28e89)
• components: Add reservation_affinity support in v1.create_custom_training_job_from_component (c84241b)
• components: add strategy to v1 GCPC custom job components/utils (1cdd648)
• components: Deprecate preview.custom_job module (abbd915)
• frontend/backend: Allow the ability to sort experiments by last run creation. Fixes #10884 (#11163) (db8669c)
• sdk: add PipelineConfig to DSL to re-implement pipeline-level config (#11112) (df4d787)
• sdk: Allow disabling default caching via a CLI flag and env var (#11222) (3f49522)
• sdk: Pin kfp-pipeline-spec==0.4.0, kfp-server-api>=2.1.0,<2.4.0 (#11192) (dfd4cc1)
• sdk: stop auto-populating metrics as dag output (#11362) (8d018af)
• sdk/backend: enable parameterization of container images (#11404) (22e85de)
• testing: use kustomize to patch deployments before deploy (#11294) (be863a8)
• add fields in SinglePlatformSpec (#11299) (a0d313e)
• sdk: support dynamic machine type parameters in pipeline task setters (#11097) (70aaf8a)
• workflows: use built images in Github workflows (#11284) (1550b36)

Bug Fixes

• backend: Allow initializing the Kubernetes client with a kubeconfig (#11443) (87bdb7c)
• backend: handle client side HTTP timeouts to fix crashes of metadata-writer. Fixes #8200 (#11361) (94a21cc)
• backend: modelToCRDTrigger was not including periodic schedule correctly (#11475) (97acacb)
• backend: randomizing output uri path to avoid overwriting. Fixes #10186 (#11243) (219725d)
• backend: remove unused function argument (#11425) (7f2278f)
• backend: return error properly (#11424) (13f83cf)
• backend: set default value to true for ForcePathStyle (#11281) (391de8c)
• backend: stop heartbeat status updates for ScheduledWorkflows. Fixes #8757 (#11363) (9ccec4c)
• backend: Synced ScheduledWorkflow CRs on apiserver startup (#11469) (d21fca6)
• backend: upgrade PyYMAL to fix metadata_writer build error (#11231) (a4119a6)
• backend: upgrade various old dependencies (#11448) (803d7a8)
• backend: Use an Argo Workflow exit lifecycle hook for exit handlers (#11470) (3059f7c)
• components: Fix create_custom_training_job_from_component default location (04d600b)
• components: remove default prediction column names in evaluation regression component to fix issues with bigquery data source (753a2f1)
• frontend: Detailed information of nodes is not displayed when clicking the node. Fixes #11325 (#11493) (028d81b)
• frontend: first time choosing a pipeline definition is VERY slow. Fixes #10897 (#11130) (cfb3b31)
• frontend: Fix the frontend image build with Node 22 (#11524) (#11525) (2e47604)
• sdk: accelerator type setting in kfp (#11373) (64e3900)
• sdk: Add error handling. Fixes #11164 (#11356) (4a64fe9)
• sdk: Backport fixes in kubeflow/pipelines#11075 (#11392) (6ebf4aa)
• tests: remove redundant integration test wf (#11322) (6a35ee5)
• ui: Disable GKE Metadata as default Fixes: #11247, fixes #11260 (#11403) (23f718d)
• ui: Fixes V1 Run detail unnecessary reloading. Fixes #10590 (#11214) (eee095e)
• workflows: patch reversed launcher / driver in workflow matrix (#11238) (ceeda01)

Other Pull Requests

• switch release/api generator images to ghcr (#11528) (#11530) (33fca73)
• add remaining dockerfiles to build workflow (#11522) (682d3ac)
• Fix the failing exit handler SDK execution tests (#11519) (8bce9c4)
• nominate reviewers for backend (#11508) (56e6116)
• nominate approver & reviewer for backend (#11507) (81ebd7f)
• feat[frontend]: implement artifact-repositories configmap support (#11354) (467f30c)
• Add-Create-Experiment-button-when-selecting-experiement-final (#11332) (c5f162d)
• adding chore as a new template. Fixes #11263 (#11317) (f256d86)
• correct lastrun unittest timestamps (#11270) (6f6c8ae)
• replaced deprecated image repos with registry.k8s.io (#11152) (d23b72b)

Features

• Add support for a Pythonic artifact authoring style #9932
• Support collecting outputs from conditional branches using dsl.OneOf #10067

Breaking changes

Deprecations

• Add notice of Python 3.7 support removal on April 23, 2024 #10139

Bug fixes and other changes

• Fix type on dsl.ParallelFor sub-DAG output when a dsl.Collected is used. Non-functional fix. #10069
• Fix bug when dsl.importer argument is provided by a dsl.ParallelFor loop variable. #10116
• Fix client authentication in notebook and iPython environments #10094

Documentation updates

Features

• api: Add new ResourceSpec proto fields to support dynamic values. (#11075) (83dcf1a)
• backend: mount EmptyDir volumes for launcher write locations (#10857) (65839ce)
• backend: move comp logic to workflow params (#10979) (0e37fd6)
• Backend + SDK: Update kfp backend and kubernetes sdk to support EmptyDir (#10913) (7506a8e)
• component: internal (383cbcd)
• component: internal (8041ea8)
• component: internal (c06d9ae)
• component: internal (ad03bb6)
• components: Add role_field_name and model_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema (de346d5)
• components: Add Starry Net forecasting pipeline to public preview (3a0566e)
• components: create infer preprocessor component (e9d6876)
• components: Create the write_user_defined_error function (454a654)
• components: create utility class for preprocessors and use it in rlhf preprocessor and infer preprocessor (cd16a33)
• components: internal (e8f0208)
• components: internal (6f62203)
• components: internal (2eb9bec)
• components: internal (c6f658b)
• components: internal (e128bdb)
• components: internal (abe5257)
• components: internal (522b593)
• components: internal (461d892)
• components: release LLM Model Evaluation image version v0.7 (4f36fe3)
• components: Retry on batch prediction internal errors in AutoSxS (5d9f4ab)
• components: Support dynamic machine parameters in preview.custom_job.utils.create_custom_training_job_from_component (e44dfa7)
• components: Support dynamic machine type paramters in CustomTrainingJobOp (#10883) (b57f9e8)
• components: Support dynamic values for boot_disk_type, boot_disk_size in preview.custom_job.utils.create_custom_training_job_from_component (7b7918e)
• components: Support parsing Gemini BP outputs in AutoSxS pipeline (b4f91a3)
• components: Update Starry Net image tags (bf5104f)
• components: Use GetModel integration test to manually test write_user_defined_error function (609c637)
• components: use preprocessor utility methods for the upload model graph (7908ed6)
• frontend&backend: Add UI support for object store customization and prefixes (#10787) (6723d3d)
• GH workflow: migrate periodic functional tests to GH actions (#10751) (c4d7ec3)
• internal: Adding proto field to support regional quota (c8f08ba)
• kubernetes_platform: Add empty dir mount (#10892) (10aaf43)
• kubernetes_platform: Update kubernetes_platform go package to include EnabledSharedMemory (#10703) (7c63599)

Bug Fixes

• components: add check and add log to call out the fallback to the default model checkpoint and remove the model checkpoint check condition in RLHF GCPC (92c3178)
• components: Add input param autorater_prompt_parameters to online_evaluation_pairwise component (cf7450b)
• components: Add staging and temp locations to prophet trainer component (00440f7)
• components: Fix to model batch explanation component for Structured Data pipelines (289f64f)
• components: Pass moddel name to eval_runner to process batch prediction's output as per the output schema of model used (0d3e79a)
• components: Remove unused import function_based from infer pipeline (e369bd3)
• components: Use instance.target_field_name format for text-bison models only, use target_field_name for gemini models (1a07ffa)
• docs: add note about protoc dependency and version (#10895) (d911c8b)
• docs: IR -> Pipeline Spec (#11031) (1e95eb6)
• frontend: fixes filter pipeline text box shows error when typing anything in it. Fixes #10241 (#11096) (51d2c92)
• frontend: reduce list run latency (#10797) (768ece4)
• frontend: retrieve archived logs from correct location (#11010) (2e6e634)
• sdk: Add required auth scopes to RegistryClient for GCP service accounts credentials (#10819) (04b4cad)
• sdk: Kfp support for pip trusted host (#11151) (3efa029)
• ui: fixes empty string value in pipeline parameters (#11175) (e9c77ec)
• Basic sample tests - sequential is flaky (#11138) (e1d172b)
• Incorrect typing in samples/cores/loop_parallism and fixing loop_parameter examples (#11062) (1612dac)
• re-enable exit hanler test. (#11100) (000ef60)
• kubernetes_platform: fix api-generator docker mount for SELinux (#10890) (e69078b)
• manifests: Move metacontroller to the top in kustmization.yaml (#10669) (4e9fe75)
• sdk: Throw 'exit_task cannot depend on any other tasks.' error when an ExitHandler has a parameter dependent on other task (#11005) (08185e7)

Other Pull Requests

• fix: reduce executor logs (#11169) (d64554b)
• correct artifact preview behavior in UI (#11059) (2c91fb7)
• docs:fixing broken links in readme (#11108) (a3adf94)
• Fix view edit cluster roles (#11067) (43cdc20)
• add dev docs for kfp/sdk (#11046) (eea7d48)
• No public description (e3a4980)
• Upgrade go version to 1.21 (#10911) (bdc3bb1)
• Expose starry_net yaml to GitHub (#10943) (f1e2314)

Features

• Support PipelineTaskFinalStatus in tasks that use .ignore_upstream_failure() #10010

Breaking changes

Deprecations

Bug fixes and other changes

Documentation updates

Features

• backend: add namespace & prefix scoped credentials to kfp-launcher config for object store paths (#10625) (5e0f9b1)
• backend: Merge kfp-tekton backend code (#10678) (60a443e)
• backend: Upgrade argo to v3.4.16 (#10568) (809d576)
• components: Add model name preprocess component; Use publisher model if user uploaded model is non-tuned (084f2c2)
• components: add resolve_machine_spec and resolve_refined_image_uri to rlhf_preprocessor component (2a8d39e)
• components: add resolve_reference_model_metadata to rlhf_preprocessor component (92a7969)
• components: add task_type as a parameter to rlaif (64d288a)
• components: Added support for text-bison@002 to preview.llm.rlhf_pipeline (2f27751)
• components: AutoSxS GA pending release (aee464c)
• components: Expand regions supported by preview.llm.rlhf_pipeline (22a98d9)
• components: internal (a4f01b7)
• components: Introduce placeholders: SERVICE_ACCOUNT_PLACEHOLDER, NETWORK_PLACEHOLDER, PERSISTENT_RESOURCE_ID_PLACEHOLDER and ENCYRPTION_SPEC_KMS_KEY_NAME_PLACEHOLDER. In addition, use PERSISTENT_RESOURCE_ID_PLACEHOLDER as the default value of persistent_resource_id for CustomTrainingJobOp and create_custom_training_job_op_from_component. With this change, custom job created without explicitly setting persistent_resource_id will inherit job level persistent_resource_id, if Persistent Resource is set as job level runtime (67d3cd6)
• components: migrate function_based convert_to_delimited_string to rlhf_preprocessor component (efefe34)
• components: migrate function_based resolve_num_microbatches to rlhf_preprocessor component (ee28c72)
• components: migrate function_based resolve_regional_endpoint to rlhf_preprocessor component (f175c71)
• components: Move AutoSxS pipeline to v1 directory (d919ae7)
• components: Move ModelImportEvaluationOp component to preview namespace (33db128)
• components: Report TensorBoard metrics for preview.llm.rlhf_pipeline in real time (3d8069b)
• components: Use larger base reward model when tuning t5-xxl with the preview.llm.rlhf_pipeline (ff7f660)
• components: Use larger base reward model when tuning text and chat variants of bison@001 with the preview.llm.rlhf_pipeline (ac39931)
• components: use rlhf_preprocessor to replace the current value_exists call in rlhf (c967d9f)
• kubernetes_platform: Update kubernetes_platform go package to include generic ephemerl volume (#10602) (2fc1492)
• kubernetes_platform: Update kubernetes_platform go package to include node affinities and pod (anti)affinities (#10583) (4f8cae2)
• sdk+backend: Add support for generic ephemeral volume (#10605) (3fb76a8)

Bug Fixes

• backend: Update backend common code and integration tests with updated API Service Params (#10640) (8b2a099)
• Backend + SDK: Add missing optional field to SecretAsVolume and … (#10550) (a78dc77)
• components: Ensure preview.llm.rlhf_pipeline runs if no tensorboard_id is provided (ff0d0a7)
• components: Fix image version parameter in rl pipelines (cef6e51)
• components: Fix model eval import error in text generation/classification eval pipeline (7630f85)
• components: Make AutoSxS autorater_prompt_parameters required (df20088)
• components: remove default prediction column names in evaluation classification component to fix incorrect column names for bigquery data source (54f2e45)
• components: Remove the unused functions from function_based (e052dc8)
• components: Remove the unused generate_default_instruction and resolve_upload_location from function_based (e9d8764)
• components: Remove the unused resolve_data_paths from function_based (c386913)
• components: Update service account comment (bf444ac)
• metadata envoy: upgrade envoy and config from 1.12 to 1.27 (#10589) (96aaad9)

Other Pull Requests

• No public description (cab99f7)
• No public description (79d0a5c)
• Update loop_output.py example for the new parallel loop type requirement (#10637) (afddae9)

Features

• Add support for dsl.If, dsl.Elif, and dsl.Else control flow context managers; deprecate dsl.Condition in favor of dsl.If #9894
• Create "dependency-free" runtime package (only typing_extensions required) for Lightweight Python Components to reduce runtime dependency resolution errors #9710, #9886

Features

Breaking changes

Deprecations

Bug fixes and other changes

Documentation updates

• Fix PyPI/README description #9668

⚠ BREAKING CHANGES

• SDK: SDK client v2beta1 API integration. Fixes #8706 (#9112)
• sdk: Implement Graph Component (#8179)
• sdk: make CLI output consistent, readable, and usable (#7739)
• sdk: promote v2 compiler command and deprecate v1 command (#7508)
• sdk: Support submitting pipeline IR in yaml format via kfp.client (#7458)
• sdk: use yaml for IR package (#7431)
• sdk: rename commands, arguments parameters to command, args (#7391)
• sdk: throw error for output_component_file (#7390)
• sdk: move v2 to main namespace (#7376)
• sdk: move v1 to deprecated folder (#7291)
• sdk: drop support for python 3.6 (#7303)
• Use Argo Emissary Executor instead of Docker by default. Partial #5718 (#7137)
• sdk.v2: Block task dependency referencing tasks inside a sibling condition or loop group. (#7050)
• sdk: Deprecate V2 compatible mode in v1 compiler (#6958)
• sdk.v2: Merge v2 experimental change back to v2 namespace (#6890)

Features

• backend: Add nodeSelector support. Closes #8959. (#9132) (276f922)
• backend: add parameter verification before submitting the workflow (#9228) (52c4c6f)
• backend: add schema version check (#9447) (08494a2)
• backend: add support for resource requests. Closes #7047. Closes #6354. (#9088) (37dac45)
• backend: avoid downcast and use ExecutionSpec if possible (#9232) (6c72d95)
• backend: Implement aws-go-sdk crendentials to support IRSA for s3 in V2 (#9502) (8a3215e)
• backend: support pvc creation, deletion and mount; fixes #8708 (#9106) (761a592)
• backend: support secrets (#9178) (b8caf81)
• backend: updates compiler to include kubernetes-specific specs in argo workflow (#9059) (a3e795a)
• component: Adds Vertex Pipelines prompt validation component (249f235)
• components: add a check for model garden models and skip eval import (1782ee2)
• components: Add a remote runner for pipelines (274ac8b)
• components: Add force_dircet_runner flag to pipeline options (f167e15)
• components: Add instructions for formatting BiasConfig input proto using json_format in Model & Data bias component (3a89044)
• components: Add logic in import_model_evaluation to accept explanations from slices (4397b8c)
• components: Add machine type and encryption key for Error Analysis Component (0293fe1)
• components: Add machine type for Feature Extractor component. Deprecate Dataflow-related inputs (4e85352)
• components: add preview directory to GCPC (b167f87)
• components: Add slicing_specs input to classification evaluation (6a55449)
• components: Add utility function for constructing the serverless custom job launcher spec (6fb10f3)
• components: Allow batch predict component to add missing bq:// schemes (8cc0550)
• components: Allow importing LLM eval metrics (ecc4ae0)
• components: Change ResolveVertexModelOp name to GetVertexModelOp (6a916e8)
• components: Create ResolveVertexModelOp component (595ae8e)
• components: Define Data Bias component for detecting data bias metrics (7ab5d97)
• components: Define Model Bias component for detecting model bias metrics (7022007)
• components: Derive image tag from Eval container version. Align Eval image version for all Eval components in GCPC (63936c0)
• components: Fix batch prediction job proto bigquery payload sanitization (a7b8fd0)
• components: Import user slices to the model evaluation importer (8c9b716)
• components: make GCPC artifacts usable in KFP Python components (6104ff6)
• components: Move model eval components (48668bc)
• components: Move model evaluation components to v1 (f2ad300)
• components: Move model evaluation feature attribution component to preview namespace (dc6f431)
• components: Provide model evaluation pipelines (9ce827e)
• components: Release the Seq2Seq and Temporal Fusion Transformer pipelines (377b961)
• components: Remove experimental evaluation components (69ea826)
• components: Remove GetVertexModelOp, to add to preview later (85f6969)
• components: Sagemaker V2 Hosting components and tests (#9243) (4818e84)
• components: SageMaker V2 model monitor component and testing (#9253) (07e67bb)
• components: SageMaker V2 model monitor component release (#9368) (29444f9)
• components: Support Instance Config in kubeflow model_batch_predict component (3166418)
• components: Support model garden models in GetVertexModelOp (0b32899)
• components: Update eval component with the latest GCPC container image version (a3418b3)
• components: Update GCPC image version in two Eval importer components (1f4d943)
• components: Update import_model_evaluation so models with 100+ labels will not import confusion matrices at every threshold (cb7d9a8)
• frontend: make container name configurable (#9433) (bb9dc67)
• sdk: support compiling components with dsl.PipelineTaskFinalStatus type (#9082) (7890227)
• sdk: support extracting input/output descriptions from component/pipeline docstrings (#9156) (946c51b)
• sdk: support indexing into top-level artifact metadata in Container Components (#9131) (5828632)
• sdk: Support PIPELINE_ROOT_PLACEHOLDER (#9134) (5e98b3b)
• sdk: Support setting cpu/memory requests. (#9121) (a6ef77b)
• sdk: Support submitting pipeline with platform config. (#9140) (952e073)
• sdk: support writing TASK_FINAL_STATUS parameter type (#9080) (c01288d)
• SDK: SDK client v2beta1 API integration. Fixes #8706 (#9112) (07d6c18)
• Add cpu_request and memory_request in pipeline_spec. (#9063) (d2db64b)
• add TASK_FINAL_STATUS to PipelineSpec ParameterTypeEnum (#9061) (902e2a9)
• API: Update IR go package (#8614) (c30648a)
• backend: add run support for v2 api (#8647) (d524464)
• backend: add support for "finished_at" and others in filter. Fixes #8654 (#8662) (6ee7677)
• backend: Add support for converting v2 recurring run to scheduled workflow in template (#8521) (7e7e1af)
• backend: add support for environment variables. Fixes #8704 (#8886) (faca1e5)
• backend: Add support for recurring run v2 api (#8634) (d0f0ab3)
• backend: Make cache-server webhook port number configurable (#8401) (3824c5b)
• backend: Refactor apiserver, add Pipeline and other v2beta services (#8633) (135dc64)
• backend: support optional and default parameters. Fixes #8716 (#8765) (89bdac9)
• backend: support yaml with platform-specific specs (#8983) (a9ead5a)
• component: add evaluation_importer_gcp_resources to evaluated annotation importer component (d17916f)
• component: change Error Analysis component to use CustomJob launcher (fd67277)
• component: change Feature Extractor component to use CustomJob launcher (a725192)
• component: Create component definition for GetVertexDataset (2d29137)
• component: Create YAML definition for Feature Extractor Error Analysis component (e7c1a16)
• component: define ImportEvaluatedAnnotation component (8c9fdee)
• component: init container backend for evaluated annotation importer (74b86dc)
• component: Merge annotation and error analysis in evaluated annotation importer (10e7c55)
• component: Update input args in dataset preprocessor and evaluated annotation (9a32f76)
• components: Add data-filter-split feature back to the ImageTrainingJOb component which was removed previously (4a4a968)
• components: Add components for natural language: training TFHub model and preprocessing component for batch prediction (60b2d5f)
• components: Add logs to Batch Prediction creation process (b3de27a)
• components: Create GCPC v2 beta (834d966)
• components: Create new component YAML definition for computing error analysis annotations (b7d03cb)
• components: Create new component YAML definition for computing evaluated annotations (fe38716)
• components: Create new component YAML definition for Evaluation Dataset Preprocessor Error Analysis (6800d21)
• components: Dataflow Flex Template component (6101ff1)
• components: Delete tfp_anomaly_detection (f2537d4)
• components: Import model evaluation slices 50 at a time (39e4c36)
• components: Propagate cost labels of GCP resources (7466803)
• components: Update evaluation components to v0.9 and add input for quantile forecasting (74237dc)
• components: update EvaluationDatasetPreprocessorOp to use KFP v2 Python definition (deed177)
• sdk: add experiment_id parameter to create run methods (#9004) (3e1f3c6)
• add kubernetes platform-specific protos, generated Go code, and Python package (#8888) (a4abf27)
• add NodeSelector message to kfp-kubernetes proto (#8981) (cd15b7a)
• add platform-specific configuration protos to kfp-pipeline-spec package (#8896) (643f421)
• implement kfp-kubernetes library (#8950) (0c11cce)
• support nodeSelector in kfp-kubernetes (#8982) (c8cb5b5)
• backend: Add ExecutionClient to access ExecutionSpec (#8050) (e8abec2)
• backend: Add v2 api for experiments (#8480) (6e364d3)
• backend: authorize readArtifacts and ReportMetrics endpoints (#7819) (1caa8e0)
• backend: Expand ExecutionSpec Interface (#7766) (06c9310)
• backend: fix partner_id in GCP Marketplace application (#8426) (3fca792)
• backend: Implement RuntimeConfig in backend (#8085) (2b556ec)
• backend: Implement RuntimeConfig in backend for jobs(RecurringRun) (#8152) (f25127f)
• backend: Include namespace in Pipeline API model. Part of #4197 (#8375) (7c114a0)
• backend: update minio-go to support irsa (#7946) (7923ba3)
• backend: Upload namespaced pipeline definitions. Part of #4197 (#8511) (931c14a)
• cloud: Update JSON templates for evaluation (88a2b80)
• components: Add DataprocBatch resource to gcp_resources output parameter (6e95e89)
• components: Add display name to import model evaluation component (9b12f6e)
• components: Add evaluation forecasting default of 0.5 for quantiles (21b8479)
• components: Add Infra Validation remote runner (a06a4e6)
• components: add KServe component flag to disable storing CRD status into output parameters (#8393) (24f8651)
• components: Add metadata field for pipeline resource name (82bf1ec)
• components: Add model input to vertex ai model evaluation component (35b13b3)
• components: add namespace to pvc creation (#8147) (031a7d9)
• components: Add nargs to allow for empty string input by component (555447f)
• components: Add special case in json_util.py where explanation_spec metadata outputs can have empty values (b8b77fd)
• components: Add text classification data processing component and training component (9675870)
• components: Add vertex_notification_email component on v1 folder (3b1bd08)
• components: Added BigQuery input support to evaluation components (bfa2dc6)
• components: Allow dataset paths list (5d06871)
• components: Create new tensorboard experiment creator component (a3c7c38)
• components: Give evaluation preprocessing components unique dataflow job names (c052bf8)
• components: Import model evaluation slices when available in the metrics (0d304c6)
• components: Internal change (bf21b7b)
• components: Introducing AWS SageMaker TrainingJob component v2 in Preview (#8158) (f162b24)
• components: Model Evaluation, rename EvaluationDataSplitterOp to TargetFieldDataRemoverOp, rename ground_truth_column to target_field, rename class_names to class_labels, and remove key_columns input (71c8e77)
• components: New sagemaker training job parameters (#8538) (6a6cfdb)
• components: propagates vertex system labels to the downstream resources (batch prediction job) (4410bf2)
• components: propagates vertex system labels to the downstream resources (bigquery job) (a506da6)
• components: propagates vertex system labels to the downstream resources (custom job) (69e1e4b)
• components: propagates vertex system labels to the downstream resources (dataflow python job) (4c2797e)
• components: propagates vertex system labels to the downstream resources (dataproc) (7fe4319)
• components: propagates vertex system labels to the downstream resources (Vertex AI Datasets) (9d9a8bb)
• components: propagates vertex system labels to the downstream resources (Vertex AI Endpoints) (f37086b)
• components: propagates vertex system labels to the downstream resources (Vertex AI HyperParameter Tuning Job) (cdd0626)
• components: propagates vertex system labels to the downstream resources (Vertex AI Model Registry) (56edebb)
• components: release Structured Data team's updated components and pipelines (b9a0499)
• components: release Structured Data team's updated components and pipelines (bd1538d)
• components: Release Structured Data team's updated components and pipelines (73034a9)
• components: Separate feature attribution from evaluation component to its own component (f9cf42a)
• components: Split evaluation component into classification, forecasting, and regression evaluation & create artifact types for google.__Metrics (8c74612)
• components: Support serving default in bq export model job op (582576f)
• components: Support uploading for model versions for ModelUploadOp (94bdce8)
• components: Sync gcpc staging code to prod (01c9ed9)
• components: Sync GCPC staging to prod to include AutoML model comparison (06761b9)
• components: Update AutoML Tables components to have latest SDK features (49fdac6)
• components: Update dataset_preprocessor component inputs (399b7be)
• components: Update evaluation container to v0.5 for data sampler and splitter preprocessing components (37ff321)
• components: Update evaluation container to v0.8.1 image (8d87bec)
• components: Update import_model_evaluation component to accept new google.___Metrics artifact types (8eb61ad)
• components: Update regression and forecasting to contain ground truth input fields (6ec9b71)
• components: Updated components and pipelines (e358dee)
• components: Updated components and pipelines (d1f1ee9)
• docs: use new sdk reference docs theme (#8212) (19cf904)
• frontend: Add KFPv2 namespace check (#8036) (9df8343)
• frontend: Add loading and error states for HTML and Markdown displays (#7989) (6cbc10b)
• frontend: Add parent labels to the Compare Table (#8024) (4be213a)
• frontend: Add pre-check for pipeline and pipeline version name at the UI (#8113) (58f3a60)
• frontend: Add refresh and run re-selection functionality (#8118) (d02857f)
• frontend: Add ROC Curve filter table and functionality (#8073) (23d549f)
• frontend: Add run parameters (#8105) (f96d3e9)
• frontend: Complete Sub-DAG implementation in KFPv2. (#8326) (21cf640)
• frontend: Create two-level dropdown (#7933) (832858b)
• frontend: Creating run from various entry points (#8190) (7d5985b)
• frontend: Display HTML and Markdown files (#7981) (cfe3278)
• frontend: Display metrics loading and error states (#8029) (4cb8562)
• frontend: Display multi-level dropdown on KFPv2 Run Comparison page (#7943) (53d8f7e)
• frontend: Display ROC curve (#8046) (cc3bd8b)
• frontend: Display scalar metrics table (#8026) (d3fe514)
• frontend: Display two-panel layout and confusion matrices (#7966) (617ad83)
• frontend: Implement aws-js-sdk crendentials to support IRSA for s3 (#8651) (9b9b3ac)
• frontend: Show error message for invalid input or missing parameters. (#8003) (303267f)
• frontend: Show pipeline details (DAG) for an existing run without pipeline / pipeline version. (#8116) (448b4ca)
• frontend: Support Clone Run with RuntimeConfig (#8093) (1257a35)
• frontend: Support cloning recurringRun in KFP v2 (#8652) (80c0dc5)
• frontend: Support creating recurringRun in KFP v2. (#8568) (a4a2488)
• frontend: Support namespaced pipelines from the UI. Closes #5084 (#8831) (6996947)
• frontend: v2 Dynamic Sub-DAG (I) (#8278) (440f4e1)
• google-cloud: add explanation_metadata_artifact input arg to custom Batch Prediction component. (ad364d9)
• google-cloud: Add ground_truth_column input argument to data splitter component. (17d62e6)
• google-cloud: Change evaluation preprocessing component output type. (059afb0)
• google-cloud: Modify GCPC evaluation templates. (176d3ff)
• google-cloud: Release eval changes with v0.3 (d2fd41b)
• google-cloud: Release notes for 1.0.12 and 1.0.13 (e42d9d2)
• google-cloud: Update eval container version for eval components (065f732)
• IR: add is_optional field for input artifacts (#8571) (998f52d)
• IR: change is_list_of_artifacts proto field to is_artifact_list (#8522) (041f446)
• samples: Full Kubeflow E2E sample using KFPv2. (#8303) (91fe29f)
• sdk: Accepts higher PyYAML versions (#8665) (6040334)
• sdk: add --build-image option to 'kfp components build' to allow users to skip docker build. Fixes #8382 for 2.0 (#8387) (5b75d75)
• sdk: Add comments to IR YAML file (#8467) (49db63c)
• sdk: add compilation logic for google artifact types (support custom artifact types pt. 2) (#8232) (e371fa3)
• sdk: Add default registry context (#7948) (aab7fda)
• sdk: add is_list_of_artifacts field to PipelineSpec [lists of artifacts support pt. 2] (#8466) (6cd7cbc)
• sdk: add pipeline job placholders for scheduled runs (#8447) (e97fe8e)
• sdk: add retry policy support to kfp v2 (#7867) (850a750)
• sdk: add runtime logic for custom artifact types (support for custom artifact types pt. 3) (#8233) (166d6bb)
• sdk: add support for lists of artifacts in Python components [support lists of artifacts pt. 3] (#8465) (08b2408)
• sdk: add support for metadata placeholders (#8151) (88a1b31)
• sdk: add workaround for google artifact (support custom artifact types pt. 4) (#8279) (8ab690f)
• sdk: Allow artifact inputs in pipeline definition. (#8044) (1dd06cd)
• sdk: deprecate .add_node_selector_constraint in favor of .set_accelerator_type (#8980) (e96754c)
• sdk: deprecate .set_gpu_limit in favor of .set_accelerator_limit (#8836) (e7ad073)
• sdk: enable dynamic importer metadata (#7660) (c353245)
• sdk: enable use of primitive placeholders in f-string (#8494) (b829e4a)
• sdk: expose IfPresentPlaceholder and ConcatPlaceholder to kfp.dsl (#8145) (c7a90a3)
• sdk: expose new ContainerSpec API (#8144) (35fccb1)
• sdk: Ignore upstream failure (#8838) (5684bfc)
• sdk: Implement Graph Component (#8179) (5ce4954)
• sdk: migrate v1 auth to v2 (#7789) (e14a784)
• sdk: rename kfp artifact attributes (support custom artifact types pt. 1) (#8191) (72c1d10)
• sdk: support compiling platform specific features (#8940) (a38d747)
• sdk: support for fan-in from dsl.ParallelFor (#8631) (b575950)
• sdk: support list of artifacts annotations [list of artifacts support pt. 1] (#8464) (e7c82c0)
• sdk: Support loading pipeline from yaml (#8209) (410e99c)
• sdk: support more than one exit handler per pipeline (#8088) (bdff332)
• sdk: support optional artifact inputs (#8623) (151e8a3)
• sdk: Support Parallelism in ParallelFor in KFP SDK V2 (#8146) (51bea09)
• sdk: Support pipeline outputs (#8204) (48574dc)
• sdk: Support using pipeline in exit handlers (#8220) (d883582)
• sdk: supporting bring your own container for arbitrary input and outputs (#8066) (04c827f)
• Upgrade argo-workflow to v3.3.8 (#8009) (8bee292)
• api: add runtime_config.parameter_values with string typed values (#7734) (5b64733)
• api: adds task name to PipelineTaskFinalStatus message (#7452) (541f2ac)
• api: Update IR with RetryPolicy (#7581) (27d5404)
• backend: Add ExecutionSpec Interface (#7531) (8cf022c)
• backend: add new endpoint that get pipeline by name (#7004) (81438f3)
• backend: Forbid unarchive runs that belog to archived experiment (#7147) (96a6a3b)
• backend: support IR YAML format in API (#7499) (4f398dc)
• backend: Upgrade go client for MLMD to 1.5.0. Partial #7029 (#7101) (ba59a64)
• backend: use cert-manager for cache server cert (#7843) (784f9fa)
• chore: add labels and error to the gcp_resources.proto (bc56df6)
• component: add BQ export model component (28c666f)
• component: adding bq evaluate model component (27d2ab5)
• component: Adds Vertex Pipelines notification email component. (e2687ce)
• component: remove training_filter_split, validation_filter_split, test_filter_split as they are not working (ff99a98)
• component: Support BigQuery drop model job components (affbc09)
• component: Support BigQuery ML advanced weights job component (f6b560a)
• component: Support BigQuery ML centroids job components (45cfb91)
• component: Support BigQuery ML reconstruction loss and trial info job components (d2f33c8)
• component: Support BigQuery ML weights job component (4183344)
• components: Add a KServe component (#6881) (72185ff)
• components: Add AutoML Tables default pipeline with evaluation (1265649)
• components: Add GetWorkerPoolSpecsOp and update sample notebook to include training new model from tuning results (30308ea)
• components: Add IsMetricBeyondThresholdOp util function for HP tuning job; add tests for all util components (37b3397)
• components: add nfs mount support for create_custom_training_job_op_from_component (0e316b6)
• components: Add util components for getting best trial/hyperparameters; delete HP Tuning Job lightweight python version to avoid potential confusion (a62bdbe)
• components: add utility library for google artifacts, and use the library for FPCs. Also fix BQML export model output param error. (a17e1d3)
• components: cleanup the deprecated fields in v1 model upload component. (9738726)
• components: create gcpc v1. (5baa487)
• components: Create new import model evaluation component (00eb97c)
• components: Expose reserved_ip_ranges to custom job component (89c741d)
• components: Implement cancellation propagation (61d9210)
• components: Implement delete_endpoint (1d3eaf7)
• components: Implement delete_model (284b46e)
• components: Implement undeploy_model component (3bc17be)
• components: Improve AutoML Tables default pipeline with evaluation component (b98a2ab)
• components: Outputting system.artifact and google.BQTable as a replacement for the google.VertexBatchPredictionJob artifact. (18b3ab8)
• components: Take problem type as input rather than schema URI (d1c0c75)
• components: Tests for import model evaluation (cdddaef)
• components: Update batch prediction component to use artifact.uri instead of artifact.metadata['artifactUri'] (7d5690a)
• components: update model upload component to accept the unmanaged model artifact (f341b67)
• components/google-cloud: Accept UnmanagedContainerModel artifact in Batch Prediction component (e39eed1)
• components/google-cloud: Add support for env variables in Custom_Job component. (5080288)
• components/google-cloud: Support parametrized input for reserved_ip_range and other Vertex Training parameters in custom job utility. (be682e4)
• conformance: adapt pipeline API tests to KF environment (#7647) (694a274)
• conformance: adapt pipeline version API tests to run in KF (#7653) (913277a)
• conformance: adapt remaining tests to KF env (#7670) (bcbdbd0)
• conformance: adapt run api tests to KF env (#7658) (8b11551)
• conformance: containerize KFP conformance test. (#7738) (3867496)
• conformance: Enable experiment API tests in KF mode. (#7596) (1265113)
• doc: Update kubeflow-e2e-mnist notebook to use KServe (#7355) (d93b390)
• frontend: Add ability to filter by visualization type (#7906) (f5a1264)
• frontend: Add V2 Run Comparison page (#7793) (f822eea)
• frontend: Automatically fetch MLMD for Run status update in V2. (#7636) (d1a12cb)
• frontend: Create metrics tabs (#7905) (e683290)
• frontend: Create sections and run list for KFPv2 Run Comparison page (#7882) (146dae7)
• frontend: Enable KFPv2 UI by default by creating a new feature key #7610 (#7624) (5fb63c7)
• frontend: NewRunV2 page and NewRunParametersV2 page. (#7769) (e246ffb)
• frontend: Request MLMD information for KFPv2 Run Comparison (#7897) (3bcdeb6)
• frontend: Support default feature flags update in localStorage. Fix #7379 (#7537) (7f4e9a4)
• frontend: Update compare page and banner states based on run versions and count (#7844) (468d780)
• frontend: Upgrade MLMD client to 1.5.0. Partial #7029 (#7102) (ef8e092)
• google-cloud: Add 'generate_feature_attribution' and executor_output to Evaluation component inputs. (25508b6)
• google-cloud: Add GCPResources to ImportModelEvaluation component. (ba82969)
• google-cloud: Add internal flag to ImportModelEvaluation component (e7985b8)
• google-cloud: Add new components 'evaluation_data_sampler' and 'evaluation_data_splitter'. ([d6f9265

KFP SDK 2.0.0 release notes are distilled to emphasize high-level improvements in major version 2. See preceding 2.x.x pre-release notes for a more comprehensive list.

Also see the KFP SDK v1 to v2 migration guide.

Features

The KFP SDK 2.0.0 release contains features present in the KFP SDK v1's v2 namespace along with considerable additional functionality. A selection of these features include:

• An improved and unified Python-based authoring experience for components and pipelines
• Support for using pipelines as components (pipeline in pipeline)
• Various additional configurations for tasks
• Compilation to an Argo-independent pipeline definition that enables pipelines to be compiled once and run anywhere
• Additional SDK client functionality
• An improved KFP CLI
• Refreshed user documentation and reference documentation

Selected contributions from pre-releases:

• Support for @component decorator for Python components #6825
• Support for loading v1 and v2 components using load_component_from_* #6822
• Support importer in KFP v2 #6917
• Add metadata field for importer #7112
• Add in filter to list_pipeline_versions client method #7223
• Support getting pipeline status in exit handler via PipelineTaskFinalStatus #7309
• Support v2 KFP API in SDK client
• Enable pip installation from custom PyPI repository #7453, #8871
• Add additional methods to kfp.client.Client #7239, #7563, #7562, #7463, #7835
• CLI improvements #7547, #7558, #7559, #7560, , #7569, #7567, #7603, #7606, #7607, #7628, #7618
• Add Registry Client #7597, #7763
• Add support for authoring Custom Container Components #8066
• Support setting parallelism in ParallelFor #8146
• Support using pipelines as components (pipeline in pipeline) #8179, #8204, #8209
• Support using pipeline in exit handlers #8220
• Support google.-namespaced artifact types #8191, #8232, #8233, #8279
• Support passing upstream outputs to importer metadata #7660
• Add ability to skip building image when using kfp component build #8387
• Support single element then and else_ arguments to IfPresentPlaceholder #8414
• Enable use of input and output placeholders in f-strings #8494
• Add comments to IR YAML file #8467
• Support fanning-in parameters #8631 and artifacts #8808 from tasks in a dsl.ParallelFor context using dsl.Collected
• Support .ignore_upstream_failure() on PipelineTask #8838
• Support setting cpu/memory requests #9121
• Support additional pipeline placeholders
• Support for platform-specific features via extension libraries #8940, #9140
• Support display_name and description in @dsl.pipeline decorator #9153
• Extract component input and output descriptions from docstring #9156
• Allow user to specify platform when building container components #9212

Breaking changes

See the KFP SDK v1 to v2 migration guide.

Deprecations

• Deprecate compiling pipelines to JSON in favor of compiling to YAML #8179
• Deprecate ability to select --engine when building components #7559
• Deprecate @dsl.component's output_component_file parameter in favor of compilation via the main Compiler #7554
• Deprecate client's *_job methods in favor of *_recurring_run methods #9112
• Deprecate pipeline task .set_gpu_limit in favor of .set_accelerator_limit #8836
• Deprecate .add_node_selector_constraint in favor of .set_accelerator_type #8980

Bug fixes and other changes

• Various changes to dependency versions relative to v1
• Various dependencies removed relative to v1
• Various bug fixes applied in KFP SDK 2.x.x pre-release versions
• Various bug fixes associated with providing backward compatibility for KFP SDK v1
• Use YAML as default serialization format for pipeline IR #7431
• Support Python 3.10 #8186 and 3.11 #8907
• Enable overriding caching options at submission time #7912
• Format file when compiling to JSON #7712
• Allow artifact inputs in pipeline definition #8044
• Support task-level retry policy #7867
• Support multiple exit handlers per pipeline #8088
• Migrate Out-Of-Band (OOB) authentication flow #8262
• CLI kfp component build generates runtime-requirements.txt #8372
• Throw exception for component parameter named Output #8367
• Block illegal IfPresentPlaceholder and ConcatPlaceholder authoring #8414
• Fix boolean default value compilation bug #8444
• Fix bug when writing to same file using gcsfuse and distributed training strategy in lightweight/containerized Python components #8544 [alternative fix after #8455 in kfp==2.0.0b8], #8607
• Pipeline compilation is now triggered from @pipeline decorator instead of Compiler.compile() method. Technically no breaking changes but compilation error could be exposed in a different (and earlier) stage #8179
• Fully support optional parameter inputs by writing isOptional field to IR #8612
• Add support for optional artifact inputs (toward feature parity with KFP SDK v1) #8623
• Fix upload_pipeline method on client when no name is provided #8695
• Enables output definitions when compiling components as pipelines #8848
• Add experiment_id parameter to create run methods #9004

Documentation updates

• Refresh KFP SDK v2 user documentation
• Refresh KFP SDK v2 reference documentation

Features

• backend: add schema version check (#9447) (08494a2)
• backend: Implement aws-go-sdk crendentials to support IRSA for s3 in V2 (#9502) (8a3215e)
• components: add a check for model garden models and skip eval import (1782ee2)
• components: Add force_dircet_runner flag to pipeline options (f167e15)
• components: Add instructions for formatting BiasConfig input proto using json_format in Model & Data bias component (3a89044)
• components: add preview directory to GCPC (b167f87)
• components: Add utility function for constructing the serverless custom job launcher spec (6fb10f3)
• components: Define Data Bias component for detecting data bias metrics (7ab5d97)
• components: Define Model Bias component for detecting model bias metrics (7022007)
• components: Derive image tag from Eval container version. Align Eval image version for all Eval components in GCPC (63936c0)
• components: make GCPC artifacts usable in KFP Python components (6104ff6)
• components: Provide model evaluation pipelines (9ce827e)
• components: Release the Seq2Seq and Temporal Fusion Transformer pipelines (377b961)
• components: Support Instance Config in kubeflow model_batch_predict component (3166418)
• components: Update eval component with the latest GCPC container image version (a3418b3)
• frontend: make container name configurable (#9433) (bb9dc67)
• sdk: support compiling components with dsl.PipelineTaskFinalStatus type (#9082) (7890227)

Bug Fixes

• backend: Add creation of pipeline and pipeline version atomically (#9416) (79d8d83)
• backend: enable cache for create/delete PVC (#9504) (eadeb30)
• backend: fix column size of StateHistory in task table. Fixes: #9553 (#9554) (0d53de7)
• backend: Fix url encoding issue. Closes #9448 (#9449) (3e9f048)
• backend: let components add default values (#9591) (dbebbde)
• backend: publish launcher execution even if it fails (#9513) (0ccdb09)
• backend: upgrade to gopkg.in/yaml.v3 to fix boolean support. Fixes #9451 (#9473) (4810b7a)
• components: Change docker base image to marketplace.gcr.io/google/ubuntu2004, which has a smaller size (d68231d)
• components: Fix a bug in job_remote_runner and refactor code (fc8865a)
• components: fix dataset components (3a7d684)
• components: fix invalid JSON payload (c4bf506)
• components: Remove print statement when importing eval metrics (de5ca7a)
• components: Skip sliced FA for non classification tasks to avoid empty input bug in Direct Runner (bf8c101)
• components: Update force_dircet_runner flag to force_runner_mode to choose the runner of evaluation pipeline (5acb016)
• frontend: Choose button is disable even if pipeline is already selected (after uploading a new pipeline) (#9499) (9e5e6c6)
• frontend: Choose version button is greyed out after closing pipeline selector (#9585) (a029a73)
• frontend: Choose version button should be disabled if no pipeline is selected (#9596) (c0e170d)
• frontend: Display experiment name if experiment is selected (#9599) (5d779e5)
• frontend: Incorrect page text for empty pipeline. (#9560) (a303e03)
• frontend: Incorrect page title after switching to another pipeline version. (#9450) (5cf4f19)
• frontend: Incorrect selected version in summary card (#9463) (c28d41b)
• frontend: Make pipeline definition in YAML tab more readable (#9511) (ef9781b)
• frontend: Mismatched error text in new run page (#9569) (745e2a6)
• frontend: Missing pipeline info for "Clone recurring run" after viewing pipeline (#9559) (a2fa55f)
• frontend: Parameter is missing after upload pipeline in pipeline selector (#9579) (8613a45)
• frontend: Parameter is missing in run creation page. Closes #9441 (#9458) (aa93e59)
• frontend: Pipeline Root is not used when provided during New Run creation (#9561) (23739e9)
• frontend: Pipeline version is not updated in new run page (#9578) (ee63f91)
• frontend: Run name should be empty if no pipeline is selected (#9497) (70219b5)
• frontend: Toolbar in pipeline details (from run) should only have "Clone run" button (#9467) (43cbb7e)
• frontend: Unexpected page crash when clicking artifact node. (#9519) (10e639e)
• sdk: fix bug when iterating over output from upstream task in nested parallelfor loops (#9580) (48ecc7d)
• sdk: Optimize wait_for_run_completion sdk function to skip an extra wait (#9407) (4d2a1cf)
• sdk: print docker error messages (#9595) (a9ac0b9)
• sdk: relax executor type checking (#9481) (2961202)

Other Pull Requests

• normalize docs version dropdown format (#9435) (f264f38)

Features

Breaking changes

Deprecations

Bug fixes and other changes

• Fix integer value not allowed as float-typed return #9481
• Change kubernetes version requirement from kubernetes>=8.0.0,<24 to kubernetes>=8.0.0,<27 #9545
• Fix bug when iterating over upstream task output in nested dsl.ParallelFor loop #9580

Documentation updates

Features

• components: Fix batch prediction job proto bigquery payload sanitization (a7b8fd0)
• components: SageMaker V2 model monitor component and testing (#9253) (07e67bb)
• components: SageMaker V2 model monitor component release (#9368) (29444f9)
• sdk: support writing TASK_FINAL_STATUS parameter type (#9080) (c01288d)

Bug Fixes

• backend: Add pipeline spec name validation and relax display name validation (#9374) (9ec4dc7)
• backend: added rbac group to commands to create pipeline/run. Fixes #8854 (#9365) (c17b681)
• backend: change format of some backend error message (#9304) (4f4f2cb)
• backend: Fix authorization logic and remove redundant DB calls (#9373) (09c0b92)
• backend: prepublish execution in the driver for create/delete pvc (#9397) (73f5f1b)
• components: Fix Model Garden Publisher model names in GetVertexModel component (2bc30e9)
• components: fix sklearn component discovery (0318caa)
• components: Move Feature Extractor and Error Analysis Annotation components from experimental folder to google folder in GCPC (a2d1f89)
• frontend: Change the empty message of archived experiment list. (#9381) (501589f)
• frontend: Filter of pipeline selector in create new run page is broken. (#9401) (62e5895)
• frontend: Make the hover text for "Archive" button more informative (#9382) (24a7fdd)
• frontend: Momentary "No graph to show" when pipeline/graph is loading. (#9399) (ec2a6fb)
• frontend: No values in "Finished at" & "Duration" when run is finished (#9379) (47fc33b)
• frontend: Remove redundant "(optional)" text in the optional input fields. (#9380) (c1958d9)
• frontend: Rename button in pipeline/run details side panel (#9383) (e2129da)
• frontend: Show summary button and summary card in pipeline details is disappeared. (#9404) (260e5eb)
• sdk: fix compilation of boolean constant passed to component (#9390) (96947e6)

Other Pull Requests

• fix(frontend) Format container information in pipeline details side panel (#9387) (66ff81e)

Features

• Support compiling primitive components with dsl.PipelineTaskFinalStatus input #9080, #9082

Breaking changes

Deprecations

Bug fixes and other changes

• Fix compilation of boolean constant passed to component #9390

Documentation updates

⚠ BREAKING CHANGES

• SDK: SDK client v2beta1 API integration. Fixes #8706 (#9112)

Features

• backend: Add nodeSelector support. Closes #8959. (#9132) (276f922)
• backend: add parameter verification before submitting the workflow (#9228) (52c4c6f)
• backend: avoid downcast and use ExecutionSpec if possible (#9232) (6c72d95)
• backend: support secrets (#9178) (b8caf81)
• components: Add a remote runner for pipelines (274ac8b)
• components: Add logic in import_model_evaluation to accept explanations from slices (4397b8c)
• components: Add slicing_specs input to classification evaluation (6a55449)
• components: Allow batch predict component to add missing bq:// schemes (8cc0550)
• components: Allow importing LLM eval metrics (ecc4ae0)
• components: Change ResolveVertexModelOp name to GetVertexModelOp (6a916e8)
• components: Create ResolveVertexModelOp component (595ae8e)
• components: Import user slices to the model evaluation importer (8c9b716)
• components: Sagemaker V2 Hosting components and tests (#9243) (4818e84)
• components: Support model garden models in GetVertexModelOp (0b32899)
• components: Update GCPC image version in two Eval importer components (1f4d943)
• sdk: support extracting input/output descriptions from component/pipeline docstrings (#9156) (946c51b)
• sdk: Support submitting pipeline with platform config. (#9140) (952e073)
• SDK: SDK client v2beta1 API integration. Fixes #8706 (#9112) (07d6c18)

Bug Fixes

• api: Regenerate v2beta1 API (#9197) (7b70cb5)
• backend: Add upsert for SQLite and MySQL dialects (#9245) (2b92f9b)
• backend: Allow empty namespace and remove default namespace. Fixes #8945. Fixes #8897. Fixes #8854. (#9286) (fbb8b39)
• backend: Cleanup pipeline creation and upload logic (#9189) (f5514c7)
• backend: fix launcher importer error (#9290) (38bf368)
• backend: Fix timestamp in task store, add disabled unit test (#9246) (a975e99)
• backend: fix volume bug (#9150) (ca63f20)
• components: Changing path for model (#9331) (aae9cb7)
• components: Evaluation metrics don't show when using TabNet on Tabular Workflows (065d31c)
• components: fix BQML encryption_spec_key_name default value bug in experimental components (12398e8)
• components: Fix dataset preprocessor to get imported from v1 and change the default value for *_dataset_resource_name from None to empty string (6677a95)
• components: Fix feature extractor component definition to make vertex dataset argument optional (cc4a103)
• components: fix type error for dict (9d2abff)
• components: fix VertexPromptValidationOp definition (2abddf0)
• components: replace special characters (curved double quotes “”) in component YAML files (f5b265e)
• components: set input arg default values for import model evaluation (52f6c97)
• components: Update Vertex Pipelines prompt validation component with new input parameters (6fd7131)
• conformance: enable healthz test in full Kubeflow. (#9297) (2cfc428)
• frontend: Buttons dislocated in narrow toolbar (#9344) (1f8fdbc)
• frontend: Change the incorrect loading context for run details (#9179) (bb50b8d)
• frontend: fail to clone SDK-created run / recurring run (#9255) (47735f7)
• frontend: Failed pipeline version creation should not generate an empty pipeline. (#9342) (9cfa3a2)
• frontend: Incorrect run type after pipeline is selected (#9339) (5ff829a)
• frontend: Logs details is not changed when selected other execution node. (#9334) (f68e780)
• frontend: Missing pipeline info while cloning recurring run. (#9231) (c8f6031)
• frontend: Parameter field is missing after choosing another v2 pipeline (#9202) (1d659d1)
• frontend: Parameter field is missing when creating run from run list (#9335) (59db283)
• frontend: Redirect to recurring run list after creating v1 recurring run. (#9280) (34c7475)
• frontend: Render graph for yaml file containing platform_spec (#9192) (d8672e6)
• frontend: Render run parameters in run details tab. (#9217) (d8f9885)
• frontend: Replace obsoleted hyperlinks in new pipeline page. (#9181) (3537504)
• release: Update release image with the latest api-generator (#9198) (8645cac)
• sdk: allow user to choose platform for component container build. Fixes #8883 (#9212) (b59b708)
• sdk: fix appengine import error (#9323) (c437004)
• sdk: fix handling of PipelineTaskFinalStatus in ir_type_ attribute hack (#9186) (41c0d78)
• sdk: fixes module not found error for containerized python components. Fixes #8385 (#9157) (3164dfb)
• SDK: fix client.run_pipeline with pipeline_id and version_id gives error (#9191) (91277b1)

Other Pull Requests

• release 2.0.0b16 (#9325) (58ebd20)
• [WIP] test: Add v2 upgrade test (#9287) (e115e0c)
• feat(components):Release the default pipeline with SLM feature selection (4ccb0ef)
• feature(frontend) Support Logs tab in KFP v2 (#9265) (430bd54)
• feat(sdk): support display_name and description in @dsl.pipeline decorator (#9153) (91abbea)

Major Features and Improvements

Breaking Changes

For Pipeline Authors

For Component Authors

• Add support for containerized component #8066

Deprecations

Bug Fixes and Other Changes

• Enable overriding caching options at submission time #7912
• Allow artifact inputs in pipeline definition. #8044
• Support task-level retry policy #7867
• Support multiple exit handlers per pipeline #8088

Documentation Updates

Features

• backend: add support for environment variables. Fixes #8704 (#8886) (faca1e5)
• backend: add support for resource requests. Closes #7047. Closes #6354. (#9088) (37dac45)
• backend: support optional and default parameters. Fixes #8716 (#8765) (89bdac9)
• backend: support pvc creation, deletion and mount; fixes #8708 (#9106) (761a592)
• backend: support yaml with platform-specific specs (#8983) (a9ead5a)
• backend: updates compiler to include kubernetes-specific specs in argo workflow (#9059) (a3e795a)
• component: add evaluation_importer_gcp_resources to evaluated annotation importer component (d17916f)
• component: Adds Vertex Pipelines prompt validation component (249f235)
• component: change Error Analysis component to use CustomJob launcher (fd67277)
• component: change Feature Extractor component to use CustomJob launcher (a725192)
• component: Create component definition for GetVertexDataset (2d29137)
• component: define ImportEvaluatedAnnotation component (8c9fdee)
• component: init container backend for evaluated annotation importer (74b86dc)
• component: Merge annotation and error analysis in evaluated annotation importer (10e7c55)
• components: Add data-filter-split feature back to the ImageTrainingJOb component which was removed previously (4a4a968)
• components: Add machine type and encryption key for Error Analysis Component (0293fe1)
• components: Add machine type for Feature Extractor component. Deprecate Dataflow-related inputs (4e85352)
• components: Dataflow Flex Template component (6101ff1)
• components: Delete tfp_anomaly_detection (f2537d4)
• components: Import model evaluation slices 50 at a time (39e4c36)
• components: Propagate cost labels of GCP resources (7466803)
• components: Update evaluation components to v0.9 and add input for quantile forecasting (74237dc)
• components: update EvaluationDatasetPreprocessorOp to use KFP v2 Python definition (deed177)
• components: Update import_model_evaluation so models with 100+ labels will not import confusion matrices at every threshold (cb7d9a8)
• frontend: Support namespaced pipelines from the UI. Closes #5084 (#8831) (6996947)
• sdk: add experiment_id parameter to create run methods (#9004) (3e1f3c6)
• sdk: deprecate .add_node_selector_constraint in favor of .set_accelerator_type (#8980) (e96754c)
• sdk: deprecate .set_gpu_limit in favor of .set_accelerator_limit (#8836) (e7ad073)
• sdk: Ignore upstream failure (#8838) (5684bfc)
• sdk: support compiling platform specific features (#8940) (a38d747)
• sdk: support indexing into top-level artifact metadata in Container Components (#9131) (5828632)
• sdk: Support PIPELINE_ROOT_PLACEHOLDER (#9134) (5e98b3b)
• sdk: Support setting cpu/memory requests. (#9121) (a6ef77b)
• Add cpu_request and memory_request in pipeline_spec. (#9063) (d2db64b)
• add kubernetes platform-specific protos, generated Go code, and Python package (#8888) (a4abf27)
• add NodeSelector message to kfp-kubernetes proto (#8981) (cd15b7a)
• add platform-specific configuration protos to kfp-pipeline-spec package (#8896) (643f421)
• add TASK_FINAL_STATUS to PipelineSpec ParameterTypeEnum (#9061) (902e2a9)
• implement kfp-kubernetes library (#8950) (0c11cce)
• support nodeSelector in kfp-kubernetes (#8982) (c8cb5b5)
• sdk: using component's pip_index_urls for Dockerfile generation. Fixes #8816 (#8871) (ed66ba3)

Bug Fixes

• api: Fix pipeline.upload.swagger.json (#9104) (4a9bc2b)
• backend: Add checking for state change in state history (#8903) (fde6b94)
• backend: Add default values to timestamps. Fixes #8845 (#8857) (beeaed0)
• backend: Add ListAllRuns endpoint (#8901) (e8f10b1)
• backend: Add pipeline version to the request body (#9056) (caf842d)
• backend: Add pipeline_id to pipeline_source in Run and RecurringRun (#9097) (e7fdb7f)
• backend: Allow empty namespace in pipelines. Fixes #8840. (#8859) (03b7752)
• backend: Allow runs and recurring runs without creator pipeline and pipeline version (#8926) (7fdb1b9)
• backend: backend accepts pipeline spec struct as well as nested (#9024) (472e922)
• backend: Deprecate experiment_id in Run API v2 (#8921) (06f1542)
• backend: Fix CreatePipelineVersion API (#9075) (7d3a6e6)
• backend: fix CVEs in persistenceagent, scheduledworkflow and cache-server. Fixes #8943 (#8944) (87deff0)
• backend: Fix missing parameters in jobs and recurring runs. Closes #9012. (#9118) (ce096dd)
• backend: Fix priority in spec source. Closes #8948 (#8949) (d0a9405)
• backend: fix UploadPipeline/UploadPipelineVersion v2 API returns v1 object. (#9064) (01c87f8)
• backend: Remove duplicate pipelines in list pipelines V1 (#8956) (1d75286)
• backend: Remove resource from workflow-controller-configmap. Closes #9087 (#9089) (59688a3)
• backend: Revert the url changes in run API v 2 (#8920) (e568148)
• backend: Set status when converting API recurring run to internal representation (#9128) (b5dae77)
• backend: support reading large doc from URL; Fixes #7652 (#9072) (bd9f74e)
• component: Assign default value to KMS key in BigqueryQueryJob component (e3c9e1f)
• components: Change base image to ubuntu OS (84d5eff)
• components: Fix default values for JsonArrays in Eval components (b87795f)
• components: fix evaluated annotation batch importer bugs (e20840a)
• components: Fix experimental components and tests to be compatible with KFP v2 (68e15b7)
• components: remove 3.10 Python type annotation from evaluated annotation importer container (31fe64f)
• components: Replace base image to reduce vulnerabilities (5e2475e)
• components: Set google-cloud-aiplatform>=1.21.0 to ultimately avoid dataset creation 900s timeout and remove the workaround. Reference: https://github.com/googleapis/python-aiplatform/issues/1870#issuecomment-1382008391 (5de4d78)
• components: Update parameter name in Feature Extractor component (0094141)
• deployment: fix custom namespace for viewer crd deployment. Fixes #7714 (#7715) (3c3ba25)
• frontend: Change the prompt and error message of parameter field into more readable text (#8824) (e6e8600)
• frontend: Remove the "finished at" information for artifact node. (#8810) (508b6a1)
• frontend: Set mode instead of status when creating a v2 recurring run (#9129) (6e5dbb0)
• manifests: disable mysql binlog (#8941) (cdc9f2e)
• proxy: fixes typo (#8839) (86c9205)
• sdk: Add outputDefinitions of root pipeline to compiled components IR YAML file (#8848) (1e1a020)
• sdk: fix attribute error for Containerized Python Components (#8887) (87062db)
• sdk: fix bug when passing data between tasks using f-strings (#8879) (31298c4)
• sdk: Fix environment variable set in component yaml lost during compilation. Fixes #8884 (#8885) (8d3e0dd)
• sdk: fix loading non-canonical type strings from v1 component YAML (#9041) (3026e45)
• sdk: fix unknown artifact string in dsl.OutputPath/dsl.InputPath (#8880) (386ff88)
• sdk: Restore github -> g3 sync by fixing broken tests (328243b)
• sdk: support accessing .dsl attribute on kfp module object (#9048) (0cbcebc)
• support setting task dependencies via kfp.kubernetes.mount_pvc (#8999) (2bbfd5e)
• sdk: Support python 3.11 (#8907) (6777e8f)

Other Pull Requests

• set _ir_type attribute (#9105) (e907b63)
• update compiler type annotations (#9090) (ad6f55e)
• Fix display_name and description fields. (#9085) (de8d800)
• Add pipeline metadata fields in the IR. (#9038) (60f2a1f)
• Internal change (9584058)
• Fix typo in IR field description (#9023) (250db6e)
• cpu and memory resource can be optional (#8992) (0fd1e69)
• Add gkcalat to kubernetes_platform/OWNERS (#8984) (4c97e57)
• Adding changelog for 2.0.0-alpha.7 in master branch (ccde7b9)
• release kfp sdk 2.0.b13 (#8904) (86554be)
• support fanning-in artifacts (#8808) (c046879)
• add 1.8.19 docs (#8866) (893a6e4)
• support list of artifact input placeholders (#8484) (4175260)
• Update API references and readme. Closes #8723 (#8828) (c816585)

Major Features and Improvements

Breaking Changes

For Pipeline Authors

For Component Authors

Deprecations

Bug Fixes and Other Changes

• Include default registry context JSON in package distribution #7987

Documentation Updates

Features

• API: Update IR go package (#8614) (c30648a)
• backend: add run support for v2 api (#8647) (d524464)
• backend: add support for "finished_at" and others in filter. Fixes #8654 (#8662) (6ee7677)
• backend: Add support for converting v2 recurring run to scheduled workflow in template (#8521) (7e7e1af)
• backend: Add support for recurring run v2 api (#8634) (d0f0ab3)
• backend: Add v2 api for experiments (#8480) (6e364d3)
• backend: fix partner_id in GCP Marketplace application (#8426) (3fca792)
• backend: Include namespace in Pipeline API model. Part of #4197 (#8375) (7c114a0)
• backend: Make cache-server webhook port number configurable (#8401) (3824c5b)
• backend: Refactor apiserver, add Pipeline and other v2beta services (#8633) (135dc64)
• backend: Upload namespaced pipeline definitions. Part of #4197 (#8511) (931c14a)
• component: Create YAML definition for Feature Extractor Error Analysis component (e7c1a16)
• component: Update input args in dataset preprocessor and evaluated annotation (9a32f76)
• components: Add components for natural language: training TFHub model and preprocessing component for batch prediction (60b2d5f)
• components: Add DataprocBatch resource to gcp_resources output parameter (6e95e89)
• components: add KServe component flag to disable storing CRD status into output parameters (#8393) (24f8651)
• components: Add logs to Batch Prediction creation process (b3de27a)
• components: Add text classification data processing component and training component (9675870)
• components: Create GCPC v2 beta (834d966)
• components: Create new component YAML definition for computing error analysis annotations (b7d03cb)
• components: Create new component YAML definition for computing evaluated annotations (fe38716)
• components: Create new component YAML definition for Evaluation Dataset Preprocessor Error Analysis (6800d21)
• components: New sagemaker training job parameters (#8538) (6a6cfdb)
• components: propagates vertex system labels to the downstream resources (batch prediction job) (4410bf2)
• components: propagates vertex system labels to the downstream resources (bigquery job) (a506da6)
• components: propagates vertex system labels to the downstream resources (custom job) (69e1e4b)
• components: propagates vertex system labels to the downstream resources (dataflow python job) (4c2797e)
• components: propagates vertex system labels to the downstream resources (dataproc) (7fe4319)
• components: propagates vertex system labels to the downstream resources (Vertex AI Datasets) (9d9a8bb)
• components: propagates vertex system labels to the downstream resources (Vertex AI Endpoints) (f37086b)
• components: propagates vertex system labels to the downstream resources (Vertex AI HyperParameter Tuning Job) (cdd0626)
• components: propagates vertex system labels to the downstream resources (Vertex AI Model Registry) (56edebb)
• components: release Structured Data team's updated components and pipelines (b9a0499)
• components: release Structured Data team's updated components and pipelines (bd1538d)
• components: Release Structured Data team's updated components and pipelines (73034a9)
• components: Support serving default in bq export model job op (582576f)
• components: Support uploading for model versions for ModelUploadOp (94bdce8)
• components: Sync GCPC staging to prod to include AutoML model comparison (06761b9)
• components: Update dataset_preprocessor component inputs (399b7be)
• components: Update evaluation container to v0.8.1 image (8d87bec)
• components: Updated components and pipelines (e358dee)
• components: Updated components and pipelines (d1f1ee9)
• frontend: Implement aws-js-sdk crendentials to support IRSA for s3 (#8651) (9b9b3ac)
• frontend: Support cloning recurringRun in KFP v2 (#8652) (80c0dc5)
• frontend: Support creating recurringRun in KFP v2. (#8568) (a4a2488)
• IR: add is_optional field for input artifacts (#8571) (998f52d)
• IR: change is_list_of_artifacts proto field to is_artifact_list (#8522) (041f446)
• samples: Full Kubeflow E2E sample using KFPv2. (#8303) (91fe29f)
• sdk: Accepts higher PyYAML versions (#8665) (6040334)
• sdk: add --build-image option to 'kfp components build' to allow users to skip docker build. Fixes #8382 for 2.0 (#8387) (5b75d75)
• sdk: Add comments to IR YAML file (#8467) (49db63c)
• sdk: add is_list_of_artifacts field to PipelineSpec [lists of artifacts support pt. 2] (#8466) (6cd7cbc)
• sdk: add pipeline job placholders for scheduled runs (#8447) (e97fe8e)
• sdk: add support for lists of artifacts in Python components [support lists of artifacts pt. 3] (#8465) (08b2408)
• sdk: enable use of primitive placeholders in f-string (#8494) (b829e4a)
• sdk: support for fan-in from dsl.ParallelFor (#8631) (b575950)
• sdk: support list of artifacts annotations [list of artifacts support pt. 1] (#8464) (e7c82c0)
• sdk: support optional artifact inputs (#8623) (151e8a3)

Bug Fixes

• backend: add retry run in v2 api. Fixes #8735 (#8804) (6522e6c)
• backend: Avoid referencing v2 images via latest label. (#8448) (7138072)
• backend: Fix recurring run for v2 pipeline (#8487) (9970f3d)
• backend: fix run, job from empty manifest. Closes #8802. (#8805) (1f748c5)
• backend: fixes a potential upgrade/downgrade error (#8515) (acaa7f0)
• components: Add missing dependency (633461b)
• components: Fix aiplatform & v1 batch predict job to work with KFP v2 (1d8272b)
• components: Fix aiplatform & v1 batch predict job to work with KFP v2 (fdd92df)
• components: Fix aiplatform serialization (de2469e)
• components: Fix list and dict types for converted aiplatform components (b166c18)
• components: Fix serialization for aiplatform components (5e929ca)
• components: Fix the payload in batch prediction unit test (1c7a618)
• components: update tags in Structured Data team's forecasting pipelines (217cbd9)
• frontend: Change the pipeline description field as optional. (#8793) (f6e0b6f)
• frontend: Create recurring run by default from recurring run entry point (#8384) (1f5d8ff)
• frontend: Disable choosing pipeline version if pipeline is empty. (#8787) (8d602bc)
• frontend: preserve run parameters on pipeline version change for cloned runs. Fixes #4944 (#8597) (2c2cfa3)
• frontend: Show pipeline details for an existing recurring run without pipeline / pipeline version. (#8681) (874e3d3)
• frontend: Use browser-compatible atob instead of Buffer for frontend graph decompression. Fixes #7655 (#8343) (3e75296)
• manifests: Change mysql default binlog expire to 10 days. Fixes #8552 (#8553) (352f1c6)
• release: Add v1 and v2 proto generation (#8819) (e5aa67c)
• samples: update sklearn package name (#8439) (684779e)
• sdk: add special handling for PipelineTaskFinalStatus to get_parameter_type (#8587) (2eef97c)
• sdk: Allow using cli kfp dsl compile to compile components (#8371) (4cc0e80)
• sdk: fix kfp component build only produces empty requirements.txt (#8372) (54922b3)
• sdk: fix boolean default value serialization bug (#8444) (9e9e108)
• sdk: fix bug when referencing component.pipeline_spec in a pipeline body (#8514) (00c6b01)
• sdk: fix deserializing v1 component yaml defaults (#8639) (84241d6)
• sdk: fix multiple workers writing with gcsfuse bug [KFP SDK v2] (#8455) (4bb57e6)
• sdk: fix nested placeholders and block illegal IfPresent form in Concat (#8414) (d33f359)
• sdk: fix upload_pipeline when no pipeline name is provided (#8695) (e50f403)
• sdk: make executor output writer more permissive [sdk v2] (#8607) (f64348f)
• sdk: new fix for multiple workers writing with gcsfuse bug [KFP SDK v2] (#8544) (d4b896d)
• sdk: print max_attempts instead of sleep_duration (#8523) (1d6a52b)
• sdk: revert premature client changes (#8507) (834bfbe)
• sdk: support writing/reading isOptional for input parameters (#8612) (5491f6f)
• sdk: Throw exception for component parameter named Output (#8367) (ac02a0e)
• sdk: unblock valid topology. (#8416) (614c231)
• test: Add frontend integration test manual run (#8767) (c79d35f)
• test: fix missing test dependencies (#8813) (fcdd738)
• test: unable to create test cluster (#8410) (100b1f3)

Other Pull Requests

• remove 2.0.0b10 docs (#8746) (8f472d9)
• bump version; update release notes; add docs (#8745) (edfb267)
• pin kfp-pipeline-spec version exactly (#8737) (1c7ab4c)
• fix release notes (#8742) (3c562b6)
• docs[manifests]: fix and standardize pns comments (#8438) (280edbe)
• add versions to docs (#8613) (579a02c)
• feat(components): support a HyperparameterTuningJobWithMetrics type to take execution_metrics path (50c8721)

Major Features and Improvements

Breaking Changes

For Pipeline Authors

For Component Authors

Deprecations

Bug Fixes and Other Changes

Documentation Updates

Features

• components: Add evaluation forecasting default of 0.5 for quantiles (21b8479)
• components: Add metadata field for pipeline resource name (82bf1ec)
• components: Add special case in json_util.py where explanation_spec metadata outputs can have empty values (b8b77fd)
• components: Added BigQuery input support to evaluation components (bfa2dc6)
• components: Allow dataset paths list (5d06871)
• components: Create new tensorboard experiment creator component (a3c7c38)
• docs: use new sdk reference docs theme (#8212) (19cf904)
• frontend: v2 Dynamic Sub-DAG (I) (#8278) (440f4e1)
• sdk: add compilation logic for google artifact types (support custom artifact types pt. 2) (#8232) (e371fa3)
• sdk: add runtime logic for custom artifact types (support for custom artifact types pt. 3) (#8233) (166d6bb)
• sdk: add workaround for google artifact (support custom artifact types pt. 4) (#8279) (8ab690f)
• sdk: enable dynamic importer metadata (#7660) (c353245)
• sdk: migrate v1 auth to v2 (#7789) (e14a784)
• sdk: Support using pipeline in exit handlers (#8220) (d883582)

Bug Fixes

• backend: Fix "Unknown execution spec" error. Fixes #8256 (#8287) (667a633)
• backend: fix backward compatibility in scheduledworkflow (#8261) (f6f8485)
• components: Fix input parameter in tensorboard experiment creator component (984d8a0)
• components: Fix missing error payload from logging (1155cd4)
• components: Make some input parameters optional to provide better user experience (a1c643f)
• components: Remove unused input in evaluation classification yaml (1a34e38)
• components: Sync GCPC staging to prod to include recent API changes (a8e9f4b)
• frontend: Fix the issue when drawing dependentTask for task-to-task edges. (#8304) (a2b169d)
• sdk: fix docs version reference (#8239) (a13ad4e)
• sdk: fix dsl control flow docstrings (#8183) (eb453d1)
• sdk: fix InputPath error in executor (#8273) (0cb0d99)
• sdk: keeping placeholder strings in command and args even if default value is set (#8259) (5dff29a)
• sdk: make sdk installable with poetry (#8263) (231c507)
• sdk: Replace OOB authentication with loopback flow (#8262) (65f2b64)

Other Pull Requests

• chore(components/pytorch) - Samples fix for PT and PTL Upgrade (#8148) (58d22d4)
• feat(component):Add tensorboard experiment creator component e2e test (8342377)
• fix v1 output deserialization (#8272) (f54f0b9)

Major Features and Improvements

• Implement Registry Client #7597, #7763
• Write compiled JSON with formatting (multiline with indentation) #7712
• Add function to sdk client for terminating run #7835
• Re-enable component compilation via @component decorator (deprecated) #7554

Breaking Changes

• Make CLI output consistent, readable, and usable #7739

For Pipeline Authors

For Component Authors

Deprecations

Bug Fixes and Other Changes

• Fix CLI upload pipeline version #7722

Documentation Updates

⚠ BREAKING CHANGES

• sdk: Implement Graph Component (#8179)

Features

• backend: Add ExecutionClient to access ExecutionSpec (#8050) (e8abec2)
• backend: authorize readArtifacts and ReportMetrics endpoints (#7819) (1caa8e0)
• backend: Implement RuntimeConfig in backend (#8085) (2b556ec)
• backend: Implement RuntimeConfig in backend for jobs(RecurringRun) (#8152) (f25127f)
• cloud: Update JSON templates for evaluation (88a2b80)
• components: add namespace to pvc creation (#8147) (031a7d9)
• components: Add nargs to allow for empty string input by component (555447f)
• components: Add vertex_notification_email component on v1 folder (3b1bd08)
• components: Give evaluation preprocessing components unique dataflow job names (c052bf8)
• components: Import model evaluation slices when available in the metrics (0d304c6)
• components: Introducing AWS SageMaker TrainingJob component v2 in Preview (#8158) (f162b24)
• components: Separate feature attribution from evaluation component to its own component (f9cf42a)
• components: Split evaluation component into classification, forecasting, and regression evaluation & create artifact types for google.__Metrics (8c74612)
• components: Sync gcpc staging code to prod (01c9ed9)
• components: Update evaluation container to v0.5 for data sampler and splitter preprocessing components (37ff321)
• components: Update import_model_evaluation component to accept new google.___Metrics artifact types (8eb61ad)
• components: Update regression and forecasting to contain ground truth input fields (6ec9b71)
• frontend: Add KFPv2 namespace check (#8036) (9df8343)
• frontend: Add parent labels to the Compare Table (#8024) (4be213a)
• frontend: Add pre-check for pipeline and pipeline version name at the UI (#8113) (58f3a60)
• frontend: Add refresh and run re-selection functionality (#8118) (d02857f)
• frontend: Add ROC Curve filter table and functionality (#8073) (23d549f)
• frontend: Add run parameters (#8105) (f96d3e9)
• frontend: Creating run from various entry points (#8190) (7d5985b)
• frontend: Display metrics loading and error states (#8029) (4cb8562)
• frontend: Display ROC curve (#8046) (cc3bd8b)
• frontend: Display scalar metrics table (#8026) (d3fe514)
• frontend: Show error message for invalid input or missing parameters. (#8003) (303267f)
• frontend: Show pipeline details (DAG) for an existing run without pipeline / pipeline version. (#8116) (448b4ca)
• frontend: Support Clone Run with RuntimeConfig (#8093) (1257a35)
• google-cloud: Add ground_truth_column input argument to data splitter component. (17d62e6)
• google-cloud: Update eval container version for eval components (065f732)
• sdk: add support for metadata placeholders (#8151) (88a1b31)
• sdk: Allow artifact inputs in pipeline definition. (#8044) (1dd06cd)
• sdk: expose IfPresentPlaceholder and ConcatPlaceholder to kfp.dsl (#8145) (c7a90a3)
• sdk: expose new ContainerSpec API (#8144) (35fccb1)
• sdk: Implement Graph Component (#8179) (5ce4954)
• sdk: rename kfp artifact attributes (support custom artifact types pt. 1) (#8191) (72c1d10)
• sdk: Support loading pipeline from yaml (#8209) (410e99c)
• sdk: support more than one exit handler per pipeline (#8088) (bdff332)
• sdk: Support Parallelism in ParallelFor in KFP SDK V2 (#8146) (51bea09)
• sdk: Support pipeline outputs (#8204) (48574dc)
• sdk: supporting bring your own container for arbitrary input and outputs (#8066) (04c827f)

Bug Fixes

• component: update the missing doc of returns in export_model component (1664895)
• components: Ensure main() is run in executor (1de9e66)
• components: Fix typo (9cfe3cf)
• components: include fix AMI issues for criteo dataset (2a48096)
• components: Reverse re.findall order of arguments to (pattern, string) in job_remote_runner (582aefa)
• frontend: Added pipeline_manifest to pipeline_spec in ApiRun to enable cloning SDK-created run. (#8134) (a7388f7)
• frontend: make frontend server config s3 endpoint url configurable via environment variable. Fixes #7995 (#8033) (e312d95)
• frontend: Replace license-checker with yarn licenses. (#8101) (2299cd7)
• frontend: Resolve all warnings when running npm run start (#8089) (369e14c)
• frontend: Show correct artifact type at the side panel. (#8083) (75c31f5)
• sdk: fix compiling components with artifact inputs (#8106) (45f3bac)
• sdk: fix constructor signature on dsl.Condition (#8182) (889637a)
• sdk: fix output type of importer (#8172) (5ccf53a)
• sdk: fix v2 sample tests for cuj3 (#8168) (121ddcb)
• Renovate json format. Fix #6772 (#8107) (b9a0523)
• sdk: fixes kfp.components module type annotations, docstrings, and string/bytes processing (#8040) (e6c9411)
• sdk: fixes requirements.txt discovery for sdk api reference docs (#8048) (777c981)

Other Pull Requests

• Fix the example pipeline in sdk/python/README.md (#8206) (f5277d3)
• Use STABLE base image to fix CVE-2022-2068 (2bee241)
• feat(component):Rearrange json and yaml files in e2e test to eliminate duplicate defining and reading (7404dd0)
• feat(frontend) Show PipelineSpec from an existing Run (#8077) (a63182b)
• fix(frontend) Missing parameter in the input/output tab of Run details side panel (#8069) (ca41925)
• Update documentation for Feature Transform Engine-related components (c7d1efc)
• Sync code to GCPC codebase to reflect bug fix in FTE component spec. (0b2116d)
• add kfp v2 api reference docs (#8042) (d8b9439)
• Auto-generate batch id if none is specified. (2060e38)
• remove tfx from dependencies and simplify requirements files (#8030) (8cf061b)

⚠ BREAKING CHANGES

• sdk: make CLI output consistent, readable, and usable (#7739)

Features

• Upgrade argo-workflow to v3.3.8 (#8009) (8bee292)
• api: add runtime_config.parameter_values with string typed values (#7734) (5b64733)
• backend: Add ExecutionSpec Interface (#7531) (8cf022c)
• backend: Expand ExecutionSpec Interface (#7766) (06c9310)
• backend: update minio-go to support irsa (#7946) (7923ba3)
• backend: use cert-manager for cache server cert (#7843) (784f9fa)
• component: Support BigQuery drop model job components (affbc09)
• component: Support BigQuery ML advanced weights job component (f6b560a)
• component: Support BigQuery ML centroids job components (45cfb91)
• component: Support BigQuery ML reconstruction loss and trial info job components (d2f33c8)
• component: Support BigQuery ML weights job component (4183344)
• components: Add display name to import model evaluation component (9b12f6e)
• components: Outputting system.artifact and google.BQTable as a replacement for the google.VertexBatchPredictionJob artifact. (18b3ab8)
• conformance: adapt remaining tests to KF env (#7670) (bcbdbd0)
• conformance: containerize KFP conformance test. (#7738) (3867496)
• frontend: Add ability to filter by visualization type (#7906) (f5a1264)
• frontend: Add loading and error states for HTML and Markdown displays (#7989) (6cbc10b)
• frontend: Add V2 Run Comparison page (#7793) (f822eea)
• frontend: Create metrics tabs (#7905) (e683290)
• frontend: Create sections and run list for KFPv2 Run Comparison page (#7882) (146dae7)
• frontend: Create two-level dropdown (#7933) (832858b)
• frontend: Display HTML and Markdown files (#7981) (cfe3278)
• frontend: Display multi-level dropdown on KFPv2 Run Comparison page (#7943) (53d8f7e)
• frontend: Display two-panel layout and confusion matrices (#7966) (617ad83)
• frontend: NewRunV2 page and NewRunParametersV2 page. (#7769) (e246ffb)
• frontend: Request MLMD information for KFPv2 Run Comparison (#7897) (3bcdeb6)
• frontend: Update compare page and banner states based on run versions and count (#7844) (468d780)
• google-cloud: add explanation_metadata_artifact input arg to custom Batch Prediction component. (ad364d9)
• google-cloud: Add GCPResources to ImportModelEvaluation component. (ba82969)
• google-cloud: Add new components 'evaluation_data_sampler' and 'evaluation_data_splitter'. (d6f9265)
• google-cloud: Change evaluation preprocessing component output type. (059afb0)
• google-cloud: Modify GCPC evaluation templates. (176d3ff)
• google-cloud: Release eval changes with v0.3 (d2fd41b)
• google-cloud: Release notes for 1.0.12 and 1.0.13 (e42d9d2)
• google-cloud: Update AutoML pipelines to reference next GCPC package release (1.0.8). (c849871)
• IR: add is_optional to ParameterSpec proto (#7704) (f960a52)
• IR: adds IteratorPolicy to PipelineTaskSpec proto for support of parallelism setting on ParallelFor (#7804) (a86ae9b)
• kfp: implement config file support for registry client (#7908) (4f01b7e)
• sdk: Add default registry context (#7948) (aab7fda)
• sdk: Add function to sdk client for terminating run (#7835) (9ffcb2e)
• sdk: add retry policy support to kfp v2 (#7867) (850a750)
• sdk: add support for IfPresentPlaceholder and ConcatPlaceholder strings (#7795) (de0b824)
• sdk: compile JSON with formatting (#7712) (2570922)
• sdk: enable compilation of primitive components (#7580) (62972ec)
• sdk: enable component compilation via cli (#7649) (c6125ff)
• sdk: enable component compilation via component decorator (#7554) (6935a47)
• sdk: enable read in component using IR (#7689) (cbc2ac5)
• sdk: Implement Registry client (#7597) (25e4c58)
• sdk: make CLI output consistent, readable, and usable (#7739) (b0db428)
• sdk: make client return values consistent (#7659) (1ae6e11)
• sdk: only use IR when saving component (#7690) (a6ec449)
• sdk: throw informative exception on uncalled @pipeline decorator (#7913) (ecb55bd)
• sdk: validate pipeline resource name before submission (#7713) (101d243)
• sdk/client: implements overriding caching options at submission (#7912) (b95c5ab)

Bug Fixes

• Update proxy agent image to fix CVE-2022-1292 (#8019) (5724849)
• components: fix launch_crd.py to be compatible with kubernetes python SDK API. Fixes #7984 (#7985) (c7bf68e)
• Update GCP marketplace deployer base image (#8018) (1d871fe)
• Update mysql image to fix CVE-2022-1292 (#8017) (3bbc1d9)
• backend: "Started at", "Finished at" and "Duration" display error. (#7876) (d11b097)
• backend: change downloaded IR from JSON to YAML. Fixes #7673 (#7768) (c7b56d6)
• backend: fixes healthz response by adding json content type. Fixes #7525 (#7532) (bd38cb5)
• backend: Upgrade grpc-gateway patch version to enable correct swagger config (#7788) (cf1b873)
• cache: Change kfp cache cert names to match cert-manager (#7538) (2a82a0f)
• components: Pin pip version to fix apache-beam[gcp] installation issue (#8016) (bb83581)
• components: Preserve empty features in explanation_spec (4c70132)
• components/google-cloud: custom job util should add the nfs_mounts as a sub node of worker_pool_spec. (6296c18)
• deps: update dependency npm to v8 [security] (#7885) (624f197)
• frontend: Change "Layer" to unclickable. (#7737) (b7cf1fb)
• frontend: Fix run comparison filter (#7833) (f41b545)
• frontend: Fix Run Comparison refresh button (#7872) (85c7901)
• frontend: Move all V2 flag check to V2_ALPHA (#7773) (6cc33f5)
• ir: fix kfp-pipeline-spec proto imports (#7754) (436d7af)
• manifests: Update snapshot to 2.0.0-alpha.2 (#7852) (d099864)
• sdk: Fix corner cases and implement validation (#7763) (eaa8ec7)
• sdk: fix extract docstring in load component (#7921) (a020c9b)
• sdk: fix ifpresent and concat placeholder compile (#7930) (f877a1f)
• sdk: fix load v1 if present placeholder (#7765) (0cf8173)
• sdk: fix placeholder read/write logic (#7928) (ad458d5)
• test: chmod +x test_load_all_components.sh (#7865) (95d0f7f)
• test: Make exit handler test install KFP SDK from source (#7785) (d25bbc8)

Other Pull Requests

• Fix the problem that AutoML Tabular pipeline could fail when using large number of features. (e.g., > 500 features). (9f6689e)
• Move new bq components from experimental to v1 (527d9e1)
• remove kms key name from the drop model interface. (660e847)
• Internal change (6372420)
• feat(frontend) Support LIST, STRUCT type in RuntimeConfig parameters (#7991) (2d81e7b)
• chore(components/pytorch):kserve migration (#7615) (49c3587)
• add registry json to package data (#7987) (7253203)
• feat(frontend) Convert string-type RuntimeConfig parameters to real-type (currently support num, bool, str) (#7919) (b01000a)
• Fix AutoML Tabular pipeline always runs evaluation. (f2d3d4d)
• feat(components):Add pipeline metadata to import evaluation component (1fe3641)
• Use FTE image directly to launch FTE component (e9221ec)
• Add automl_tabular_pipeline pipeline for Tabular Workflow. Pin Tabular Workflow images to 20220629_0725_RC00. (5373af5)
• Make generate analyze/transform data and split materialized data as components (9d3ac26)
• For built-in algorithms, the transform_config input is expected to be a GCS file path. The file contains transformations to apply. (736f7fe)
• weight_column_name -> weight_column and target_column_name -> target_column for Tables v1 and skip_architecture_search pipelines (a4a7462)
• Make calculation logic in SDK helper function run inside a component for Tables v1 and skip_architecture_search pipelines (9b23034)
• Use feature transform engine docker image for related components (7dc6448)
• add algorithm to pipeline, also switch the default alg to be AMI. (4ca9993)
• Update skip_architecture_search pipeline (a8aa75d)
• google-cloud-pipeline-components-1.0.11 release notes (aedd748)
• Added dataflow_service_account to specify custom service account to run dataflow jobs for stats_and_example_gen and transform components. (745abc2)
• Update Tabular workflow to reference 1.0.10 launcher image. (419c8fa)
• Change JSON to primitive types for Tables v1, built-in algorithm and internal pipelines (daf40bb)
• move generating feature ranking to utils to be available in sdk (50ea394)
• The added field transformations_path enables users to specify transformations in a file and provide a path to the file as an input. (5e39259)
• add algorithm parameter to be configurable by users. (c5df3e8)
• Add a study_spec_parameters_override parameter to Tables v1 and stage_1 component pipelines and update handling logic (7f886db)
• Add explanation metadata to upload model component for L2L pipelines (5898a19)
• Use 'unmanaged_container_model' instead of 'model' in infra validator component for all pipelines (768d7a9)
• Remove use of input_directionary_to_parameter in v1 Tables and component-specific pipelines only and read from executor_input instead (958a181)
• Feature transform engine config planner to generate training schema & instance baseline (b0a7ba5)
• Decouple transform config planner from metadata (fcb8f5f)
• Make Tables ensemble also output explanation_metadata artifact. (ed8e7e9)
• Internal change (26fb337)
• Internal change (5b74b21)
• Merge pull request #7776 from evanseabrook:fix/empty-explanation-features (e3fc7cd)
• Fix AutoML Tables pipeline and builtin pipelines on VPC-SC environment. (023680d)
• Make stats-gen component to support running with example-gen only mode (f85dbe6)
• Remove trailing whitespace (cf0a2ca)
• KFP component for ml.detect_anomalies (d05110e)
• Adding ML.GLOBAL_EXPLAIN KFP BQ Component. (b1c803b)
• Add distill + evaluation pipeline for Tables (83e618e)
• KFP component for ml.forecast (79b3335)
• KFP component for ml.explain_forecast (44d9575)
• Add ML.ARIMA_EVALUATE in component.yaml (dd8ccd2)
• Adding ML.Recommend KFP BQ component. (56c9111)
• Add ML.ARIMA_COEFFICIENTS in component.yaml (7af13b6)
• Adding ML.FEATURE_IMPORTANCE KFP BQ Component. (b067fc6)
• Adding ML.PRINCIPAL_COMPONENTS and ML.PRINCIPAL_COMPONENT_INFO KFP BQ component. (e7dbbcb)
• Adding ML. ROC_CURVE KFP BQ Component. (640ef6f)
• Merge distill_skip_evaluation and skip_evaluation pipelines with default pipeline using dsl.Condition (e95b9f5)
• Adding ML.FEATURE_INFO KFP BQ Component. (cdd5760)
• Adding ML.CONFUSION_MATRIX KFP BQ Component. (8b9fe02)
• Wide and Deep and Tabnet models both now use the Feature Transform Engine pipeline instead of the Transform component. (1365f83)
• fix(frontend) Regenerate the Typescript client library from swagger to match the backend changes in #7788 (#7847) (2c065d3)
• add additional experiments in distillation pipeline. (db9ee59)
• Adding ML.TRAINING_INFO KFP and ML.EXPLAIN_PREDICT BQ Component. (a6d0b86)
• FTE now outputs training schema. (be317d0)
• FTE transform config passed as path to config file instead of directly as string to FTE. (b83327f)
• Add model eval component to built-in algorithm default pipelines (8829400)
• add README with project description (#7782) (2c27475)
• Renamed "Feature Transform Engine Transform Configuration" component to "Transform Configuration Planner" for clarity. (e38df3a)
• feat[deployment]: add startup probe to pipeline API deployment (#7741) (dd98b1e)
• Update default machine type to c2-standard-16 for built-in algorithms Custom and HyperparameterTuning Jobs (93ea81b)
• Removed default location setting from AutoML components and documentation. (0932905)
• Add window config to ARIMA pipeline. (f77ebc8)
• Stats Gen and Feature Transform Engine pipeline integration. (1e105d3)
• Uses BigQuery batch queries in ARIMA pipeline after first 50 queries. (9e7de60)
• Typo fix (99b278f)
• Allow ARIMA pipeline to overwrite output tables. (6efb7e5)
• Replace custom copy_table component with BQ first-party query component. (59c8c6f)
• support vpc in feature selection (a446ec6)
• Fix failure when running distillation pipeline due to unexpected additional_experiments pipeline parameter. (076b050)
• fix cli upload pipeline version (#7722) (49cdb42)
• Create Python-based component to set study_spec_parameters for Wide & Deep HyperparameterTuningJobs (4d4b88b)
• Create Python-based component to set study_spec_parameters for TabNet HyperparameterTuningJobs (8fa1235)
• Add default Wide & Deep study_spec_parameters configs and add helper function to utils.py to get parameters (638a38e)
• Internal testing change (669374b)
• Update google-cloud-pipeline-components version and release notes (10d9fe3)
• Updates Workbench Executor component for Pipelines to support kernel_spec. (b40f450)
• Add default TabNet study_spec_parameters configs for different dataset sizes and search space modes and helper function to get the parameters (f877b6e)
• Add init file to container/experimental directory to recognize as a python module. (8c20a2c)
• add E2E test cases for classification type feature attribution pipeline. (0eb9d3b)
• Minor changes to the feature transform engine and transform configuration component specs to support their integration. (ab2a84e)

Major Features and Improvements

• feat(sdk): add .list_pipeline_versions and .unarchive_experiment methods to Client #7563
• Add additional methods to kfp.client.Client #7562, #7463
• Migrate V1 CLI to V2, with improvements #7547, #7558, #7559, #7560, , #7569, #7567, #7603, #7606, #7607, #7628, #7618

Breaking Changes

For Pipeline Authors

For Component Authors

Deprecations

Bug Fixes and Other Changes

• Accepts typing-extensions>=4,<5 in addition to typing-extensions>=3.7.4,<4 #7632
• Remove dependency on pydantic #7639

Documentation Updates

⚠ BREAKING CHANGES

• sdk: promote v2 compiler command and deprecate v1 command (#7508)

Features

• api: Update IR with RetryPolicy (#7581) (27d5404)
• components: Add AutoML Tables default pipeline with evaluation (1265649)
• components: Create new import model evaluation component (00eb97c)
• components: Improve AutoML Tables default pipeline with evaluation component (b98a2ab)
• components: Take problem type as input rather than schema URI (d1c0c75)
• components: Tests for import model evaluation (cdddaef)
• components/google-cloud: Add support for env variables in Custom_Job component. (5080288)
• components/google-cloud: Support parametrized input for reserved_ip_range and other Vertex Training parameters in custom job utility. (be682e4)
• conformance: adapt pipeline API tests to KF environment (#7647) (694a274)
• conformance: adapt pipeline version API tests to run in KF (#7653) (913277a)
• conformance: adapt run api tests to KF env (#7658) (8b11551)
• conformance: Enable experiment API tests in KF mode. (#7596) (1265113)
• frontend: Automatically fetch MLMD for Run status update in V2. (#7636) (d1a12cb)
• frontend: Enable KFPv2 UI by default by creating a new feature key #7610 (#7624) (5fb63c7)
• frontend: Support default feature flags update in localStorage. Fix #7379 (#7537) (7f4e9a4)
• google-cloud: Add internal flag to ImportModelEvaluation component (e7985b8)
• google-cloud: Add new pipeline template for Vertex Evaluation (8b81223)
• sdk: Accept newer typing-extensions versions (#7632) (1118f48)
• sdk: add .list_pipeline_versions and .unarchive_experiment methods to Client (#7563) (2494447)
• sdk: add archive, unarchive, and delete run methods and tidy docstrings (#7562) (2636727)
• sdk: add autocomplete and version options to kfp cli (#7567) (fbfeadd)
• sdk: add client methods to cli (#7606) (027ac32)
• sdk: add noun aliasing to cli (#7569) (e077d01)
• sdk: improve cli help text (#7618) (ec00813)
• sdk: port cli code from v1 to v2 (#7547) (d46fafe)
• sdk: promote v2 compiler command and deprecate v1 command (#7508) (4eeb420)
• sdk: rename CLI methods to 'create' (#7607) (652ec3b)
• sdk: use custom basemodel and remove pydantic (#7639) (5da3826)
• sdk: use dsl noun group for compile CLI commands (#7603) (e12ac39)

Bug Fixes

• backend: fix simple loop bug (#7578) (1180e10)
• components: Add retries to queries hitting BQ write quota. (c19facc)
• frontend: Convert PipelineSpec format from json to yaml #7444 (#7591) (ec32711)
• frontend: Update sample pipelines in mock-backend to use yaml format (#7570) (cdbfbe8)
• frontend: Use yaml mode for PipelineSpec editor view (#7637) (7d6edee)
• sdk: some cli fixes (#7668) (18f0842)

Other Pull Requests

• Update test data for evaluation classification feature attribution pipeline. (d822f6a)
• Internal change (6ceea6f)
• Pipeline level exit handler and v2 test (#7642) (98a58c7)
• generate feature selection tuning pipeline and test utils. (c6aaaef)
• Internal change. (afe07f1)
• Update TabNet and Wide & Deep parameter descriptions (b30fcee)
• add deprecation warning to dsl-compile (#7628) (6cecfd9)
• Update automl first-party pipeline jsons. (bc73718)
• internal change. (2038336)
• Add additional experiment flag in AutoML Tables related pipelines. (0dea8ba)
• Fix the problem that skip_architecture_search pipeline is using the wrong compiled json. (9e4bd6a)
• use click for components commands (#7559) (2db431b)
• use click for dsl-compile command (#7560) (a554844)
• Add feature transform engine component definition. (94a4459)
• Update evaluation yaml to v0.2 (d452261)
• Expose eval_frequency_secs and eval_steps as separate inputs in the built-in algorithm HPT component (88e4066)
• Add e2e tests for ARIMA pipeline and release to GCPC experimental dir. (5a8202e)
• Add template JSON pipeline spec for running evaluation on a managed GCP Vertex model. (0c91279)
• Update documentation for Dataproc Serverless components v1.0. (c64879f)
• Use if:cond:then when specifying image name in built-in algorithm hyperparameter tuning job component and add separate hyperparameter tuning job default pipelines for TabNet and Wide & Deep (493ce18)
• Add gcp_resources in the eval component output (f97a8aa)
• Add downsampled_test_split_json to example_and_stats_gen component. It is a JSON object that contains the downsampled test split information. (7884f46)
• Dataproc Serverless components v1.0 launch. (6949964)
• add typestub libraries (#7451) (d74d7b9)

Major Features and Improvements

• Enable pip installation from custom PyPI repository #7453

Breaking Changes

For Pipeline Authors

For Component Authors

Deprecations

Bug Fixes and Other Changes

• Fix wrong kfp import causes wrong sdk_version being set in pipeline_spec. #7433
• Use YAML as default serialization format for package IR #7431
• Support submitting pipeline IR in yaml format via kfp.client. #7458
• Add pipeline_task_name to PipelineTaskFinalStatus #7464
• Depends on kfp-pipeline-spec>=0.1.14,<0.2.0 #7464
• Depends on google-cloud-storage>=2.2.1,<3 #7493

Documentation Updates