NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #1196 most downloaded on PyPI
Provider package apache-airflow-providers-google for Apache Airflow
Last release today
24 Sep 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
1 version withdrawn
withdrawn after publishing
6 years old
226 releases · first in 2020
One column per quarter.
- Deprecate start_sql_job in DataflowHook
Release Date: 2025-04-19
Add ORC support to GCSToBigQueryOperator and test for external tables (#49188)
Fix default value wait_for_operation_result in OperationHelper (#49121)
Fix get_confluent_token callback for Managed Kafka service (#48926)
Fix: BigQuery connection UI defaults 'Use Legacy SQL' to true on edit (#49038)
remove superfluous else block (#49199)
Deprecate start_sql_job in DataflowHook (#48663)
type: annotate miss type in 'provider' (#49130)
Add ORC support to GCSToBigQueryOperator and test for external tables (#49188)
Fix default value wait_for_operation_result in OperationHelper (#49121)
Fix get_confluent_token callback for Managed Kafka service (#48926)
Fix: BigQuery connection UI defaults 'Use Legacy SQL' to true on edit (#49038)
remove superfluous else block (#49199)
Deprecate start_sql_job in DataflowHook (#48663)
type: annotate miss type in 'provider' (#49130)
Nothing published for this version
- Deprecate Life Science service
Release Date: 2025-04-14
Deprecate Life Science service (#48862)
Add missing license comment in google docs (#48985)
Deprecate Life Science service (#48862)
Add missing license comment in google docs (#48985)
Nothing published for this version
Deprecated classes, parameters and features have been removed from the Google provider package. The following breaking changes were introduced:
Release Date: 2025-04-09
Warning
Deprecated classes, parameters and features have been removed from the Google provider package. The following breaking changes were introduced:
Operators
Removed MLEngineStartBatchPredictionJobOperator use CreateBatchPredictionJobOperator instead
Removed MLEngineManageModelOperator use MLEngineCreateModelOperator or GetModelOperator from Vertex AI instead
Removed MLEngineDeleteModelOperator use DeleteModelOperator from Vertex AI instead
Removed MLEngineManageVersionOperator use appropriate operator from Vertex AI instead, e.g AddVersionAliasesOnModelOperator
Removed MLEngineCreateVersionOperator use parent_model parameter for VertexAI operators
Removed MLEngineSetDefaultVersionOperator use SetDefaultVersionOnModelOperator from Vertex AI instead
Removed MLEngineListVersionsOperator use ListModelVersionsOperator from Vertex AI instead
Removed MLEngineDeleteVersionOperator use DeleteModelVersionOperator from Vertex AI instead
Removed MLEngineStartTrainingJobOperator use CreateCustomPythonPackageTrainingJobOperator from Vertex AI instead
Removed MLEngineTrainingCancelJobOperator use CancelCustomTrainingJobOperator from Vertex AI instead
Removed DataProcScaleClusterOperator use DataprocUpdateClusterOperator instead
Removed page_size parameter from GoogleAdsToGcsOperator
Removed DeleteAutoMLTrainingJobOperator.training_pipeline method use training_pipeline_id parameter instead
Removed DeleteCustomTrainingJobOperator.training_pipeline method use training_pipeline_id parameter instead
Removed DeleteCustomTrainingJobOperator.custom_job method use custom_job_id parameter instead
Links
Removed DataFusionPipelineLinkHelper use airflow.providers.google.cloud.utils.helpers.resource_path_to_dict instead
Hooks
Removed page_size parameter from GoogleAdsHook.search , GoogleAdsHook.search_proto_plus , GoogleAdsHook._search methods
Removed CloudBuildHook.create_build method use CloudBuildHook.create_build_without_waiting_for_result method
Removed DataflowHook.start_java_dataflow method please use airflow.providers.apache.beam.hooks.beam.start.start_java_pipeline method to start pipeline and providers.google.cloud.hooks.dataflow.DataflowHook.wait_for_done method to wait for the required pipeline state instead
Removed DataflowHook.start_python_dataflow method please use airflow.providers.apache.beam.hooks.beam.start.start_python_pipeline method to start pipeline and providers.google.cloud.hooks.dataflow.DataflowHook.wait_for_done method to wait for the required pipeline state instead
Removed CustomJobHook.cancel_pipeline_job method use PipelineJobHook.cancel_pipeline_job method instead
Removed CustomJobHook.create_pipeline_job method use PipelineJobHook.create_pipeline_job method instead
Removed CustomJobHook.get_pipeline_job method use PipelineJobHook.get_pipeline_job method instead
Removed CustomJobHook.list_pipeline_jobs method use PipelineJobHook.list_pipeline_jobs method instead
Removed CustomJobHook.delete_pipeline_job method use PipelineJobHook.delete_pipeline_job method instead
Removed GoogleBaseHook.get_client_info method use airflow.providers.google.common.consts.CLIENT_INFO instead
LifeSciencesHook life period extended to July 08, 2025
Utils
Removed create_evaluate_ops method
Remove deprecated for February and March 2025 in providers/google. #48018
SFTPToGCSOperator: Added option to stream data directly to GCS without saving to disk (#48107)
fix: remove extra slash character from file URI formation in 'MetastoreHivePartitionSensor.poke' method. (#48731)
Disallowing types-protobuf 5.29.1.20250402 for google to fix mypy (#48666)
Add 'ti' to the RemoteLogIO read and upload methods (#48804)
Rework remote task log handling for the structlog era. (#48491)
Move bases classes to 'airflow.sdk.bases' (#48487)
Move 'BaseSensorOperator' to TaskSDK definitions (#48244)
Nothing published for this version
- Add DNS endpoint support for GKE Hooks and Operators
Release Date: 2025-03-31
Add DNS endpoint support for GKE Hooks and Operators (#48075)
feat: allow to set task/dag labels for 'DataprocCreateBatchOperator' (#46781)
Create operators for working with Consumer Groups for GCP Apache Kafka (#47056)
FIX: BigQuery traceability labels missing in TaskGroup (#47583)
Vendor-in json-merge-patch and add '--no-dev' to direct uv tool install (#48210)
serialize http transports contained in composite transport (#47444)
AIP-72: Handle Custom XCom Backend on Task SDK (#47339)
misc: update API version of Google ADS (#47772)
Introduce operation helper class and refactor (#45354)
Remove auth backends from core Airflow (#47399)
Mark Google Oauth2 backend as Airflow 2 only (#47622)
Add Dataflow Apache Beam Java streaming system test (#47209)
Add legacy namespace packages to airflow.providers (#47064)
Add DNS endpoint support for GKE Hooks and Operators (#48075)
feat: allow to set task/dag labels for 'DataprocCreateBatchOperator' (#46781)
Create operators for working with Consumer Groups for GCP Apache Kafka (#47056)
FIX: BigQuery traceability labels missing in TaskGroup (#47583)
Vendor-in json-merge-patch and add '--no-dev' to direct uv tool install (#48210)
serialize http transports contained in composite transport (#47444)
AIP-72: Handle Custom XCom Backend on Task SDK (#47339)
misc: update API version of Google ADS (#47772)
Introduce operation helper class and refactor (#45354)
Remove auth backends from core Airflow (#47399)
Mark Google Oauth2 backend as Airflow 2 only (#47622)
Add Dataflow Apache Beam Java streaming system test (#47209)
Add legacy namespace packages to airflow.providers (#47064)
Nothing published for this version
This version contains no code changes. It was released to replace a previous version that was yanked due to a packaging issue.
Release Date: 2025-02-26
Note
This version contains no code changes. It was released to replace a previous version that was yanked due to a packaging issue.
Nothing published for this version
Deprecated classes, parameters and features have been removed from the Google provider package. The following breaking changes were introduced:
Release Date: 2025-02-21
Warning
Deprecated classes, parameters and features have been removed from the Google provider package. The following breaking changes were introduced:
Operators
Removed AutoMLBatchPredictOperator . Please use the operators from airflow.providers.google.cloud.operators.vertex_ai.batch_prediction_job instead
Removed DataflowStartSqlJobOperator . Please DataflowStartYamlJobOperator instead
Removed PromptLanguageModelOperator . Please TextGenerationModelPredictOperator instead
Removed GenerateTextEmbeddingsOperator . Please TextEmbeddingModelGetEmbeddingsOperator instead
Removed PromptMultimodalModelOperator . Please GenerativeModelGenerateContentOperator instead
Removed PromptMultimodalModelWithMediaOperator . Please GenerativeModelGenerateContentOperator instead
Hooks
Removed GenerativeModelHook.prompt_multimodal_model_with_media() . Please use GenerativeModelHook.generative_model_generate_content() instead
Removed GenerativeModelHook.prompt_multimodal_model() . Please use GenerativeModelHook.generative_model_generate_content() instead
Removed GenerativeModelHook.get_generative_model_part() . Please use GenerativeModelHook.generative_model_generate_content() instead
Removed GenerativeModelHook.prompt_language_model() . Please use GenerativeModelHook.text_generation_model_predict() instead
Removed GenerativeModelHook.generate_text_embeddings() . Please use GenerativeModelHook.text_generation_model_predict() instead
Remove deprecated functionality from google provider (#46235)
Create operators for working with Clusters for GCP Apache Kafka (#45795)
Deprecate old bigquery operators and introduce CreateTableOperator (#46673)
Add operators for Entry resource (#46647)
Add deferrable mode to the PubSubPullOperator (#45835)
Add deferrable mode to google cloud storage transfer sensor and operators (#45754)
Add operators for AspectType resource (#46240)
feat: Add OpenLineage support for BigQueryDataTransferServiceStartTransferRunsOperator (#45801)
feat: Add OpenLineage support for CloudSQLExecuteQueryOperator (#45182)
Add new operators for Entry Type (#45799)
Add Dataplex Catalog Entry Group operators (#45751)
feat: Add Hook Level Lineage support for BigQueryHook (#45706)
feat: Add OpenLineage support for MsSqlHook and MSSQLToGCSOperator (#45637)
Adding holiday_region parameter to create_auto_ml_forecasting_training_job in AutoMl hook (#45465)
feat: automatically inject OL transport info into spark jobs (#45326)
feat: Add OpenLineage support for transfer operators between GCS and SFTP (#45485)
Implement AlloyDB create/update/delete user and backups operators (#45378)
feat: Add OpenLineage support for some SQL to GCS operators (#45242)
feat: Add OpenLineage support for some BQ operators (#45422)
feat: Add OpenLineage support for non-query jobs in BigQueryInsertJobOperator (#44996)
feat: automatically inject OL info into spark job in DataprocInstantiateInlineWorkflowTemplateOperator (#44697)
feat: Add CLL to OpenLineage in BigQueryInsertJobOperator (#44872)
feat: automatically inject OL info into spark job in DataprocCreateBatchOperator (#44612)
Implement AlloyDB create/update/delete instance operators (#45241)
Add BigQuery job link (#45020) (#45222)
Introduce gcp translation(V3) glossaries providers (#45085)
Add dependencies section to gcp cloud build instance response (#46947)
Create operators for working with Topics for GCP Apache Kafka (#46865)
fix return job_id when async finish before deffed in BigQueryInsertJobOperator (#46753)
fix: Dataproc operators fail to import without OpenLineage (#46561)
Add backward compatibility for old Airflow version for CloudComposerDAGRunSensor (#46011)
Improve google credentials error message (#45553)
Add case of aarch64 environments in CloudSqlProxyRunner (#45488)
fix GCSToGCSOperator bug when copy single object with replace to False (#45181)
Rework the TriggererJobRunner to run triggers in a process without DB access (#46677)
AIP-72: Improving Operator Links Interface to Prevent User Code Execution in Webserver (#46613)
Remove Airflow 3 Deprecation Warning in Google Provider, Move to Private Warning (#46599)
Add evaluation extra to google-cloud-aiplatform (#46270)
Limit 'google-cloud-aiplatform' to fix issues in CI (#46242)
Improve grammar in cloud_composer.rst (#45708)
AIP-72: Support better type-hinting for Context dict in SDK (#45583)
Deprecate page_size parameter for Google Ads API v17 and later versions (#45239)
Remove obsolete pandas specfication for pre-python 3.9 (#45399)
Deprecation policy for apache-airflow-providers-google package (#44985)
Add lower bound for alloydb (#45180)
Deprecate gcp AutoML module (#44875)
Refactor deferrable mode for BeamRunPythonPipelineOperator and BeamRunJavaPipelineOperator (#46678)
Bump cloud build to 3.31.0 (#46953)
Warning
Deprecated classes, parameters and features have been removed from the Google provider package. The following breaking changes were introduced:
Operators
Removed AutoMLBatchPredictOperator. Please use the operators from airflow.providers.google.cloud.operators.vertex_ai.batch_prediction_job instead
Removed DataflowStartSqlJobOperator. Please DataflowStartYamlJobOperator instead
Removed PromptLanguageModelOperator. Please TextGenerationModelPredictOperator instead
Removed GenerateTextEmbeddingsOperator. Please TextEmbeddingModelGetEmbeddingsOperator instead
Removed PromptMultimodalModelOperator. Please GenerativeModelGenerateContentOperator instead
Removed PromptMultimodalModelWithMediaOperator. Please GenerativeModelGenerateContentOperator instead
Hooks
Removed GenerativeModelHook.prompt_multimodal_model_with_media(). Please use GenerativeModelHook.generative_model_generate_content() instead
Removed GenerativeModelHook.prompt_multimodal_model(). Please use GenerativeModelHook.generative_model_generate_content() instead
Removed GenerativeModelHook.get_generative_model_part(). Please use GenerativeModelHook.generative_model_generate_content() instead
Removed GenerativeModelHook.prompt_language_model(). Please use GenerativeModelHook.text_generation_model_predict() instead
Removed GenerativeModelHook.generate_text_embeddings(). Please use GenerativeModelHook.text_generation_model_predict() instead
Remove deprecated functionality from google provider (#46235)
Create operators for working with Clusters for GCP Apache Kafka (#45795)
Deprecate old bigquery operators and introduce CreateTableOperator (#46673)
Add operators for Entry resource (#46647)
Add deferrable mode to the PubSubPullOperator (#45835)
Add deferrable mode to google cloud storage transfer sensor and operators (#45754)
Add operators for AspectType resource (#46240)
feat: Add OpenLineage support for BigQueryDataTransferServiceStartTransferRunsOperator (#45801)
feat: Add OpenLineage support for CloudSQLExecuteQueryOperator (#45182)
Add new operators for Entry Type (#45799)
Add Dataplex Catalog Entry Group operators (#45751)
feat: Add Hook Level Lineage support for BigQueryHook (#45706)
feat: Add OpenLineage support for MsSqlHook and MSSQLToGCSOperator (#45637)
Adding holiday_region parameter to create_auto_ml_forecasting_training_job in AutoMl hook (#45465)
feat: automatically inject OL transport info into spark jobs (#45326)
feat: Add OpenLineage support for transfer operators between GCS and SFTP (#45485)
Implement AlloyDB create/update/delete user and backups operators (#45378)
feat: Add OpenLineage support for some SQL to GCS operators (#45242)
feat: Add OpenLineage support for some BQ operators (#45422)
feat: Add OpenLineage support for non-query jobs in BigQueryInsertJobOperator (#44996)
feat: automatically inject OL info into spark job in DataprocInstantiateInlineWorkflowTemplateOperator (#44697)
feat: Add CLL to OpenLineage in BigQueryInsertJobOperator (#44872)
feat: automatically inject OL info into spark job in DataprocCreateBatchOperator (#44612)
Implement AlloyDB create/update/delete instance operators (#45241)
Add BigQuery job link (#45020) (#45222)
Introduce gcp translation(V3) glossaries providers (#45085)
Add dependencies section to gcp cloud build instance response (#46947)
Create operators for working with Topics for GCP Apache Kafka (#46865)
fix return job_id when async finish before deffed in BigQueryInsertJobOperator (#46753)
fix: Dataproc operators fail to import without OpenLineage (#46561)
Add backward compatibility for old Airflow version for CloudComposerDAGRunSensor (#46011)
Improve google credentials error message (#45553)
Add case of aarch64 environments in CloudSqlProxyRunner (#45488)
fix GCSToGCSOperator bug when copy single object with replace to False (#45181)
Rework the TriggererJobRunner to run triggers in a process without DB access (#46677)
AIP-72: Improving Operator Links Interface to Prevent User Code Execution in Webserver (#46613)
Remove Airflow 3 Deprecation Warning in Google Provider, Move to Private Warning (#46599)
Add evaluation extra to google-cloud-aiplatform (#46270)
Limit 'google-cloud-aiplatform' to fix issues in CI (#46242)
Improve grammar in cloud_composer.rst (#45708)
AIP-72: Support better type-hinting for Context dict in SDK (#45583)
Deprecate page_size parameter for Google Ads API v17 and later versions (#45239)
Remove obsolete pandas specfication for pre-python 3.9 (#45399)
Deprecation policy for apache-airflow-providers-google package (#44985)
Add lower bound for alloydb (#45180)
Deprecate gcp AutoML module (#44875)
Refactor deferrable mode for BeamRunPythonPipelineOperator and BeamRunJavaPipelineOperator (#46678)
Bump cloud build to 3.31.0 (#46953)
Deprecated classes, parameters and features have been removed from the Google provider package. The following breaking changes were introduced:
Release Date: 2024-12-26
Note
This release of provider is only available for Airflow 2.9+ as explained in the Apache Airflow providers support policy .
Warning
Deprecated classes, parameters and features have been removed from the Google provider package. The following breaking changes were introduced:
Operators
Removed CreateDataPipelineOperator . Please use the DataflowCreatePipelineOperator instead
Removed RunDataPipelineOperator . Please use the DataflowRunPipelineOperator instead
Removed CreateDataPipelineOperator . Please use the DataflowCreatePipelineOperator instead
Removed RunDataPipelineOperator . Please use the DataflowRunPipelineOperator instead
Hooks
Removed DataPipelineHook . Please use the DataflowHook instead
Removed DataPipelineHook . Please use the DataflowHook instead
Google provider delete deprecated reaching removal date (December 2024) (#45084)
Add Google Vertex AI Feature Store - Feature View Sync Operators, Sensor (#44891)
Introduce gcp translation(V3), translate document providers (#44971)
Introduce the translation API v3 (advanced) models operators. (#44627)
Support multiple SQL queries in Dataproc SQL job (#44890)
feat: add OpenLineage support for BigQuery Create Table operators (#44783)
feat: add OpenLineage support for S3ToGCSOperator (#44426)
feat: automatically inject OL info into spark job in DataprocSubmitJobOperator (#44477)
Implement AlloyDB operators: create/update/delete clusters (#45027)
Fix MetastoreHivePartitionSensor failing due to duplicate aliases (#45001)
Fix failing OpenLineage emission for InsertBigQueryOperator (#44650)
BigQueryInsertJobOperator: log transient error and check job state before marking task as success (#44279)
Make 'CloudBatchSubmitJobOperator' fail when job fails (#44425)
Remove references to AIRFLOW_V_2_9_PLUS (#44987)
Consistent way of checking Airflow version in providers (#44686)
Fix yoda-conditions (#44466)
feat: add OpenLineage support for transfer operators between gcs and local (#44417)
Bump minimum Airflow version in providers to Airflow 2.9.0 (#44956)
chore: remove deprecated bigquery facets from OpenLineage utils (#44838)
Implement KubernetesInstallKueueOperator (#44568)
Fix docs for dataform operators (#44729)
Deprecate VertexAI PaLM text generative model (#44719)
Show prominent warning for deprecations in docs (#44479)
chore: unify handling of gcs paths (#44410)
Nothing published for this version
Nothing published for this version
Deprecated classes, parameters and features have been removed from the Google provider package. The following breaking changes were introduced:
Release Date: 2024-11-27
Warning
Deprecated classes, parameters and features have been removed from the Google provider package. The following breaking changes were introduced:
Operators
Removed CreateAutoMLTextTrainingJobOperator . Use SupervisedFineTuningTrainOperator instead
Removed BigQueryExecuteQueryOperator . Use BigQueryInsertJobOperator instead
Removed BigQueryPatchDatasetOperator . Use BigQueryUpdateDatasetOperator instead
Removed DataflowCreateJavaJobOperator . Use BeamRunJavaPipelineOperator instead
Removed DataflowCreatePythonJobOperator . Use BeamRunPythonPipelineOperator instead
Removed DataprocSubmitPigJobOperator . Use DataprocSubmitJobOperator instead
Removed DataprocSubmitHiveJobOperator . Use DataprocSubmitJobOperator instead
Removed DataprocSubmitSparkSqlJobOperator . Use DataprocSubmitJobOperator instead
Removed DataprocSubmitSparkJobOperator . Use DataprocSubmitJobOperator instead
Removed DataprocSubmitHadoopJobOperator . Use DataprocSubmitJobOperator instead
Removed DataprocSubmitPySparkJobOperator . Use DataprocSubmitJobOperator instead
Removed GoogleAnalyticsListAccountsOperator . Use GoogleAnalyticsAdminListAccountsOperator instead
Removed GoogleAnalyticsGetAdsLinkOperator . Use GoogleAnalyticsAdminGetGoogleAdsLinkOperator instead
Removed GoogleAnalyticsRetrieveAdsLinksListOperator . Use GoogleAnalyticsAdminListGoogleAdsLinksOperator instead
Removed GoogleAnalyticsDataImportUploadOperator . Use GoogleAnalyticsAdminCreateDataStreamOperator instead
Removed GoogleAnalyticsDeletePreviousDataUploadsOperator . Use GoogleAnalyticsAdminDeleteDataStreamOperator instead
Removed GoogleAnalyticsModifyFileHeadersDataImportOperator . The class is no longer in actual use due to Google Analytics API v3 has reached sunset, and thus the covered use case is no longer relevant
Removed GoogleCampaignManagerDeleteReportOperator.delegate_to . Use GoogleCampaignManagerDeleteReportOperator.impersonation_chain instead
Removed GoogleCampaignManagerDownloadReportOperator.delegate_to . Use GoogleCampaignManagerDownloadReportOperator.impersonation_chain instead
Removed GoogleCampaignManagerInsertReportOperator.delegate_to . Use GoogleCampaignManagerInsertReportOperator.impersonation_chain instead
Removed GoogleCampaignManagerRunReportOperator.delegate_to . Use GoogleCampaignManagerRunReportOperator.impersonation_chain instead
Removed GoogleCampaignManagerBatchInsertConversionsOperator.delegate_to . Use GoogleCampaignManagerBatchInsertConversionsOperator.impersonation_chain instead
Removed GoogleCampaignManagerBatchUpdateConversionsOperator.delegate_to . Use GoogleCampaignManagerBatchUpdateConversionsOperator.impersonation_chain instead
Removed GoogleDisplayVideo360CreateQueryOperator.delegate_to . Use GoogleDisplayVideo360CreateQueryOperator.impersonation_chain instead
Removed GoogleDisplayVideo360DeleteReportOperator.delegate_to . Use GoogleDisplayVideo360DeleteReportOperator.impersonation_chain instead
Removed GoogleDisplayVideo360DownloadReportV2Operator.delegate_to . Use GoogleDisplayVideo360DownloadReportV2Operator.impersonation_chain instead
Removed GoogleDisplayVideo360RunQueryOperator.delegate_to . Use GoogleDisplayVideo360RunQueryOperator.impersonation_chain instead
Removed GoogleDisplayVideo360DownloadLineItemsOperator.delegate_to . Use GoogleDisplayVideo360DownloadLineItemsOperator.impersonation_chain instead
Removed GoogleDisplayVideo360UploadLineItemsOperator.delegate_to . Use GoogleDisplayVideo360UploadLineItemsOperator.impersonation_chain instead
Removed GoogleDisplayVideo360CreateSDFDownloadTaskOperator.delegate_to . Use GoogleDisplayVideo360CreateSDFDownloadTaskOperator.impersonation_chain instead
Removed GoogleDisplayVideo360SDFtoGCSOperator.delegate_to . Use GoogleDisplayVideo360SDFtoGCSOperator.impersonation_chain instead
Removed GoogleSheetsCreateSpreadsheetOperator.delegate_to . Use GoogleSheetsCreateSpreadsheetOperator.impersonation_chain instead
Removed GCSToGoogleDriveOperator.delegate_to . Use GCSToGoogleDriveOperator.impersonation_chain instead
Removed GCSToGoogleSheetsOperator.delegate_to . Use GCSToGoogleSheetsOperator.impersonation_chain instead
Removed LocalFilesystemToGoogleDriveOperator.delegate_to . Use LocalFilesystemToGoogleDriveOperator.impersonation_chain instead
Removed SQLToGoogleSheetsOperator.delegate_to . Use SQLToGoogleSheetsOperator.impersonation_chain instead
Removed CreateBatchPredictionJobOperator.sync . This parameter is not in actual use
Removed CreateHyperparameterTuningJobOperator.sync . This parameter is not in actual use
Removed CustomTrainingJobBaseOperator.sync . This parameter is not in actual use
Removed GKEStartPodOperator.get_gke_config_file() . Please use GKEStartPodOperator.fetch_cluster_info() instead
Triggers
Removed support of delegate_to field in the GCSCheckBlobUpdateTimeTrigger.hook_params parameter
Sensors
Removed BigQueryTableExistenceAsyncSensor . Use BigQueryTableExistenceSensor and set deferrable attribute to True instead
Removed BigQueryTableExistencePartitionAsyncSensor . Use BigQueryTablePartitionExistenceSensor and set deferrable attribute to True instead
Removed CloudComposerEnvironmentSensor . Use CloudComposerCreateEnvironmentOperator , CloudComposerUpdateEnvironmentOperator , or CloudComposerDeleteEnvironmentOperator instead
Removed GCSObjectExistenceAsyncSensor . Use GCSObjectExistenceSensor and set deferrable attribute to True instead
Removed GoogleCampaignManagerReportSensor.delegate_to . Use GoogleCampaignManagerReportSensor.impersonation_chain instead
Removed GoogleDisplayVideo360GetSDFDownloadOperationSensor.delegate_to . Use GoogleDisplayVideo360GetSDFDownloadOperationSensor.impersonation_chain instead
Removed GoogleDisplayVideo360RunQuerySensor.delegate_to . Use GoogleDisplayVideo360RunQuerySensor.impersonation_chain instead
Removed GoogleDriveFileExistenceSensor.delegate_to . Use GoogleDriveFileExistenceSensor.impersonation_chain instead
Links
Removed BigQueryConsoleIndexableLink . This property is no longer in actual use
Removed BigQueryConsoleLink . This property is no longer in actual use
Hooks
Removed GKEDeploymentHook . Use GKEKubernetesHook instead
Removed GKECustomResourceHook . Use GKEKubernetesHook instead
Removed GKEPodHook . Use GKEKubernetesHook instead
Removed GKEJobHook . Use GKEKubernetesHook instead
Removed GKEPodAsyncHook . Use GKEKubernetesAsyncHook instead
Removed SecretsManagerHook . Use GoogleCloudSecretManagerHook instead
Removed GoogleAnalyticsHook . The class is no longer in actual use due to Google Analytics API v3 has reached sunset
Removed GoogleBaseHook.delegate_to . Please use GoogleBaseHook.impersonation_chain . Please note that the delegate_to parameter used to be inherited in all Google hooks, and from now its support is removed everywhere within the Google provider
Removed GoogleDiscoveryApiHook.delegate_to . Please use GoogleDiscoveryApiHook.impersonation_chain instead
Removed GoogleCampaignManagerHook.delegate_to . Please use GoogleCampaignManagerHook.impersonation_chain instead
Removed GoogleDisplayVideo360Hook.delegate_to . Please use GoogleDisplayVideo360Hook.impersonation_chain instead
Removed GoogleSearchAdsHook.delegate_to . Please use GoogleSearchAdsHook.impersonation_chain instead
Removed GoogleCalendarHook.delegate_to . Please use GoogleCalendarHook.impersonation_chain instead
Removed GoogleDriveHook.delegate_to . Please use GoogleDriveHook.impersonation_chain instead
Removed GSheetsHook.delegate_to . Please use GSheetsHook.impersonation_chain instead
Removed BigQueryHook.credentials_path . This property is no longer in actual use
Removed GKEHook.get_conn() . Please use GKEHook.get_cluster_manager_client() instead
Removed GKEHook.get_client() . Please use GKEHook.get_cluster_manager_client() instead
Removed BigQueryHook.patch_table() . Please use BigQueryHook.update_table() instead
Removed BigQueryHook.patch_dataset() . Please use BigQueryHook.update_dataset() instead
Removed BigQueryHook.get_dataset_tables_list() . Please use BigQueryHook.get_dataset_tables() instead
Removed BigQueryHook.run_table_delete() . Please use BigQueryHook.delete_table() instead
Removed BigQueryHook.get_tabledata() . Please use BigQueryHook.list_rows() instead
Removed BigQueryHook.cancel_query() . Please use BigQueryHook.cancel_job() instead
Removed BigQueryHook.run_with_configuration() . Please use BigQueryHook.insert_job() instead
Removed BigQueryHook.run_load() . Please use BigQueryHook.insert_job() instead
Removed BigQueryHook.run_copy() . Please use BigQueryHook.insert_job() instead
Removed BigQueryHook.run_extract() . Please use BigQueryHook.insert_job() instead
Removed BigQueryHook.run_query() . Please use BigQueryHook.insert_job() instead
Removed BigQueryHook.create_external_table() . Please use BigQueryHook.create_empty_table() instead
Removed BigQueryHook.get_service() . Please use BigQueryHook.get_client() instead
Backends
Removed CloudSecretManagerBackend.get_conn_uri() . Please use CloudSecretManagerBackend.get_conn_value() instead
Other deprecations
Removed BigQueryBaseCursor.create_empty_table() . Please use BigQueryHook.create_empty_table() instead
Removed BigQueryBaseCursor.create_empty_dataset() . Please use BigQueryHook.create_empty_dataset() instead
Removed BigQueryBaseCursor.get_dataset_tables() . Please use BigQueryHook.get_dataset_tables() instead
Removed BigQueryBaseCursor.delete_dataset() . Please use BigQueryHook.delete_dataset() instead
Removed BigQueryBaseCursor.create_external_table() . Please use BigQueryHook.create_empty_table() instead
Removed BigQueryBaseCursor.patch_table() . Please use BigQueryHook.update_table() instead
Removed BigQueryBaseCursor.insert_all() . Please use BigQueryHook.insert_all() instead
Removed BigQueryBaseCursor.update_dataset() . Please use BigQueryHook.update_dataset() instead
Removed BigQueryBaseCursor.patch_dataset() . Please use BigQueryHook.update_dataset() instead
Removed BigQueryBaseCursor.get_dataset_tables_list() . Please use BigQueryHook.get_dataset_tables() instead
Removed BigQueryBaseCursor.get_datasets_list() . Please use BigQueryHook.get_datasets_list() instead
Removed BigQueryBaseCursor.get_dataset() . Please use BigQueryHook.get_dataset() instead
Removed BigQueryBaseCursor.run_grant_dataset_view_access() . Please use BigQueryHook.run_grant_dataset_view_access() instead
Removed BigQueryBaseCursor.run_table_upsert() . Please use BigQueryHook.run_table_upsert() instead
Removed BigQueryBaseCursor.run_table_delete() . Please use BigQueryHook.delete_table() instead
Removed BigQueryBaseCursor.get_tabledata() . Please use BigQueryHook.list_rows() instead
Removed BigQueryBaseCursor.get_schema() . Please use BigQueryHook.get_schema() instead
Removed BigQueryBaseCursor.poll_job_complete() . Please use BigQueryHook.poll_job_complete() instead
Removed BigQueryBaseCursor.cancel_query() . Please use BigQueryHook.cancel_job() instead
Removed BigQueryBaseCursor.run_with_configuration() . Please use BigQueryHook.insert_job() instead
Removed BigQueryBaseCursor.run_load() . Please use BigQueryHook.insert_job() instead
Removed BigQueryBaseCursor.run_copy() . Please use BigQueryHook.insert_job() instead
Removed BigQueryBaseCursor.run_extract() . Please use BigQueryHook.insert_job() instead
Removed BigQueryBaseCursor.run_query() . Please use BigQueryHook.insert_job() instead
Remove deprecated functionality from Google provider (#43953)
feat: add OpenLineage support for BigQueryToBigQueryOperator (#44214)
Introduce gcp advance API (V3) translate native datasets operators (#44271)
Introduce new gcp TranslateText and TranslateTextBatch operators (#43860)
Add gcloud command to DataprocCreateClusterOperator to be able to create dataproc on GKE cluster (#44185)
Fix incorrect query in 'BigQueryAsyncHook.create_job_for_partition_get'. (#44225)
Fix Dataplex Data Quality partial update (#44262)
Bump 'google-cloud-translate' to '3.16' (#44297)
Bump 'google-cloud-datacatalog' to '3.23' (#44281)
Rename execution_date to logical_date across codebase (#43902)
Nothing published for this version
- Add support for IAM database authentication for CloudSQL connection
Release Date: 2024-11-18
Add support for IAM database authentication for CloudSQL connection (#43631)
Provide option to 'force_delete' for 'GCSToBigQueryOperator' (#43785)
Unify reattach_states parameter logic across BigQuery operators (#43259)
Remove non-existing field from the ListCustomTrainingJobOperator's template_fields (#43924)
Fix validating 'parent_model' parameter in 'UploadModelOperator' (#43473)
Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916)
Remove Airflow 2.1 compat code in Google provider (#43952)
Explain how to use uv with airflow virtualenv and make it works (#43604)
Move python operator to Standard provider (#42081)
Update version of Google ADS (#43474)
Add support for IAM database authentication for CloudSQL connection (#43631)
Provide option to 'force_delete' for 'GCSToBigQueryOperator' (#43785)
Unify reattach_states parameter logic across BigQuery operators (#43259)
Remove non-existing field from the ListCustomTrainingJobOperator's template_fields (#43924)
Fix validating 'parent_model' parameter in 'UploadModelOperator' (#43473)
Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916)
Remove Airflow 2.1 compat code in Google provider (#43952)
Explain how to use uv with airflow virtualenv and make it works (#43604)
Move python operator to Standard provider (#42081)
Update version of Google ADS (#43474)
Nothing published for this version
- feat: add Hook Level Lineage support for GCSHook
Release Date: 2024-10-31
feat: add Hook Level Lineage support for GCSHook (#42507)
feat: sensor to check status of Dataform action (#43055)
Create Operators for Google Cloud Vertex AI Context Caching (#43008)
Fix outdated CloudRunExecuteJobOperator docs (#43195)
Fix TestTranslationLegacyModelPredictLink dataset_id error (#42463)
Add a debug log for creating batch workloads in dataproc (#43265)
add min version to plyvel (#43129)
vertex ai training operators: add display_name to rendered fields (#43028)
Make google provider pyarrow dependency explicit (#42996)
Restrict looker-sdk version 24.18.0 and microsoft-kiota-http 1.3.4 (#42954)
feat: add Hook Level Lineage support for GCSHook (#42507)
feat: sensor to check status of Dataform action (#43055)
Create Operators for Google Cloud Vertex AI Context Caching (#43008)
Fix outdated CloudRunExecuteJobOperator docs (#43195)
Fix TestTranslationLegacyModelPredictLink dataset_id error (#42463)
Add a debug log for creating batch workloads in dataproc (#43265)
add min version to plyvel (#43129)
vertex ai training operators: add display_name to rendered fields (#43028)
Make google provider pyarrow dependency explicit (#42996)
Restrict looker-sdk version 24.18.0 and microsoft-kiota-http 1.3.4 (#42954)
Nothing published for this version
- Deprecate AutoMLBatchPredictOperator and refactor AutoMl system tests
Release Date: 2024-10-14
Add 'retry_if_resource_not_ready' logic for DataprocCreateClusterOperator and DataprocCreateBatchOperator (#42703)
Publish Dataproc Serverless Batch link after it starts if batch_id was provided (#41153)
fix PubSubAsyncHook in PubsubPullTrigger to use gcp_conn_id (#42671)
Fix consistent return response from PubSubPullSensor (#42080)
Undo partition exclusion from the table name when splitting a full BigQuery table name (#42541)
Fix gcp text to speech uri fetch (#42309)
Refactor ''bucket.get_blob'' calls in ''GCSHook'' to handle validation for non-existent objects. (#42474)
Bugfix/dataflow job location passing (#41887)
Removed conditional check for task context logging in airflow version 2.8.0 and above (#42764)
Rename dataset related python variable names to asset (#41348)
Deprecate AutoMLBatchPredictOperator and refactor AutoMl system tests (#42260)
Add 'retry_if_resource_not_ready' logic for DataprocCreateClusterOperator and DataprocCreateBatchOperator (#42703)
Publish Dataproc Serverless Batch link after it starts if batch_id was provided (#41153)
fix PubSubAsyncHook in PubsubPullTrigger to use gcp_conn_id (#42671)
Fix consistent return response from PubSubPullSensor (#42080)
Undo partition exclusion from the table name when splitting a full BigQuery table name (#42541)
Fix gcp text to speech uri fetch (#42309)
Refactor ''bucket.get_blob'' calls in ''GCSHook'' to handle validation for non-existent objects. (#42474)
Bugfix/dataflow job location passing (#41887)
Removed conditional check for task context logging in airflow version 2.8.0 and above (#42764)
Rename dataset related python variable names to asset (#41348)
Deprecate AutoMLBatchPredictOperator and refactor AutoMl system tests (#42260)
Nothing published for this version
- Enforce deprecation message format with EOL for google provider package
Release Date: 2024-09-24
Add ability to create Flink Jobs in dataproc cluster (#42342)
Add new Google Search 360 Reporting Operators (#42255)
Add return_immediately as argument to the PubSubPullSensor class (#41842)
Add parent_model param in 'UploadModelOperator' (#42091)
Add DataflowStartYamlJobOperator (#41576)
Add RunEvaluationOperator for Google Vertex AI Rapid Evaluation API (#41940)
Add CountTokensOperator for Google Generative AI CountTokensAPI (#41908)
Add Supervised Fine Tuning Train Operator, Hook, Tests, Docs (#41807)
Minor fixes to ensure successful Vertex AI LLMops pipeline (#41997)
Exclude partition from BigQuery table name (#42130)
[Fix #41763]: Redundant forward slash in SFTPToGCSOperator when destination_path is not specified or have default value (#41928)
Fix poll_interval in GKEJobTrigger (#41712)
update pattern for dataflow job id extraction (#41794)
Enforce deprecation message format with EOL for google provider package (#41637)
Fix 'do_xcom_push' and 'get_logs' functionality for KubernetesJobOperator (#40814)
Mark VertexAI AutoMLText deprecation (#42251)
Exclude google-cloud-spanner 3.49.0 (#42011)
Remove system test for derepcated Google analytics operators (#41946)
Update min version of google-cloud-bigquery package (#41882)
Unpin google-cloud-bigquery package version for Google provider (#41839)
Move away from deprecated DAG.following_schedule() method (#41773)
remove deprecated soft_fail from providers (#41710)
Update the version of google-ads (#41638)
Remove deprecated log handler argument filename_template (#41552)
Nothing published for this version
This release of provider is only available for Airflow 2.8+ as explained in the Apache Airflow providers support policy .
Release Date: 2024-08-22
Note
This release of provider is only available for Airflow 2.8+ as explained in the Apache Airflow providers support policy .
Add 'CloudRunServiceHook' and 'CloudRunCreateServiceOperator' (#40008)
fix(providers/google): add missing sync_hook_class to CloudDataTransferServiceAsyncHook (#41417)
Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)
Refactor 'DataprocCreateBatchOperator' (#41527)
Upgrade package gcloud-aio-auth>=5.2.0 (#41262)
Note
This release of provider is only available for Airflow 2.8+ as explained in the Apache Airflow providers support policy.
Add 'CloudRunServiceHook' and 'CloudRunCreateServiceOperator' (#40008)
fix(providers/google): add missing sync_hook_class to CloudDataTransferServiceAsyncHook (#41417)
Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)
Refactor 'DataprocCreateBatchOperator' (#41527)
Upgrade package gcloud-aio-auth>=5.2.0 (#41262)
Nothing published for this version
- Resolve CloudSQLDatabaseHook deprecation warning
Release Date: 2024-08-06
fix unnecessary imports for CloudSQL hook (#41009)
Move sensitive information to the secret manager for the system test google_analytics_admin (#40951)
Fix Custom Training Job operators to accept results without managed model (#40685)
Fix behavior for reattach_state parameter in BigQueryInsertJobOperator (#40664)
Fix CloudSQLDatabaseHook temp file handling (#41092)
Refactor dataproc system tests (#40720)
openlineage: migrate OpenLineage provider to V2 facets. (#39530)
Resolve CloudSQLDatabaseHook deprecation warning (#40834)
Fix BeamRunJavaPipelineOperator fails without job_name set (#40645)
fix unnecessary imports for CloudSQL hook (#41009)
Move sensitive information to the secret manager for the system test google_analytics_admin (#40951)
Fix Custom Training Job operators to accept results without managed model (#40685)
Fix behavior for reattach_state parameter in BigQueryInsertJobOperator (#40664)
Fix CloudSQLDatabaseHook temp file handling (#41092)
Refactor dataproc system tests (#40720)
openlineage: migrate OpenLineage provider to V2 facets. (#39530)
Resolve CloudSQLDatabaseHook deprecation warning (#40834)
Fix BeamRunJavaPipelineOperator fails without job_name set (#40645)
Nothing published for this version
Nothing published for this version
The change Fix 'GCSToGCSOperator' behavior difference for moving single object (#40162) has been reverted as it turned out to break too much existing
Release Date: 2024-07-12
Note
The change Fix 'GCSToGCSOperator' behavior difference for moving single object (#40162) has been reverted as it turned out to break too much existing workflows. The behavior of the GCSToGCSOperator has been restored to the previous behavior.
Update Google Cloud Generative Model Hooks/Operators to bring parity with Vertex AI API (#40484)
DataflowStartFlexTemplateOperator. Check for Dataflow job type each check cycle. (#40584)
Added chunk_size parameter to LocalFilesystemToGCSOperator (#40379)
Add support for query parameters to BigQueryCheckOperator (#40558)
Add link button to dataproc job in DataprocCreateBatchOperator (#40643)
Revert "Fix 'GCSToGCSOperator' behavior difference for moving single object (#40162)" (#40577)
fix BigQueryInsertJobOperator's return value and openlineage extraction in deferrable mode (#40457)
fix OpenLineage extraction for GCP deferrable operators (#40521)
fix respect project_id in CloudBatchSubmitJobOperator (#40560)
Note
The change Fix 'GCSToGCSOperator' behavior difference for moving single object (#40162) has been reverted as it turned out to break too much existing workflows. The behavior of the GCSToGCSOperator has been restored to the previous behavior.
Update Google Cloud Generative Model Hooks/Operators to bring parity with Vertex AI API (#40484)
DataflowStartFlexTemplateOperator. Check for Dataflow job type each check cycle. (#40584)
Added chunk_size parameter to LocalFilesystemToGCSOperator (#40379)
Add support for query parameters to BigQueryCheckOperator (#40558)
Add link button to dataproc job in DataprocCreateBatchOperator (#40643)
Revert "Fix 'GCSToGCSOperator' behavior difference for moving single object (#40162)" (#40577)
fix BigQueryInsertJobOperator's return value and openlineage extraction in deferrable mode (#40457)
fix OpenLineage extraction for GCP deferrable operators (#40521)
fix respect project_id in CloudBatchSubmitJobOperator (#40560)
Nothing published for this version
The GCSToGCSOperator now retains the nested folder structure when moving or copying a single object, aligning its behavior with the behavior for multi
Release Date: 2024-06-27
Note
The GCSToGCSOperator now retains the nested folder structure when moving or copying a single object, aligning its behavior with the behavior for multiple objects. If this change impacts your workflows, you may need to adjust your source_object parameter to include the full path up to the folder containing your single file and specify destination_object explicitly to ignore nested folders. For example, if you previously used source_object='folder/nested_folder/' , to move file 'folder/nested_folder/second_nested_folder/file' you should now use source_object='folder/nested_folder/second_nested_folder/' and specify destination_object='folder/nested_folder/' . This would move the file to 'folder/nested_folder/file' instead of the fixed behavior of moving it to 'folder/nested_folder/second_nested_folder/file' .
Warning
The change above has been reverted in the 10.21.0 release. The behavior of the GCSToGCSOperator has been restored to the previous behavior.
Add generation_config and safety_settings to google cloud multimodal model operators (#40126)
Add missing location param to 'BigQueryUpdateTableSchemaOperator' (#40237)
Add support for external IdP OIDC token retrieval for Google Cloud Operators. (#39873)
Add encryption_configuration parameter to BigQuery operators (#40063)
Add default gcp_conn_id to GoogleBaseAsyncHook (#40080)
Add ordering key option for PubSubPublishMessageOperator GCP Operator (#39955)
Add method to get metadata from GCS blob in GCSHook (#38398)
Add window parameters to create_auto_ml_forecasting_training_job in AutoMLHook (#39767)
Implement CloudComposerDAGRunSensor (#40088)
Implement 'CloudDataTransferServiceRunJobOperator' (#39154)
Fetch intermediate log async GKEStartPod (#39348)
Add OpenLineage support for AzureBlobStorageToGCSOperator in google provider package (#40290)
Fix hive_partition_sensor system test (#40023)
Fix openai 1.32 breaking openai tests (#40110)
Fix credentials initialization revealed by mypy version of google auth (#40108)
Fix regular expression to exclude double quote and newline in DataflowHook (#39991)
Fix replace parameter for BigQueryToPostgresOperator (#40278)
Fix 'GCSToGCSOperator' behavior difference for moving single object (#40162)
Refactor datapipeline operators (#39716)
Update pandas minimum requirement for Python 3.12 (#40272)
implement per-provider tests with lowest-direct dependency resolution (#39946)
openlineage: execute extraction and message sending in separate process (#40078)
Bump minimum version of google-auth to 2.29.0 (#40190)
Bump google-ads version to use v17 by default (#40158)
google: move openlineage imports inside methods (#40062)
Add job_id as template_field in DataplexGetDataQualityScanResultOperator (#40041)
Add dependency to httpx >= 0.25.0 everywhere (#40256)
Nothing published for this version
- Deprecate AutoML Tables operators
Release Date: 2024-05-30
Note
Several AutoML operators have stopped being supported following the shutdown of a legacy version of AutoML Natural Language, Tables, Vision, and Video Intelligence services. This includes AutoMLDeployModelOperator , AutoMLTablesUpdateDatasetOperator , AutoMLTablesListTableSpecsOperator and AutoMLTablesListColumnSpecsOperator . Please refer to the operator documentation to find out about available alternatives, if any. For additional information regarding the AutoML shutdown see:
AutoML Natural Language
AutoML Tables
AutoML Vision
AutoML Video Intelligence
Introduce anonymous credentials in GCP base hook (#39695)
Remove parent_model version suffix if it is passed to Vertex AI operators (#39640)
Fix BigQueryCursor execute method if the location is missing (#39659)
Fix acknowledged functionality in deferrable mode for PubSubPullSensor (#39711)
Reroute AutoML operator links to Google Translation links (#39668)
Pin google-cloud-bigquery to < 3.21.0 (#39583)
Remove 'openlineage.common' dependencies in Google and Snowflake providers. (#39614)
Deprecate AutoML Tables operators (#39752)
Resolve deprecation warnings in Azure FileShare-to-GCS tests (#39599)
typo: wrong OpenLineage facet key in spec (#39782)
removed stale code from StackdriverTaskHandler (#39744)
Nothing published for this version
- Add deprecation warnings and raise exception for already deprecated ones
Release Date: 2024-05-17
Note
This release of provider is only available for Airflow 2.7+ as explained in the Apache Airflow providers support policy .
add templated fields for google llm operators (#39174)
Add logic to handle on_kill for BigQueryInsertJobOperator when deferrable=True (#38912)
Create 'CloudComposerRunAirflowCLICommandOperator' operator (#38965)
Deferrable mode for Dataflow sensors (#37693)
Deferrable mode for Custom Training Job operators (#38584)
Enhancement for SSL-support in CloudSQLExecuteQueryOperator (#38894)
Create GKESuspendJobOperator and GKEResumeJobOperator operators (#38677)
Add support for role arn for aws creds in Google Transfer Service operator (#38911)
Add encryption_configuration parameter to BigQueryCheckOperator and BigQueryTableCheckOperator (#39432)
Add 'job_id' parameter to 'BigQueryGetDataOperator' (#39315)
Fix deferrable mode for DataflowTemplatedJobStartOperator and DataflowStartFlexTemplateOperator (#39018)
Fix batching for BigQueryToPostgresOperator (#39233)
Fix DataprocSubmitJobOperator in deferrable mode=True when task is marked as failed. (#39230)
Fix GCSObjectExistenceSensor operator to return the same XCOM value in deferrable and non-deferrable mode (#39206)
Fix conn_id BigQueryToMsSqlOperator (#39171)
Fix add retry logic in case of google auth refresh credential error (#38961)
Fix BigQueryCheckOperator skipped value and error check in deferrable mode (#38408)
Fix Use prefixes instead of all file paths for OpenLineage datasets in GCSDeleteObjectsOperator (#39059)
Fix Use prefixes instead of full file paths for OpenLineage datasets in GCSToGCSOperator (#39058)
Fix OpenLineage datasets in GCSTimeSpanFileTransformOperator (#39064)
Fix generation temp filename in 'DataprocSubmitPySparkJobOperator' (#39498)
Fix logic to cancel the external job if the TaskInstance is not in a running or deferred state for DataprocSubmitJobOperator (#39447)
Fix logic to cancel the external job if the TaskInstance is not in a running or deferred state for BigQueryInsertJobOperator (#39442)
Fix logic to cancel the external job if the TaskInstance is not in a running or deferred state for DataprocCreateClusterOperator (#39446)
Fix 'DataprocCreateBatchOperator' with 'result_retry' raises 'AttributeError' (#39462)
Fix yaml parsing for GKEStartKueueInsideClusterOperator (#39234)
Fix validation of label values in BigQueryInsertJobOperator (#39568)
Bump minimum Airflow version in providers to Airflow 2.7.0 (#39240)
Improve 'DataprocCreateClusterOperator' Triggers for Better Error Handling and Resource Cleanup (#39130)
Adding MSGraphOperator in Microsoft Azure provider (#38111)
Apply PROVIDE_PROJECT_ID mypy workaround across Google provider (#39129)
handle KubernetesDeleteJobOperator import (#39036)
Remove Airflow 2.6 back compact code (#39558)
Reapply templates for all providers (#39554)
Faster 'airflow_version' imports (#39552)
Add deprecation warnings and raise exception for already deprecated ones (#38673)
Simplify 'airflow_version' imports (#39497)
Disconnect GKE operators from deprecated hooks (#39434)
Nothing published for this version
Nothing published for this version
- Add 'impersonation_scopes' to BigQuery
Release Date: 2024-04-13
Add 'impersonation_scopes' to BigQuery (#38169)
Add the deferrable mode to RunPipelineJobOperator (#37969)
Add GKECreateCustomResourceOperator and GKEDeleteCustomResourceOperator operators (#37616)
Add VertexAI Language Model and Multimodal Model Operators for Google Cloud Generative AI use (#37721)
Add GKEListJobsOperator and GKEDescribeJobOperator (#37598)
Create GKEStartKueueJobOperator operator (#37477)
Create DeleteKubernetesJobOperator and GKEDeleteJobOperator operators (#37793)
Update GCS hook to get crc32c hash for CMEK-protected objects (#38191)
Set job labels for traceability in BigQuery jobs (#37736)
Deferrable mode for CreateBatchPredictionJobOperator (#37818)
Fix BigQuery connection and add docs (#38430)
fix(google,log): Avoid log name overriding (#38071)
Fix credentials error for S3ToGCSOperator trigger (#37518)
Fix 'parent_model' parameter in GCP Vertex AI AutoML and Custom Job operators (#38417)
fix(google): add return statement to yield within a while loop in triggers (#38394)
Fix cursor unique name surpasses Postgres identifier limit in 'PostgresToGCSOperator' (#38040)
Fix gcs Anonymous user issue because none token (#38102)
Fix BigQueryTablePartitionExistenceTrigger partition query (#37655)
Add google-cloud-bigquery as explicit google-provider dependency (#38753)
Avoid to use 'functools.lru_cache' in class methods in 'google' provider (#38652)
Refactor GKE hooks (#38404)
Remove unused loop variable from airflow package (#38308)
templated fields logic checks for cloud_storage_transfer_service (#37519)
Rename mlengine's operators' fields' names to comply with templated fields validation (#38053)
Rename Vertex AI AutoML operators fields' names to comply with templated fields validation (#38049)
Rename 'DeleteCustomTrainingJobOperator''s fields' names to comply with templated fields validation (#38048)
Restore delegate_to for Google Transfer Operators retrieving from Google Cloud. (#37925)
Refactor CreateHyperparameterTuningJobOperator (#37938)
Upgrade google-ads version (#37787)
Add 'impersonation_scopes' to BigQuery (#38169)
Add the deferrable mode to RunPipelineJobOperator (#37969)
Add GKECreateCustomResourceOperator and GKEDeleteCustomResourceOperator operators (#37616)
Add VertexAI Language Model and Multimodal Model Operators for Google Cloud Generative AI use (#37721)
Add GKEListJobsOperator and GKEDescribeJobOperator (#37598)
Create GKEStartKueueJobOperator operator (#37477)
Create DeleteKubernetesJobOperator and GKEDeleteJobOperator operators (#37793)
Update GCS hook to get crc32c hash for CMEK-protected objects (#38191)
Set job labels for traceability in BigQuery jobs (#37736)
Deferrable mode for CreateBatchPredictionJobOperator (#37818)
Fix BigQuery connection and add docs (#38430)
fix(google,log): Avoid log name overriding (#38071)
Fix credentials error for S3ToGCSOperator trigger (#37518)
Fix 'parent_model' parameter in GCP Vertex AI AutoML and Custom Job operators (#38417)
fix(google): add return statement to yield within a while loop in triggers (#38394)
Fix cursor unique name surpasses Postgres identifier limit in 'PostgresToGCSOperator' (#38040)
Fix gcs Anonymous user issue because none token (#38102)
Fix BigQueryTablePartitionExistenceTrigger partition query (#37655)
Add google-cloud-bigquery as explicit google-provider dependency (#38753)
Avoid to use 'functools.lru_cache' in class methods in 'google' provider (#38652)
Refactor GKE hooks (#38404)
Remove unused loop variable from airflow package (#38308)
templated fields logic checks for cloud_storage_transfer_service (#37519)
Rename mlengine's operators' fields' names to comply with templated fields validation (#38053)
Rename Vertex AI AutoML operators fields' names to comply with templated fields validation (#38049)
Rename 'DeleteCustomTrainingJobOperator''s fields' names to comply with templated fields validation (#38048)
Restore delegate_to for Google Transfer Operators retrieving from Google Cloud. (#37925)
Refactor CreateHyperparameterTuningJobOperator (#37938)
Upgrade google-ads version (#37787)
Nothing published for this version
- Fix invalid deprecation of 'DataFusionPipelineLinkHelper'
Release Date: 2024-03-08
'CloudRunExecuteJobOperator': Add project_id to hook.get_job calls (#37201)
Add developer token as authentication method to GoogleAdsHook (#37417)
Add GKEStartKueueInsideClusterOperator (#37072)
Add optional 'location' parameter to the BigQueryInsertJobTrigger (#37282)
feat(GKEPodAsyncHook): use async credentials token implementation (#37486)
Create GKEStartJobOperator and KubernetesJobOperator (#36847)
Fix invalid deprecation of 'DataFusionPipelineLinkHelper' (#37755)
fix templated field assignment 'google/cloud/operators/compute.py' (#37659)
fix bq_to_mysql init checks (#37653)
Fix Async GCSObjectsWithPrefixExistenceSensor xcom push (#37634)
Fix GCSSynchronizeBucketsOperator timeout error (#37237)
fix: Signature of insert_rows incompatible with supertype DbApiHook (#37391)
Use offset-naive datetime in _CredentialsToken (#37539)
Use wait_for_operation in DataprocInstantiateInlineWorkflowTemplateOperator (#37145)
Fix typo on DataflowStartFlexTemplateOperator documentation (#37595)
Make 'executemany' keyword arguments only in 'DbApiHook.insert_rows' (#37840)
Unify 'aws_conn_id' type to always be 'str | None' (#37768)
Limit 'pandas' to '<2.2' (#37748)
Remove broken deprecated fallback into the Google provider operators (#37740)
Implement AIP-60 Dataset URI formats (#37005)
resolve template fields init checks for 'bigquery' (#37586)
Update docs for the DataprocCreateBatchOperator (#37562)
Replace usage of 'datetime.utcnow' and 'datetime.utcfromtimestamp' in providers (#37138)
'CloudRunExecuteJobOperator': Add project_id to hook.get_job calls (#37201)
Add developer token as authentication method to GoogleAdsHook (#37417)
Add GKEStartKueueInsideClusterOperator (#37072)
Add optional 'location' parameter to the BigQueryInsertJobTrigger (#37282)
feat(GKEPodAsyncHook): use async credentials token implementation (#37486)
Create GKEStartJobOperator and KubernetesJobOperator (#36847)
Fix invalid deprecation of 'DataFusionPipelineLinkHelper' (#37755)
fix templated field assignment 'google/cloud/operators/compute.py' (#37659)
fix bq_to_mysql init checks (#37653)
Fix Async GCSObjectsWithPrefixExistenceSensor xcom push (#37634)
Fix GCSSynchronizeBucketsOperator timeout error (#37237)
fix: Signature of insert_rows incompatible with supertype DbApiHook (#37391)
Use offset-naive datetime in _CredentialsToken (#37539)
Use wait_for_operation in DataprocInstantiateInlineWorkflowTemplateOperator (#37145)
Fix typo on DataflowStartFlexTemplateOperator documentation (#37595)
Make 'executemany' keyword arguments only in 'DbApiHook.insert_rows' (#37840)
Unify 'aws_conn_id' type to always be 'str | None' (#37768)
Limit 'pandas' to '<2.2' (#37748)
Remove broken deprecated fallback into the Google provider operators (#37740)
Implement AIP-60 Dataset URI formats (#37005)
resolve template fields init checks for 'bigquery' (#37586)
Update docs for the DataprocCreateBatchOperator (#37562)
Replace usage of 'datetime.utcnow' and 'datetime.utcfromtimestamp' in providers (#37138)
Nothing published for this version
- feat: Switch all class, functions, methods deprecations to decorators
Release Date: 2024-02-15
add service_file support to GKEPodAsyncHook (#37081)
Update GCP Dataproc ClusterGenerator to support GPU params (#37036)
Create DataprocStartClusterOperator and DataprocStopClusterOperator (#36996)
Implement deferrable mode for CreateHyperparameterTuningJobOperator (#36594)
Enable '_enable_tcp_keepalive' functionality for GKEPodHook (#36999)
fix(providers/google): fix how GKEPodAsyncHook.service_file_as_context is used (#37306)
Fix metadata override for ComputeEngineSSHHook (#37192)
Fix assignment of template field in 'init' in 'custom_job' (#36789)
Fix location requirement in DataflowTemplatedJobStartOperator (#37069)
Fix assignment of template field in 'init' in 'CloudDataTransferServiceCreateJobOperator' (#36909)
Fixed the hardcoded default namespace value for GCP Data Fusion links. (#35379)
Do not ignore the internal_ip_only if set to false in Dataproc cluster config (#37014)
Revert protection against back-compatibility issue with google-core-api (#37111)
feat: Switch all class, functions, methods deprecations to decorators (#36876)
add service_file support to GKEPodAsyncHook (#37081)
Update GCP Dataproc ClusterGenerator to support GPU params (#37036)
Create DataprocStartClusterOperator and DataprocStopClusterOperator (#36996)
Implement deferrable mode for CreateHyperparameterTuningJobOperator (#36594)
Enable '_enable_tcp_keepalive' functionality for GKEPodHook (#36999)
fix(providers/google): fix how GKEPodAsyncHook.service_file_as_context is used (#37306)
Fix metadata override for ComputeEngineSSHHook (#37192)
Fix assignment of template field in '__init__' in 'custom_job' (#36789)
Fix location requirement in DataflowTemplatedJobStartOperator (#37069)
Fix assignment of template field in '__init__' in 'CloudDataTransferServiceCreateJobOperator' (#36909)
Fixed the hardcoded default namespace value for GCP Data Fusion links. (#35379)
Do not ignore the internal_ip_only if set to false in Dataproc cluster config (#37014)
Revert protection against back-compatibility issue with google-core-api (#37111)
feat: Switch all class, functions, methods deprecations to decorators (#36876)
Nothing published for this version
- Fix deprecations into the GCP Dataproc links
Release Date: 2024-01-27
Note
The default value of parquet_row_group_size in BaseSQLToGCSOperator has changed from 1 to 100000, in order to have a default that provides better compression efficiency and performance of reading the data in the output Parquet files. In many cases, the previous value of 1 resulted in very large files, long task durations and out of memory issues. A default value of 100000 may require more memory to execute the operator, in which case users can override the parquet_row_group_size parameter in the operator. All operators that are derived from BaseSQLToGCSOperator are affected when export_format is parquet : MySQLToGCSOperator , PrestoToGCSOperator , OracleToGCSOperator , TrinoToGCSOperator , MSSQLToGCSOperator and PostgresToGCSOperator . Due to the above we treat this change as bug fix.
Add templated fields to 'BigQueryToSqlBaseOperator' from 'BigQueryToPostgresOperator' (#36663)
Added Check for Cancel Workflow Invocation and added new Query Workflow Invocation operator (#36351)
Implement Google Analytics Admin (GA4) operators (#36276)
Add operator to diagnose cluster (#36899)
Add scopes into a GCP token (#36974)
feat: full support for google credentials in gcloud-aio clients (#36849)
fix templating field to super constructor (#36934)
fix: respect connection ID and impersonation in GKEStartPodOperator (#36861)
Fix stacklevel in warnings.warn into the providers (#36831)
Fix deprecations into the GCP Dataproc links (#36834)
fix assignment of templated field in constructor (#36603)
Check cluster state before defer Dataproc operators to trigger (#36892)
prevent templated field logic checks in operators init (#36489)
Preserve ASCII control characters directly through the BigQuery load API (#36533)
Change default 'parquet_row_group_size' in 'BaseSQLToGCSOperator' (#36817)
Fix google operators handling of impersonation chain (#36903)
style(providers/google): improve BigQueryInsertJobOperator type hinting (#36894)
Deprecate AutoMLTrainModelOperator for Vision and Video (#36473)
Remove backward compatibility check for KubernetesPodOperator module (#36724)
Remove backward compatibility check for KubernetesPodTrigger module (#36721)
Set min pandas dependency to 1.2.5 for all providers and airflow (#36698)
remove unnecessary templated field (#36491)
docs(providers/google): reword GoogleBaseHookAsync as GoogleBaseAsyncHook in docstring (#36946)
Note
The default value of parquet_row_group_size in BaseSQLToGCSOperator has changed from 1 to 100000, in order to have a default that provides better compression efficiency and performance of reading the data in the output Parquet files. In many cases, the previous value of 1 resulted in very large files, long task durations and out of memory issues. A default value of 100000 may require more memory to execute the operator, in which case users can override the parquet_row_group_size parameter in the operator. All operators that are derived from BaseSQLToGCSOperator are affected when export_format is parquet: MySQLToGCSOperator, PrestoToGCSOperator, OracleToGCSOperator, TrinoToGCSOperator, MSSQLToGCSOperator and PostgresToGCSOperator. Due to the above we treat this change as bug fix.
Add templated fields to 'BigQueryToSqlBaseOperator' from 'BigQueryToPostgresOperator' (#36663)
Added Check for Cancel Workflow Invocation and added new Query Workflow Invocation operator (#36351)
Implement Google Analytics Admin (GA4) operators (#36276)
Add operator to diagnose cluster (#36899)
Add scopes into a GCP token (#36974)
feat: full support for google credentials in gcloud-aio clients (#36849)
fix templating field to super constructor (#36934)
fix: respect connection ID and impersonation in GKEStartPodOperator (#36861)
Fix stacklevel in warnings.warn into the providers (#36831)
Fix deprecations into the GCP Dataproc links (#36834)
fix assignment of templated field in constructor (#36603)
Check cluster state before defer Dataproc operators to trigger (#36892)
prevent templated field logic checks in operators __init__ (#36489)
Preserve ASCII control characters directly through the BigQuery load API (#36533)
Change default 'parquet_row_group_size' in 'BaseSQLToGCSOperator' (#36817)
Fix google operators handling of impersonation chain (#36903)
style(providers/google): improve BigQueryInsertJobOperator type hinting (#36894)
Deprecate AutoMLTrainModelOperator for Vision and Video (#36473)
Remove backward compatibility check for KubernetesPodOperator module (#36724)
Remove backward compatibility check for KubernetesPodTrigger module (#36721)
Set min pandas dependency to 1.2.5 for all providers and airflow (#36698)
remove unnecessary templated field (#36491)
docs(providers/google): reword GoogleBaseHookAsync as GoogleBaseAsyncHook in docstring (#36946)
Nothing published for this version
Nothing published for this version
- Remove backcompat code for stackdriver
Release Date: 2023-12-31
Remove backcompat code for stackdriver (#36442)
Remove unused '_parse_version' function (#36450)
Remove remaining Airflow 2.5 backcompat code from GCS Task Handler (#36443) (#36457)
Remove backcompat code for stackdriver (#36442)
Remove unused '_parse_version' function (#36450)
Remove remaining Airflow 2.5 backcompat code from GCS Task Handler (#36443) (#36457)
Nothing published for this version
- Deprecate 'CloudComposerEnvironmentSensor' in favor of 'CloudComposerCreateEnvironmentOperator' with defer mode
Release Date: 2023-12-27
Note
This release of provider is only available for Airflow 2.6+ as explained in the Apache Airflow providers support policy .
GCP Secrets Backend Impersonation (#36072)
Add OpenLineage support to GcsOperators - Delete, Transform and TimeSpanTransform (#35838)
Add support for service account impersonation with computeEngineSSHHook (google provider) and IAP tunnel (#35136)
Add Datascan Profiling (#35696)
Add overrides to template fields of Google Cloud Run Jobs Execute Operator (#36133)
Implement deferrable mode for BeamRunJavaPipelineOperator (#36122)
Add ability to run streaming Job for BeamRunPythonPipelineOperator in non deferrable mode (#36108)
Add use_glob to GCSObjectExistenceSensor (#34137)
Fix DataprocSubmitJobOperator to retrieve failed job error message (#36053)
Fix CloudRunExecuteJobOperator not able to retrieve the Cloud Run job status in deferrable mode (#36012)
Fix gcs listing - ensure blobs are loaded (#34919)
allow multiple elements in impersonation chain (#35694)
Change retry type for Google Dataflow Client to async one (#36141)
Minor fix to DataprocCreateClusterOperator operator docs. (#36322)
fix(bigquery.py): pass correct project_id to triggerer (#35200)
iterate through blobs before checking prefixes (#36202)
Update 'retry' param typing in PubSubAsyncHook (#36198)
Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017)
Deprecate 'CloudComposerEnvironmentSensor' in favor of 'CloudComposerCreateEnvironmentOperator' with defer mode (#35775)
Follow BaseHook connection fields method signature in child classes (#36086)
Allow storage options to be passed (#35820)
Add feature to build "chicken-egg" packages from sources (#35890)
Remove remaining Airflow 2.5 backcompat code from Google Provider (#36366)
Move KubernetesPodTrigger hook to a cached property (#36290)
Add code snippet formatting in docstrings via Ruff (#36262)
Note
This release of provider is only available for Airflow 2.6+ as explained in the Apache Airflow providers support policy.
GCP Secrets Backend Impersonation (#36072)
Add OpenLineage support to GcsOperators - Delete, Transform and TimeSpanTransform (#35838)
Add support for service account impersonation with computeEngineSSHHook (google provider) and IAP tunnel (#35136)
Add Datascan Profiling (#35696)
Add overrides to template fields of Google Cloud Run Jobs Execute Operator (#36133)
Implement deferrable mode for BeamRunJavaPipelineOperator (#36122)
Add ability to run streaming Job for BeamRunPythonPipelineOperator in non deferrable mode (#36108)
Add use_glob to GCSObjectExistenceSensor (#34137)
Fix DataprocSubmitJobOperator to retrieve failed job error message (#36053)
Fix CloudRunExecuteJobOperator not able to retrieve the Cloud Run job status in deferrable mode (#36012)
Fix gcs listing - ensure blobs are loaded (#34919)
allow multiple elements in impersonation chain (#35694)
Change retry type for Google Dataflow Client to async one (#36141)
Minor fix to DataprocCreateClusterOperator operator docs. (#36322)
fix(bigquery.py): pass correct project_id to triggerer (#35200)
iterate through blobs before checking prefixes (#36202)
Update 'retry' param typing in PubSubAsyncHook (#36198)
Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017)
Deprecate 'CloudComposerEnvironmentSensor' in favor of 'CloudComposerCreateEnvironmentOperator' with defer mode (#35775)
Follow BaseHook connection fields method signature in child classes (#36086)
Allow storage options to be passed (#35820)
Add feature to build "chicken-egg" packages from sources (#35890)
Remove remaining Airflow 2.5 backcompat code from Google Provider (#36366)
Move KubernetesPodTrigger hook to a cached property (#36290)
Add code snippet formatting in docstrings via Ruff (#36262)
Nothing published for this version
Nothing published for this version
Nothing published for this version
- Remove usage of deprecated method from BigQueryToBigQueryOperator
Release Date: 2023-11-29
added Topic params for schema_settings and message_retention_duration. (#35767)
Add OpenLineage support to GCSToBigQueryOperator (#35778)
Add OpenLineage support to BigQueryToGCSOperator (#35660)
Add support for driver pool, instance flexibility policy, and min_num_instances for Dataproc (#34172)
Add "NON_PREEMPTIBLE" as a valid preemptibility type for Dataproc workers (#35669)
Add ability to pass impersonation_chain to BigQuery triggers (#35629)
Add a filter for local files in GoogleDisplayVideo360CreateQueryOperator (#35635)
Extend task context logging support for remote logging using GCP GCS (#32970)
Fix and reapply templates for provider documentation (#35686)
Fix the logic of checking dataflow job state (#34785)
Remove usage of deprecated method from BigQueryToBigQueryOperator (#35605)
Check attr on parent not self re TaskContextLogger set_context (#35780)
Remove backcompat with Airflow 2.3/2.4 in providers (#35727)
Restore delegate_to param in GoogleDiscoveryApiHook (#35728)
Remove usage of deprecated methods from BigQueryCursor (#35606)
Align documentation of 'MSSQLToGCSOperator' (#35715)
added Topic params for schema_settings and message_retention_duration. (#35767)
Add OpenLineage support to GCSToBigQueryOperator (#35778)
Add OpenLineage support to BigQueryToGCSOperator (#35660)
Add support for driver pool, instance flexibility policy, and min_num_instances for Dataproc (#34172)
Add "NON_PREEMPTIBLE" as a valid preemptibility type for Dataproc workers (#35669)
Add ability to pass impersonation_chain to BigQuery triggers (#35629)
Add a filter for local files in GoogleDisplayVideo360CreateQueryOperator (#35635)
Extend task context logging support for remote logging using GCP GCS (#32970)
Fix and reapply templates for provider documentation (#35686)
Fix the logic of checking dataflow job state (#34785)
Remove usage of deprecated method from BigQueryToBigQueryOperator (#35605)
Check attr on parent not self re TaskContextLogger set_context (#35780)
Remove backcompat with Airflow 2.3/2.4 in providers (#35727)
Restore delegate_to param in GoogleDiscoveryApiHook (#35728)
Remove usage of deprecated methods from BigQueryCursor (#35606)
Align documentation of 'MSSQLToGCSOperator' (#35715)
Nothing published for this version
- Update Google Ads API version from v14 to v15
Release Date: 2023-11-12
Update Google Ads API version from v14 to v15 (#35295)
Nothing published for this version
- AIP-58: Add Airflow ObjectStore (AFS)
Release Date: 2023-10-31
AIP-58: Add Airflow ObjectStore (AFS) (#34729)
Improve Dataprep hook (#34880)
Added 'overrides' parameter to CloudRunExecuteJobOperator (#34874)
AIP-58: Add Airflow ObjectStore (AFS) (#34729)
Improve Dataprep hook (#34880)
Added 'overrides' parameter to CloudRunExecuteJobOperator (#34874)
Nothing published for this version
- Add links between documentation related to Google Cloud Storage
Release Date: 2023-10-21
Add links between documentation related to Google Cloud Storage (#34994)
Migrate legacy version of AI Platform Prediction to VertexAI (#34922)
Cancel workflow in on_kill in DataprocInstantiate{Inline}WorkflowTemplateOperator (#34957)
Your coding agent can read these notes before it upgrades. Set up the MCP server →