NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #1540 most downloaded on PyPI
Provider package apache-airflow-providers-google for Apache Airflow
Last release 6 days ago
29 Sep 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
1 version withdrawn
withdrawn after publishing
6 years old
227 releases · first in 2020
One column per quarter.
- Add links between documentation related to Google Cloud Storage
Release Date: 2023-10-21
Add links between documentation related to Google Cloud Storage (#34994)
Migrate legacy version of AI Platform Prediction to VertexAI (#34922)
Cancel workflow in on_kill in DataprocInstantiate{Inline}WorkflowTemplateOperator (#34957)
Nothing published for this version
- Deprecate Life Sciences Operator and Hook
Release Date: 2023-10-17
Note
This release of provider is only available for Airflow 2.5+ as explained in the Apache Airflow providers support policy .
improvement: introduce project_id in BigQueryIntervalCheckOperator (#34573)
respect soft_fail argument when exception is raised for google sensors (#34501)
Fix GCSToGoogleDriveOperator and gdrive system tests (#34545)
Fix LookerHook serialize missing 1 argument error (#34678)
Fix Dataform system tests (#34329)
Bump min airflow version of providers (#34728)
Refactor DataFusionInstanceLink usage (#34514)
Use 'airflow.models.dag.DAG' in Google Provider examples (#34614)
Deprecate Life Sciences Operator and Hook (#34549)
Use 'airflow.exceptions.AirflowException' in providers (#34511)
Note
This release of provider is only available for Airflow 2.5+ as explained in the Apache Airflow providers support policy.
improvement: introduce project_id in BigQueryIntervalCheckOperator (#34573)
respect soft_fail argument when exception is raised for google sensors (#34501)
Fix GCSToGoogleDriveOperator and gdrive system tests (#34545)
Fix LookerHook serialize missing 1 argument error (#34678)
Fix Dataform system tests (#34329)
Bump min airflow version of providers (#34728)
Refactor DataFusionInstanceLink usage (#34514)
Use 'airflow.models.dag.DAG' in Google Provider examples (#34614)
Deprecate Life Sciences Operator and Hook (#34549)
Use 'airflow.exceptions.AirflowException' in providers (#34511)
Nothing published for this version
- Deprecate AutoMLTrainModelOperator for NL
Release Date: 2023-09-18
Add explicit support of stream (realtime) pipelines for CloudDataFusionStartPipelineOperator (#34271)
Add 'expected_terminal_state' parameter to Dataflow operators (#34217)
Fix 'ComputeEngineInsertInstanceOperator' doesn't respect jinja-templated instance name when given in body argument (#34171)
fix: BigQuery job error message (#34208)
GKEPodHook ignores gcp_conn_id parameter. (#34194)
Bump min common-sql provider version for Google provider (#34257)
Remove unnecessary call to keys() method on dictionaries (#34260)
Refactor: Think positively in providers (#34279)
Refactor: Simplify code in providers/google (#33229)
Refactor: Simplify comparisons (#34181)
Deprecate AutoMLTrainModelOperator for NL (#34212)
Simplify to bool(...) (#34258)
Make Google Dataform operators templated_fields more consistent (#34187)
Nothing published for this version
- Remove a deprecated option from 'BigQueryHook.get_pandas_df'
Release Date: 2023-09-12
Add deferrable mode to Dataplex DataQuality. (#33954)
allow impersonation_chain to be set on Google Cloud connection (#33715)
fix(providers/google-marketing-platform): respect soft_fail argument when exception is raised (#34165)
fix: docstring in endpoint_service.py (#34135)
Fix BigQueryValueCheckOperator deferrable mode optimisation (#34018)
Dynamic setting up of artifact versions for Datafusion pipelines (#34068)
Early delete a Dataproc cluster if started in the ERROR state. (#33668)
Avoid blocking event loop when using DataFusionAsyncHook by replacing sleep by asyncio.sleep (#33756)
Consolidate importing of os.path.* (#34060)
Refactor regex in providers (#33898)
Move the try outside the loop when this is possible in Google provider (#33976)
Combine similar if logics in providers (#33987)
Remove useless string join from providers (#33968)
Update Azure fileshare hook to use azure-storage-file-share instead of azure-storage-file (#33904)
Refactor unneeded jumps in providers (#33833)
replace loop by any when looking for a positive value in providers (#33984)
Replace try - except pass by contextlib.suppress in providers (#33980)
Remove some useless try/except from providers code (#33967)
Replace sequence concatenation by unpacking in Airflow providers (#33933)
Remove a deprecated option from 'BigQueryHook.get_pandas_df' (#33819)
replace unnecessary dict comprehension by dict() in providers (#33857)
Improve modules import in google provider by move some of them into a type-checking block (#33783)
Use a single statement with multiple contexts instead of nested statements in providers (#33768)
Use literal dict instead of calling dict() in providers (#33761)
remove unnecessary and rewrite it using list in providers (#33763)
Refactor: Simplify a few loops (#33736)
E731: replace lambda by a def method in Airflow providers (#33757)
Use f-string instead of in Airflow providers (#33752)
Add deferrable mode to Dataplex DataQuality. (#33954)
allow impersonation_chain to be set on Google Cloud connection (#33715)
fix(providers/google-marketing-platform): respect soft_fail argument when exception is raised (#34165)
fix: docstring in endpoint_service.py (#34135)
Fix BigQueryValueCheckOperator deferrable mode optimisation (#34018)
Dynamic setting up of artifact versions for Datafusion pipelines (#34068)
Early delete a Dataproc cluster if started in the ERROR state. (#33668)
Avoid blocking event loop when using DataFusionAsyncHook by replacing sleep by asyncio.sleep (#33756)
Consolidate importing of os.path.* (#34060)
Refactor regex in providers (#33898)
Move the try outside the loop when this is possible in Google provider (#33976)
Combine similar if logics in providers (#33987)
Remove useless string join from providers (#33968)
Update Azure fileshare hook to use azure-storage-file-share instead of azure-storage-file (#33904)
Refactor unneeded jumps in providers (#33833)
replace loop by any when looking for a positive value in providers (#33984)
Replace try - except pass by contextlib.suppress in providers (#33980)
Remove some useless try/except from providers code (#33967)
Replace sequence concatenation by unpacking in Airflow providers (#33933)
Remove a deprecated option from 'BigQueryHook.get_pandas_df' (#33819)
replace unnecessary dict comprehension by dict() in providers (#33857)
Improve modules import in google provider by move some of them into a type-checking block (#33783)
Use a single statement with multiple contexts instead of nested statements in providers (#33768)
Use literal dict instead of calling dict() in providers (#33761)
remove unnecessary and rewrite it using list in providers (#33763)
Refactor: Simplify a few loops (#33736)
E731: replace lambda by a def method in Airflow providers (#33757)
Use f-string instead of in Airflow providers (#33752)
Nothing published for this version
- Add CloudRunHook and operators
Release Date: 2023-08-29
Add CloudRunHook and operators (#33067)
Add 'CloudBatchHook' and operators (#32606)
Adding Support for Google Cloud's Data Pipelines Run Operator (#32846)
Add parameter sftp_prefetch to SFTPToGCSOperator (#33274)
Add Google Cloud's Data Pipelines Create Operator (#32843)
Add Dataplex Data Quality operators. (#32256)
Fix BigQueryCreateExternalTableOperator when using a format different to CSV (#33540)
Fix DataplexDataQualityJobStatusSensor and add unit tests (#33440)
Avoid importing pandas and numpy in runtime and module level (#33483)
Add missing template fields to DataformCreateCompilationResultOperator (#33585)
Consolidate import and usage of pandas (#33480)
Import utc from datetime and normalize its import (#33450)
Add CloudRunHook and operators (#33067)
Add 'CloudBatchHook' and operators (#32606)
Adding Support for Google Cloud's Data Pipelines Run Operator (#32846)
Add parameter sftp_prefetch to SFTPToGCSOperator (#33274)
Add Google Cloud's Data Pipelines Create Operator (#32843)
Add Dataplex Data Quality operators. (#32256)
Fix BigQueryCreateExternalTableOperator when using a format different to CSV (#33540)
Fix DataplexDataQualityJobStatusSensor and add unit tests (#33440)
Avoid importing pandas and numpy in runtime and module level (#33483)
Add missing template fields to DataformCreateCompilationResultOperator (#33585)
Consolidate import and usage of pandas (#33480)
Import utc from datetime and normalize its import (#33450)
Nothing published for this version
- refactor(providers.google): use module level __getattr__ for DATAPROC_JOB_LOG_LINK to DATAPROC_JOB_LINK and add deprecation warning
Release Date: 2023-08-10
openlineage, bigquery: add openlineage method support for BigQueryExecuteQueryOperator (#31293)
Add GCS Requester Pays bucket support to GCSToS3Operator (#32760)
Add system test and docs for CloudDataTransferServiceGCSToGCSOperator (#32960)
Add a new parameter to SQL operators to specify conn id field (#30784)
Fix 'DataFusionAsyncHook' catch 404 (#32855)
Fix system test for MetastoreHivePartitionSensor (#32861)
Fix catching 409 error (#33173)
make 'sql' a cached property in 'BigQueryInsertJobOperator' (#33218)
refactor(providers.google): use module level getattr for DATAPROC_JOB_LOG_LINK to DATAPROC_JOB_LINK and add deprecation warning (#33189)
openlineage, bigquery: add openlineage method support for BigQueryExecuteQueryOperator (#31293)
Add GCS Requester Pays bucket support to GCSToS3Operator (#32760)
Add system test and docs for CloudDataTransferServiceGCSToGCSOperator (#32960)
Add a new parameter to SQL operators to specify conn id field (#30784)
Fix 'DataFusionAsyncHook' catch 404 (#32855)
Fix system test for MetastoreHivePartitionSensor (#32861)
Fix catching 409 error (#33173)
make 'sql' a cached property in 'BigQueryInsertJobOperator' (#33218)
refactor(providers.google): use module level __getattr__ for DATAPROC_JOB_LOG_LINK to DATAPROC_JOB_LINK and add deprecation warning (#33189)
Nothing published for this version
Nothing published for this version
Nothing published for this version
- Fixup docstring for deprecated DataprocSubmitSparkJobOperator and refactoring system tests
Release Date: 2023-08-01
openlineage, gcs: add openlineage methods for GcsToGcsOperator (#31350)
Add Spot Instances support with Dataproc Operators (#31644)
Install sqlalchemy-spanner package into Google provider (#31925)
Filtering and ordering results of DataprocListBatchesOperator (#32500)
Fix BigQueryGetDataOperator where project_id is not being respected in deferrable mode (#32488)
Refresh GKE OAuth2 tokens (#32673)
Fix 'BigQueryInsertJobOperator' not exiting deferred state (#31591)
Fixup docstring for deprecated DataprocSubmitSparkJobOperator and refactoring system tests (#32743)
Add more accurate typing for DbApiHook.run method (#31846)
Add deprecation info to the providers modules and classes docstring (#32536)
Fixup docstring for deprecated DataprocSubmitHiveJobOperator (#32723)
Fixup docstring for deprecated DataprocSubmitPigJobOperator (#32739)
Fix Datafusion system tests (#32749)
openlineage, gcs: add openlineage methods for GcsToGcsOperator (#31350)
Add Spot Instances support with Dataproc Operators (#31644)
Install sqlalchemy-spanner package into Google provider (#31925)
Filtering and ordering results of DataprocListBatchesOperator (#32500)
Fix BigQueryGetDataOperator where project_id is not being respected in deferrable mode (#32488)
Refresh GKE OAuth2 tokens (#32673)
Fix 'BigQueryInsertJobOperator' not exiting deferred state (#31591)
Fixup docstring for deprecated DataprocSubmitSparkJobOperator and refactoring system tests (#32743)
Add more accurate typing for DbApiHook.run method (#31846)
Add deprecation info to the providers modules and classes docstring (#32536)
Fixup docstring for deprecated DataprocSubmitHiveJobOperator (#32723)
Fixup docstring for deprecated DataprocSubmitPigJobOperator (#32739)
Fix Datafusion system tests (#32749)
Nothing published for this version
- Implement deferrable mode for S3ToGCSOperator
Release Date: 2023-07-16
Implement deferrable mode for S3ToGCSOperator (#29462)
Bugfix GCSToGCSOperator when copy files to folder without wildcard (#32486)
Fix 'cache_control' parameter of upload function in 'GCSHook' (#32440)
Fix BigQuery transfer operators to respect project_id arguments (#32232)
Fix the gcp_gcs_delete_objects on empty list (#32383)
Fix endless loop of defer in cloud_build (#32387)
Fix GCSToGCSOperator copy without wildcard and exact_match=True (#32376)
Allow a destination folder to be provided (#31885)
Moves 'AzureBlobStorageToGCSOperator' from Azure to Google provider (#32306)
Give better link to job configuration docs in BigQueryInsertJobOperator (#31736)
Implement deferrable mode for S3ToGCSOperator (#29462)
Bugfix GCSToGCSOperator when copy files to folder without wildcard (#32486)
Fix 'cache_control' parameter of upload function in 'GCSHook' (#32440)
Fix BigQuery transfer operators to respect project_id arguments (#32232)
Fix the gcp_gcs_delete_objects on empty list (#32383)
Fix endless loop of defer in cloud_build (#32387)
Fix GCSToGCSOperator copy without wildcard and exact_match=True (#32376)
Allow a destination folder to be provided (#31885)
Moves 'AzureBlobStorageToGCSOperator' from Azure to Google provider (#32306)
Give better link to job configuration docs in BigQueryInsertJobOperator (#31736)
Nothing published for this version
- Deprecate 'delimiter' param and source object's wildcards in GCS, introduce 'match_glob' param.
Release Date: 2023-07-09
Add 'on_finish_action' to 'KubernetesPodOperator' (#30718)
Add deferrable mode to CloudSQLExportInstanceOperator (#30852)
Adding 'src_fmt_configs' to the list of template fields. (#32097)
[Issue-32069] Fix name format in the batch requests (#32070)
Fix 'BigQueryInsertJobOperator' error handling in deferrable mode (#32034)
Fix 'BIGQUERY_JOB_DETAILS_LINK_FMT' in 'BigQueryConsoleLink' (#31953)
Make the deferrable version of DataprocCreateBatchOperator handle a batch_id that already exists (#32216)
Switch Google Ads API version from v13 to v14 (#32028)
Deprecate 'delimiter' param and source object's wildcards in GCS, introduce 'match_glob' param. (#31261)
Refactor GKECreateClusterOperator's body validation (#31923)
Optimize deferrable mode execution for 'BigQueryValueCheckOperator' (#31872)
Add default_deferrable config (#31712)
Add 'on_finish_action' to 'KubernetesPodOperator' (#30718)
Add deferrable mode to CloudSQLExportInstanceOperator (#30852)
Adding 'src_fmt_configs' to the list of template fields. (#32097)
[Issue-32069] Fix name format in the batch requests (#32070)
Fix 'BigQueryInsertJobOperator' error handling in deferrable mode (#32034)
Fix 'BIGQUERY_JOB_DETAILS_LINK_FMT' in 'BigQueryConsoleLink' (#31953)
Make the deferrable version of DataprocCreateBatchOperator handle a batch_id that already exists (#32216)
Switch Google Ads API version from v13 to v14 (#32028)
Deprecate 'delimiter' param and source object's wildcards in GCS, introduce 'match_glob' param. (#31261)
Refactor GKECreateClusterOperator's body validation (#31923)
Optimize deferrable mode execution for 'BigQueryValueCheckOperator' (#31872)
Add default_deferrable config (#31712)
Nothing published for this version
Nothing published for this version
This release dropped support for Python 3.7
Release Date: 2023-06-23
Note
This release dropped support for Python 3.7
add a return when the event is yielded in a loop to stop the execution (#31985)
Add deferrable mode to PubsubPullSensor (#31284)
Add a new param to set parquet row group size in 'BaseSQLToGCSOperator' (#31831)
Add 'cacheControl' field to google cloud storage (#31338)
Add 'preserveAsciiControlCharacters' to 'src_fmt_configs' (#31643)
Add support for credential configuration file auth to Google Secrets Manager secrets backend (#31597)
Add credential configuration file support to Google Cloud Hook (#31548)
Add deferrable mode to 'GCSUploadSessionCompleteSensor' (#31081)
Add append_job_name parameter in DataflowStartFlexTemplateOperator (#31511)
FIPS environments: Mark uses of md5 as "not-used-for-security" (#31171)
Implement MetastoreHivePartitionSensor (#31016)
Bigquery: fix links for already existing tables and datasets. (#31589)
Provide missing project id and creds for TabularDataset (#31991)
Optimize deferrable mode execution for 'DataprocSubmitJobOperator' (#31317)
Optimize deferrable mode execution for 'BigQueryInsertJobOperator' (#31249)
Remove return statement after yield from triggers class (#31703)
Replace unicodecsv with standard csv library (#31693)
Optimize deferrable mode (#31758)
Remove Python 3.7 support (#30963)
Note
This release dropped support for Python 3.7
add a return when the event is yielded in a loop to stop the execution (#31985)
Add deferrable mode to PubsubPullSensor (#31284)
Add a new param to set parquet row group size in 'BaseSQLToGCSOperator' (#31831)
Add 'cacheControl' field to google cloud storage (#31338)
Add 'preserveAsciiControlCharacters' to 'src_fmt_configs' (#31643)
Add support for credential configuration file auth to Google Secrets Manager secrets backend (#31597)
Add credential configuration file support to Google Cloud Hook (#31548)
Add deferrable mode to 'GCSUploadSessionCompleteSensor' (#31081)
Add append_job_name parameter in DataflowStartFlexTemplateOperator (#31511)
FIPS environments: Mark uses of md5 as "not-used-for-security" (#31171)
Implement MetastoreHivePartitionSensor (#31016)
Bigquery: fix links for already existing tables and datasets. (#31589)
Provide missing project id and creds for TabularDataset (#31991)
Optimize deferrable mode execution for 'DataprocSubmitJobOperator' (#31317)
Optimize deferrable mode execution for 'BigQueryInsertJobOperator' (#31249)
Remove return statement after yield from triggers class (#31703)
Replace unicodecsv with standard csv library (#31693)
Optimize deferrable mode (#31758)
Remove Python 3.7 support (#30963)
Nothing published for this version
- Fix accessing a GKE cluster through the private endpoint in 'GKEStartPodOperator'
Release Date: 2023-05-27
Fix accessing a GKE cluster through the private endpoint in 'GKEStartPodOperator' (#31391)
Fix 'BigQueryGetDataOperator''s query job bugs in deferrable mode (#31433)
Nothing published for this version
Since v12 is deprecated by Google and soon to be removed we are not consider this to be a breaking change in Airflow.
Release Date: 2023-05-22
Note
This release of provider is only available for Airflow 2.4+ as explained in the Apache Airflow providers support policy .
Note
This release changed default Google ads to v13. Since v12 is deprecated by Google and soon to be removed we are not consider this to be a breaking change in Airflow.
Note
This version of the provider bumped many Google packages. Please review packages change logs
Add deferrable mode to DataprocInstantiateInlineWorkflowTemplateOperator (#30878)
Add deferrable mode to 'GCSObjectUpdateSensor' (#30579)
Add protocol to define methods relied upon by KubernetesPodOperator (#31298)
Add BigQueryToPostgresOperator (#30658)
'DataflowTemplatedJobStartOperator' fix overwriting of location with default value, when a region is provided. (#31082)
Poke once before defer for GCSObjectsWithPrefixExistenceSensor (#30939)
Add deferrable mode to 'GCSObjectsWithPrefixExistenceSensor' (#30618)
allow multiple prefixes in gcs delete/list hooks and operators (#30815)
Fix removed delegate_to parameter in deferrable GCS sensor (#30810)
Add 'use_legacy_sql' param to 'BigQueryGetDataOperator' (#31190)
Add 'as_dict' param to 'BigQueryGetDataOperator' (#30887)
Add flag apply_gcs_prefix to S3ToGCSOperator (b/245077385) (#31127)
Add 'priority' parameter to BigQueryHook (#30655)
Bump minimum Airflow version in providers (#30917)
implement gcs_schema_object for BigQueryCreateExternalTableOperator (#30961)
Optimize deferred execution mode (#30946)
Optimize deferrable mode execution (#30920)
Optimize deferrable mode in 'GCSObjectExistenceSensor' (#30901)
'CreateBatchPredictionJobOperator' Add batch_size param for Vertex AI BatchPredictionJob objects (#31118)
GKEPodHook needs to have all methods KPO calls (#31266)
Add CloudBuild build id log (#30516)
Switch default Google ads to v13 (#31382)
Switch to google ads v13 (#31369)
Update SDKs for google provider package (#30067)
Note
This release of provider is only available for Airflow 2.4+ as explained in the Apache Airflow providers support policy.
Note
This release changed default Google ads to v13. Since v12 is deprecated by Google and soon to be removed we are not consider this to be a breaking change in Airflow.
Note
This version of the provider bumped many Google packages. Please review packages change logs
Add deferrable mode to DataprocInstantiateInlineWorkflowTemplateOperator (#30878)
Add deferrable mode to 'GCSObjectUpdateSensor' (#30579)
Add protocol to define methods relied upon by KubernetesPodOperator (#31298)
Add BigQueryToPostgresOperator (#30658)
'DataflowTemplatedJobStartOperator' fix overwriting of location with default value, when a region is provided. (#31082)
Poke once before defer for GCSObjectsWithPrefixExistenceSensor (#30939)
Add deferrable mode to 'GCSObjectsWithPrefixExistenceSensor' (#30618)
allow multiple prefixes in gcs delete/list hooks and operators (#30815)
Fix removed delegate_to parameter in deferrable GCS sensor (#30810)
Add 'use_legacy_sql' param to 'BigQueryGetDataOperator' (#31190)
Add 'as_dict' param to 'BigQueryGetDataOperator' (#30887)
Add flag apply_gcs_prefix to S3ToGCSOperator (b/245077385) (#31127)
Add 'priority' parameter to BigQueryHook (#30655)
Bump minimum Airflow version in providers (#30917)
implement gcs_schema_object for BigQueryCreateExternalTableOperator (#30961)
Optimize deferred execution mode (#30946)
Optimize deferrable mode execution (#30920)
Optimize deferrable mode in 'GCSObjectExistenceSensor' (#30901)
'CreateBatchPredictionJobOperator' Add batch_size param for Vertex AI BatchPredictionJob objects (#31118)
GKEPodHook needs to have all methods KPO calls (#31266)
Add CloudBuild build id log (#30516)
Switch default Google ads to v13 (#31382)
Switch to google ads v13 (#31369)
Update SDKs for google provider package (#30067)
Nothing published for this version
Nothing published for this version
For more information please check: https://developers.google.com/doubleclick-advertisers/deprecation . As a result, the default api version for Campai…
Release Date: 2023-04-24
Google has announced sunset of Campaign Manager 360 v3.5 by Apr 20, 2023. For more information please check: https://developers.google.com/doubleclick-advertisers/deprecation . As a result, the default api version for Campaign Manager 360 operator was updated to the latest v4 version.
Warning
In this version of the provider, deprecated delegate_to param is removed from all GCP operators, hooks, and triggers, as well as from firestore and gsuite transfer operators that interact with GCS. Impersonation can be achieved instead by utilizing the impersonation_chain param. The delegate_to param will still be available only in gsuite and marketing platform hooks and operators, that don’t interact with Google Cloud.
remove delegate_to from GCP operators and hooks (#30748)
Update Google Campaign Manager360 operators to use API v4 (#30598)
Update DataprocCreateCluster operator to use 'label' parameter properly (#30741)
add missing project_id in BigQueryGetDataOperator (#30651)
Display Video 360 cleanup v1 API usage (#30577)
Nothing published for this version
Google announced sunset of Bid manager API v1 and v1.1 by April 27, 2023 for more information please check: docs As a result default value of api_vers
Release Date: 2023-04-13
Google announced sunset of Bid manager API v1 and v1.1 by April 27, 2023 for more information please check: docs As a result default value of api_version in GoogleDisplayVideo360Hook and related operators updated to v2
This version of provider contains a temporary workaround to issue with v11 version of google-ads API being discontinued, while the google provider dependencies preventing installing any google-ads client supporting v12 API. This version contains vendored-in version of google-ads library 20.0.0 v12 support only. The workaround (and vendored-in library) will be removed as soon as dependencies of the provider will allow to use google-ads supporting newer API versions of google-ads.
Note
ONLY v12 version of google ads is supported. You should set v12 when your create an operator or client.
Update DV360 operators to use API v2 (#30326)
Fix dynamic imports in google ads vendored in library (#30544)
Fix one more dynamic import needed for vendored-in google ads (#30564)
Add deferrable mode to GKEStartPodOperator (#29266)
BigQueryHook list_rows/get_datasets_list can return iterator (#30543)
Fix cloud build async credentials (#30441)
Google announced sunset of Bid manager API v1 and v1.1 by April 27, 2023 for more information please check: docs As a result default value of api_version in GoogleDisplayVideo360Hook and related operators updated to v2
This version of provider contains a temporary workaround to issue with v11 version of google-ads API being discontinued, while the google provider dependencies preventing installing any google-ads client supporting v12 API. This version contains vendored-in version of google-ads library 20.0.0 v12 support only. The workaround (and vendored-in library) will be removed as soon as dependencies of the provider will allow to use google-ads supporting newer API versions of google-ads.
Note
ONLY v12 version of google ads is supported. You should set v12 when your create an operator or client.
Update DV360 operators to use API v2 (#30326)
Fix dynamic imports in google ads vendored in library (#30544)
Fix one more dynamic import needed for vendored-in google ads (#30564)
Add deferrable mode to GKEStartPodOperator (#29266)
BigQueryHook list_rows/get_datasets_list can return iterator (#30543)
Fix cloud build async credentials (#30441)
Nothing published for this version
Nothing published for this version
- Add missing 'poll_interval' in Bigquery operator
Release Date: 2023-04-06
Add missing 'poll_interval' in Bigquery operator (#30132)
Add poll_interval param in BigQueryInsertJobOperator (#30091)
Add 'job_id' to 'BigQueryToGCSOperator' templated_fields (#30006)
Support deleting the local log files when using remote logging (#29772)
fix setting project_id for gs to bq and bq to gs (#30053)
Fix location on cloud build operators (#29937)
'GoogleDriveHook': Fixing log message + adding more verbose documentation (#29694)
Add "BOOLEAN" to type_map of MSSQLToGCSOperator, fix incorrect bit->int type conversion by specifying BIT fields explicitly (#29902)
Google Cloud Providers - Fix _MethodDefault deepcopy failure (#29518)
Handling project location param on async BigQuery dts trigger (#29786)
Support CloudDataTransferServiceJobStatusSensor without specifying a project_id (#30035)
Wait insert_job result in normal mode (#29925)
merge BigQueryTableExistenceAsyncSensor into BigQueryTableExistenceSensor (#30235)
Remove unnecessary upper constraints from google provider (#29915)
Merge BigQueryTableExistencePartitionAsyncSensor into BigQueryTableExistencePartitionSensor (#30231)
Merge GCSObjectExistenceAsyncSensor logic to GCSObjectExistenceSensor (#30014)
Align cncf provider file names with AIP-21 (#29905)
Switch to using vendored-in google ads. (#30410)
Merging of the google ads vendored-in code. (#30399)
Add missing 'poll_interval' in Bigquery operator (#30132)
Add poll_interval param in BigQueryInsertJobOperator (#30091)
Add 'job_id' to 'BigQueryToGCSOperator' templated_fields (#30006)
Support deleting the local log files when using remote logging (#29772)
fix setting project_id for gs to bq and bq to gs (#30053)
Fix location on cloud build operators (#29937)
'GoogleDriveHook': Fixing log message + adding more verbose documentation (#29694)
Add "BOOLEAN" to type_map of MSSQLToGCSOperator, fix incorrect bit->int type conversion by specifying BIT fields explicitly (#29902)
Google Cloud Providers - Fix _MethodDefault deepcopy failure (#29518)
Handling project location param on async BigQuery dts trigger (#29786)
Support CloudDataTransferServiceJobStatusSensor without specifying a project_id (#30035)
Wait insert_job result in normal mode (#29925)
merge BigQueryTableExistenceAsyncSensor into BigQueryTableExistenceSensor (#30235)
Remove unnecessary upper constraints from google provider (#29915)
Merge BigQueryTableExistencePartitionAsyncSensor into BigQueryTableExistencePartitionSensor (#30231)
Merge GCSObjectExistenceAsyncSensor logic to GCSObjectExistenceSensor (#30014)
Align cncf provider file names with AIP-21 (#29905)
Switch to using vendored-in google ads. (#30410)
Merging of the google ads vendored-in code. (#30399)
Nothing published for this version
- Add deferrable mode to BigQueryTablePartitionExistenceSensor.
Release Date: 2023-03-06
Add deferrable mode to BigQueryTablePartitionExistenceSensor. (#29735)
Add a new param for BigQuery operators to support additional actions when resource exists (#29394)
Add deferrable mode to DataprocInstantiateWorkflowTemplateOperator (#28618)
Dataproc batches (#29136)
Add 'CloudSQLCloneInstanceOperator' (#29726)
Fix 'NoneType' object is not subscriptable. (#29820)
Fix and augment 'check-for-inclusive-language' CI check (#29549)
Don't push secret in XCOM in BigQueryCreateDataTransferOperator (#29348)
Google Cloud Providers - Introduce GoogleCloudBaseOperator (#29680)
Update google cloud dlp package and adjust hook and operators (#29234)
Refactor Dataproc Trigger (#29364)
Remove <2.0.0 limit on google-cloud-bigtable (#29644)
Move help message to the google auth code (#29888)
Nothing published for this version
- Add defer mode to GKECreateClusterOperator and GKEDeleteClusterOperator
Release Date: 2023-02-21
Add defer mode to GKECreateClusterOperator and GKEDeleteClusterOperator (#28406)
Move cloud_sql_binary_path from connection to Hook (#29499)
Check that cloud sql provider version is valid (#29497)
'GoogleDriveHook': Add folder_id param to upload_file (#29477)
Add documentation for BigQuery transfer operators (#29466)
Add defer mode to GKECreateClusterOperator and GKEDeleteClusterOperator (#28406)
Move cloud_sql_binary_path from connection to Hook (#29499)
Check that cloud sql provider version is valid (#29497)
'GoogleDriveHook': Add folder_id param to upload_file (#29477)
Add documentation for BigQuery transfer operators (#29466)
Nothing published for this version
- Deprecate 'delegate_to' param in GCP operators and update docs
Release Date: 2023-02-11
Add deferrable capability to existing ''DataprocDeleteClusterOperator'' (#29349)
Add deferrable mode to dataflow operators (#27776)
Add deferrable mode to DataprocCreateBatchOperator (#28457)
Add deferrable mode to DataprocCreateClusterOperator and DataprocUpdateClusterOperator (#28529)
Add deferrable mode to MLEngineStartTrainingJobOperator (#27405)
Add deferrable mode to DataFusionStartPipelineOperator (#28690)
Add deferrable mode for Big Query Transfer operator (#27833)
Add support for write_on_empty in BaseSQLToGCSOperator (#28959)
Add DataprocCancelOperationOperator (#28456)
Enable individual trigger logging (#27758)
Auto ML assets (#25466)
Fix GoogleDriveHook writing files to trashed folders on upload v2 (#29119)
fix Google provider CHANGELOG.rst (#29122)
fix Google provider CHANGELOG.rst (#29114)
Keyfile dict can be dict not str (#29135)
GCSTaskHandler may use remote log conn id (#29117)
Deprecate 'delegate_to' param in GCP operators and update docs (#29088)
Nothing published for this version
- Add deferrable ''GCSObjectExistenceSensorAsync''
Release Date: 2023-01-17
Add deferrable ''GCSObjectExistenceSensorAsync'' (#28763)
Support partition_columns in BaseSQLToGCSOperator (#28677)
'BigQueryCreateExternalTableOperator' fix field delimiter not working with csv (#28856)
Fix using private _get_credentials instead of public get_credentials (#28588)
Fix'GoogleCampaignManagerReportSensor' with 'QUEUED' status (#28735)
Fix BigQueryColumnCheckOperator runtime error (#28796)
assign "datasetReference" attribute to dataset_reference dict. by default if not already set in create_empty_dataset method of bigquery hook (#28782)
Add deferrable ''GCSObjectExistenceSensorAsync'' (#28763)
Support partition_columns in BaseSQLToGCSOperator (#28677)
'BigQueryCreateExternalTableOperator' fix field delimiter not working with csv (#28856)
Fix using private _get_credentials instead of public get_credentials (#28588)
Fix'GoogleCampaignManagerReportSensor' with 'QUEUED' status (#28735)
Fix BigQueryColumnCheckOperator runtime error (#28796)
assign "datasetReference" attribute to dataset_reference dict. by default if not already set in create_empty_dataset method of bigquery hook (#28782)
Nothing published for this version
- Remove deprecated AIPlatformConsoleLinkk from google/provider.yaml
Release Date: 2023-01-05
Add table_resource to template fields for BigQueryCreateEmptyTableOperator (#28235)
Add retry param in GCSObjectExistenceSensor (#27943)
Add preserveAsciiControlCharacters to src_fmt_configs (#27679)
Add deferrable mode to CloudBuildCreateBuildOperator (#27783)
GCSToBigQueryOperator allows autodetect None and infers schema (#28564)
Improve memory usage in Dataproc deferrable operators (#28117)
Push job_id in xcom for dataproc submit job op (#28639)
Fix for issue with reading schema fields for JSON files in GCSToBigQueryOperator (#28284)
Fix GCSToBigQueryOperator not respecting schema_obj (#28444)
Fix GCSToGCSOperator copying list of objects without wildcard (#28111)
Fix: re-enable use of parameters in gcs_to_bq which had been disabled (#27961)
Set bigquery ''use_legacy_sql'' param in job config correctly (#28522)
Remove 'pylint' messages control instructions (#28555)
Remove deprecated AIPlatformConsoleLinkk from google/provider.yaml (#28449)
Use object instead of array in config.yml for config template (#28417)
[misc] Get rid of 'pass' statement in conditions (#27775)
Change log level to DEBUG when secret not found for google secret manager (#27856)
[misc] Replace XOR '^' conditions by 'exactly_one' helper in providers (#27858)
Add table_resource to template fields for BigQueryCreateEmptyTableOperator (#28235)
Add retry param in GCSObjectExistenceSensor (#27943)
Add preserveAsciiControlCharacters to src_fmt_configs (#27679)
Add deferrable mode to CloudBuildCreateBuildOperator (#27783)
GCSToBigQueryOperator allows autodetect None and infers schema (#28564)
Improve memory usage in Dataproc deferrable operators (#28117)
Push job_id in xcom for dataproc submit job op (#28639)
Fix for issue with reading schema fields for JSON files in GCSToBigQueryOperator (#28284)
Fix GCSToBigQueryOperator not respecting schema_obj (#28444)
Fix GCSToGCSOperator copying list of objects without wildcard (#28111)
Fix: re-enable use of parameters in gcs_to_bq which had been disabled (#27961)
Set bigquery ''use_legacy_sql'' param in job config correctly (#28522)
Remove 'pylint' messages control instructions (#28555)
Remove deprecated AIPlatformConsoleLinkk from google/provider.yaml (#28449)
Use object instead of array in config.yml for config template (#28417)
[misc] Get rid of 'pass' statement in conditions (#27775)
Change log level to DEBUG when secret not found for google secret manager (#27856)
[misc] Replace XOR '^' conditions by 'exactly_one' helper in providers (#27858)
Nothing published for this version
Nothing published for this version
- Persist DataprocLink for workflow operators regardless of job status
Release Date: 2022-11-29
Persist DataprocLink for workflow operators regardless of job status (#26986)
Deferrable mode for BigQueryToGCSOperator (#27683)
Add Export Format to Template Fields in BigQueryToGCSOperator (#27910)
Fix to read location parameter properly in BigQueryToBigQueryOperator (#27661)
Bump common.sql provider to 1.3.1 (#27888)
Persist DataprocLink for workflow operators regardless of job status (#26986)
Deferrable mode for BigQueryToGCSOperator (#27683)
Add Export Format to Template Fields in BigQueryToGCSOperator (#27910)
Fix to read location parameter properly in BigQueryToBigQueryOperator (#27661)
Bump common.sql provider to 1.3.1 (#27888)
Nothing published for this version
Nothing published for this version
- Use non-deprecated method for on_kill in BigQueryHook
Release Date: 2022-11-18
Note
This release of provider is only available for Airflow 2.3+ as explained in the Apache Airflow providers support policy .
Move min airflow version to 2.3.0 for all providers (#27196)
Rename hook bigquery function '_bq_cast' to 'bq_cast' (#27543)
Use non-deprecated method for on_kill in BigQueryHook (#27547)
Typecast biquery job response col value (#27236)
Remove <2 limit on google-cloud-storage (#26922)
Replace urlparse with urlsplit (#27389)
When defining a connection in environment variables or secrets backend, previously extra fields needed to be defined with prefix extra__google_cloud_platform__ . Now this is no longer required. So for example you may store the keyfile json as keyfile_dict instead of extra__google_cloud_platform__keyfile_dict . If both are present, the short name will be preferred.
Add backward compatibility with old versions of Apache Beam (#27263)
Add deferrable mode to GCSToBigQueryOperator + tests (#27052)
Add system tests for Vertex AI operators in new approach (#27053)
Dataform operators, links, update system tests and docs (#27144)
Allow values in WorkflowsCreateExecutionOperator execution argument to be dicts (#27361)
DataflowStopJobOperator Operator (#27033)
Allow for the overriding of stringify_dict for json/jsonb column data type in Postgres #26875 (#26876)
Allow and prefer non-prefixed extra fields for dataprep hook (#27039)
Update google hooks to prefer non-prefixed extra fields (#27023)
Add new Compute Engine Operators and fix system tests (#25608)
Common sql bugfixes and improvements (#26761)
Fix delay in Dataproc CreateBatch operator (#26126)
Remove unnecessary newlines around single arg in signature (#27525)
set project_id and location when canceling BigQuery job (#27521)
use the proper key to retrieve the dataflow job_id (#27336)
Make GSheetsHook return an empty list when there are no values (#27261)
Cloud ML Engine operators assets (#26836)
Note
This release of provider is only available for Airflow 2.3+ as explained in the Apache Airflow providers support policy.
Move min airflow version to 2.3.0 for all providers (#27196)
Rename hook bigquery function '_bq_cast' to 'bq_cast' (#27543)
Use non-deprecated method for on_kill in BigQueryHook (#27547)
Typecast biquery job response col value (#27236)
Remove <2 limit on google-cloud-storage (#26922)
Replace urlparse with urlsplit (#27389)
When defining a connection in environment variables or secrets backend, previously extra fields needed to be defined with prefix extra__google_cloud_platform__. Now this is no longer required. So for example you may store the keyfile json as keyfile_dict instead of extra__google_cloud_platform__keyfile_dict. If both are present, the short name will be preferred.
Add backward compatibility with old versions of Apache Beam (#27263)
Add deferrable mode to GCSToBigQueryOperator + tests (#27052)
Add system tests for Vertex AI operators in new approach (#27053)
Dataform operators, links, update system tests and docs (#27144)
Allow values in WorkflowsCreateExecutionOperator execution argument to be dicts (#27361)
DataflowStopJobOperator Operator (#27033)
Allow for the overriding of stringify_dict for json/jsonb column data type in Postgres #26875 (#26876)
Allow and prefer non-prefixed extra fields for dataprep hook (#27039)
Update google hooks to prefer non-prefixed extra fields (#27023)
Add new Compute Engine Operators and fix system tests (#25608)
Common sql bugfixes and improvements (#26761)
Fix delay in Dataproc CreateBatch operator (#26126)
Remove unnecessary newlines around single arg in signature (#27525)
set project_id and location when canceling BigQuery job (#27521)
use the proper key to retrieve the dataflow job_id (#27336)
Make GSheetsHook return an empty list when there are no values (#27261)
Cloud ML Engine operators assets (#26836)
Nothing published for this version
- Add BigQuery Column and Table Check Operators
Release Date: 2022-10-06
Add BigQuery Column and Table Check Operators (#26368)
Add deferrable big query operators and sensors (#26156)
Add 'output' property to MappedOperator (#25604)
Added append_job_name parameter to DataflowTemplatedJobStartOperator (#25746)
Adding a parameter for exclusion of trashed files in GoogleDriveHook (#25675)
Cloud Data Loss Prevention Operators assets (#26618)
Cloud Storage Transfer Operators assets & system tests migration (AIP-47) (#26072)
Merge deferrable BigQuery operators to exisitng one (#26433)
specifying project id when calling wait_for_operation in delete/create cluster (#26418)
Auto tail file logs in Web UI (#26169)
Cloud Functions Operators assets & system tests migration (AIP-47) (#26073)
GCSToBigQueryOperator Resolve 'max_id_key' job retrieval and xcom return (#26285)
Allow for the overriding of 'stringify_dict' for json export format on BaseSQLToGCSOperator (#26277)
Append GoogleLink base in the link class (#26057)
Cloud Video Intelligence Operators assets & system tests migration (AIP-47) (#26132)
Life Science assets & system tests migration (AIP-47) (#25548)
GCSToBigQueryOperator allow for schema_object in alternate GCS Bucket (#26190)
Use AsyncClient for Composer Operators in deferrable mode (#25951)
Use project_id to get authenticated client (#25984)
Cloud Build assets & system tests migration (AIP-47) (#25895)
Dataproc submit job operator async (#25302)
Support project_id argument in BigQueryGetDataOperator (#25782)
Fix JSONDecodeError in Datafusion operators (#26202)
Fixed never ending loop to in CreateWorkflowInvocation (#25737)
Update gcs.py (#26570)
Don't throw an exception when a BQ cusor job has no schema (#26096)
Google Cloud Tasks Sensor for queue being empty (#25622)
Correcting the transfer config name. (#25719)
Fix parsing of optional 'mode' field in BigQuery Result Schema (#26786)
Fix MaxID logic for GCSToBigQueryOperator (#26768)
Sql to GSC operators update docs for parquet format (#25878)
Limit Google Protobuf for compatibility with biggtable client (#25886)
Make GoogleBaseHook credentials functions public (#25785)
Consolidate to one 'schedule' param (#25410)
Add BigQuery Column and Table Check Operators (#26368)
Add deferrable big query operators and sensors (#26156)
Add 'output' property to MappedOperator (#25604)
Added append_job_name parameter to DataflowTemplatedJobStartOperator (#25746)
Adding a parameter for exclusion of trashed files in GoogleDriveHook (#25675)
Cloud Data Loss Prevention Operators assets (#26618)
Cloud Storage Transfer Operators assets & system tests migration (AIP-47) (#26072)
Merge deferrable BigQuery operators to exisitng one (#26433)
specifying project id when calling wait_for_operation in delete/create cluster (#26418)
Auto tail file logs in Web UI (#26169)
Cloud Functions Operators assets & system tests migration (AIP-47) (#26073)
GCSToBigQueryOperator Resolve 'max_id_key' job retrieval and xcom return (#26285)
Allow for the overriding of 'stringify_dict' for json export format on BaseSQLToGCSOperator (#26277)
Append GoogleLink base in the link class (#26057)
Cloud Video Intelligence Operators assets & system tests migration (AIP-47) (#26132)
Life Science assets & system tests migration (AIP-47) (#25548)
GCSToBigQueryOperator allow for schema_object in alternate GCS Bucket (#26190)
Use AsyncClient for Composer Operators in deferrable mode (#25951)
Use project_id to get authenticated client (#25984)
Cloud Build assets & system tests migration (AIP-47) (#25895)
Dataproc submit job operator async (#25302)
Support project_id argument in BigQueryGetDataOperator (#25782)
Fix JSONDecodeError in Datafusion operators (#26202)
Fixed never ending loop to in CreateWorkflowInvocation (#25737)
Update gcs.py (#26570)
Don't throw an exception when a BQ cusor job has no schema (#26096)
Google Cloud Tasks Sensor for queue being empty (#25622)
Correcting the transfer config name. (#25719)
Fix parsing of optional 'mode' field in BigQuery Result Schema (#26786)
Fix MaxID logic for GCSToBigQueryOperator (#26768)
Sql to GSC operators update docs for parquet format (#25878)
Limit Google Protobuf for compatibility with biggtable client (#25886)
Make GoogleBaseHook credentials functions public (#25785)
Consolidate to one 'schedule' param (#25410)
Nothing published for this version
Nothing published for this version
- Remove deprecated modules in Amazon provider
Release Date: 2022-08-14
add description method in BigQueryCursor class (#25366)
Add project_id as a templated variable in two BQ operators (#24768)
Remove deprecated modules in Amazon provider (#25543)
Move all "old" SQL operators to common.sql providers (#25350)
Improve taskflow type hints with ParamSpec (#25173)
Unify DbApiHook.run() method with the methods which override it (#23971)
Bump typing-extensions and mypy for ParamSpec (#25088)
Deprecate hql parameters and synchronize DBApiHook method APIs (#25299)
Dataform operators (#25587)
Fix GCSListObjectsOperator docstring (#25614)
Fix BigQueryInsertJobOperator cancel_on_kill (#25342)
Fix BaseSQLToGCSOperator approx_max_file_size_bytes (#25469)
Fix PostgresToGCSOperat bool dtype (#25475)
Fix Vertex AI Custom Job training issue (#25367)
Fix Flask Login user setting for Flask 2.2 and Flask-Login 0.6.2 (#25318)
add description method in BigQueryCursor class (#25366)
Add project_id as a templated variable in two BQ operators (#24768)
Remove deprecated modules in Amazon provider (#25543)
Move all "old" SQL operators to common.sql providers (#25350)
Improve taskflow type hints with ParamSpec (#25173)
Unify DbApiHook.run() method with the methods which override it (#23971)
Bump typing-extensions and mypy for ParamSpec (#25088)
Deprecate hql parameters and synchronize DBApiHook method APIs (#25299)
Dataform operators (#25587)
Fix GCSListObjectsOperator docstring (#25614)
Fix BigQueryInsertJobOperator cancel_on_kill (#25342)
Fix BaseSQLToGCSOperator approx_max_file_size_bytes (#25469)
Fix PostgresToGCSOperat bool dtype (#25475)
Fix Vertex AI Custom Job training issue (#25367)
Fix Flask Login user setting for Flask 2.2 and Flask-Login 0.6.2 (#25318)
Nothing published for this version
Nothing published for this version
Nothing published for this version
- Add DeprecationWarning for column_transformations parameter in AutoML
Release Date: 2022-07-16
PubSub assets & system tests migration (AIP-47) (#24867)
Add handling state of existing Dataproc batch (#24924)
Add links for Google Kubernetes Engine operators (#24786)
Add test_connection method to 'GoogleBaseHook' (#24682)
Add gcp_conn_id argument to GoogleDriveToLocalOperator (#24622)
Add DeprecationWarning for column_transformations parameter in AutoML (#24467)
Modify BigQueryCreateExternalTableOperator to use updated hook function (#24363)
Move all SQL classes to common-sql provider (#24836)
Datacatalog assets & system tests migration (AIP-47) (#24600)
Upgrade FAB to 4.1.1 (#24399)
GCSDeleteObjectsOperator empty prefix bug fix (#24353)
perf(BigQuery): pass table_id as str type (#23141)
Update providers to use functools compat for ''cached_property'' (#24582)
PubSub assets & system tests migration (AIP-47) (#24867)
Add handling state of existing Dataproc batch (#24924)
Add links for Google Kubernetes Engine operators (#24786)
Add test_connection method to 'GoogleBaseHook' (#24682)
Add gcp_conn_id argument to GoogleDriveToLocalOperator (#24622)
Add DeprecationWarning for column_transformations parameter in AutoML (#24467)
Modify BigQueryCreateExternalTableOperator to use updated hook function (#24363)
Move all SQL classes to common-sql provider (#24836)
Datacatalog assets & system tests migration (AIP-47) (#24600)
Upgrade FAB to 4.1.1 (#24399)
GCSDeleteObjectsOperator empty prefix bug fix (#24353)
perf(BigQuery): pass table_id as str type (#23141)
Update providers to use functools compat for ''cached_property'' (#24582)
Nothing published for this version
Your coding agent can read these notes before it upgrades. Set up the MCP server →