Added

• #1568 Allow to store Quickstart dockers data in a folder for persistence @afranzi
• #1602 feat: support for Kubernetes-based deployment @bharatak
• #1608 add lineage hive @clojurians-org
• #1609 add support for kubernetes helm packaging @bharatak
• #1611 init jdbc generator @clojurians-org
• #1613 add oracle driver @clojurians-org
• #1629 feat: Converting MCE to a Spring boot Application @arunvasudevan
• #1635 feat: convert MAE application to springboot @arunvasudevan
• #1637 add postgresql support and force utf8 encode on non-utf8 locale @clojurians-org
• #1647 Add openldap-etl script and instruction @loftyet
• #1673 add DataProcess Urn @loftyet
• #1678 refactor(pdl): convert all pdsc to pdl @mars-lan
• #1677 feat(urn): add AzkabanFlow and AzkabanJob urn @hshahoss

Changed

• #1601 build: bypass testing datahub-web when running idea gradle task @mars-lan
• 6ab2ab6f78ebc8aec20a4e8b58094a3e19f42eba build(mysql): Change mysql dependency from latest to 5.7 @keremsahin1
• #1610 metadata-models 54.0.1 -> 58.0.1 @jywadhwani
• #1616 metadata-models 58.0.1 -> 62.0.3 @jywadhwani
• #1619 refactor(gms): move gms restli resources @jywadhwani
• #1624 build(gms): rename JettyRunWar task to run @mars-lan
• #1626 refactor(frontend): fails loudly to help debug gms issue @mars-lan
• #1633 add field for ui and parser reference @clojurians-org
• #1641 migrate hive generator @clojurians-org
• #1662 style: add checkstyle and IDEA code style config @mars-lan
• #1664 build: update pegasus to v28 to add PDL support @mars-lan
• #1667 refactor: change the default log location @mars-lan
• #1669 refactor: use named volume instead of bind mount in quickstart @mars-lan

Deprecated

• #1618 refactor: remove unused legacy model & move DatasetSearchConfig @mars-lan

Removed

• #1614 refactor: remove unused legacy models @mars-lan

Fixed

• #1605 specify explicit avro lib for compatibility issue @jhsenjaliya
• d1cf62854d3bce29c15e197b4cd0929c31a96bc1 Fix: Docker Quickstart - Sample Data Loading Error @RealChrisL
• ba33c7a5cd916ad822acb4087a5c9cbee080fd89 Specify python version in mce-cli requirement.txt @RealChrisL
• #1621 fix: elasticsearch not starting on Mac @mars-lan
• #1622 build: pegasus plugin doesn't work well with gradle caching @mars-lan
• #1625 fix(gms): unable to find registered resources @mars-lan
• #1630 fix: Reduce gms & frontend docker image sizes @keremsahin1
• #1631 fix(Docker): Fixing 'dockerize not found' issue while starting @keremsahin1
• #1632 fix: Reduce mae-consumer & mce-consumer docker image sizes @bharatak
• #1646 fix(metadata-ingestion): pass schema_record to mce-cli cosumer @RealChrisL
• #1657 fix(quickstart): set utf8mb4 for mysql @e11it
• #1661 fix(urn): Move UrnCoercer into corresponding Urn class @mars-lan
• #1665 fix: use semantic instead of literal comparison in DefaultEqualityTester @mars-lan
• #1670 build: start enforcing checkstyle and fix all violations @mars-lan
• #1672 fix(frontend): Extract lastModified field from downstream/upstream aspect @keremsahin1

---

description: "DataHub Cloud v0.3.4 release notes from July 2024 with Looker SQL parsing, Grafana and Azure Blob connectors, and Snowflake queries source."

v0.3.4

---

Release Availability Date

29-Jul-2024

Recommended CLI/SDK

• v0.13.3.4 with release notes at https://github.com/acryldata/datahub/releases/tag/v0.13.3.4

If you are using an older CLI/SDK version, then please upgrade it. This applies for all CLI/SDK usages, if you are using it through your terminal, GitHub Actions, Airflow, in Python SDK somewhere, Java SDK, etc. This is a strong recommendation to upgrade, as we keep on pushing fixes in the CLI, and it helps us support you better.

Release Changelog

---

• Product changes

  • [notifications] You are now able to send test notifications to Slack user or group from settings.
  • [cross-instance share] Share failures should now correctly display handle URLs.
  • [siblings] Fixed siblings split feature when used for autocomplete; DataHub now disallows creation of assertions on merged sibling page; Fixed a bug in editing an assertion on a merged lineage node.
  • [lineage - v2] Minor fixes to lineage counts and column-level-lineage resolution by respecting soft deleted assets in the lineage graph.

• Ingestion changes

  • Added support for SQL parsing in Looker source to more correctly represent column-level-lineage. Please report any edge cases issues to the DataHub team as we will be closely monitoring this integration.
  • Added cli version in system metadata at the aspect level for tracking purposes.
  • Added async REST sink capabilities in CLI.
  • Added documentation on how to cleanup obsolete airflow pipelines and tasks from DataHub
  • Soft deleted entities using the airflow plugin should now come back once they re-appear.
  • Added 2 new connectors: Grafana & Azure Blob Storage (ABS)
  • Glue to s3 lineage should now be generated automatically in the glue source.
  • Added a snowflake queries source which does lineage, usage, queries, and operations all in one go.
  • Added dataset profiling to PowerBI.
  • Added a file source integration (S3 / Local Files), this is to add files as datasets. Not to be confused with the metadata file source that loads MCPs into DataHub.

• Platform changes

  • Added initial support for custom MCP mutator hooks.

• Since v0.3.3, these changes from OSS DataHub have been pulled in: OSS DataHub Changes.

---

description: "DataHub Cloud v0.3.3 release notes from June 2024 with on-demand assertions, richer Slack incident messages, and assertion-level subscriptions."

v0.3.3

---

Release Availability Date

25-June-2024

Recommended CLI/SDK

• v0.13.3 with release notes at https://github.com/acryldata/datahub/releases/tag/v0.13.3

If you are using an older CLI/SDK version then please upgrade it. This applies for all CLI/SDK usages, if you are using it through your terminal, github actions, airflow, in python SDK somewhere, Java SKD etc. This is a strong recommendation to upgrade as we keep on pushing fixes in the CLI and it helps us support you better.

Release Changelog

---

• Ability to run assertions "on-demand" from the DataHub Cloud UI (Reach out to DataHub team to enable this feature)
• Richer Incident Slack Messages - Resolve and reopen incidents from Slack. Reflect the latest status of incidents in the same slack message. Richer details about incident when raised via Assertion
• Assertion-Level Subscriptions - Subscribe to be notified when a specific assertion passes, fails, or errors out.
• Misc fixes with 2.0 UI (new loading indicators, fixed text overflows, consistent entity health badges)
• Structured Property Schema Change & Delete
• Since v0.3.2 these changes from OSS DataHub https://github.com/datahub-project/datahub/compare/6ed21bd1bc70a3ceb7dddb43ea7db4ca56874547...92e9a5823bc14e81f0f21c55a68c493c3bbe87b9 have been pulled in.

---

description: "DataHub Cloud v0.3.2 release notes from May 2024 introducing SCIM, schema assertions, Business Attributes, and new lineage GraphQL endpoints."

v0.3.2

---

Release Availability Date

16-May-2024

Recommended CLI/SDK

• v0.13.2.3 with release notes at https://github.com/acryldata/datahub/releases/tag/v0.13.2.3

If you are using an older CLI/SDK version then please upgrade it. This applies for all CLI/SDK usages, if you are using it through your terminal, github actions, airflow, in python SDK somewhere, Java SKD etc. This is a strong recommendation to upgrade as we keep on pushing fixes in the CLI and it helps us support you better.

Release Changelog

---

• Since v0.3.1 these changes from OSS DataHub https://github.com/datahub-project/datahub/compare/57de905c66b6992aefb2051708fa83898fa82cec...6ed21bd1bc70a3ceb7dddb43ea7db4ca56874547 have been pulled in.
• OpenAPI Entitites v3 (Entity-registry generated endpoints)
• Business Attributes Support (disabled by default)
• Graph Retriever (validators, mutators, side-effects)
• New Lineage Graphql Endpoints (optimizations for the UI)
• Metadata Tests Tracks Unique Hash
• SCIM Support
• V3 CLL Across DataJob Nodes Upgrade
• Subscribe to assertion errors: Get notified when assertion errors occur
• Group owners are labeled in slack incident notifications
• Field assertions now raise incidents when they go in error (if configured to do so)
• Assertion timeline viz handles missing data more gracefully
• Freshness assertions on Snowflake using the AUDIT_LOG will no longer consider no-op queries as valid 'updates' to the dataset
• [NEW] Schema Assertions: Allows users to configure a set of columns and expected types and verify that they actually exist in the tabe schema. Disabled by default. Please reach out to DataHub team if you wish to have this enabled.
• Adding polish and fixing bugs throughout the new UI
• Ability to add and remove lineage manually in the new UI
• Ability to configure the default time filter on lineage
• Ability to collapse lineage in the lineage graph (new UI)
• Support rich text on metadata form descriptions

Added

• 3765c1d234acea1b60093878a9e1a900e280415a Enable parallel Gradle build @keremsahin1
• #1575 Enable Failed Metadata Change Event for MCE Processor @arunvasudevan
• #1570 Use pictureLink property to show person picture @afranzi
• #1569 Show Dataset description in Dataset view @afranzi
• #1597 Ingestion tool to load JSON data to DataHub (in /contrib) @clojurians-org
• #1585 Nix sandbox (in /contrib) @clojurians-org
• 71f2d14503ddb810a930789764310998550e3442 Added EventUtilsTest @keremsahin1

Changed

• 36a5d23e42d393c587f167647f1856f1153ab396 Migrate to getSnapshot API & remove dataset snapshot @keremsahin1
• b17b91f24a950d30b4e5ae6339d0514747aebf34 Bump gradle to 5.6.4 and pegasus to 27.7.18 @keremsahin1
• Documentation

Removed

• #1581 Drop LinkedIn internal fabrics @mars-lan
• 1fff6c94be102a40df53743f8fde7bf74558e496 Cleanup unused snapshot resources for corp users & groups @keremsahin1

Fixed

• #1590 Gradle Build Fails When Run in Parallel @RyanHolstien
• #1574 Fix typo and watchman error @clojurians-org
• #1564 Allow dashes in user urn @ben5448
• 3d64c45fa6d3d4dbac283c5e78073c6fef91d501 Fix browse result pagination @keremsahin1
• fba5cd8c5e40309fe435f6cb10db525b3f23426b Handle optional aspects/fields for CorpUser gracefully @keremsahin1

• Onboarded people as a top level entity
• Enabled people search
• Created Docker image for running ingestion pipeline
• Misc bug fixes
• Documentation updates
• Code cleanup

---

description: "DataHub Cloud v0.2.3 release notes from March 2023 covering MCL restating, dataset description proposals, and lineage performance improvements."

v0.2.3

---

Release Availability Date

14-Mar-2023

Release Changelog

---

• Since v0.2.2 no changes from OSS DataHub have been pulled in.
• fix(mcl): only restate Lineage MCL's - This should help with some lag issues being seen
• feat(proposals): Add ability to propose descriptions on datasets
• Hotfix 2023 03 06 - Some Miscellaneous search improvements
• fix(bootstrap): only ingest default metadata tests once - This should help with some deleted metadata tests re-appearing.
• refactor(lineage): Fix & optimize getAndUpdatePaths - The impact should be a reduced page load time for the lineage-intensive entities
• refactor(ui): Loading schema dynamically for datasets
• fix(lineage): nullpointer exceptions - should fix some errors related to lineage search
• chore(ci): add daylight savings timezone for tests, fix daylight saving bug in analytics charts - Should fix gaps in Monthly charts for people with daylight savings

---

description: "DataHub Cloud v0.2.2 release notes from March 2023 with fixes for lineage filtering, recommendations, analytics, search, and ingestion runs."

v0.2.2

---

Release Availability Date

01-Mar-2023

Release Changelog

---

• Since v0.2.1 no changes from OSS DataHub have been pulled in.
• fix(lineage): fix filtering for Timeline Lineage, regression for Search Ingestion Summaries
• fix(recommendations): recommendations now display on the homepage for recently viewed, searched, and most popular.
• fix(analytics): chart smoothing and date range fixes
• fix(search): case-sensitive exact match
• fix(search): fix handling of 2 character search terms when not a prefix or exact match
• fix(ingestion): fix ingestion run summary showing no results

---

description: "DataHub Cloud v0.2.1 release notes from February 2023 covering metadata test properties, accessibility fixes, and tag and term proposals."

v0.2.1

---

Release Availability Date

23-Feb-2023

Release Changlog

---

• Since v0.2.0 these changes from OSS DataHub https://github.com/datahub-project/datahub/compare/cf1e627e55431fc69d72918b2bcc3c5f3a1d5002...36037cf288eea12f1760dd0718255eeb1d7039c7 have been pulled in.
• Add first, last synched + last updated properties to metadata tests.
• Update link colors to pass accessibility.
• Extend tag and term proposals to other entity types besides datasets. This allows proposals to work on entities other than datasets.
• We are skipping running metadata tests in real-time processing as it was not scaling out and causing issues in ingestion
• Re-enabling hard-deletes which was temporarily disabled

---

description: "DataHub Cloud v0.2.0 release notes from February 2023 with Elasticsearch reindex, improved docs editor, time-windowed lineage, and SSO updates."

v0.2.0

---

Release Availability Date

09 Feb 2023

Update Downtime

During release installation the Elasticsearch indices will be reindex to improve search capabilities. While the upgrade is in progress DataHub will be set to a read-only mode. Once this operation is completed, the upgrade will proceed normally. Depending on index sizes and infrastructure this process can take 5 minutes to hours however as a rough estimate 1 hour for every 2.3 million entities.

Release Changlog

---

• Since v0.1.73 these changes from OSS DataHub https://github.com/datahub-project/datahub/compare/36afdec3946df2fb4166ac27a89b933ced87d00e...v0.10.0 have been pulled in
  • Improved documentation editor
  • Filter lineage graphs based on time windows
  • Improvements in Search
  • Metadata Ingestion
    • Redshift: You can now extract lineage information from unload queries
    • PowerBI: Ingestion now maps Workspaces to DataHub Containers
    • BigQuery: You can now extract lineage metadata from the Catalog
    • Glue: Ingestion now uses table name as the human-readable name
• SSO Preferred Algorithm Setting