dagster-cloud-cli

New

• The selection argument of define_asset_job now accepts lists of AssetKeys or AssetsDefinitions.
• RunRequest now takes a stale_assets_only flag that filters the full set of assets that would be materialized by a job to stale assets only. This can be used in schedules and sensors.
• Dagit will now choose a different open port on the local machine to run on when no port is specified to the dagit command and the default port 3000 is already in use.
• The grpcio pin in Dagster to <1.48.1 has been removed for Python versions 3.10 and 3.11. Python 3.7, 3.8, and 3.9 are still pinned to <1.48.1 due to a bug in the grpc library that is causing the process to sometimes hang.
• When it is likely that an op process was killed due to running out of memory, a clearer error message is now displayed in Dagit.
• When a sensor tick fails due to taking longer than 60 seconds to execute, a clearer error message is displayed on the sensor timeline in Dagit.
• When you view compute logs on a run in Dagit, we now locally track whether you choose the stdout or stderr tab. The next time you view compute logs, you will see that tab first by default.
• The executor and loggers arguments on Definitions are no longer experimental.
• [dagster-dbt] When json_log_format is set to False when using the dbt_cli_resource, logs will be emitted at the appropriate log level in some situations. Previously, all logs would be emitted at the INFO level.
• [dagster-snowflake] The Snowflake IO Manager and Snowflake Resource now support private key authentication. Thanks Josh Taylor!
• [dagster-airbyte] Users can now specify freshness policies when generating Airbyte assets.
• [dagster-airbyte] When using managed Airbyte ingestion, users can now specify a destination table prefix.

Bugfixes

• Fixed a bug that caused backfills launched from the asset graph page not to work with code locations running versions of Dagster less than 1.1.8.
• Fixed a bug that reverted to the default partition mappings in situations where asset dependencies were resolved based on group instead of asset key.
• The way skips are propagate through the graph when using dynamic outputs are used has been fixed.
• Fixed a bug affecting the download link for cloud-based compute log manager implementations (e.g. dagster-azure / dagster-aws / dagster-gcp)
• Fixed a bug that would cause errors when using build_asset_reconciliation_sensor with asset graphs that contained references to source assets without the associated SourceAsset objects (which may happen when using load_assets_from_dbt_*).
• [dagit] Fixed an issue where an error appeared in dagit when a code server stopped and restarted.
• [dagit] Previously, when restarting the dagit process, the Dagit frontend kept cached versions of certain queries even after the code location finished loading. This could lead to display of stale versions of jobs or other code objects. These objects will now be correctly retrieved anew from the backend.
• [dagster-dbt] Fixed a bug with the DbtManifestAssetSelection which could result in KeyErrors when selecting from projects with sources defined.
• [dagster-k8s] Fixed a bug where disabling run worker crash recovery by setting maxResumeRunAttempts to 0 in the Helm chart had no effect.
• [dagster-airflow] Fixed a bug where transformed Airflow DAG schedules would always use UTC for their timezone.

Breaking Changes

• [dagit] The /instance and /workspace path prefixes were removed in previous version, but redirects were left in place. These redirects have now been removed.

Community Contributions

• The new StaticPartitionMapping enables explicitly defining the dependencies between partitions in two StaticPartitionsDefinitions. Thanks Alexander VR!
• Fixed a typo in the Dagster Instance documentation header - thanks Cushnir Grigore!
• Fixed a typo in the Dagster Instance documentation body - thanks Chris Zubak-Skees!
• Fixed docstring for static_partitioned_config - thanks Sylvain Lesage!
• Fix dead link in the docs to the Slack community - thanks Emil Christensen!

Documentation

• The Ops and jobs tutorial has been moved to the Guides section. Clicking "Tutorial" in the sidenav will open the Assets tutorial.

All Changes

https://github.com/dagster-io/dagster/compare/1.1.9...1.1.10

<details><summary>See All Contributors</summary>

• aabd812 - docs for new tag_concurrency_limits feature on executor (#11499) by @gibsondan
• f898164 - [dagster-airflow] warn on airflow dataset (#11498) by @Ramshackle-Jamathon
• d8afae3 - Add typehints to snapshot arguments to create_run (#11454) by @schrockn
• 4696ca0 - Typehint and do invariants for root_run_id, parent_run_id, step_keys_to_execute and execution_plan_snapshot (#11455) by @schrockn
• 17a5af8 - Typehint and validate tags and status params to create_run (#11456) by @schrockn
• bccc196 - Require kwargs on create_reexecuted_run (#11460) by @schrockn
• 4cf382a - Make frozentags inherit from Mapping[str, str] and correctly type _check_execute_external_pipeline_args (#11461) by @schrockn
• cecf84f - Add support for loading cacheable assets from module (#10389) by @benpankow
• f6add35 - enable filtering by asset tag when asset tags table is not present (#11509) by @sryza
• 7775c0a - only count materializations within backfill (#11506) by @sryza
• 78afde9 - telemetry: add num_assets_in_repo in log_repo_stats metadata (#11513) by @yuhan
• 1cbda0c - telemetry: add location_name in repo-level metadata (#11514) by @yuhan
• ecf8753 - [dagit] Clean up /instance and /workspace redirects (#11503) by @hellendag
• 7100ffe - Fix getting partition mappings when asset dependencies are resolved (#11483) by @sryza
• c6bf10b - docstring examples for AssetSelection.keys (#11477) by @sryza
• e9822ed - [dagster-airflow] add make_dagster_definitions_from_airflow_dags_pathandmake_dagster_definitions_from_airflow_dag_bag apis (#11441) by @Ramshackle-Jamathon
• 6590ed9 - Fix maxResumeRunAttempts being explicitly enabled even if you set it to 0 (#11511) by @gibsondan
• 6048df9 - dagster-graphql tests for permissions failures (#11486) by @gibsondan
• f229aa1 - Hook for Cloud dagit into PythonErrorInfo (#11467) by @salazarm
• 1c80d73 - Changelog 1.1.8 (#11526) by @clairelin135
• a392f23 - Add experimental examples directory (#11505) by @schrockn
• 7c2d72d - [dagster-airbyte] Allow specifying freshness policies on Airbyte assets (#11525) by @benpankow
• 5cfad3b - Migrate airflow guide to definitions (#11518) by @petehunt
• 7fcf021 - Automation: versioned docs for 1.1.8 by @elementl-devtools
• 36ac1d3 - Remove packaging v22 cap (#11212) by @smackesey
• 84fddc1 - remove pragmas in docs snippets (#11335) by @smackesey
• 5ddb9a9 - update black 22.3.0 -> 22.12.0 and enable --preview (#10900) by @smackesey
• 78df951 - Adopt ruff as linter (#10901) by @smackesey
• 157a1ed - Add .git to excludes for ruff in pyproject.toml (#11534) by @schrockn
• 692b566 - chore(ruff): add pre-commit config (#11538) by @rexledesma
• f81e051 - chore(dagit): remove unnecessary dependencies (#11532) by @rexledesma
• 8cf4f94 - docs: add links to follow LinkedIn page (#11533) by @rexledesma
• ea8934d - [dagit] Use cache-and-network as default watchQuery fetch policy (#11523) by @hellendag
• 974752f - [docs] - Add GSC site verification file (#11547) by @erinkcochran87
• c272f68 - disable executing airflow_example_http_operator in tests (#11544) by @Ramshackle-Jamathon
• 313481f - Add and apply ruff docstring rules (#10907) by @smackesey
• 7b2c1b0 - Ruff docstrings D209 (NewLineAfterLastParagraph) (#10993) by @smackesey
• 9a0a8e7 - Use Config class instead of base class kwargs in structured config to not break early pydantic versions (#11546) by @schrockn
• 351e713 - [fix] Fix builtin type error related to resource annotations (#11551) by @benpankow
• 25e1197 - [changelog] 1.1.9 (#11556) by @benpankow
• b24b5b4 - add NullMetadataValue (#10969) by @smackesey
• 40bb609 - Ruff docstrings D210 (NoSurroundingWhitespace) (#10994) by @smackesey
• c13c992 - [changelog] 1.1.9 round 2 (#11560) by @benpankow
• f4dfd29 - Make JsonMetadataValue accept lists (#11550) by @smackesey
• f4e84b5 - Ruff docstrings D212,D214 (MultilineSummaryFirstLine,SectionNotOverIndented) (#10995) by @smackesey
• 94549f6 - Automation: versioned docs for 1.1.9 by @elementl-devtools
• 5d3851e - Ruff docstring D300 (UsesTripleQuotes) (#10996) by @smackesey
• 1884a92 - Add a graphql field and API for location-scoped permissions (#11097) by @gibsondan
• 0961570 - [@dagster-io/eslint-config] Remove GraphQL type parameter rule (#11479) by @hellendag
• 89b7c68 - Ruff docstrings D301 (Raw string for backslash in docstring)) (#11329) by @smackesey
• 11b457a - [dagit] Update eslint-config dep in dagster-io/ui (#11563) by @hellendag
• 5f860b9 - [dagit] Stop generating docblocks on schema.graphql and codegen (#11564) by @hellendag
• ee71a58 - [dagit] Track most recent preference for computed log type view (#11527) by @hellendag
• db0d80a - Fix graphql permisisons checks, fix black (#11566) by @gibsondan
• 260e3e3 - [docs] - Move ops + jobs tutorial to Guides (#11528) by @erinkcochran87
• 29b9d48 - [dynamic] fix how skips cascade (#11561) by @alangenfeld
• 27ddfe4 - [dagit] Add GraphQL configuration for VSCode (#11565) by @hellendag
• 00b8a7a - [dagster-dbt] add DbtManifestAssetSelection to API docs (#11562) by @OwenKephart
• 89ae9b6 - define_asset_job accept AssetKey and AssetsDefinition selections (#11568) by @sryza
• 653490e - Ruff docstrings D403,D405 (FirstLineCapitalized,CapitalizeSectionName) (#10997) by @smackesey
• a08fc29 - fix remote captured log manager complete check (#11552) by @prha
• 1f58720 - misc docs fixes in tutorial / getting started / assets (#11575) by @gibsondan
• 41ba92f - docs: fix typo Telementry -> Telemetry (#11571) by @chriszs
• 2f1db63 - fix docstring for static_partitioned_config (#11535) by @severo
• f670262 - Add tip when the multiprocess executor hits a likely OOM error (#11570) by @gibsondan
• 8036414 - Ruff docstrings D414 (NonEmptySection) (#10999) by @smackesey
• c5a8ee3 - Ruff docstrings D419 (NonEmpty) (#11000) by @smackesey
• 3b1ff76 - [@dagster-io/ui] Output cjs (#11594) by @hellendag
• 5f18862 - Show clearer grpc errors, as well as a custom failure for sensor timeouts in particular since those are so common (#11576) by @gibsondan
• 1f60700 - Adds OSS email collection dialog (#11480) by @salazarm
• d34e087 - Ruff docstrings D202 (NoBlankLineAfterFunction) (#10992) by @smackesey
• e579f9b - Ruff docstrings D411,D412 (BlankLineBeforeSection,NoBlankLinesBetweenHeaderAndContent) (#10998) by @smackesey
• 46a4a6f - Change exclude to extend-exclude in pyproject.toml settings for ruff (#11591) by @schrockn
• e46519a - refactor(dbt-cloud): remove v2 suffix (#11586) by @rexledesma
• ff2481d - feat(dbt-cloud): allow env vars to be retrieved and set (#11592) by @rexledesma
• 79bdf57 - Ruff docstrings (misc fixes) (#11002) by @smackesey
• 31bf0d7 - [docs] - Remove accidental comments (#11601) by @erinkcochran87
• 51f21f9 - docs: fix dead link to Slack community (#11602) by @EmilRex
• 88ae9ce - Fix graphene enum not being instantiated (#11605) by @gibsondan
• 1bd5924 - [dagit] Add location-scoped permissions to Dagit (#11587) by @hellendag
• 458936b - [dagster-dbt] parse log level when json_log_level is not set (#11596) by @OwenKephart
• 2a00a0c - use Python type annotations in loading-from-config docs examples (#11609) by @sryza
• ce974ad - [dagit] Use updated react-scripts fork for codegen babel plugin (#11583) by @hellendag
• ea523f5 - Fall back to a different port when 3000 is in use instead of failing (#11610) by @gibsondan
• 09ec94b - [dagit] Some Permissions tweaks to support Cloud (#11612) by @hellendag
• 7a8ba5c - Ruff docstrings D415 (EndsInPunctuation) (#11001) by @smackesey
• c8d9efe - [dagster-dbt] allow for non-model nodes to be part of the DbtManifestAssetSelection (#11617) by @OwenKephart
• 67a915b - Update OSS Nux to receive size from iframe via postmessage (#11625) by @salazarm
• 40d678f - [freshness-policies] fix get_non_source_roots (#11619) by @OwenKephart
• 6e626c1 - remove reexecute_pipeline_iterator (#11608) by @alangenfeld
• ec9b417 - [eslint-config] Enable ignoreExternal on no-cycle rule (#11632) by @hellendag
• 61eeaa7 - [@dagster-io/ui] Bump eslint-config (#11634) by @hellendag
• 9358501 - add example to TimeWindowPartitionMapping apidoc (#11615) by @sryza
• 29e3981 - Fix typo in Dagster instance documentation (#11621) by @GrigoriiKushnir
• 86685e4 - use run tags instead of asset tags to filter materializations within backfill (#11555) by @sryza
• 6a4f466 - test fixes for python 3.7 (#11636) by @gibsondan
• ae8005d - Restrict the grpcio pin to only py3.7/3.8/3.9 (#11635) by @gibsondan
• e618f20 - Remove experimental flag on executor and loggers parameters of Definitions and add more detail to docblock (#11626) by @schrockn
• cb48769 - [dagster-snowflake-pandas] update doc string to use Definitions (#11606) by @jamiedemaria
• e4336f3 - [dagster-snowflake] Add private key authentication for Snowflake (#10750) by @joshuataylor
• db92447 - Load OSS Nux iframe via https (#11638) by @salazarm
• df4e0d4 - telemetry: add step start/success/failure event + client timestamp (#11624) by @yuhan
• e03f3fb - Add StaticPartitionMapping as a built-in PartitionMapping (#11521) by @AlexanderVR
• 2acea53 - [dagster-airbyte] Enable setting a destination table prefix using managed ingestion (#11057) by @benpankow
• 14e21c4 - telemetry: add is_known_ci_env (#11642) by @yuhan
• 8848f6e - stale asset materialization for schedules and sensors (#11516) by @smackesey
• 95b2824 - telemetry: avoid race condition of dagit and daemon creating same dir (#11652) by @yuhan
• c78add2 - [Community Nux] Don't use blueprint dialog (#11660) by @salazarm
• a97e2bf - fix code location status when a new location is added in a loading state (#11654) by @prha
• b1669d2 - [dagster-airflow] pass airflow dag timezone to dagster schedule definition (#11663) by @Ramshackle-Jamathon
• f7344dd - Revert "test fixes for python 3.7 (#11636)" and "Fall back to a different port when 3000 is in use instead of failing (#11610)" (#11671) by @gibsondan
• de8f48b - Remove community nux for now (#11672) by @salazarm
• 5213131 - Unbreak windows tests (and likely the windows multiprocess executor in general) (#11679) by @gibsondan
• 8cc6ccc - 1.1.10 by @elementl-devtools

</details>

New

• Asset backfills launched from the asset graph now respect partition mappings. For example, if partition N of asset2 depends on partition N-1 of asset1, and both of those partitions are included in a backfill, asset2’s partition N won’t be backfilled until asset1’s partition N-1 has been materialized.
• Asset backfills launched from the asset graph will now only materialize each non-partitioned asset once - after all upstream partitions within the backfill have been materialized.
• Executors can now be configured with a tag_concurrency_limits key that allows you to specify limits on the number of ops with certain tags that can be executing at once within a single run. See the docs for more information.
• ExecuteInProcessResult, the type returned by materialize, materialize_to_memory, and execute_in_process, now has an asset_value method that allows you to fetch output values by asset key.
• AssetIns can now accept Nothing for their dagster_type, which allows omitting the input from the parameters of the @asset- or @multi_asset- decorated function. This is useful when you want to specify a partition mapping or metadata for a non-managed input.
• The start_offset and end_offset arguments of TimeWindowPartitionMapping now work across TimeWindowPartitionsDefinitions with different start dates and times.
• If add_output_metadata is called multiple times within an op, asset, or IO manager handle_output, the values will now be merged, instead of later dictionaries overwriting earlier ones.
• materialize and materialize_to_memory now both accept a tags argument.
• Added SingleDimensionDependencyMapping, a PartitionMapping object that defines a correspondence between an upstream single-dimensional partitions definition and a downstream MultiPartitionsDefinition.
• The RUN_DEQUEUED event has been removed from the event log, since it was duplicative with the RUN_STARTING event.
• When an Exception is raised during the execution of an op or asset, Dagit will now include the original Exception that was raised, even if it was caught and another Exception was raised instead. Previously, Dagit would only show exception chains if the Exception was included using the raise Exception() from e syntax.
• [dagit] The Asset Catalog table in Dagit is now a virtualized infinite-scroll table. It is searchable and filterable just as before, and you can now choose assets for bulk materialization without having to select across pages.
• [dagit] Restored some metadata to the Code Locations table, including image, python file, and module name.
• [dagit] Viewing a partition on the asset details page now shows both the latest materialization and also all observations about that materialization.
• [dagit] Improved performance of the loading time for the backfills page
• [dagit] Improved performance when materializing assets with very large partition sets
• [dagit] Moving around asset and op graphs while selecting nodes is easier - drag gestures no longer clear your selection.
• [dagster-k8s] The Dagster Helm chart now allows you to set an arbitrary kubernetes config dictionary to be included in the launched job and pod for each run, using the runK8sConfig key in the k8sRunLauncher section. See the docs for more information.
• [dagster-k8s] securityContext can now be set in the k8sRunLauncher section of the Dagster Helm chart.
• [dagster-aws] The EcsRunLauncher can now be configured with cpu and memory resources for each launched job. Previously, individual jobs needed to be tagged with CPU and memory resources. See the docs for more information.
• [dagster-aws] The S3ComputeLogManager now takes in an argument upload_extra_args which are passed through as the ExtraArgs parameter to the file upload call.
• [dagster-airflow] added make_dagster_definitions_from_airflow_dags_path and make_dagster_definitions_from_airflow_dag_bag which are passed through as the ExtraArgs parameter to the file upload call.

Bugfixes

• Fixed a bug where ad-hoc materializations of assets were not correctly retrieving metadata of upstream assets.
• Fixed a bug that caused ExperimentalWarnings related to LogicalVersions to appear even when version-based staleness was not in use.
• Fixed a bug in the asset reconciliation sensor that caused multi-assets to be reconciled when some, but not all, of the assets they depended on, were reconciled.
• Fixed a bug in the asset reconciliation sensor that caused it to only act on one materialization per asset per tick, even when multiple partitions of an asset were materialized.
• Fixed a bug in the asset reconciliation sensor that caused it to never attempt to rematerialize assets which failed in their last execution. Now, it will launch the next materialization for a given asset at the same time that it would have if the original run had completed successfully.
• The load_assets_from_modules and load_assets_from_package_module utilities now will also load cacheable assets from the specified modules.
• The dequeue_num_workers config setting on QueuedRunCoordinatoris now respected.
• [dagit] Fixed a bug that caused a “Maximum recursion depth exceeded” error when viewing partitioned assets with self-dependencies.
• [dagit] Fixed a bug where “Definitions loaded” notifications would constantly show up in cases where there were multiple dagit hosts running.
• [dagit] Assets that are partitioned no longer erroneously appear "Stale" in the asset graph.
• [dagit] Assets with a freshness policy no longer appear stale when they are still meeting their freshness policy.
• [dagit] Viewing Dagit in Firefox no longer results in erroneous truncation of labels in the left sidebar.
• [dagit] Timestamps on the asset graph are smaller and have an appropriate click target.
• [dagster-databricks] The databricks_pyspark_step_launcher will now cancel the relevant databricks job if the Dagster step execution is interrupted.
• [dagster-databricks] Previously, the databricks_pyspark_step_launcher could exit with an unhelpful error after receiving an HTTPError from databricks with an empty message. This has been fixed.
• [dagster-snowflake] Fixed a bug where calling execute_queries or execute_query on a snowflake_resource would raise an error unless the parameters argument was explicitly set.
• [dagster-aws] Fixed a bug in the EcsRunLauncher when launching many runs in parallel. Previously, each run risked hitting a ClientError in AWS for registering too many concurrent changes to the same task definition family. Now, the EcsRunLauncher recovers gracefully from this error by retrying it with backoff.
• [dagster-airflow] Added make_dagster_definitions_from_airflow_dags_path and make_dagster_definitions_from_airflow_dag_bag for creating Dagster definitions from a given airflow Dag file path or DagBag

Community Contributions

• Fixed a metadata loading error in UPathIOManager, thanks @danielgafni!
• [dagster-aws]FakeS3Session now includes additional functions and improvements to align with the boto3 S3 client API, thanks @asharov!
• Typo fix from @vpicavet, thank you!
• Repository license file year and company update, thanks @vwbusguy!

Experimental

• Added experimental BranchingIOManager to model use case where you wish to read upstream assets from production environments and write them into a development environment.
• Add create_repository_using_definitions_args to allow for the creation of named repositories.
• Added the ability to use Python 3 typing to define and access op and asset config.
• [dagster-dbt] Added DbtManifestAssetSelection, which allows you to define selections of assets loaded from a dbt manifest using dbt selection syntax (e.g. tag:foo,path:marts/finance).

Documentation

• There’s now only one Dagster Cloud Getting Started guide, which includes instructions for both Hybrid and Serverless deployment setups.
• Lots of updates throughout the docs to clean up remaining references to @repository, replacing them with Definitions.
• Lots of updates to the dagster-airflow documentation, a tutorial for getting started with Dagster from an airflow background, a migration guide for going to Dagster from Airflow and a terminology/concept map for Airflow onto Dagster.

All Changes

https://github.com/dagster-io/dagster/compare/1.1.7...1.1.8

<details><summary>See All Contributors</summary>

• 95950df - Adjust resources guide to be in a Definitions world by @schrockn
• f245696 - add thread name prefix to run dequeue workers (#11155) by @alangenfeld
• 45b4c26 - make schedules produced by build_schedule_from_partitioned_job more p… (#11147) by @sryza
• 2d3c914 - [k8s launcher] security context (#9788) by @alangenfeld
• b655ca6 - [run coordinator] fix threaded tests (#11139) by @alangenfeld
• 1f7980e - [docs] - [definitions] Update Dagit + tutorial screenshots (#11089) by @erinkcochran87
• c5c6eff - [docs] - [definitions] Update Partitions concept docs (#11030) by @erinkcochran87
• 6ef79f0 - Port asset sensor guide to Definitions by @schrockn
• 8cf4dba - Use buildkite_deps.txt to declare explicit buildkite deps (#11025) by @jmsanders
• 52c726f - Trigger builds when .ini files change (#11161) by @jmsanders
• 7480066 - Revert "keep track of max timestamps client side for code location up… (#11162) by @prha
• 679d2e1 - [docs] - Fix links (#11163) by @erinkcochran87
• b16f30c - Add create_repository_using_definitions_args by @schrockn
• b4f8705 - Change Graph-backed asset guide code examples to be on Definitions by @schrockn
• e71dacf - Delete repository unit testing using load_all_definitions in testing guide by @schrockn
• 2ae2ed6 - 1.1.7 Changelog (#11165) by @jamiedemaria
• c0d8a9a - Do a single pip install in tox suites [OSS] (#11164) by @gibsondan
• a763ec9 - Update declarative scheduling guide to include Definitions by @schrockn
• 4467bc7 - add thread name prefix to grpc server (#11158) by @alangenfeld
• fb59fda - Automation: versioned docs for 1.1.7 by @elementl-devtools
• 1518150 - Fixup rename of requirements.txt -> buildkite_deps.txt (#11160) by @jmsanders
• 9a46201 - fix precedence ordering when merging dictionaries in container context (#11169) by @gibsondan
• bf4222c - support in and len on PartitionsSubsets (#11172) by @sryza
• d64b050 - Disable breaking azure tests in master (#11183) by @schrockn
• 722c3b6 - [docs] Fix typo in title (#11156) by @vpicavet
• 630d9c6 - Fix regression with passing in None to snowflake resource (#11182) by @gibsondan
• b2bb257 - [dagit] Add tests for partition health data parsing / accessors (#11114) by @bengotow
• 3922523 - Allow setting resources on EcsContainerContext and EcsRunLauncher (#11170) by @gibsondan
• d8555aa - fix a small bug in UPathIOManager (#11110) by @danielgafni
• 15b0f27 - [dagster-dbt] in tests, pin dbt rpc < 0.3.0 (#11196) by @OwenKephart
• baadabe - Move core_tests/storage_tests to storage_tests (#11180) by @schrockn
• bcd1229 - skip sqlite env var test on windows (#11195) by @gibsondan
• 59faa75 - Move old_sqlalchemy_tests to only run on storage_tests (#11181) by @schrockn
• fb3d4f9 - [dagster-airflow] airflow terminology mapping (#11015) by @Ramshackle-Jamathon
• c4e398b - Move core_tests/definitions_tests to definitions_tests (#11184) by @schrockn
• ab3498a - fix dequeue_num_workers setting (#11198) by @alangenfeld
• 3db32c1 - Move core_tests/asset_defs_tests to asset_defs_tests (#11186) by @schrockn
• 8ae492c - update timeout in sensor run tests (#11193) by @jamiedemaria
• 80557d2 - Move core_tests/launcher_tests to launcher_tests (#11187) by @schrockn
• 4961453 - add assets example to AssetSelection apidoc (#11194) by @sryza
• b242ed7 - Move various logging tests into logging_tests (#11188) by @schrockn
• ed912c3 - feat(dbt-cloud): add Dagster run id to dbt Cloud run (#11005) by @rexledesma
• d521afa - [docs] snowflake reference page (#10985) by @jamiedemaria
• 6acf983 - [declarative-scheduling] Fix bug with declarative scheduling where repeated calls to get_latest_materialization_record could return incorrect results (#11214) by @OwenKephart
• 732ab85 - only reconcile multi-assets if all parents are reconciled (#11190) by @sryza
• 1ed7d08 - [dagit] Fix dragging on the DAG clearing your selection / clicking links (#11202) by @bengotow
• e446201 - [dagster-aws] Extend fake S3 resource (#11105) by @asharov
• b99fd34 - [dagit] Optimizations to backfill UI for large partition key sets (#11201) by @bengotow
• 4d79412 - do not compute projected logical versions of partitioned assets (#11204) by @sryza
• ae3ed2d - factor out asset reconciliation graph traversal into util (#11206) by @sryza
• 6ee425f - [dagit] Fresh + Stale should not show “Stale” on the Asset Graph (#11234) by @bengotow
• c5cb42a - [dagit] Round middle truncate calculations for Firefox (#11203) by @bengotow
• cbf21af - support tags argument on materialize and materialize_to_memory (#11225) by @sryza
• 530e118 - docs: update license to include year and company (#11231) by @vwbusguy
• 1895517 - [docs] - [definitions] Update OSS deployment overview (#11199) by @erinkcochran87
• d94cff0 - split backfill table so partition status is fetched lazily (#11205) by @prha
• 934f7c3 - fix tslint (#11242) by @alangenfeld
• 76b00ba - [dagit] webpack-bundle-analyzer (#11230) by @hellendag
• 8f72eb0 - [dagit] Virtualized Asset Catalog (#11168) by @hellendag
• cd523d9 - Can remove hardcoded_resource from project_fully_featured because of Definitions (#11243) by @schrockn
• ced6042 - [dagit] Replace moment-timezone (#11197) by @hellendag
• 4cb9a82 - Remove unused backfillStatus, which loads individual run status (#11246) by @prha
• e763c59 - Retry registering task definitions (#11192) by @jmsanders
• 9e5df46 - Rerun dagster_tests with --snapshot-update (#11208) by @schrockn
• 17f13ea - Add test case for binding assets before passing to Definitions (#11216) by @schrockn
• 2c03dfd - [dagster-airflow] from airflow to dagster guide updates (#11218) by @Ramshackle-Jamathon
• 86338fa - [graphql test] share schema instance (#11236) by @alangenfeld
• 9460504 - Use FromSourceAsset instead of FromRootInputManager when loading assets with input managers (#11233) by @jamiedemaria
• 8850c6a - Use bare objects for the hacker news resources (#11249) by @schrockn
• 34a5f5a - Use bare I/O manager in fully featured (#11250) by @schrockn
• 3f30430 - Delete unused fixed_s3_pickle_io_manager (#11251) by @schrockn
• 5df60bf - Consolidate _resolve_bound_config and have logger and resource use the same one (#11209) by @schrockn
• df13fc6 - Delete op version of _resolve_bound_config and call generic one (#11211) by @schrockn
• 2ecec63 - Skip race condition tests (#11286) by @jmsanders
• eca24df - Skip flaky test (#11292) by @jmsanders
• 5d83868 - [dagit] Replace remaining moment usage (#11278) by @hellendag
• a28f5d5 - Update op-retries.mdx - no solid (#11284) by @yuhan
• 11c1640 - [docs] airbyte guide repository -> definitions (#11296) by @yuhan
• 8be6dc3 - [docs] fivetran guide repository -> definitions (#11297) by @yuhan
• 547a6cb - [docs] dbt guide repository -> definitions (#11298) by @yuhan
• ed212ed - [docs] dbt cloud guide repository -> definitions (#11299) by @yuhan
• 52336d7 - [declarative-scheduling] Rework constraint passing (#11229) by @OwenKephart
• 58a31a0 - [dagster-airflow] basic airflow migration guide (#11012) by @Ramshackle-Jamathon
• d46f969 - [dagstermill] add retries to flaky tests (#11291) by @jamiedemaria
• c8f3a0c - Product tour component (#11227) by @salazarm
• bd372fd - [dagit] Utility for timezone-aware date/time formatting (#11285) by @hellendag
• 8ca865f - [dagit] Restore some metadata on Code Locations page (#11281) by @hellendag
• d8b5c3b - [docs] - [definitions] Update Loggers Concept docs (#11171) by @erinkcochran87
• b09e95a - [docs] - Update ECS deployment guide (#11289) by @erinkcochran87
• d235619 - [docs] - [definitions] Update Dagster instance docs (#11241) by @erinkcochran87
• be59c3e - Pass duckdb_path to __init__ rather than relying on context (#11300) by @schrockn
• 40cd16a - [structured config] Base structured config implementation (#11268) by @benpankow
• a0152b8 - [structured config] Add support for default values (#11272) by @benpankow
• a21feb4 - Add gql pin (#11312) by @gibsondan
• d1ae079 - [structured config] Add support for class, field descriptions (#11274) by @benpankow
• ec47d7c - Move env var injection earlier in step command (#11239) by @gibsondan
• 3e47f52 - accrete metadata with multiple calls to add_output_metadata (#9518) by @sryza
• 534e38d - [docs] - [definitions] - Update Executors docs (#11247) by @erinkcochran87
• 905ef7a - [docs] - [definitions] Update Helm guide (#11320) by @erinkcochran87
• a3963b1 - Eliminating unnecessary output_context.resource_config check (#11313) by @schrockn
• 49dd2c7 - [docs] - Consolidate sections in Resources concept doc (#11316) by @erinkcochran87
• 0e670b6 - [docs] - [definitions] Update SDA Concept docs (#11018) by @erinkcochran87
• b22c163 - [docs] - [definitions] Update Run launchers guide (#11200) by @erinkcochran87
• 9632313 - [docs] - Re-do Cloud Getting Started guides (#10429) by @erinkcochran87
• f099158 - [docs] - [definitions] Update Docker guide (#11295) by @erinkcochran87
• 09c13a5 - backfill perf: swap backfill requested for num cancelable (#11304) by @prha
• a3c0e71 - allow upload config to pass through s3 compute log manager (#11317) by @prha
• b911f34 - [apidoc] define_asset_job repository -> Definitions (#11302) by @yuhan
• dbca38a - Ignore stale timestamps from code location updates (#11173) by @prha
• 43d4283 - [structured config] Fix usage with Assets (#11327) by @benpankow
• e72a3f5 - Include parent exceptions in Dagster errors, even if they weren't explicitly raised (#11306) by @gibsondan
• ec38705 - [docs] - [definitions] Update Dagster daemon docs (#11226) by @erinkcochran87
• e8e3c57 - Docs release backfill 1.1.7 (#11328) by @yuhan
• e92ce74 - Use dark mode logo on README.md (#11326) by @hellendag
• a88a38e - AssetGraph.from_external_assets().get_required_multi_asset_keys() (#11318) by @sryza
• 8f80959 - [declarative-scheduling] Update retry logic to attempt to retry failed materializations after some time has passed (#11294) by @OwenKephart
• c90056e - Add instance property to InputContext (#11331) by @schrockn
• dbfc06a - chore: add method to strip error stack trace (#11307) by @rexledesma
• 661b4e4 - Hoist schema and database to DbIOManager constructor (#11301) by @schrockn
• 3b0fe94 - Rename _resolve_bound_config to resolve_bound_config (#11287) by @schrockn
• 7839ee3 - Add asset_materialization property to EventLogEntry (#11340) by @schrockn
• 3a11f32 - store repository on external asset graph (#11332) by @sryza
• e830ac6 - Revert "Use dark mode logo on README.md (#11326)" (#11345) by @hellendag
• 6e8d060 - [dagster-io/eslint-config] v1.0.6 (#11324) by @hellendag
• abfc7b5 - Add get_implicit_global_asset_job on Definitions and RepositoryDefinition (#11279) by @schrockn
• 73e6a02 - feat: add retry number and url to integration api call failure (#11308) by @rexledesma
• 538e5b3 - Fix dagster-graphql circular import (BK broken) (#11339) by @smackesey
• ccf7468 - Skipping dbt rpc resource tests (#11357) by @schrockn
• 7434f5e - skip test_threaded_ephemeral_instance (#11359) by @schrockn
• 6eb8673 - Allow specifying resources in Op/Asset params list (#11322) by @benpankow
• 945eb03 - change version placeholder 0+dev -> 1!0+dev (#11334) by @smackesey
• 2975629 - [structured config] Structured-config-backed Resources (#11321) by @benpankow
• a34a02d - [structured config] Add traditional resource wrapper base class (#11337) by @benpankow
• a842b2b - [structured config] Fix test importing functools.cached_property on py37 tests (#11361) by @benpankow
• b8a7b0c - [structured config] Structured-config-backed IO managers (#11343) by @benpankow
• 62e1527 - fix some bugs in ExternalAssetGraph (#11350) by @sryza
• 02ed6c2 - more refactors to reconciliation sensor (#11223) by @sryza
• 9d25371 - [dagster-io/eslint-config] v1.0.7 (#11347) by @hellendag
• 586f15b - Add Materialize Button hook (#11319) by @salazarm
• 1a57141 - Allow setting raw k8s config at the run launcher / container context level (#11333) by @gibsondan
• af2ddc6 - Make it more likely that we hit our lock (#11290) by @jmsanders
• 860ffe6 - fix __contains__ of TimeWindowPartitionsSubset (#11380) by @sryza
• 2394272 - [docs] - add note on pandas integration to redirect users interested in pandas w/out validation (#11342) by @slopp
• 4dbf286 - fix asset reconciliation bug that ignores earlier partitions (#11336) by @sryza
• 0263eab - fix typo on partitions concepts page (#11349) by @sryza
• 9832545 - Clicking on top-level concept sections takes you to a page (#10140) by @sryza
• 81b192d - Add single dimension -> multidimension partition mapping (#10910) by @clairelin135
• d761f14 - [RFC] Update cached_status_data column 1/2 (#10821) by @clairelin135
• b60b9a1 - Remove stray console.log (#11410) by @gibsondan
• 1c5733d - ExecuteInProcessResult.asset_value (#11403) by @sryza
• 9504c26 - Elminate PIPELINE_DEQUEUED event (#11393) by @gibsondan
• 551a532 - [dagit] Begin adding new graphql-codegen (#11411) by @hellendag
• b88880c - Docs for new ECS resource options (#11391) by @gibsondan
• 26e9853 - Docs for new k8s configuration options (#11390) by @gibsondan
• 7854e62 - Fix malformed dagster_cloud.yaml in code location docs (#11416) by @gibsondan
• be844f7 - [dagit] Refactor mutations to new GraphQL codegen (#11414) by @hellendag
• ffe89bf - [dagit] Refactor queries in src/workspace (#11418) by @hellendag
• f29e001 - [dagit] dedupeFragments (#11426) by @hellendag
• 2bc5146 - relax matching criteria for TimeWindowPartitionMappings w/offsets (#11422) by @sryza
• 5ba8f4e - [dagster-databricks] handle empty responses (#11430) by @OwenKephart
• b5fbfed - Export RunRecord in the public API (#11427) by @gibsondan
• 67c7650 - [dagit] Refactor queries in src/runs (#11419) by @hellendag
• 03fcf9b - [dagit] Refactor queries in Assets (#11423) by @hellendag
• b7b7d06 - [dagit] Refactor queries in instance, instigation, launchpad (#11429) by @hellendag
• 384fe6b - Branching I/O Manager (#11315) by @schrockn
• a4ea49f - Fix issues with a SerializableErrorInfo being coerced to a GraphenePythonError (#11437) by @gibsondan
• 186e2c7 - export product tour component (#11314) by @salazarm
• f12798f - [dagit] Refactor queries in remaining src (#11435) by @hellendag
• ba3a381 - Eliminate unused config_or_config_fn arg in copy_for_configured (#11433) by @schrockn
• a6dcbfe - [dagit] Don’t show assets as stale if projectedLogicalVersion is null (#11224) by @bengotow
• 9ca91c5 - [dagit] Show observations about the latest materialization on Asset > Partitions (#11288) by @bengotow
• 157d901 - asset backfill core logic (#11377) by @sryza
• d72e34c - Allow AssetIn(dagster_type=Nothing) (#11436) by @sryza
• 5ab885d - Rename and refactor in structured_config.py for clarity (#11372) by @schrockn
• f00cec7 - Consolidate all logic and trickery to support private cached properties in pydantic in a single class (#11373) by @schrockn
• 5a82bdc - deprecate job-level memoization in docs (#11392) by @sryza
• 09a66b9 - Add StructuredIOManagerAdapter (#11383) by @schrockn
• eb5e973 - [dagit] Fix line height and size of timestamps on the asset graph (#11465) by @bengotow
• afaeb13 - Default new-style config mappings to *Source equivalents rather than raw scalars (#11386) by @schrockn
• f2915c5 - Unexperimentalize LogicalVersion and quell warning messages whenever assets are materialized (#11407) by @sryza
• 7d69c44 - Passthrough pydantic.ModelField.required to dagster.Field.is_required (#11388) by @schrockn
• 8c7b92f - Support pydantic aliases in config schema mapping (#11389) by @schrockn
• cdcfedb - Make conversion to *Source types work on direct annotations of the config parameter (#11469) by @schrockn
• dafbf49 - [dagster-databricks] handle execution interrupts (#11421) by @OwenKephart
• d74dcf4 - Skip flaky race condition test (#11471) by @jmsanders
• 08d0b0a - remove AssetStoreHandle (#11412) by @sryza
• ebd9fb2 - refactor: sequester job backfill code (#11252) by @sryza
• 8ca3874 - Be more resilient to user code errors when dequeuing runs (#11406) by @gibsondan
• 0ff7c7f - Make all arguments on DagsterInstance.create_run keyword-only (#11446) by @schrockn
• 9dd7579 - Elimiinate default value for asset_selection param on DagsterInstance.create_run (#11447) by @schrockn
• 5f23b2f - Eliminate default values for solid_selection, external_pipeline_origin, and pipeline_code_origin on create_run (#11448) by @schrockn
• 662077b - rename create_run to create_run_for_test on TestQueuedRunCoordinator to increase greppability of create_run (#11449) by @schrockn
• 4d205f5 - [dagit] Remove old GraphQL Codegen (#11474) by @hellendag
• fe1620e - [dagit] Delete apollo CLI dep (#11478) by @hellendag
• 359e16f - Add tag_concurrency_limits config to executors (#11472) by @gibsondan
• 8ca33ca - add run filter for updated before (#11481) by @prha
• 8d50157 - docs(dagster-dbt): clarify that the integration supports arbitrary dbt profiles (#11351) by @rexledesma
• 057b369 - [structured config] Add support for Permissive fields (#11275) by @benpankow
• 7751e64 - handle pure asset backfills in backfill daemon (#11378) by @sryza
• 5bdc212 - Fix missing callsite of tag_concurrency_limits (#11496) by @gibsondan
• 936660e - fix(docs): format docs (#11501) by @rexledesma
• d7e687e - [dagster-dbt] add DbtManifestAssetSelection (#11473) by @OwenKephart
• 41b616b - Add parameter invariants around external_pipeline_origin and pipeline_code_origin arguments (#11450) by @schrockn
• 6a6bb02 - supply solid_selection to fix submitting runs from pure asset backfills (#11502) by @sryza
• 75b9084 - telemetry: add num_assets_in_repo in repo-level metadata (#11490) by @yuhan
• 5144eab - fix(docs): run mdx-format again (#11504) by @rexledesma
• 0e654f6 - Tighten solids_to_execute and solid_selection invariant (#11451) by @schrockn
• 7d69dad - Bump json5 from 1.0.1 to 1.0.2 in /js_modules/dagit (#11485) by @dependabot[bot]
• 39c0573 - Add invariant for asset_selection (#11452) by @schrockn
• 72654df - Typehint pipeline_name, run_id, and mode (#11453) by @schrockn
• 4b91684 - graphql for pure asset backfills (#11379) by @sryza
• ba75a8f - docs for new tag_concurrency_limits feature on executor (#11499) by @gibsondan
• 8e47dc2 - Add support for loading cacheable assets from module (#10389) by @benpankow
• ea185ef - enable filtering by asset tag when asset tags table is not present (#11509) by @sryza
• c578c55 - only count materializations within backfill (#11506) by @sryza
• 862ca6d - telemetry: add num_assets_in_repo in log_repo_stats metadata (#11513) by @yuhan
• e6a1a94 - telemetry: add location_name in repo-level metadata (#11514) by @yuhan
• 8349be9 - [dagster-airflow] warn on airflow dataset (#11498) by @Ramshackle-Jamathon
• 893ed12 - [dagster-airflow] add make_dagster_definitions_from_airflow_dags_pathandmake_dagster_definitions_from_airflow_dag_bag apis (#11441) by @Ramshackle-Jamathon
• 07c5d7d - Changelog 1.1.8 (#11526) by @clairelin135
• 88c01f1 - 1.1.8 by @elementl-devtools

</details>

New

• Definitions is no longer marked as experimental and is the preferred API over @repository for new users of Dagster. Examples, tutorials, and documentation have largely ported to this new API. No migration is needed. Please see GitHub discussion for more details.
• The “Workspace” section of Dagit has been removed. All definitions for your code locations can be accessed via the “Deployment” section of the app. Just as in the old Workspace summary page, each code location will show counts of its available jobs, assets, schedules, and sensors. Additionally, the code locations page is now available at /locations.
• Lagged / rolling window partition mappings: TimeWindowPartitionMapping now accepts start_offset and end_offset arguments that allow specifying that time partitions depend on earlier or later time partitions of upstream assets.
• Asset partitions can now depend on earlier time partitions of the same asset. The asset reconciliation sensor will respect these dependencies when requesting runs.
• dagit can now accept multiple arguments for the -m and -f flags. For each argument a new code location is loaded.
• Schedules created by build_schedule_from_partitioned_job now execute more performantly - in constant time, rather than linear in the number of partitions.
• The QueuedRunCoordinator now supports options dequeue_use_threads and dequeue_num_workers options to enable concurrent run dequeue operations for greater throughput.
• [dagster-dbt] load_assets_from_dbt_project, load_assets_from_dbt_manifest, and load_assets_from_dbt_cloud_job now support applying freshness policies to loaded nodes. To do so, you can apply dagster_freshness_policy config directly in your dbt project, i.e. config(dagster_freshness_policy={"maximum_lag_minutes": 60}) would result in the corresponding asset being assigned a FreshnessPolicy(maximum_lag_minutes=60).
• The DAGSTER_RUN_JOB_NAME environment variable is now set in containerized environments spun up by our run launchers and executor.
• [dagster-airflow] make_dagster_repo_from_airflow_dags_path ,make_dagster_job_from_airflow_dag and make_dagster_repo_from_airflow_dag_bag have a new connections parameter which allows for configuring the airflow connections used by migrated dags.

Bugfixes

• Fixed a bug where the log property was not available on the RunStatusSensorContext context object provided for run status sensors for sensor logging.

• Fixed a bug where the re-execute button on runs of asset jobs would incorrectly show warning icon, indicating that the pipeline code may have changed since you last ran it.

• Fixed an issue which would cause metadata supplied to graph-backed assets to not be viewable in the UI.

• Fixed an issue where schedules often took up to 5 seconds to start after their tick time.

• Fixed an issue where Dagster failed to load a dagster.yaml file that specified the folder to use for sqlite storage in the dagster.yaml file using an environment variable.

• Fixed an issue which would cause the k8s/docker executors to unnecessarily reload CacheableAssetsDefinitions (such as those created when using load_assets_from_dbt_cloud_job) on each step execution.

• [dagster-airbyte] Fixed an issue where Python-defined Airbyte sources and destinations were occasionally recreated unnecessarily.

• Fixed an issue with build_asset_reconciliation_sensor that would cause it to ignore in-progress runs in some cases.

• Fixed a bug where GQL errors would be thrown in the asset explorer when a previously materialized asset had its dependencies changed.

• [dagster-airbyte] Fixed an error when generating assets for normalization table for connections with non-object streams.

• [dagster-dbt] Fixed an error where dbt Cloud jobs with dbt run and dbt run-operation were incorrectly validated.

• [dagster-airflow] use_ephemeral_airflow_db now works when running within a PEX deployment artifact.

Documentation

• New documentation for Code locations and how to define one using Definitions
• Lots of updates throughout the docs to reflect the recommended usage of Definitions. Any content not ported to Definitions in this release is in the process of being updated.
• New documentation for dagster-airflow on how to start writing dagster code from an airflow background.

All Changes

https://github.com/dagster-io/dagster/compare/1.1.6...1.1.7

<details><summary>See All Contributors</summary>

• 858b9d2 - Non isolated runs docs (#10860) by @johannkm
• 24bff5f - [dagit] Fix Gantt chart rendering of per-step resource init log messages (#10943) by @bengotow
• cde76d9 - [dagit] Fix the “Assets” label on large asset runs (#10932) by @bengotow
• 6cd734a - [dagit] Fix “Job In” label regression in Chrome v109 (#10934) by @bengotow
• 523edb0 - [dagit] Pass repository tag when loading runs for Partitions page (#10948) by @bengotow
• 49f7c4f - [dagit] Updated asset DAG styles, added additional compute tags (#10931) by @bengotow
• fb574c0 - [docs] - add a guide for scheduling assets (#10949) by @slopp
• 66959b7 - cap packaging requirement at 22.0 (#10968) by @smackesey
• 5a45dd1 - Execution result typing (#10919) by @smackesey
• 7d18ed2 - solid -> node method renames (#10920) by @smackesey
• f19c6ea - [dagster-airflow] re-enable airflow 2.5.0 tests (#10966) by @Ramshackle-Jamathon
• 0d80225 - [dagster-airbyte][docs] Use dagster-airbyte CLI alias in docs (#10955) by @benpankow
• 162e791 - [dagster-slack] create slack_on_freshness_policy_sensor (#10960) by @OwenKephart
• faf4f30 - [docs ] - fix image dimensions in hello-dagster materialize (#10977) by @slopp
• 4babd9d - 1.1.6 Changelog (#10978) by @OwenKephart
• 577e3eb - Automation: versioned docs for 1.1.6 by @elementl-devtools
• 0b59fa9 - [convert-environment-variables-and-secrets-guide-stack-2] Convert env vars and secrets guide from repository to Definitions by @schrockn
• 352d077 - unexperimentalize PartitionMapping (#10980) by @sryza
• 231ddfa - remove validation in AssetGraph.get_child_partition_keys_of_parent an… (#10981) by @sryza
• 305f1a2 - [convert-development-to-production-1] Move repository to __init__.py by @schrockn
• d1054f9 - [convert-development-to-production-2] Changing development to production to use snowflake_pandas_io_manager by @schrockn
• 7d5fabc - [convert-deployment-to-production-3] Convert @repository to Definitions by @schrockn
• 196c086 - [convert-development-to-production-4] Use base object instead of resource by @schrockn
• 435ddcd - [convert-development-to-production-6] Use pyproject.toml instead of workspace.yaml by @schrockn
• 888660d - [convert-development-to-production-7] Convert guide to use Definitions by @schrockn
• ad41a9f - [dagit] New Code Locations table (#10975) by @hellendag
• 3008420 - Pin graphene to <3.2 by @schrockn
• 5373a74 - fix missing metadata in dagit on graph-backed assets (#10988) by @OwenKephart
• 7bbcb8a - [dagit] Export a few Code Location components for Cloud (#11008) by @hellendag
• 39c84bb - Temporarily disable some Azure test suites (#11007) by @jmsanders
• 41101f7 - [graphql] fix for graphene 3.2 (#11011) by @alangenfeld
• a0e5c3d - [code-location-selector-stack] Code location sensor tests 1/N. Rename workspace_load_target function to create_workspace_load_target by @schrockn
• 160aba2 - [code-location-selector-stack] Code location sensor tests 2/N Make instance_with_multiple_repos_with_sensors workspace_load_target parameterizable by @schrockn
• d6ad971 - [code-location-selector-stack] Code location sensor tests 3/N. Refactor instance_with_multiple_repos_with_sensors to handle multiple code locations by @schrockn
• 6e305c1 - [code-location-selector-stack] Code location sensor tests 4/N Actually add test to test cross code location selector by @schrockn
• 5fc415a - [code-location-selector-stack Add CodeLocationSelector; Have run_status_sensor accept it by @schrockn
• 3c2367f - Fix test_persistent by @schrockn
• bf83710 - chore: auto-assign dependabot pull requests (#10953) by @rexledesma
• a91525b - fix(dbt-cloud): parse command string to find materialization commands (#10989) by @rexledesma
• 3be1547 - [code-location-selector-stack] Change typehint on make_slack_on_run_failure_sensor to accept CodeLocationSelector by @schrockn
• c156ab3 - [docs] re-org snowflake integration guide (#10984) by @jamiedemaria
• 33affcf - Check scheduler ticks right after each minute boundary instead of once every 5 seconds (#10886) by @gibsondan
• 3b93947 - [docs] - [definitions] Update Configured API concept doc (#11020) by @erinkcochran87
• 3cfc612 - Fix bug with re-execution snapshot ids (#10967) by @OwenKephart
• 3fb76d9 - [bugfix] UPathIOManger load_input type checking (#11022) by @danielgafni
• 546ece8 - Pathspec typing fix (#11036) by @smackesey
• c729760 - [dagit] /code-locations -> /locations (#11024) by @hellendag
• f606250 - Replace partition ranges with subsets (#10909) by @clairelin135
• cec9f43 - [dagit] With multiple assets selected, backfill “missing” should include partially materialized partitions (#11027) by @bengotow
• ae946c0 - [dagit] Add empty value string for invalid tag input in Runs filter (#11044) by @hellendag
• 57e5def - [docs] - [definitions] Update Repository page for Definitions (#10986) by @erinkcochran87
• 9300fc7 - [definitions-accessors] Add get_job_def to Definitions. by @schrockn
• 1014add - 1/ definitions in create new project: update dagster project CLI (#10829) by @yuhan
• 550ec43 - 2/ definitions in create new project: update create-new-project docs (#10830) by @yuhan
• 7eb520d - [docs] - [definitions] - Update dbt tutorial to use Definitions (#10842) by @erinkcochran87
• 891dc89 - 2.1/ definitions in in create new project: remove unnecessary asset dir in scaffold + update docs (#10831) by @yuhan
• ad1fc35 - update isolated run docs to say that it's enabled by default (#11039) by @gibsondan
• a4cb91e - [docs] - remove repository from hello dagster as its not needed (#11048) by @slopp
• 90b3e01 - [dagster-fivetran][ez] Add missing docstring entry for build_fivetran_assets (#11049) by @benpankow
• b4e5a80 - Add start_offset and end_offset to TimeWindowPartitionMapping (#10979) by @sryza
• ec1d974 - fix subsettable multi asset case (#10878) by @OwenKephart
• e98e5fb - move ecs task tagging to run_task call (#11037) by @prha
• c1d1224 - Add custom resource key to load_assets_from_dbt_project (#10827) by @dpeng817
• cc1925f - [dagit] Highlight Assets in top nav for global asset graph (#11052) by @hellendag
• fc1ab1f - [dagster-airbyte][docs][ez] Fix AirbyteConnection apidoc rendering (#11059) by @benpankow
• 795ae3f - convert-examples 1.1/ assets_dbt_python defs + pyproject (#11060) by @yuhan
• c4d6792 - convert-examples 2.1/ assets_modern_data_stack repo -> defs, pyproject (#11062) by @yuhan
• 09ac5ed - convert-examples 5/ assets_smoke_test repository -> definitions (#11071) by @yuhan
• 48c7ae3 - convert-examples 6.1/ feature_graph_backed_assets pyproject, @repository -> Definitions (#11072) by @yuhan
• 8a12bf3 - convert-examples 7.1/ quickstart_etl workspace->pyproject, repo->defs (#11074) by @yuhan
• d85d59f - convert-examples 8.1/ with_great_expectations workspace->pyproject, repo->defs (#11077) by @yuhan
• e4099ec - convert-examples 1.2/ assets_dbt_python file renames (#11061) by @yuhan
• 4d38088 - convert-examples 6.2/ feature_graph_backed_assets file rename and reorder (#11073) by @yuhan
• 58e0d88 - convert-examples 2.2/ assets_modern_data_stack file renames (#11063) by @yuhan
• 99a62a1 - Removes codecov commands from tox (#10635) by @dpeng817
• 0a675c6 - [dagit] Disable Re-execute menu item in run action menu for perms (#11026) by @hellendag
• db21352 - Support loading multiple modules in single command line invocation by @schrockn
• e06b9ae - Enable multiple files in CLI tools by @schrockn
• 4d4aecb - Add get_sensor_def and get_schedule_def to Definitions by @schrockn
• ba87836 - convert-examples 10/ with_pyspark_emr repository.py -> defs.py, update [docs] (#11080) by @yuhan
• 32c3742 - convert-examples 9/ with_pyspark repository.py -> defs.py, update [docs] (#11079) by @yuhan
• 0234c90 - Add load_asset_value and get_asset_value_loader to Definitions by @schrockn
• c3a962a - [dagster-airflow] reorg dagster airflow api docs (#11009) by @Ramshackle-Jamathon
• f2a705c - convert-examples 3.1/ assets_pandas_pyspark definitons, pyproject (#11067) by @yuhan
• e1a93e1 - convert-examples 8.2/ with_great_expectations file rename (#11078) by @yuhan
• b0bcc14 - convert-examples 7.2/ quickstart_etl file rename + update README (#11075) by @yuhan
• e9651fc - Support module_name in tool.dagster section of pyproject.toml by @schrockn
• f0ed982 - Move all examples to use module_name in pyproject.toml by @schrockn
• 0510adf - Pin tox on windows to < 4 (#11013) by @jmsanders
• b023c47 - [docs] - [definitions] Code location concept page (#10843) by @erinkcochran87
• 606ff79 - Add instance for test to top level API (#10709) by @dpeng817
• 86a4fe1 - [example] recomment code in dagstermill tutorial (#11098) by @jamiedemaria
• e43e653 - Make JobSelector work on cross code location by @schrockn
• 3693ccf - [convert-branch-deployments-stack] Convert branch deployments guide to Definitions by @schrockn
• a541020 - Eliminate package_name from pyproject.toml loading spec by @schrockn
• e1f51b0 - [docs] - [definitions] Update Enriching with SDAs guide (#10849) by @erinkcochran87
• c706db1 - [dagit] Don't show View link on code location toast if already there (#11095) by @hellendag
• 559ebb6 - set DAGSTER_RUN_JOB_NAME env var (#10888) by @alangenfeld
• 9e58100 - remove parameter tag from dagstermill notebook (#11109) by @jamiedemaria
• 195f975 - sort task definition config secrets and env vars to ensure consistent ordering (#10982) by @gibsondan
• b32e5cb - [dagster-airflow] from airflow to dagster guide (#10923) by @Ramshackle-Jamathon
• aefd575 - [dagit] Improvements to partition range selection interactions (#11017) by @bengotow
• 07b6d4e - add tests for legacy storage (#11047) by @prha
• 8884177 - keep track of max timestamps client side for code location updates (#11100) by @prha
• 5caeb9f - in default IO manager, handle PartitionMappings that return 0 partitions (#11065) by @sryza
• 8552f8f - Fix bug with logical versions for assets with changed deps (#11083) by @smackesey
• cbf36be - Adding apidocs for Definitions by @schrockn
• 822f1e8 - convert-examples 3.2/ asset_pandas_pyspark re-arrange files + [docs] (#11068) by @yuhan
• 5ebe03e - [dagit] Persist AssetPartitions state to URL, open Materialize panel with same selection (#11042) by @bengotow
• ceb2050 - [dagit] Render assets with self-dependencies in Dagit (#11043) by @bengotow
• ccb2b47 - run queue daemon refactor (#11090) by @alangenfeld
• fa6b48b - Allow assets to depend on earlier partitions of themselves (#11066) by @sryza
• 1811e25 - [docs] - [definitions] Update Schedules documentation (#11046) by @erinkcochran87
• afc161c - Re-add section for customizing Docker images pre-fast-deploys (#11112) by @shalabhc
• 8a5ade0 - Allow setting base_dir as a StringSource in sqlite storage (#11031) by @gibsondan
• c1ca6ef - handle self-dependencies in reconciliation sensor (#11085) by @sryza
• 621f9fb - [docs] - [definitions] Update Workspaces concept page (#10944) by @erinkcochran87
• d5feb2b - Coerce IOManager objects into IOManagerDefinitions in Definitions by @schrockn
• 1c7542a - Revert "[docs] - [definitions] Update Workspaces concept page (#10944)" by @alangenfeld
• 1292df7 - Allow overriding the default max_concurrent for the default executor via env var (#11116) by @gibsondan
• 2ec20a3 - convert-examples 4.1/ assets_pandas_type_metadata definitions, pyproject (#11069) by @yuhan
• 1f9c37c - [docs] [definitions] convert snowflake guide to definitions (#11029) by @jamiedemaria
• 132ce0d - [docs] - Fix workspace link (#11121) by @erinkcochran87
• e64f0de - convert-examples 4.2/ assets_pandas_type_metadata repository.py -> __init__.py (new) (#11127) by @yuhan
• d36a5e6 - Add executor and loggers to Definitions by @schrockn
• ab743c4 - [docs] - [definitions] Update Sensors documentation (#11050) by @erinkcochran87
• 4506c12 - threaded run queue daemon (#11113) by @alangenfeld
• c87bf8d - convert-examples 11.1/ project_fully_featured defs, pyproject, no with_resources (#11081) by @yuhan
• 9491833 - [dagster-airbyte] Fix normalization table generation, materializations for non-object Streams (#10899) by @benpankow
• c221574 - update assets_modern_data_stack example readme (#11122) by @yuhan
• f0ff38c - [CacheableAssets] Fix bug that would cause definitions to be recomputed when using a StepDelegatingExecutor (#11086) by @OwenKephart
• bd0801b - convert-examples 11.2/ project_fully_featured repository.py -> __init__.py (#11082) by @yuhan
• 4e72f19 - update assets_dbt_python example readme (#11126) by @yuhan
• 7d92ffc - [dagster-dbt] Enable setting FreshnessPolicies on dbt assets (#11103) by @OwenKephart
• c14d793 - Mark Definitions as not experimental by @schrockn
• a0268ff - [declarative-scheduling] Fix issue where in-progress runs were not properly handled (#11118) by @OwenKephart
• 2a66629 - [dagster-airbyte] Avoid unnecessarily recreating sources, destinations w/ managed ingestion (#11117) by @benpankow
• 7ab1263 - [dagster-airflow] pex compatibility for ephemeral db (#11115) by @Ramshackle-Jamathon
• 47e3c31 - [dagster-airflow] provide interface for passing connection models directly to dagster (#11006) by @Ramshackle-Jamathon
• 372aae8 - [dagit] Left nav: show entire code location string for non-dunder repos (#11133) by @hellendag
• 439a32a - move quickstart_aws back to mono repo (#11130) by @yuhan
• 650f837 - move quickstart_gcp back to mono repo (#11131) by @yuhan
• 30bd794 - move quickstart_snowflake back to mono repo (#11132) by @yuhan
• 479ab1a - Update Definitions docs by @schrockn
• a5a8c84 - Use wrap_resources_for_execution in Definitions by @schrockn
• 8182da0 - Use i/o manager coercion in assets_pandas_pyspark example by @schrockn
• 5fdecbd - convert-examples-repo-to-defs cloud nux quickstart_aws (#11135) by @yuhan
• 2169bf8 - convert-examples-repo-to-defs cloud nux quickstart_gcp (#11136) by @yuhan
• 96750d3 - convert-examples-repo-to-defs cloud nux quickstart_snowflake (#11137) by @yuhan
• 841efdc - enable run status sensor logging (#11145) by @prha
• defd785 - Move jobs concept guide to refer to Definitions by @schrockn
• ef723a6 - Move I/O Manager Guide to be on Definitions by @schrockn
• 2d9ff0b - add thread name prefix to run dequeue workers (#11155) by @alangenfeld
• 1670918 - Adjust resources guide to be in a Definitions world by @schrockn
• 61b43cb - make schedules produced by build_schedule_from_partitioned_job more p… (#11147) by @sryza
• 4760e16 - [docs] - [definitions] Update Dagit + tutorial screenshots (#11089) by @erinkcochran87
• 9df72f3 - [docs] - [definitions] Update Partitions concept docs (#11030) by @erinkcochran87
• 799d333 - Port asset sensor guide to Definitions by @schrockn
• 9290f54 - Revert "keep track of max timestamps client side for code location up… (#11162) by @prha
• 468f39b - [docs] - Fix links (#11163) by @erinkcochran87
• ecd4895 - Merge branch 'release-1.1.7' of github.com:dagster-io/dagster into release-1.1.7 by @jamiedemaria
• de634ac - 1.1.7 Changelog (#11165) by @jamiedemaria
• 9882250 - 1.1.7 by @elementl-devtools

</details>

New

• [dagit] Throughout Dagit, when the default repository name __repository__ is used for a repo, only the code location name will be shown. This change also applies to URL paths.
• [dagster-dbt] When attempting to generate software-defined assets from a dbt Cloud job, an error is now raised if none are created.
• [dagster-dbt] Software-defined assets can now be generated for dbt Cloud jobs that execute multiple commands.

Bugfixes

• Fixed a bug that caused load_asset_value to error with the default IO manager when a partition_key argument was provided.
• Previously, trying to access context.partition_key or context.asset_partition_key_for_output when invoking an asset directly (e.g. in a unit test) would result in an error. This has been fixed.
• Failure hooks now receive the original exception instead of RetryRequested when using a retry policy.
• The LocationStateChange GraphQL subscription has been fixed (thanks @****roeij !)**
• Fixed a bug where a sqlite3.ProgrammingError error was raised when creating an ephemeral DagsterInstance, most commonly when build_resources was called without passing in an instance parameter.
• [dagstermill] Jupyter notebooks now correctly render in Dagit on Windows machines.
• [dagster-duckdb-pyspark] New duckdb_pyspark_io_manager helper to automatically create a DuckDB I/O manager that can store and load PySpark DataFrames.
• [dagster-mysql] Fixed a bug where versions of mysql < 8.0.31 would raise an error on some run queries.
• [dagster-postgres] connection url param “options“ are no longer overwritten in dagit.
• [dagit] Dagit now allows backfills to be launched for asset jobs that have partitions and required config.
• [dagit] Dagit no longer renders the "Job in repo@location" label incorrectly in Chrome v109.
• [dagit] Dagit's run list now shows improved labels on asset group runs of more than three assets
• [dagit] Dagit's run gantt chart now renders per-step resource initialization markers correctly.
• [dagit] In op and asset descriptions in Dagit, rendered markdown no longer includes extraneous escape slashes.
• Assorted typos and omissions fixed in the docs — thanks @C0DK and @akan72!

Experimental

• As an optional replacement of the workspace/repository concepts, a new Definitions entrypoint for tools and the UI has been added. A single Definitions object per code location may be instantiated, and accepts typed, named arguments, rather than the heterogenous list of definitions returned from an @repository-decorated function. To learn more about this feature, and provide feedback, please refer to the Github Discussion.
• [dagster-slack] A new make_slack_on_freshness_policy_status_change_sensor allows you to create a sensor to alert you when an asset is out of date with respect to its freshness policy (and when it’s back on time!)

Documentation

• Refreshed dagstermill guide and reference page https://docs.dagster.io/integrations/dagstermill
• New declarative scheduling guide: https://docs.dagster.io/guides/dagster/scheduling-assets
• New dagster-snowflake guide: https://docs.dagster.io/integrations/snowflake
• Added docs for asset code versioning: https://docs.dagster.io/concepts/assets/software-defined-assets#asset-code-versions
• Added docs for observable source assets: https://docs.dagster.io/concepts/assets/asset-observations#observable-source-assets

Bugfixes

• [dagit] Fixed an issue where the Partitions tab sometimes failed to load for asset jobs.

Community Contributions

• Fixed a typo in GCSComputeLogManager docstring (thanks reidab)!
• [dagster-airbyte] job cancellation on run termination is now optional. (Thanks adam-bloom)!
• [dagster-snowflake] Can now specify snowflake role in config to snowflake io manager (Thanks binhnefits)!
• [dagster-aws] A new AWS systems manager resource (thanks zyd14)!
• [dagstermill] Retry policy can now be set on dagstermill assets (thanks nickvazz)!
• Corrected typo in docs on metadata (thanks C0DK)!

New

• Added a job_name parameter to InputContext.
• Fixed inconsistent io manager behavior when using execute_in_process on a GraphDefinition (it would use the fs_io_manager instead of the in-memory io manager).
• Compute logs will now load in Dagit even when websocket connections are not supported.
• [dagit] A handful of changes have been made to our URLs:
  • The /instance URL path prefix has been removed. E.g. /instance/runs can now be found at /runs.
  • The /workspace URL path prefix has been changed to /locations. E.g. the URL for job my_job in repository foo@bar can now be found at /locations/foo@bar/jobs/my_job.
• [dagit] The “Workspace” navigation item in the top nav has been moved to be a tab under the “Deployment” section of the app, and is renamed to “Definitions”.
• [dagstermill] Dagster events can now be yielded from asset notebooks using dagstermill.yield_event.
• [dagstermill] Failed notebooks can be saved for inspection and debugging using the new save_on_notebook_failure parameter.
• [dagster-airflow] Added a new option use_ephemeral_airflow_db which will create a job run scoped airflow db for airflow dags running in dagster
• [dagster-dbt] Materializing software-defined assets using dbt Cloud jobs now supports partitions.
• [dagster-dbt] Materializing software-defined assets using dbt Cloud jobs now supports subsetting. Individual dbt Cloud models can be materialized, and the proper filters will be passed down to the dbt Cloud job.
• [dagster-dbt] Software-defined assets from dbt Cloud jobs now support configurable group names.
• [dagster-dbt] Software-defined assets from dbt Cloud jobs now support configurable AssetKeys.

Bugfixes

• Fixed regression starting in 1.0.16 for some compute log managers where an exception in the compute log manager setup/teardown would cause runs to fail.
• The S3 / GCS / Azure compute log managers now sanitize the optional prefix argument to prevent badly constructed paths.
• [dagit] The run filter typeahead no longer surfaces key-value pairs when searching for tag:. This resolves an issue where retrieving the available tags could cause significant performance problems. Tags can still be searched with freeform text, and by adding them via click on individual run rows.
• [dagit] Fixed an issue in the Runs tab for job snapshots, where the query would fail and no runs were shown.
• [dagit] Schedules defined with cron unions displayed “Invalid cron string” in Dagit. This has been resolved, and human-readable versions of all members of the union will now be shown.

Breaking Changes

• You can no longer set an output’s asset key by overriding get_output_asset_key on the IOManager handling the output. Previously, this was experimental and undocumented.

Experimental

• Sensor and schedule evaluation contexts now have an experimental log property, which log events that can later be viewed in Dagit. To enable these log views in dagit, navigate to the user settings and enable the Experimental schedule/sensor logging view option. Log links will now be available for sensor/schedule ticks where logs were emitted. Note: this feature is not available for users using the NoOpComputeLogManager.

Bugfixes

• Fixed a bug with the asset reconciliation sensor that caused duplicate runs to be submitted in situations where an asset has a different partitioning than its parents.
• Fixed a bug with the asset reconciliation sensor that caused it to error on time-partitioned assets.
• [dagster-snowflake] Fixed a bug when materializing partitions with the Snowflake I/O manager where sql BETWEEN was used to determine the section of the table to replace. BETWEEN included values from the next partition causing the I/O manager to erroneously delete those entries.
• [dagster-duckdb] Fixed a bug when materializing partitions with the DuckDB I/O manager where sql BETWEEN was used to determine the section of the table to replace. BETWEEN included values from the next partition causing the I/O manager to erroneously delete those entries.

Bugfixes

• In Dagit, assets that had been materialized prior to upgrading to 1.1.1 were showing as "Stale". This is now fixed.
• Schedules that were constructed with a list of cron strings previously rendered with an error in Dagit. This is now fixed.
• For users running dagit version >= 1.0.17 (or dagster-cloud) with dagster version < 1.0.17, errors could occur when hitting "Materialize All" and some other asset-related interactions. This has been fixed.

Major Changes since 1.0.0 (core) / 0.16.0 (libraries)

Core

• You can now create multi-dimensional partitions definitions for software-defined assets, through the MultiPartitionsDefinition API. In Dagit, you can filter and materialize certain partitions by providing ranges per-dimension, and view your materializations by dimension.
• The new asset reconciliation sensor automatically materializes assets that have never been materialized or whose upstream assets have changed since the last time they were materialized. It works with partitioned assets too. You can construct it using build_asset_reconciliation_sensor.
• You can now add a FreshnessPolicy to any of your software-defined assets, to specify how up-to-date you expect that asset to be. You can view the freshness status of each asset in Dagit, alert when assets are missing their targets using the @freshness_policy_sensor, and use the build_asset_reconciliation_sensor to make a sensor that automatically kick off runs to materialize assets based on their freshness policies.
• You can now version your asset ops and source assets to help you track which of your assets are stale. You can do this by assigning op_version s to software-defined assets or observation_fn s to SourceAssets. When a set of assets is versioned in this way, their “Upstream Changed” status will be based on whether upstream versions have changed, rather than on whether upstream assets have been re-materialized. You can launch runs that materialize only stale assets.
• The new @multi_asset_sensor decorator enables defining custom sensors that trigger based on the materializations of multiple assets. The context object supplied to the decorated function has methods to fetch latest materializations by asset key, as well as built-in cursor management to mark specific materializations as “consumed”, so that they won’t be returned in future ticks. It can also fetch materializations by partition and mark individual partitions as consumed.
• RepositoryDefinition now exposes a load_asset_value method, which accepts an asset key and invokes the asset’s I/O manager’s load_input function to load the asset as a Python object. This can be used in notebooks to do exploratory data analysis on assets.
• With the new asset_selection parameter on @sensor and SensorDefinition, you can now define a sensor that directly targets a selection of assets, instead of targeting a job.
• When running dagit or dagster-daemon locally, environment variables included in a .env file in the form KEY=value in the same folder as the command will be automatically included in the environment of any Dagster code that runs, allowing you to easily use environment variables during local development.

Dagit

• The Asset Graph has been redesigned to make better use of color to communicate asset health. New status indicators make it easy to spot missing and stale assets (even on large graphs!) and the UI updates in real-time as displayed assets are materialized.
• The Asset Details page has been redesigned and features a new side-by-side UI that makes it easier to inspect event metadata. A color-coded timeline on the partitions view allows you to drag-select a time range and inspect the metadata and status quickly. The new view also supports assets that have been partitioned across multiple dimensions.
• The new Workspace page helps you quickly find and navigate between all your Dagster definitions. It’s also been re-architected to load significantly faster when you have thousands of definitions.
• The Overview page is the new home for the live run timeline and helps you understand the status of all the jobs, schedules, sensors, and backfills across your entire deployment. The timeline is now grouped by repository and shows a run status rollup for each group.

Integrations

• dagster-dbt now supports generating software-defined assets from your dbt Cloud jobs.
• dagster-airbyte and dagster-fivetran now support automatically generating assets from your ETL connections using load_assets_from_airbyte_instance and load_assets_from_fivetran_instance.
• New dagster-duckdb integration: build_duckdb_io_manager allows you to build an I/O manager that stores and loads Pandas and PySpark DataFrames in DuckDB.

Database migration

• Optional database schema migration, which can be run via dagster instance migrate:
  • Improves Dagit performance by adding database indexes which should speed up the run view as well as a range of asset-based queries.
  • Enables multi-dimensional asset partitions and asset versioning.

Breaking Changes and Deprecations

• define_dagstermill_solid, a legacy API, has been removed from dagstermill. Use define_dagstermill_op or define_dagstermill_asset instead to create an op or asset from a Jupyter notebook, respectively.
• The internal ComputeLogManager API is marked as deprecated in favor of an updated interface: CapturedLogManager. It will be removed in 1.2.0. This should only affect dagster instances that have implemented a custom compute log manager.

Dependency Changes

• dagster-graphql and dagit now use version 3 of graphene

Since 1.0.17

New

• The new UPathIOManager base class is now a top-level Dagster export. This enables you to write a custom I/O manager that plugs stores data in any filesystem supported by universal-pathlib and uses different serialization format than pickle (Thanks Daniel Gafni!).
• The default fs_io_manager now inherits from the UPathIOManager, which means that its base_dir can be a path on any filesystem supported by universal-pathlib (Thanks Daniel Gafni!).
• build_asset_reconciliation_sensor now works with support partitioned assets.
• build_asset_reconciliation_sensor now launches runs to keep assets in line with their defined FreshnessPolicies.
• The FreshnessPolicy object is now exported from the top level dagster package.
• For assets with a FreshnessPolicy defined, their current freshness status will be rendered in the asset graph and asset details pages.
• The AWS, GCS, and Azure compute log managers now take an additional config argument upload_interval which specifies in seconds, the interval in which partial logs will be uploaded to the respective cloud storage. This can be used to display compute logs for long-running compute steps.
• When running dagit or dagster-daemon locally, environment variables included in a .env file in the form KEY=value in the same folder as the command will be automatically included in the environment of any Dagster code that runs, allowing you to easily test environment variables during local development.
• observable_source_asset decorator creates a SourceAsset with an associated observation_fn that should return a LogicalVersion, a new class that wraps a string expressing a version of an asset’s data value.
• [dagit] The asset graph now shows branded compute_kind tags for dbt, Airbyte, Fivetran, Python and more.
• [dagit] The asset details page now features a redesigned event viewer, and separate tabs for Partitions, Events, and Plots. This UI was previously behind a feature flag and is now generally available.
• [dagit] The asset graph UI has been revamped and makes better use of color to communicate asset status, especially in the zoomed-out view.
• [dagit] The asset catalog now shows freshness policies in the “Latest Run” column when they are defined on your assets.
• [dagit] The UI for launching backfills in Dagit has been simplified. Rather than selecting detailed ranges, the new UI allows you to select a large “range of interest” and materialize only the partitions of certain statuses within that range.
• [dagit] The partitions page of asset jobs has been updated to show per-asset status rather than per-op status, so that it shares the same terminology and color coding as other asset health views.
• [dagster-k8s] Added an execute_k8s_job function that can be called within any op to run an image within a Kubernetes job. The implementation is similar to the build-in k8s_job_op , but allows additional customization - for example, you can incorporate the output of a previous op into the launched Kubernetes job by passing it into execute_k8s_job. See the dagster-k8s API docs for more information.
• [dagster-databricks] Environment variables used by dagster cloud are now automatically set when submitting databricks jobs if they exist, thank you @zyd14!
• [dagstermill] define_dagstermill_asset now supports RetryPolicy . Thanks @nickvazz!
• [dagster-airbyte] When loading assets from an Airbyte instance using load_assets_from_airbyte_instance, users can now optionally customize asset names using connector_to_asset_key_fn.
• [dagster-fivetran] When loading assets from a Fivetran instance using load_assets_from_fivetran_instance, users can now alter the IO manager using io_manager_key or connector_to_io_manager_key_fn, and customize asset names using connector_to_asset_key_fn.

Bugfixes

• Fixed a bug where terminating runs from a backfill would fail without notice.
• Executing a subset of ops within a job that specifies its config value directly on the job, it no longer attempts to use that config value as the default. The default is still presented in the editable interface in dagit.
• [dagit] The partition step run matrix now reflects historical step status instead of just the last run’s step status for a particular partition.

Documentation

• Updated Environment variables and secrets docs with info/instructions for using local .env files
• Added a new example test to the Testing docs. This test verifies if Dagster code loads correctly by loading a Dagster repository and its definitions.

New

• With the new asset_selection parameter on @sensor and SensorDefinition, you can now define a sensor that directly targets a selection of assets, instead of targeting a job.
• materialize and materialize_to_memory now accept a raise_on_error argument, which allows you to determine whether to raise an Error if the run hits an error or just return as failed.
• (experimental) Dagster now supports multi-dimensional asset partitions, through a new MultiPartitionsDefinition object. An optional schema migration enables support for this feature (run via dagster instance migrate). Users who are not using this feature do not need to run the migration.
• You can now launch a run that targets a range of asset partitions, by supplying the "dagster/asset_partition_range_start" and "dagster/asset_partition_range_end" tags.
• [dagit] Asset and op graphs in Dagit now show integration logos, making it easier to identify assets backed by notebooks, DBT, Airbyte, and more.
• [dagit] a -db-pool-recycle cli flag (and dbPoolRecycle helm option) have been added to control how long the pooled connection dagit uses persists before recycle. The default of 1 hour is now respected by postgres (mysql previously already had a hard coded 1hr setting). Thanks @adam-bloom!
• [dagster-airbyte] Introduced the ability to specify output IO managers when using load_assets_from_airbyte_instance and load_assets_from_airbyte_project.
• [dagster-dbt] the dbt_cloud_resource resource configuration account_id can now be sourced from the environment. Thanks @sowusu-ba!
• [dagster-duckdb] The DuckDB integration improvements: PySpark DataFrames are now fully supported, “schema” can be specified via IO Manager config, and API documentation has been improved to include more examples
• [dagster-fivetran] Introduced experimental load_assets_from_fivetran_instance helper which automatically pulls assets from a Fivetran instance.
• [dagster-k8s] Fixed an issue where setting the securityContext configuration of the Dagit pod in the Helm chart didn’t apply to one of its containers. Thanks @jblawatt!

Bugfixes

• Fixed a bug that caused the asset_selection parameter of RunRequest to not be respected when used inside a schedule.
• Fixed a bug with health checks during delayed Op retries with the k8s_executor and docker_executor.
• [dagit] The asset graph now live-updates when assets fail to materialize due to op failures.
• [dagit] The "Materialize" button now respects the backfill permission for multi-run materializations.
• [dagit] Materializations without metadata are padded correctly in the run logs.
• [dagster-aws] Fixed an issue where setting the value of task_definition field in the EcsRunLauncher to an environment variable stopped working.
• [dagster-dbt] Add exposures in load_assets_from_dbt_manifest. This fixed then error when load_assets_from_dbt_manifest failed to load from dbt manifest with exposures. Thanks @sowusu-ba!
• [dagster-duckdb] In some examples, the duckdb config was incorrectly specified. This has been fixed.

Breaking Changes

• The behavior of the experimental asset reconciliation sensor, which is accessible via build_asset_reconciliation_sensor has changed to be more focused on reconciliation. It now materializes assets that have never been materialized before and avoids materializing assets that are “Upstream changed”. The build_asset_reconciliation_sensor API no longer accepts wait_for_in_progress_runs and wait_for_all_upstream arguments.

Documentation

• Added documentation outlining environment variable declaration and usage in Dagster code, including how to pass secrets.
• Fixed a typo on Dagster Instance page. Thanks @domsj!

All Changes

https://github.com/dagster-io/dagster/compare/1.0.16...1.0.17

<details><summary>See All Contributors</summary>

• 7e9d8f6 - [jog] host_representation_tests, instance_tests, selector_tests (#10256) by @dpeng817
• ec07488 - [jog] resource_tests (#10257) by @dpeng817
• 8e44cf5 - [dagstermill] notebook backed assets (#10277) by @jamiedemaria
• ce52573 - notebook and assets example project (#10315) by @jamiedemaria
• 10187df - [dagster-airbyte][managed-elements] Explicit handling of reconciling secret values (#10195) by @benpankow
• faece2a - [duckdb] integration improvements (#10114) by @jamiedemaria
• b094d26 - Fix Helm package skipping (#10313) by @jmsanders
• 0500534 - Remove special casing for graphql tests (#10327) by @jmsanders
• 0c89026 - Add client ID to dagit (#10316) by @dpeng817
• 2cc3545 - [dagster-airflow] add airflow 2 support to make_dagster_job_from_airflow_dag + xcom mock option (#10337) by @Ramshackle-Jamathon
• a403cfc - 1.0.16 changelog (#10340) by @alangenfeld
• 3bbca03 - disentangle asset reconciliation sensor from MultiAssetSensorDefinition (#10258) by @sryza
• 1130ec0 - Mock tqdm in tests to avoid segfaults (#10343) by @jmsanders
• fa4a445 - Automation: versioned docs for 1.0.16 by @elementl-devtools
• 3b07b06 - update 1.0.16 changelog (#10346) by @yuhan
• 8b44cda - Better fix for tooltips around disabled buttons + gitlab icon (#10338) by @salazarm
• 08f4e2d - Back-compat fix for setting the task definition in the EcsRunLauncher to an env var (#10341) by @gibsondan
• 8289da9 - Mock each callsite of tqdm (#10368) by @gibsondan
• 15f1b09 - Build integration steps before test project steps (#10367) by @jmsanders
• 7fa824b - Packages with test changes only shouldn't run deps (#10309) by @jmsanders
• a8d457a - Remove test-project-core (#10310) by @jmsanders
• 369fb96 - chore: add project urls to dagster pypi (#10328) by @rexledesma
• 5690af5 - fix: add comma (#10373) by @rexledesma
• 22cf4db - add securityContext to dagit init-user-deployment initContainers (#10369) by @gibsondan
• adf924f - JB - Docs Typography Follow-ups (#10344) by @braunjj
• 49081bd - [dagit] Add more logos to tags (#10355) by @jamiedemaria
• 713d873 - [Multi-dimensional partitions 1/n] Add schema migration for asset event tags table (#10001) by @clairelin135
• 0244116 - Revert "Mock each callsite of tqdm" (#10379) by @jmsanders
• a34ffc5 - [Multi-dimensional partitions 2/n] Read/write from asset event tags table (#10056) by @clairelin135
• a8f8be0 - Prevent segfaults (#10372) by @jmsanders
• d709c11 - [apidocs] clarify usage of create_databricks_job_op (#10299) by @sryza
• cefab82 - Mark celery-k8s-test-suite as flaky (#10204) by @jmsanders
• 6d1429c - [Multi-dimensional partitions 3/n] Output multi-dimensional materializations (#10082) by @clairelin135
• 454a570 - fix duckdb configs in examples (#10378) by @jamiedemaria
• 79d1f12 - Revert "Mark celery-k8s-test-suite as flaky" (#10384) by @jmsanders
• b4ccac3 - Unpin grpc (#10386) by @gibsondan
• 22d94cd - some refactorings to asset reconciliation sensor (#10233) by @sryza
• 3065bad - fix mysql heading level + typo (#10336) by @domsj
• 620d6cf - [dagster-fivetran] Enable loading of Fivetran connection assets from instance (#10290) by @benpankow
• edcdcba - Resolver -> AssetGraph (#10401) by @sryza
• a35654b - Enable accessing base jobs off of repository definitions (#10405) by @sryza
• ef4a4bb - add pyrightconfig.json to .gitignore (#10403) by @sryza
• e9fcb96 - [dagster-dbt] add exposures in load_assets_from_dbt_manifest (#10395) by @sowusu-ba
• 339d14c - operate on keys instead of strings in AssetSelection and AssetGraph (#10402) by @sryza
• 9607192 - Updated "Loaded with Errors" link (#10421) by @gibsondan
• 7c205d5 - [dagster-io/ui] Add stroke color to TextArea (#10422) by @hellendag
• 4900d22 - Install project-fully-featured (#10419) by @jmsanders
• 353ed2a - More process cleanup in grpc tests (#10423) by @gibsondan
• b6aea90 - Add methods to AssetGraph for traversing asset partitions graph (#10413) by @sryza
• 29b24f1 - [dagit] Update partner compute_kind tags to latest designs (#10394) by @bengotow
• 4b207e5 - [dagit] Fix materialization log padding when no metadata is present (#10350) by @bengotow
• cd5af37 - [dagit] Respect backfill permission in asset backfill modal (#10387) by @bengotow
• 174a958 - [freshness-policy] add latency freshness policy (#10393) by @OwenKephart
• 50026a1 - Revert "Unpin grpc (#10386)" (#10428) by @gibsondan
• 44c9ddb - support RunRequest.asset_selection in schedules (#10400) by @sryza
• 4df96a7 - support raise_on_error in materialize and materialize_to_memory (#10397) by @sryza
• 42c47a8 - docs(cacheable-assets): mark unimplemented methods as abstract (#10424) by @rexledesma
• c694526 - [easy] fix incorrect header in docker guide (#10427) by @gibsondan
• 36a8383 - [dagit] Fix bug in asset live reloading after step failures, clean up AssetView (#10349) by @bengotow
• 3f75ae6 - allow sensors to target asset selections instead of jobs (#10417) by @sryza
• 1365944 - Implement abstract methods (#10434) by @jmsanders
• 46e41ad - [graphql] handle null int metadata val (#10436) by @alangenfeld
• dbbfffb - [jog] convert runtime_types_tests (#10317) by @dpeng817
• bbe1378 - [jog] convert core_test/selector_tests (#10318) by @dpeng817
• 649ea0e - [easy][dagster-airbyte] Treat SSH key as secret for managed stack (#10390) by @benpankow
• dba7b8b - [jog] Convert snap tests (#10319) by @dpeng817
• 9e28855 - Expose sqlalchemy pool recycle option for mysql/postgres (#10416) by @adam-bloom
• 1e7e9a7 - cached_method decorator (#10398) by @sryza
• 7f6a458 - Enable action logging in dagit (#10342) by @dpeng817
• 1e9af55 - Temporarily skip celery-k8s tests (#10438) by @jmsanders
• b6237b6 - Run builds on changes to .txt files (#10433) by @jmsanders
• ff77d5e - Hook for loading secrets in grpc/run/step entry points (#10089) by @gibsondan
• f03a491 - Read account_idfrom an environment variable indbt_cloud_resource (#10324) by @sowusu-ba
• 7bfe9f1 - Add container_name to EcsContainerContext (#10446) by @gibsondan
• 15f0f2d - Make asset reconciliation sensor more reconcile-y (#10435) by @sryza
• 198d234 - [dagster-airbyte] Generate src/dst Python classes for managed stacks (#10272) by @benpankow
• 28739e8 - [docs] - Guides for configuring environment variables (#10034) by @erinkcochran87
• e579a21 - Revert "specify schema in pandas to_sql" (#10450) by @jamiedemaria
• 79eaa72 - feat: bind command-K to search (#10449) by @rexledesma
• 91487d7 - [dagster-airbyte] Add ability to specify output IO managers (#10217) by @benpankow
• 7d0cc3e - [dagster-airbyte] When loading managed connections, ignore those which are not managed (#10388) by @benpankow
• efab894 - enable launching a run that targets a range of asset partitions (#10441) by @sryza
• 356f768 - chore: remove support for github dark mode (#10329) by @rexledesma
• f7da04d - [dagit] Clarify notebook buttons (#10451) by @jamiedemaria
• 2a3f8f2 - [dagstermill] allow custom io manager key specification (#10448) by @jamiedemaria
• 87c8389 - [dagster-airbyte][docs] Reorganize Airbyte guide (#10215) by @benpankow
• 00e4491 - Fix check_step_health with delayed op retries (#10458) by @johannkm
• fd6411f - changelog 1.0.17 (#10464) by @yuhan
• 4c51207 - 1.0.17 by @elementl-devtools

</details>

New

• [dagit] The new Overview and Workspace pages have been enabled for all users, after being gated with a feature flag for the last several releases. These changes include design updates, virtualized tables, and more performant querying.
  • The top navigation has been updated to improve space allocation, with main nav links moved to the left.
  • “Overview” is the new Dagit home page and “factory floor” view, were you can find the run timeline, which now offers time-based pagination. The Overview section also contains pages with all of your jobs, schedules, sensors, and backfills. You can filter objects by name, and collapse or expand repository sections.
  • “Workspace” has been redesigned to offer a better summary of your repositories, and to use the same performant table views, querying, and filtering as in the Overview pages.
• @asset and @multi_asset now accept a retry_policy argument. (Thanks Adam Bloom!)
• When loading an input that depends on multiple partitions of an upstream asset, the fs_io_manager will now return a dictionary that maps partition keys to the stored values for those partitions. (Thanks andrewgryan!).
• JobDefinition.execute_in_process now accepts a run_config argument even when the job is partitioned. If supplied, the run config will be used instead of any config provided by the job’s PartitionedConfig.
• The run_request_for_partition method on jobs now accepts a run_config argument. If supplied, the run config will be used instead of any config provided by the job’s PartitionedConfig.
• The new NotebookMetadataValue can be used to report the location of executed jupyter notebooks, and Dagit will be able to render the notebook.
• Resolving asset dependencies within a group now works with multi-assets, as long as all the assets within the multi-asset are in the same group. (Thanks @peay!)
• UPathIOManager, a filesystem-agnostic IOManager base class has been added - (Thanks @danielgafni!)
• A threadpool option has been added for the scheduler daemon. This can be enabled via your dagster.yaml file; check out the docs.
• The default LocalComputeLogManager will capture compute logs by process instead of by step. This means that for the in_process executor, where all steps are executed in the same process, the captured compute logs for all steps in a run will be captured in the same file.
• [dagster-airflow] make_dagster_job_from_airflow_dag now supports airflow 2, there is also a new mock_xcom parameter that will mock all calls to made by operators to xcom.
• [helm] volume and volumeMount sections have been added for the dagit and daemon sections of the helm chart.

Bugfixes

• For partitioned asset jobs whose config is a hardcoded dictionary (rather than a PartitionedConfig), previously run_request_for_partition would produce a run with no config. Now, the run has the hardcoded dictionary as its config.
• Previously, asset inputs would be resolved to upstream assets in the same group that had the same name, even if the asset input already had a key prefix. Now, asset inputs are only resolved to upstream assets in the same group if the input path only has a single component.
• Previously, asset inputs could get resolved to outputs of the same AssetsDefinition, through group-based asset dependency resolution, which would later error because of a circular dependency. This has been fixed.
• Previously, the “Partition Status” and “Backfill Status” fields on the Backfill page in dagit were always incomplete and showed missing partitions. This has been fixed to accurately show the status of the backfill runs.
• [dagit] When viewing the config dialog for a run with a very long config, scrolling was broken and the “copy” button was not visible. This has been fixed.
• Executors now compress step worker arguments to avoid CLI length limits with large DAGs.
• [dagster-msteams] Longer messages can now be used in Teams HeroCard - (Thanks @jayhale!)

Documentation

• API docs for InputContext have been improved - (Thanks @peay!)
• [dagster-snowflake] Improved documentation for the Snowflake IO manager

All Changes

https://github.com/dagster-io/dagster/compare/1.0.15...1.0.16

<details><summary>See All Contributors</summary>

• ee39fcd - rm all_types from config_context (#10203) by @alangenfeld
• bad2e44 - Also skip docs next (#10178) by @jmsanders
• bd6e269 - Make handleLaunchResult agnostic to the query that returned the data (#10179) by @salazarm
• b5212c7 - Make PythonPackages.get() more flexible to _/- (#10184) by @jmsanders
• 16bc4cf - [dagit] Invert stored state for expand/collapse in Overview pages (#10212) by @hellendag
• af58bf9 - [config] memoize ConfigType snap creation (#10210) by @alangenfeld
• 2eebc3c - [config] avoid double init on field cache objects (#10214) by @alangenfeld
• fcda5f3 - Override default io manager in more places (#10202) by @johannkm
• 59ae17e - Fix black (#10216) by @johannkm
• d4334f1 - type annotations on backfill-related code paths (#9402) by @sryza
• b56013e - Fix lint (#10218) by @johannkm
• 3176dcc - Add docs for customizing the serverless base image (#9571) by @petehunt
• a24298a - Skip test-project image builds (#10099) by @jmsanders
• f7c058a - [dagit] Updated asset details event view (#10143) by @bengotow
• 4ef9d27 - [dagit] Fix AssetView.test key warnings (#10226) by @hellendag
• 2bbf998 - Also trigger builds when tox.ini changes (#10209) by @jmsanders
• c1ff910 - 1.0.15 changelog (#10227) by @jamiedemaria
• 688a5d8 - Automation: versioned docs for 1.0.15 by @elementl-devtools
• 0478723 - add materialization property on dagster event (#10230) by @prha
• cf8ad4a - Parse HEAD commit in addition to BUILDKITE_MESSAGE (#10208) by @jmsanders
• 58ee17c - Allow default io_manager load_input to support partitions of differing frequencies (#10172) by @andrewgryan
• 67313e8 - Install the correct dagster-buildkite CLI (#10211) by @jmsanders
• cf84896 - Type annotations in dagster-graphql (#10005) by @smackesey
• 174dd99 - fix black, mypy (#10235) by @prha
• 3c89b70 - [freshness-policy] [1/n] FreshnessPolicy object (#10024) by @OwenKephart
• 281595b - Break up sensor_definition.py (#10181) by @sryza
• ffa01e1 - Highlight config entry when being hovered in yaml editor (#10239) by @salazarm
• a5f0fa9 - Resolve multi-asset deps when they have the same group (#10222) by @peay
• 77faa3c - update apidoc for postgres (#10241) by @prha
• 35b8f8b - fix schedules threading config (#10247) by @alangenfeld
• 8296cf7 - [dagit] Ship Overview/Workspace (#10245) by @hellendag
• 24af700 - Fix stray references to define_assets_job (#10199) by @bengotow
• 631c3ef - captured log manager (#9429) by @prha
• 65b47a0 - Allow for longer messages in Teams HeroCard (#10234) by @jayhale
• ee43f93 - [dagit] Clean up code post-Overview changes (#10249) by @hellendag
• a097c43 - [dagit] Move the refresh indicator on asset details pages to avoid flicker (#10253) by @bengotow
• ce7afb7 - [dagit] Fix issues with embedding fonts in downloaded DAG SVGs (#10252) by @bengotow
• 37f2c95 - [dagit] Fix daylight savings issue in humanCronString.test (#10263) by @hellendag
• 4ca8da0 - [dagit] Make 50-level colors opaque (#10186) by @hellendag
• 997fdbb - [dagit] Add query countdown/refresh to Timeline (#10250) by @hellendag
• da7555c - Support overriding run config for partitions with execute_in_process (#10246) by @sryza
• a2bb0cf - Add calendar icon (#10267) by @salazarm
• 5de98f5 - UPathIOManager - filesystem-agnostic IOManager base (#10193) by @danielgafni
• 40bf6c7 - rename logKey to fileKey / logFileKey for disambiguation with new API (#9956) by @prha
• dc667ef - fix some cases in group-based asset dep resolution (#10266) by @sryza
• ef678e6 - dagster-io/ui release notes (#10269) by @salazarm
• 966bfb7 - fix UnresolvedAssetJobDefinition.run_request_for_partition when confi… (#10238) by @sryza
• 36f7c38 - Improve doc for InputContext.{dagster_type,metadata} (#10242) by @peay
• 69bfd0c - Fetch asset materialization planned event from index shard in Sqlite (#10248) by @clairelin135
• cb62bc4 - fix #9193: add retry policy to @asset and @multi_asset (#10150) by @adam-bloom
• ca18259 - adds dagster-aws install to dockerfile in docker deployment guide (#10225) by @jamiedemaria
• e6444be - keep storage name for serialized event backcompat (#10280) by @prha
• 16b3cc3 - graphql for captured log subscription (#9957) by @prha
• 9ccfe7c - Add NotebookMetadataValue (#10278) by @jamiedemaria
• b88f309 - Add volumes and volumeMounts to dagit and daemon in OSS helm chart (#10285) by @gibsondan
• 1830880 - support new captured log API for process-based execution (#9958) by @prha
• a0a1340 - add new capture APIs in frontend queries (#9959) by @prha
• 621b57a - specify schema in pandas to_sql (#10289) by @jamiedemaria
• 7dd241f - [dagit] Fix AssetView flakiness (#10293) by @hellendag
• fc6ea04 - [dagit] Middle-truncate asset key path in virtualized list (#10275) by @hellendag
• b4c0930 - Snowflake IO Manager API docs (#10175) by @jamiedemaria
• 2b3280f - Compress execute_step args (#10244) by @johannkm
• 1dd602a - fix black formatting (#10298) by @prha
• 9a3b5bc - fix backfill table status (#10295) by @prha
• f7f1e13 - run_config argument on run_request_for_partition (#10279) by @sryza
• 0e5386b - [dagit] NotebookMetadataValue support (#10287) by @jamiedemaria
• f966490 - [dagit] Fix very tall configs in run config dialog (#10301) by @hellendag
• f466eb7 - Observable source asset decorator (#9899) by @smackesey
• 70c5f8d - convert NoOpComputeLogManager to support captured log API (#9960) by @prha
• 4df6e55 - convert S3 compute log manager to support new captured log API (#9961) by @prha
• c8dee2d - Test createdBefore run filter (#10270) by @benpankow
• 7c38a8c - fix build; isort; black (#10303) by @prha
• 082b407 - Fix example scaffold (#10003) by @smackesey
• 6459b0f - Add version to asset decorator (#10167) by @smackesey
• 3f99335 - Fix logic to run lints on builds (#10304) by @jmsanders
• 98fe688 - Don't repeat skip logs (#10307) by @jmsanders
• 0fc9f84 - [jog] Add a utility method to execute op inside of graph (#10255) by @dpeng817
• 6b333d5 - [jog] execution_tests,hook_tests (#10254) by @dpeng817
• 1283e63 - Revert "convert S3 compute log manager to support new captured log API (#9961) (#10311) by @prha
• 834abba - [docs] Automatically toggle tab components to display URL hash/anchor (#10231) by @benpankow
• 5b0d959 - [dagstermill] notebook backed assets (#10277) by @jamiedemaria
• 345f0b3 - notebook and assets example project (#10315) by @jamiedemaria
• 9157ebb - [dagster-airbyte][managed-elements] Explicit handling of reconciling secret values (#10195) by @benpankow
• 251b666 - [dagster-airflow] add airflow 2 support to make_dagster_job_from_airflow_dag + xcom mock option (#10337) by @Ramshackle-Jamathon
• 89948a3 - 1.0.16 changelog (#10340) by @alangenfeld
• afb20aa - 1.0.16 by @elementl-devtools

</details>

New

• [dagit] The run timeline now shows all future schedule ticks for the visible time window, not just the next ten ticks.
• [dagit] Asset graph views in Dagit refresh as materialization events arrive, making it easier to watch your assets update in real-time.
• [dagster-airbyte] Added support for basic auth login to the Airbyte resource.
• Configuring a Python Log Level will now also apply to system logs created by Dagster during a run.

Bugfixes

• Fixed a bug that broke asset partition mappings when using the key_prefix with methods like load_assets_from_modules.
• [dagster-dbt] When running dbt Cloud jobs with the dbt_cloud_run_op, the op would emit a failure if the targeted job did not create a run_results.json artifact, even if this was the expected behavior. This has been fixed.
• Improved performance by adding database indexes which should speed up the run view as well as a range of asset-based queries. These migrations can be applied by running dagster instance migrate.
• An issue that would cause schedule/sensor latency in the daemon during workspace refreshes has been resolved.
• [dagit] Shift-clicking Materialize for partitioned assets now shows the asset launchpad, allowing you to launch execution of a partition with config.

Community Contributions

• Fixed a bug where asset keys with - were not being properly sanitized in some situations. Thanks @peay!
• [dagster-airbyte] A list of connection directories can now be specified in load_assets_from_airbyte_project. Thanks @adam-bloom!
• [dagster-gcp] Dagster will now retry connecting to GCS if it gets a ServiceUnavailable error. Thanks @cavila-evoliq!
• [dagster-postgres] Use of SQLAlchemy engine instead of psycopg2 when subscribing to PostgreSQL events. Thanks @peay!

Experimental

• [dagster-dbt] Added a display_raw_sql flag to the dbt asset loading functions. If set to False, this will remove the raw sql blobs from the asset descriptions. For large dbt projects, this can significantly reduce the size of the generated workspace snapshots.
• [dagit] A “New asset detail pages” feature flag available in Dagit’s settings allows you to preview some upcoming changes to the way historical materializations and partitions are viewed.

All Changes

https://github.com/dagster-io/dagster/compare/1.0.14...1.0.15

<details><summary>See All Contributors</summary>

• 899e8d5 - [dagit] Add actions to run timeline empty state (#10092) by @hellendag
• 2e8fae8 - fix future ticks cursor when no ticks are available (#10095) by @prha
• 2d8ada6 - Revert "Make the default_job_io_manager overridable via env (#9950)" by @johannkm
• 80f1022 - Add helm vals for scheduler threading (#10094) by @dpeng817
• e0b61df - [dagit] Shift-clicking Materialize should show asset launchpad even if assets are partitioned (#10047) by @bengotow
• bcf581a - Remove references to services in docker-compose down command test fixture (#10042) by @dpeng817
• 426fc52 - 1.0.14 changelog (#10106) by @dpeng817
• 4a22317 - 1.0.14 changelog addendum (#10107) by @gibsondan
• 2af0b8a - [dagit] Show all future ticks in the timeline window (#10097) by @hellendag
• e8d8525 - [dagster-postgres] Use SQLAlchemy engine in pynotify instead of psycopg2 directly (#10090) by @peay
• c76d049 - Jb/Docs UI Cleanup (#10050) by @braunjj
• a0489f9 - [mypy] fixes for uvicorn types (#10108) by @alangenfeld
• ec3c509 - Revert "Revert "Make the default_job_io_manager overridable via env (#9950)"" by @johannkm
• 07c02c5 - [dagit] Use gray tags in repo headers (#10078) by @hellendag
• 927054f - Run make mdx-format (#10113) by @jmsanders
• 565105a - Upgrade kind (#9992) by @jmsanders
• a06a8db - rebuild event log indices to include id (#10105) by @prha
• 05a8aa0 - [dagit] Fix repo bucket sorting in virtualized tables (#10101) by @hellendag
• bd690a6 - Automation: versioned docs for 1.0.14 by @elementl-devtools
• 288f692 - docs(slack-alert-policies): add slack bot name to setup instructions (#10116) by @rexledesma
• ff07bf9 - [dagit] Some typographic and spacing tweaks on virtualized tables (#10118) by @hellendag
• 328f4a1 - Don't gate on elementl emails (#10119) by @jmsanders
• 71e0129 - Skip docs changes (#10030) by @jmsanders
• bc284fc - [docs] - improve partitioned asset examples on partitions concepts page (#10084) by @sryza
• ab42f31 - Walk the Python dependency tree (#10096) by @jmsanders
• 54dbe2d - [dagit] Add some right padding to clearable text input (#10129) by @hellendag
• 76c9674 - [dagster-airbyte] Docs for how to use basic auth in current release (#10131) by @benpankow
• 876dfab - Cache docstring check (#10127) by @schrockn
• a8e67fc - [dagster-airbyte] Docs fix using with_resources with cacheable asset (#10132) by @benpankow
• 6614290 - Retry on Service Unavailable (#10110) by @cavila-evoliq
• a90ea64 - Make AssetKey.__eq__ more efficient (#10139) by @schrockn
• 6af2ba9 - [docs formatter] Drop mypy ignore lines in docs (#10138) by @benpankow
• 0772a12 - [refactor] take logic for base asset jobs out of AssetGroup (#10071) by @sryza
• 9df49da - [dagit] Subscribe to in progress asset runs, refresh on relevant events (#10028) by @bengotow
• 38f5291 - docs: add h1 header to Dagster readme (#10156) by @rexledesma
• 3a8f4e1 - docs: update @dagsterio -> @dagster (#10158) by @rexledesma
• 85cc0ca - Use instance.python_log_level when determining default system logger level (#10073) by @gibsondan
• a20c2b7 - in hacker news demo, fix column filtering with partitions and snowfla… (#10141) by @sryza
• 30c0751 - Allow overriding svg with color (#10163) by @salazarm
• fb2b580 - [dagster-airbyte] Add top-level support for basic auth username & password (#10130) by @benpankow
• c9dca01 - improve api doc examples for cloud object store IO managers (#10153) by @sryza
• eeb6c8f - switch order of assets and jobs in partitions doc (#10142) by @sryza
• 99f3579 - Fix AssetsDefinition.with_prefix_or_group to update partition mappings (#10164) by @peay
• 05a08b8 - Fix assets with dashes in their path and io_manager_def (#10087) by @peay
• 9694525 - Refactor db io manager to core (#10128) by @jamiedemaria
• 0f2e30c - docs design: fix icons. make icon changes backcompat (#10174) by @yuhan
• bf3e941 - Support multi-repo builds (#10135) by @jmsanders
• cb8c7d6 - [docs] Out -> AssetOut docs tweak (#10146) by @benpankow
• ab0e946 - Conditionally skip dagit builds (#10165) by @jmsanders
• 0556f09 - [dagster-managed-elements] Dagster-managed-elements CLI, APIs (#10011) by @benpankow
• ab92934 - EventLogRecord.partition_key and EventLogRecord.asset_key (#10180) by @sryza
• 01edf1d - [dagster-airbyte] Airbyte managed elements impl (#10013) by @benpankow
• a4b045d - [dagster-airbyte] allow filtering connections by directory names (#10151) by @adam-bloom
• 0781a28 - make_airflow_dag airflow 2 compatibility (#10115) by @Ramshackle-Jamathon
• 60e8f6f - [dagit] Bump TypeScript (#10133) by @hellendag
• 8bfb2e3 - [@dagster-io/ui] Release v1.0.6 (#10188) by @hellendag
• c08c0d9 - [dagster-dbt] allow missing run_results.json (#10187) by @OwenKephart
• 5e74834 - docs(dagster-cloud-agent): update IAM role link (#10189) by @rexledesma
• 32c316d - [dagit] Expand run config dialog (#10173) by @hellendag
• 767946a - Replace custom in-memory logic with in-memory sqlite connection (#10154) by @prha
• e4a2357 - add cpu and memory to DagsterEcsTaskDefinitionConfig (#10198) by @gibsondan
• 4938f9b - Allow silencing failures with default io manager override (#10067) by @johannkm
• 5e32b41 - [dagster-airbyte] Managed elements typo (#10196) by @benpankow
• 385c28a - [dagit] Consolidate expand/collapse state in Overview tables (#10194) by @hellendag
• 4bd995f - reduce locking in workspace reload (#10192) by @alangenfeld
• 6d0f696 - [dagster-dbt] add flag to make asset snapshots smaller (#10213) by @OwenKephart
• 63d00fd - Extend default timeout (#10205) by @jmsanders
• 886da61 - [dagster-airbyte] Fix recreating source/destination logic (#10197) by @benpankow
• a5db43e - Revert 4938f9beb9 and ec3c5099c2 by @johannkm
• 6bdd409 - Make handleLaunchResult agnostic to the query that returned the data (#10179) by @salazarm
• 4caf010 - rm all_types from config_context (#10203) by @alangenfeld
• 8e2aec7 - [config] memoize ConfigType snap creation (#10210) by @alangenfeld
• 3f8002d - [config] avoid double init on field cache objects (#10214) by @alangenfeld
• b8b973c - Fix black (#10216) by @johannkm
• 5f6a5ea - [dagit] Invert stored state for expand/collapse in Overview pages (#10212) by @hellendag
• cb3bb61 - [dagit] Updated asset details event view (#10143) by @bengotow
• 124098c - 1.0.15 changelog (#10227) by @jamiedemaria
• f816bb9 - 1.0.15 by @elementl-devtools

</details>

New

• Tags can now be provided to an asset reconciliation sensor and will be applied to all RunRequests returned by the sensor.
• If you don’t explicitly specify a DagsterType on a graph input, but all the inner inputs that the graph input maps to have the same DagsterType, the graph input’s DagsterType will be set to the the DagsterType of the inner inputs.
• [dagster-airbyte] load_assets_from_airbyte_project now caches the project data generated at repo load time so it does not have to be regenerated in subprocesses.
• [dagster-airbyte] Output table schema metadata is now generated at asset definition time when using load_assets_from_airbyte_instance or load_assets_from_airbyte_project.
• [dagit] The run timeline now groups all jobs by repository. You can collapse or expand each repository in this view by clicking the repository name. This state will be preserved locally. You can also hold Shift while clicking the repository name, and all repository groups will be collapsed or expanded accordingly.
• [dagit] In the launchpad view, a “Remove all” button is now available once you have accrued three or more tabs for that job, to make it easier to clear stale configuration tabs from view.
• [dagit] When scrolling through the asset catalog, the toolbar is now sticky. This makes it simpler to select multiple assets and materialize them without requiring you to scroll back to the top of the page.
• [dagit] A “Materialize” option has been added to the action menu on individual rows in the asset catalog view.
• [dagster-aws] The EcsRunLauncher now allows you to pass in a dictionary in the task_definition config field that specifies configuration for the task definition of the launched run, including role ARNs and a list of sidecar containers to include. Previously, the task definition could only be configured by passing in a task definition ARN or by basing the the task definition off of the task definition of the ECS task launching the run. See the docs for the full set of available config.

Bugfixes

• Previously, yielding a SkipReason within a multi-asset sensor (experimental) would raise an error. This has been fixed.
• [dagit] Previously, if you had a partitioned asset job and supplied a hardcoded dictionary of config to define_asset_job, you would run into a CheckError when launching the job from Dagit. This has been fixed.
• [dagit] When viewing the Runs section of Dagit, the counts displayed in the tabs (e.g. “In progress”, “Queued”, etc.) were not updating on a poll interval. This has been fixed.

All Changes

https://github.com/dagster-io/dagster/compare/1.0.13...1.0.14

<details><summary>See All Contributors</summary>

• 350b61d - Fix materialization count by partition (#9979) by @clairelin135
• e297cc0 - [dagit] Turn on timeline run bucketing for everyone (#9993) by @hellendag
• c132917 - [dagit] "Remove all" tab button to clear Launchpad tabs (#9981) by @hellendag
• 80e10e5 - [dagit] Add polling to Overview pages (#9996) by @hellendag
• dc4fe9b - Return each Python PackageSpec's distribution (#9889) by @jmsanders
• 57b8c64 - [dagster-airbyte] Support union types while generating normalization tables (#9937) by @benpankow
• b745984 - [dagster-airbyte] Add option to specify custom API request params to Airbyte resource (#10000) by @benpankow
• 38dfd67 - View Notebook button opens notebook as link if it's a url (#9894) by @jamiedemaria
• e3a57ec - IO manager concept doc improvements (#9987) by @sryza
• c35332b - [dagster-airbyte] Add optional connection name filter when generating assets (#9975) by @benpankow
• 0f09fa1 - [docs] Update deployment settings reference with SSO default role (#9984) by @benpankow
• 865026b - [docs] - dbt-focused intro tutorial (#9853) by @jamiedemaria
• bbc2139 - Configure tokens -> Tokens (#9999) by @salazarm
• 2a2de19 - add a link to discuss.dagster.io on README (#10002) by @yuhan
• 1443f57 - 1.0.13 changelog (#10010) by @yuhan
• b289ba6 - [fix] fix docs bk error (#10014) by @benpankow
• d0402ba - Only add buildkite steps for affected changes (#9897) by @jmsanders
• c2538db - [dagit] Add daemon alerts to Overview schedules/sensors pages (#9972) by @hellendag
• 661f3e7 - [dagit] Make asset catalog toolbar sticky (#9974) by @hellendag
• 4064b6d - [dagit] Update workspace flag label (#9994) by @hellendag
• 7b8ada8 - fix code example in create_databricks_job_op (#10012) by @sryza
• f30c4c3 - Fixup setup paths (#10019) by @jmsanders
• 8bbbcca - Skip Python checks if no Python files change (#10016) by @jmsanders
• 3c493a5 - Skip package steps instead of excluding them (#10018) by @jmsanders
• ef0ffae - Don't raise an exception when changing the EcsRunLauncher's container name to a new name (#10026) by @gibsondan
• f25658a - [event log tests] add origin to run (#10017) by @alangenfeld
• ca177b6 - move the big honkin asset graph to latest asset APIs (#10025) by @sryza
• 0a80c4b - Conditionally skip helm steps (#10031) by @jmsanders
• 1373823 - [dagit] Fix run tab counts not updating with poll interval (#9929) by @hellendag
• 8c63dc6 - Automation: versioned docs for 1.0.13 by @elementl-devtools
• ca87aa3 - Also run helm on merges to main (#10038) by @jmsanders
• 2347024 - [dagit] Split asset graphs onto a new “Plots” tab (#9735) by @bengotow
• 96ce2b6 - [dagit] Fetch isAssetJob in WorkspaceContext, rm Launchpad tab flicker (#9770) by @bengotow
• 1f68be0 - Workaround for upstream snowflake-sqlalchemy issue (#10049) by @gibsondan
• 1d93edc - [dagster-airbyte] Generate schema metadata at load time when loading from project or instance (#9939) by @benpankow
• e3e0e61 - [cacheable-assets] Enable use of with_resources, with_prefix_or_group on cacheable assets (#9978) by @benpankow
• dd73d98 - [docs] Fix for asset preview image next to code block (#9881) by @benpankow
• 9bdd9ab - fix credential helper links and volume path (#10046) by @gibsondan
• 3343c9d - Refactor build python package skipping (#10037) by @jmsanders
• 389134a - replace tag joins with subquery over tag intersection queries (#10036) by @prha
• 23d5bb1 - Also correctly skip dagster_buildkite (#10053) by @jmsanders
• 0b0ac95 - pass tags to asset reconciliation sensor (#10032) by @jamiedemaria
• 04ba619 - infer graph input types from inner input types (#9658) by @sryza
• c648f67 - Add hook for launch pad root execution button (#9976) by @salazarm
• 064a376 - Make the default_job_io_manager overridable via env (#9950) by @johannkm
• bc59e18 - [dagit] Jest cleanup for missing canvas context (#10035) by @hellendag
• e4b3498 - [easy] fix incorrect cli link (#10058) by @gibsondan
• f84dc66 - Skip integration tests (#10051) by @jmsanders
• 55b6903 - consistent capitalization in LaunchAssetChoosePartitionsDialog (#10033) by @sryza
• e4c8a86 - Fix package skipping for dagster-test (#10063) by @jmsanders
• 0694f75 - fix graph type inference with fan-in (#10064) by @sryza
• 7ed03bd - Fix runs yielded error for multi asset sensor (#10059) by @clairelin135
• f1e1c94 - [dagit] Add "Materialize" action item to asset action menu (#10065) by @hellendag
• e2235c3 - [dagit] Update Backfill table styles (#10070) by @hellendag
• a94dc9a - Raise if we can't infer distribution (#10066) by @jmsanders
• 40df711 - fix Dagit execution of partitioned asset jobs with hardcoded config (#10057) by @sryza
• 5727e7d - Skip grapqhl and mysql checks (#10054) by @jmsanders
• 8c468c2 - [windows tests] fix start time == end time issue (#10021) by @alangenfeld
• 5f9879b - [dagster-io/ui] Calculate available width for MiddleTruncate (#10052) by @hellendag
• 149fcd2 - [dagit] Add Shift+click to expand/collapse all repos in run timeline and tables (#10076) by @hellendag
• a016e84 - [dagit] Fix repo row expanded icon (#10077) by @hellendag
• f6f58a0 - Allow passing in task definition config to the EcsRunLauncher instead of just a task definition ARN (#10044) by @gibsondan
• b695b9b - bump test_execute_schedule_on_celery_k8s to 3m (#10088) by @alangenfeld
• e97c876 - Expose permissions loading state (#10093) by @salazarm
• 65a3bbe - Revert "Make the default_job_io_manager overridable via env (#9950)" by @johannkm
• c43b595 - Add helm vals for scheduler threading (#10094) by @dpeng817
• 5ca8694 - [dagit] Shift-clicking Materialize should show asset launchpad even if assets are partitioned (#10047) by @bengotow
• bdf7468 - 1.0.14 changelog (#10106) by @dpeng817
• e7b7848 - 1.0.14 by @elementl-devtools

</details>

New

• AssetMaterialization now has a metadata property, which allows accessing the materialization’s metadata as a dictionary.
• DagsterInstance now has a get_latest_materialization_event method, which allows fetching the most recent materialization event for a particular asset key.
• RepositoryDefinition.load_asset_value and AssetValueLoader.load_asset_value now work with IO managers whose load_input implementation accesses the op_def and name attributes on the InputContext.
• RepositoryDefinition.load_asset_value and AssetValueLoader.load_asset_value now respect the DAGSTER_HOME environment variable.
• InMemoryIOManager, the IOManager that backs mem_io_manager, has been added to the public API.
• The multi_asset_sensor (experimental) now supports marking individual partitioned materializations as “consumed”. Unconsumed materializations will appear in future calls to partitioned context methods.
• The build_multi_asset_sensor_context testing method (experimental) now contains a flag to set the cursor to the newest events in the Dagster instance.
• TableSchema now has a static constructor that enables building it from a dictionary of column names to column types.
• Added a new CLI command dagster run migrate-repository which lets you migrate the run history for a given job from one repository to another. This is useful to preserve run history for a job when you have renamed a repository, for example.
• [dagit] The run timeline view now shows jobs grouped by repository, with each repository section collapsible. This feature was previously gated by a feature flag, and is now turned on for everyone.
• [dagster-airbyte] Added option to specify custom request params to the Airbyte resource, which can be used for auth purposes.
• [dagster-airbyte] When loading Airbyte assets from an instance or from YAML, a filter function can be specified to ignore certain connections.
• [dagster-airflow] DagsterCloudOperator and DagsterOperator now support Airflow 2. Previously, installing the library on Airflow 2 would break due to an import error.
• [dagster-duckdb] A new integration with DuckDB allows you to store op outputs and assets in an in-process database.

Bugfixes

• Previously, if retries were exceeded when running with execute_in_process, no error would be raised. Now, a DagsterMaxRetriesExceededError will be launched off.
• [dagster-airbyte] Fixed generating assets for Airbyte normalization tables corresponding with nested union types.
• [dagster-dbt] When running assets with load_assets_from_...(..., use_build=True), AssetObservation events would be emitted for each test. These events would have metadata fields which shared names with the fields added to the AssetMaterialization events, causing confusing historical graphs for fields such as Compilation Time. This has been fixed.
• [dagster-dbt] The name for the underlying op for load_assets_from_... was generated in a way which was non-deterministic for dbt projects which pulled in external packages, leading to errors when executing across multiple processes. This has been fixed.

Dependency changes

• [dagster-dbt] The package no longer depends on pandas and dagster-pandas.

Community Contributions

• [dagster-airbyte] Added possibility to change request timeout value when calling Airbyte. Thanks @FransDel!
• [dagster-airflow] Fixed an import error in dagster_airflow.hooks. Thanks @bollwyvl!
• [dagster-gcp] Unpin Google dependencies. dagster-gcp now supports google-api-python-client 2.x. Thanks @amarrella!
• [dagstermill] Fixed an issue where DagsterTranslator was missing an argument required by newer versions of papermill. Thanks @tizz98!

Documentation

• Added an example, underneath examples/assets_smoke_test, that shows how to write a smoke test that feeds empty data to all the transformations in a data pipeline.
• Added documentation for build_asset_reconciliation_sensor.
• Added documentation for monitoring partitioned materializations using the multi_asset_sensor and kicking off subsequent partitioned runs.
• [dagster-cloud] Added documentation for running the Dagster Cloud Docker agent with Docker credential helpers.
• [dagster-dbt] The class methods of the dbt_cli_resource are now visible in the API docs for the dagster-dbt library.
• [dagster-dbt] Added a step-by-step tutorial for using dbt models with Dagster software-defined assets

All Changes

https://github.com/dagster-io/dagster/compare/1.0.12...1.0.13

<details><summary>See All Contributors</summary>

• ab4ae1d - [dagit] Run timeline: reduce scheduled tick width, reduce chunk min width (#9913) by @hellendag
• 421997f - [dagster-airflow] Add DagsterOperator and associated airflow abstractions (#9780) by @Ramshackle-Jamathon
• 43538db - partition_key and upstream_output.asset_key in load_asset_value (#9914) by @sryza
• 76df23d - Pin dask-kubernetes until we switch KubeCluster (#9918) by @jmsanders
• 5c1f889 - Fix executor test (#9919) by @gibsondan
• f9a9bd7 - Automation: versioned docs for 1.0.12 by @elementl-devtools
• 72b3b9c - change log for 1.0.12 (#9923) by @sryza
• 1c26868 - respect DAGSTER_HOME in AssetValueLoader (#9922) by @sryza
• 448a776 - [dagit] Create Overview root (#9907) by @hellendag
• d619ab6 - fix 1.0.12 changelog (#9925) by @sryza
• 3dd81f0 - [dagit] Bucketed virtualized tables for Jobs, Schedules, Sensors (#9909) by @hellendag
• c88933e - [Feature] Add possibility to change request timeout value when calling airbyte (#9906) by @FransDel
• ebd90c1 - asset selection diff apidoc (#9917) by @sryza
• b5ba488 - add InMemoryIOManager to public API (#9882) by @sryza
• 185769c - add SDAs to fs_io_manager docstring examples (#9872) by @sryza
• 350e03f - Explicitly error when retries are exceeded if raise_on_error is set (#9934) by @dpeng817
• bb3bf95 - add cli command to migrate job runs from one repo to another (#9376) by @prha
• 82a7cb6 - add empty init.pyto makedagster_airflow.hooks, .links importable (#9932) by @bollwyvl
• f71cfc1 - Add metadata hooks for making grpc client calls (#9825) by @gibsondan
• c68397d - TableSchema.from_name_type_dict (#9926) by @sryza
• fb847aa - linting fixups for dagster-airflow (#9948) by @Ramshackle-Jamathon
• 0afa38c - add name and op_def to asset value load context (#9942) by @sryza
• eaaf0e0 - [dagster-io/ui] Middle truncation (#9933) by @hellendag
• d17e1b2 - Unpin google dependencies (#9319) by @amarrella
• ccc82ff - [dagster-dbt] Fix dbt asset op name (#9963) by @OwenKephart
• a05ac4c - Add asset_selection arg to execute_job (#9876) by @dpeng817
• 3b200e5 - Convert definitions tests to use graph/job/op APIs (#9736) by @dpeng817
• 08932da - make it easier to fetch asset materialization metadata (#9951) by @sryza
• 0035ebc - Data pipeline smoke test example (#9945) by @sryza
• 4cf2f60 - remove dagster-dbt deps on pandas and dagster-pandas (#9953) by @sryza
• c65b6be - Add skipped events to multi asset sensor context (#9903) by @clairelin135
• 384e2be - [dagstermill] update DagsterTranslator to support newer versions of papermill (#9901) by @tizz98
• bf04ebd - [dagster-io/ui] Test and comments for middle truncation search (#9955) by @hellendag
• fca13bc - [dagster-dbt] fix dbt test metadata (#9965) by @OwenKephart
• f6c2341 - Show noteable logo on noteable backed assets/ops (#9916) by @jamiedemaria
• 1d3e016 - Flag to set cursor to latest materializations on build_multi_asset_sensor_context (#9814) by @clairelin135
• 3505b55 - [docs] asset reconciliation sensor concept page (#9912) by @jamiedemaria
• f714be6 - support asset keys and asset selection for multi asset sensors (#9954) by @jamiedemaria
• f91dc3b - [docs] - Partitioned multi asset sensor examples (#9722) by @clairelin135
• 4a2fb05 - Update multi asset sensor docstring (#9971) by @jamiedemaria
• 2a8855c - Clarify with_resources error and update docs (#9784) by @clairelin135
• 0edd7c3 - Add threading to scheduler daemon (#9885) by @dpeng817
• cf8d611 - Ignore warnings sent from the dagster module (#9577) by @dpeng817
• cab3946 - [dagster-airflow] remove default param for parent init (#9966) by @Ramshackle-Jamathon
• 345ceb1 - Fix incorrect secrets_tags docs (#9980) by @gibsondan
• e42c37d - Document cloud Docker credential helpers (#9982) by @jmsanders
• ee1a1de - [dagster-dbt] boldly ignore type hints (#9989) by @OwenKephart
• c449d95 - Increase timeout in asset sensor tests (#9990) by @jamiedemaria
• 132252b - [dagster-dbt] make dbt cli resource methods public (#9973) by @OwenKephart
• c480f17 - duckdb integration library (#9869) by @jamiedemaria
• ddea744 - Fix materialization count by partition (#9979) by @clairelin135
• ab26eaa - [dagit] Turn on timeline run bucketing for everyone (#9993) by @hellendag
• ab8c21d - [dagster-airbyte] Support union types while generating normalization tables (#9937) by @benpankow
• 25d0917 - [dagster-airbyte] Add option to specify custom API request params to Airbyte resource (#10000) by @benpankow
• d8617f6 - [dagster-airbyte] Add optional connection name filter when generating assets (#9975) by @benpankow
• ad4943a - [docs] Update deployment settings reference with SSO default role (#9984) by @benpankow
• 0abdbb8 - [docs] - dbt-focused intro tutorial (#9853) by @jamiedemaria
• 4244cea - 1.0.13 changelog (#10010) by @yuhan
• 6aedfe1 - Don't raise an exception when changing the EcsRunLauncher's container name to a new name (#10026) by @gibsondan
• 141abe9 - 1.0.13 by @elementl-devtools

</details>

New

• The multi_asset_sensor (experimental) now accepts an AssetSelection of assets to monitor. There are also minor API updates for the multi-asset sensor context.
• AssetValueLoader, the type returned by RepositoryDefinition.get_asset_value_loader is now part of Dagster’s public API.
• RepositoryDefinition.load_asset_value and AssetValueLoader.load_asset_value now support a partition_key argument.
• RepositoryDefinition.load_asset_value and AssetValueLoader.load_asset_value now work with I/O managers that invoke context.upstream_output.asset_key.
• When running Dagster locally, the default amount of time that the system waits when importing user code has been increased from 60 seconds to 180 seconds, to avoid false positives when importing code with heavy dependencies or large numbers of assets. This timeout can be configured in dagster.yaml as follows:

code_servers:
  local_startup_timeout: 120

• [dagit] The “Status” section has been renamed to “Deployment”, to better reflect that this section of the app shows deployment-wide information.
• [dagit] When viewing the compute logs for a run and choosing a step to filter on, there is now a search input to make it easier to find the step you’re looking for.
• [dagster-aws] The EcsRunLauncher can now launch runs in ECS clusters using both Fargate and EC2 capacity providers. See the Deploying to ECS docs for more information.
• [dagster-airbyte] Added the load_assets_from_airbyte_instance function which automatically generates asset definitions from an Airbyte instance. For more details, see the new Airbyte integration guide.
• [dagster-airflow] Added the DagsterCloudOperator and DagsterOperator , which are airflow operators that enable orchestrating dagster jobs, running on either cloud or OSS dagit instances, from Apache Airflow.

Bugfixes

• Fixed a bug where if resource initialization failed for a dynamic op, causing other dynamic steps to be skipped, those skipped dynamic steps would be ignored when retrying from failure.
• Previously, some invocations within the Dagster framework would result in warnings about deprecated metadata APIs. Now, users should only see warnings if their code uses deprecated metadata APIs.
• How the daemon process manages its understanding of user code artifacts has been reworked to improve memory consumption.
• [dagit] The partition selection UI in the Asset Materialization modal now allows for mouse selection and matches the UI used for partitioned op jobs.
• [dagit] Sidebars in Dagit shrink more gracefully on small screens where headers and labels need to be truncated.
• [dagit] Improved performance for loading runs with >10,000 logs
• [dagster-airbyte] Previously, the port configuration in the airbyte_resource was marked as not required, but if it was not supplied, an error would occur. It is now marked as required.
• [dagster-dbt] A change made to the manifest.json schema in dbt 1.3 would result in an error when using load_assets_from_dbt_project or load_assets_from_manifest_json. This has been fixed.
• [dagster-postgres] connections that fail due to sqlalchemy.exc.TimeoutError now retry

Breaking Changes

• [dagster-aws] The redshift_resource no longer accepts a schema configuration parameter. Previously, this parameter would error whenever used, because Redshift connections do not support this parameter.

Community Contributions

• We now reference the correct method in the "loading asset values outside of Dagster runs" example (thank you Peter A. I. Forsyth!)
• We now reference the correct test directory in the “Create a New Project” documentation (thank you Peter A. I. Forsyth!)
• [dagster-pyspark] dagster-pyspark now contains a LazyPysparkResource that only initializes a spark session once it’s accessed (thank you @zyd14!)

Experimental

• The new build_asset_reconciliation_sensor function accepts a set of software-defined assets and returns a sensor that automatically materializes those assets after their parents are materialized.
• [dagit] A new "groups-only" asset graph feature flag allows you to zoom way out on the global asset graph, collapsing asset groups into smaller nodes you can double-click to expand.

New

• RepositoryDefinition now exposes a load_asset_value method, which accepts an asset key and invokes the asset’s I/O manager’s load_input function to load the asset as a Python object. This can be used in notebooks to do exploratory data analysis on assets.
• Methods to fetch a list of partition keys from an input/output PartitionKeyRange now exist on the op execution context and input/output context.
• [dagit] On the Instance Overview page, batched runs in the run timeline view will now proportionally reflect the status of the runs in the batch instead of reducing all run statuses to a single color.
• [dagster-dbt] [dagster-snowflake] You can now use the Snowflake IO manager with dbt assets, which allows them to be loaded from Snowflake into Pandas DataFrames in downstream steps.
• The dagster package’s pin of the alembic package is now much less restrictive.

Bugfixes

• The sensor daemon when using threads will no longer evaluate the next tick for a sensor if the previous one is still in flight. This resolves a memory leak in the daemon process.
• The scheduler will no longer remove tracked state for automatically running schedules when they are absent due to a workspace load error.
• The way user code severs manage repository definitions has been changed to more efficiently serve requests.
• The @multi_asset decorator now respects its config_schema parameter.
• [dagit] Config supplied to define_asset_job is now prefilled in the modal that pops up when you click the Materialize button on an asset job page, so you can quickly adjust the defaults.
• [dagster-dbt] Previously, DagsterDbtCliErrors produced from the dagster-dbt library would contain large serialized objects representing the raw unparsed logs from the relevant cli command. Now, these messages will contain only the parsed version of these messages.
• Fixed an issue where the deploy_ecs example didn’t work when built and deployed on an M1 Mac.

Community Contributions

• [dagster-fivetran] The resync_parameters configuration on the fivetran_resync_op is now optional, enabling triggering historical re*syncs for connectors. Thanks @dwallace0723!

Documentation

• Improved API documentation for the Snowflake resource.

All Changes

https://github.com/dagster-io/dagster/compare/1.0.10...1.0.11 <details><summary>See All Contributors</summary>

• d7d5879 - [dagster-dbt] fix star issue for asset input names (#9763) by @OwenKephart
• d6f08a1 - api docs for snowflake resource (#9717) by @jamiedemaria
• 233a298 - [dagit] Hide hidden asset group job from Instance Overview (#9742) by @hellendag
• fdd2ff4 - [dagit] Workspace Overview page (#9744) by @hellendag
• de43e65 - [dagit] Virtualized table for assets (#9759) by @hellendag
• 4ef2c08 - 1.0.10 Changelog (#9771) by @OwenKephart
• 3b4d8f4 - add api doc for run_request_for_partition (#9645) by @sryza
• 380aadb - [fixit] Add library versions to docs (#9768) by @smackesey
• 19f8d74 - Automation: versioned docs for 1.0.10 by @elementl-devtools
• 3d88c86 - Remove unused workspace args from schedule wipe command (#9753) by @gibsondan
• 882f506 - [dagster-fivetran] Optional Fivetran Historical Resync Parameters (#9774) by @dwallace0723
• ffae039 - Helm template tests support raw dict (#9773) by @johannkm
• c8ddaa8 - [dagit] Prefill default job config when materializing assets (#9769) by @bengotow
• 3cf2492 - [dagit] Multi-colored run batch backgrounds (#9775) by @hellendag
• 1d823b6 - [dagit] Virtualized graphs table, reuse ops view (#9778) by @hellendag
• a6efcaa - [dagster-dbt] remove logs and raw output from dbt errors (#9779) by @OwenKephart
• a833f28 - starlette TestClient deps (#9791) by @alangenfeld
• 5b66025 - Extend ECS stub to include new cpu/mem (#9786) by @jmsanders
• 1e8f629 - Get asset partition keys from IOManager and Op Contexts (#9776) by @clairelin135
• 49a2c57 - [dagit] Fix lint (#9798) by @hellendag
• a33935b - [dagit] Top tooltip for backfill segments (#9783) by @hellendag
• 273e820 - [dagit] Add reload button to WorkspaceHeader (#9789) by @hellendag
• cb01ddb - [dagit] Add top-level polling to virtualized table pages (#9797) by @hellendag
• d8596e4 - fix 4xx in docs.dagster.io CLOUD-1843 (#9802) by @yuhan
• 26257e7 - [scheduler] dont clean auto running states for error locations (#9805) by @alangenfeld
• 586c037 - [perf] resolve lru_cache usage issues (#9782) by @alangenfeld
• 5b3d15b - A few tweaks to concept docs (#9793) by @sryza
• 462db9e - Specify platform to build in deploy_ecs example (#9815) by @gibsondan
• a3c3743 - Add way to build and deploy the deploy_ecs project locally (#9760) by @gibsondan
• 2e0be96 - rm dead code in GrpcServerRepositoryLocation (#9811) by @alangenfeld
• c23c5d8 - Revert "remove dagster cli api subcommand from docs (#9165)" (#9740) by @gibsondan
• 8752ee7 - load asset values outside of a run (#9792) by @sryza
• 5a928ad - [dagit] Make asset group row clickable (#9810) by @hellendag
• 2d9ad46 - [dagit] Update homepage and top nav for new workspace flag (#9819) by @hellendag
• 0af6f5e - [dagit] Align virtualized table cells to top (#9809) by @hellendag
• a2d3ad1 - [dagit] Hover/active state for top nav items (#9816) by @hellendag
• 158269f - handle None outputs in snowflake IO manager (#9818) by @sryza
• 4e9e8b4 - [docs] - simplify multi-asset sensor examples and fix formatting (#9754) by @sryza
• 1f2633d - mark dbt outputs as Nothing (#9822) by @sryza
• 888935b - make methods of PartitionMapping public (#9747) by @sryza
• f43d582 - Make alembic pin much less restrictive (#9830) by @gibsondan
• 246f8f2 - [dagit] Export the new Workspace grid for Cloud use (#9824) by @hellendag
• 3d931ac - fix config schema on multi-asset (#9828) by @sryza
• 30db4d3 - 1.0.11 by @elementl-devtools

</details>

New

• Run status sensors can now monitor all runs in a Dagster Instance, rather than just runs from jobs within a single repository. You can enable this behavior by setting monitor_all_repositories=True in the run status sensor decorator.
• The run_key argument on RunRequest and run_request_for_partition is now optional.
• [dagster-databricks] A new “verbose_logs” config option on the databricks_pyspark_step_launcher makes it possible to silence non-critical logs from your external steps, which can be helpful for long-running, or highly parallel operations (thanks @zyd14!)
• [dagit] It is now possible to delete a run in Dagit directly from the run page. The option is available in the dropdown menu on the top right of the page.
• [dagit] The run timeline on the Workspace Overview page in Dagit now includes ad hoc asset materialization runs.

Bugfixes

• Fixed a set of bugs in multi_asset_sensor where the cursor would fail to update, and materializations would be returned out of order for latest_materialization_records_by_partition.
• Fixed a bug that caused failures in runs with time-partitioned asset dependencies when the PartitionsDefinition had an offset that wasn’t included in the date format. E.g. a daily-partitioned asset with an hour offset, whose date format was %Y-%m-%d.
• An issue causing code loaded by file path to import repeatedly has been resolved.
• To align with best practices, singleton comparisons throughout the codebase have been converted from (e.g.) foo == None to foo is None (thanks @chrisRedwine!).
• [dagit] In backfill jobs, the “Partition Set” column would sometimes show an internal __ASSET_JOB name, rather than a comprehensible set of asset keys. This has been fixed.
• [dagit] It is now possible to collapse all Asset Observation rows on the AssetDetails page.
• [dagster-dbt] Fixed issue that would cause an error when loading assets from dbt projects in which a source had a “*” character in its name (e.g. BigQuery sharded tables)
• [dagster-k8s] Fixed an issue where the k8s_job_op would sometimes fail if the Kubernetes job that it creates takes a long time to create a pod.
• Fixed an issue where links to the compute logs for a run would sometimes fail to load.
• [dagster-k8s] The k8s_job_executor now uses environment variables in place of CLI arguments to avoid limits on argument size with large dynamic jobs.

Documentation

• Docs added to explain subsetting graph-backed assets. You can use this feature following the documentation here.
• UI updated to reflect separate version schemes for mature core Dagster packages and less mature integration libraries

All Changes

https://github.com/dagster-io/dagster/compare/1.0.9...1.0.10

<details><summary>See All Contributors</summary>

• 8c12fef - [dagster-airbyte] Add config to avoid pulling/forwarding Airbyte logs to compute log (#9649) by @benpankow
• 274449b - [dagster-airbyte] Support representing normalization-created tables as Assets (#9655) by @benpankow
• 1787549 - isolate backfill partition status from partition status (#9698) by @prha
• 28ff3ff - [dagster-airbyte] Add ability to generate Airbyte assets from YAML files (#9664) by @benpankow
• 7c6b020 - [grpc] deferred snapshot support (#9630) by @alangenfeld
• 956c9fc - docstring updates for build_op_context (#9690) by @sryza
• a428f5c - tags argument on build_schedule_from_partitioned_job (#9699) by @sryza
• c9f8553 - [dagit] Fix Run Timeline support for multiple jobs with the same name (#9684) by @bengotow
• 05d76a4 - [dagit] Defensively pretty-print JSON in run logs (#9682) by @bengotow
• b52b9ca - replace run count with a run status indicator (#9700) by @prha
• d81ff02 - Add asset selection flag to run_request_for_partition (#9679) by @clairelin135
• e268231 - [dagster-dbt] fix missing select when using dbt_cli_resource (#9680) by @OwenKephart
• 62199e5 - [dagit] remove mode and graphName from workspace context (#9701) by @alangenfeld
• 4d89df7 - [dagster-databricks] fix library install version (#9696) by @OwenKephart
• 2fe1778 - [dagit] Fix run.assets returning duplicate asset keys, tags appearing twice (#9681) by @bengotow
• 755baed - Enable step retries on docker executor (#9675) by @johannkm
• bbdfbc5 - [docs] - Clean up Guides sidenav + category page (#9247) by @erinkcochran87
• f81d4ed - [docs] - Dagster Cloud full deployment docs (#9643) by @erinkcochran87
• f4a7783 - [grpc] include timeout length in timeout error (#9492) by @alangenfeld
• 3ce6ba5 - 1.0.9 Changelog (#9704) by @smackesey
• 770fb61 - Automation: versioned docs for 1.0.9 by @elementl-devtools
• 313b867 - run status sensor that can monitor the whole instance (#9666) by @jamiedemaria
• 328033f - improve dagster debug help text (#9712) by @alangenfeld
• 99db50c - [dagit] Use workspace-level repo filtering in virtualized jobs table (#9691) by @hellendag
• 81b1fc9 - [dagster-graphql] dont batch if only 1 target (#9710) by @alangenfeld
• a00e660 - [dagit] Virtualized schedules table (#9706) by @hellendag
• 10e2bbd - Fixes for multi_asset_sensor (#9708) by @clairelin135
• 568de7c - Retry while waiting for the k8s_job_op job to have pods (#9709) by @gibsondan
• d5231d4 - [dagit] Virtualized sensor table (#9718) by @hellendag
• cebbb9f - stop populating output_description on ExternalAssetNode (#9721) by @OwenKephart
• 6422c92 - [docs] - Graph-backed asset subsetting docs (#9695) by @clairelin135
• 6e2efcd - fix repeat file import (#9719) by @alangenfeld
• 8f67c72 - [docs] - Fix dbt > dbt Cloud in docs sidenav (#9720) by @erinkcochran87
• 1426361 - remove unused modes from factory floor query (#9724) by @gibsondan
• 986faad - making it possible to skip debug log within polling loop (#9716) by @zyd14
• d333c33 - [dagit] Add unloadable alert to virtualized schedule/sensor pages (#9725) by @hellendag
• cb1319a - [dagit] Avoid skipping subscription for compute log panel (#9734) by @hellendag
• 4714e48 - [dagit] Show ad hoc asset materializations in run timeline (#9726) by @hellendag
• 44b9d07 - Convert singleton comparisons from equality to identity (#9587) by @chrisRedwine
• 1f0a66c - modernize dynamic_tests (#9732) by @alangenfeld
• 5227a9e - [dagit] Allow deleting a run from the Run page itself (#9729) by @hellendag
• 20f740c - [dagit] Don't link to nowhere on Backfill history table (#9733) by @hellendag
• b9bc94a - k8s executor pass args via env var (#8564) by @johannkm
• 2bdb627 - [docs] - Fix IS_BRANCH_DEPLOYMENT docs examples (#9730) by @gibsondan
• 5976bde - [dagit] Allow collapsing the asset event row that is expanded by default (#9683) by @bengotow
• 577803d - [dagit] Repair some spacing on requested runs dialog (#9746) by @hellendag
• 0fd0a48 - make run keys optional on run requests (#9748) by @sryza
• aa8b753 - fix time window partition mapping with offset and lossy format (#9751) by @sryza
• 8d8daea - [dagit] Avoid showing hidden asset jobs on the backfill page (#9737) by @bengotow
• 42a9e91 - [dagster-dbt] fix star issue for asset input names (#9763) by @OwenKephart
• bc54e45 - 1.0.10 Changelog (#9771) by @OwenKephart
• 8421a58 - [fixit] Add library versions to docs (#9768) by @smackesey
• 2ec0561 - 1.0.10 by @elementl-devtools

</details>

New

• The multi_asset_sensor (experimental) now has improved capabilities to monitor asset partitions via a latest_materialization_records_by_partition method.
• Performance improvements for the Partitions page in Dagit.

Bugfixes

• Fixed a bug that caused the op_config argument of dagstermill.get_context to be ignored
• Fixed a bug that caused errors when loading the asset details page for assets with time window partitions definitions
• Fixed a bug where assets sometimes didn’t appear in the Asset Catalog while in Folder view.
• [dagit] Opening the asset lineage tab no longer scrolls the page header off screen in some scenarios
• [dagit] The asset lineage tab no longer attempts to materialize source assets included in the upstream / downstream views.
• [dagit] The Instance page Run Timeline no longer commingles runs with the same job name in different repositories
• [dagit] Emitting materializations with JSON metadata that cannot be parsed as JSON no longer crashes the run details page
• [dagit] Viewing the assets related to a run no longer shows the same assets multiple times in some scenarios
• [dagster-k8s] Fixed a bug with timeouts causing errors in k8s_job_op
• [dagster-docker] Fixed a bug with Op retries causing errors with the docker_executor

Community Contributions

• [dagster-aws] Thanks @Vivanov98 for adding the list_objects method to S3FakeSession!

Experimental

• [dagster-airbyte] Added an experimental function to automatically generate Airbyte assets from project YAML files. For more information, see the dagster-airbyte docs.
• [dagster-airbyte] Added the forward_logs option to AirbyteResource, allowing users to disble forwarding of Airbyte logs to the compute log, which can be expensive for long-running syncs.
• [dagster-airbyte] Added the ability to generate Airbyte assets for basic normalization tables generated as part of a sync.

Documentation

• [dagster-dbt] Added a new guide focused on the dbt Cloud integration.
• Fixed a bug that was hiding display of some public methods in the API docs.
• Added documentation for managing full deployments in Dagster Cloud, including a reference for deployment configuration options.

New

• With the new cron_schedule argument to TimeWindowPartitionsDefinition, you can now supply arbitrary cron expressions to define time window-based partition sets.
• Graph-backed assets can now be subsetted for execution via AssetsDefinition.from_graph(my_graph, can_subset=True).
• RunsFilter is now exported in the public API.
• [dagster-k8s] The dagster-user-deployments.deployments[].schedulerName Helm value for specifying custom Kubernetes schedulers will now also apply to run and step workers launched for the given user deployment. Previously it would only apply to the grpc server.

Bugfixes

• In some situations, default asset config was ignored when a subset of assets were selected for execution. This has been fixed.
• Added a pin to grpcio in dagster to address an issue with the recent 0.48.1 grpcio release that was sometimes causing Dagster code servers to hang.
• Fixed an issue where the “Latest run” column on the Instance Status page sometimes displayed an older run instead of the most recent run.

Community Contributions

• In addition to a single cron string, cron_schedule now also accepts a sequence of cron strings. If a sequence is provided, the schedule will run for the union of all execution times for the provided cron strings, e.g., ['45 23 * * 6', '30 9 * * 0] for a schedule that runs at 11:45 PM every Saturday and 9:30 AM every Sunday. Thanks @erinov1!
• Added an optional boolean config install_default_libraries to databricks_pyspark_step_launcher . It allows to run Databricks jobs without installing the default Dagster libraries .Thanks @nvinhphuc!

Experimental

• [dagster-k8s] Added additional configuration fields (container_config, pod_template_spec_metadata, pod_spec_config, job_metadata, and job_spec_config) to the experimental k8s_job_op that can be used to add additional configuration to the Kubernetes pod that is launched within the op.

New

• Several updates to the Dagit run timeline view: your time window preference will now be preserved locally, there is a clearer “Now” label to delineate the current time, and upcoming scheduled ticks will no longer be batched with existing runs.
• [dagster-k8s] ingress.labels is now available in the Helm chart. Any provided labels are appended to the default labels on each object (helm.sh/chart, app.kubernetes.io/version, and app.kubernetes.io/managed-by).
• [dagster-dbt] Added support for two types of dbt nodes: metrics, and ephemeral models
• When constructing a GraphDefinition manually, InputMapping and OutputMapping objects should be directly constructed.
• [dagit] The launchpad tab is no longer shown for Asset jobs. Asset jobs can be launched via the “Materialize All” button shown on the Overview tab. To provide optional configuration, hold shift when clicking “Materialize”.

Bugfixes

• [dagster-snowflake] Pandas is no longer imported when dagster_snowflake is imported. Instead, it’s only imported when using functionality inside dagster-snowflake that depends on pandas.
• Recent changes to run status sensors caused sensors that only monitored jobs in external repositories to also monitor all jobs in the current repository. This has been fixed.
• Fixed an issue where "unhashable type" errors could be spawned from sensor executions.
• [dagit] Clicking between assets in different repositories from asset groups and asset jobs now works as expected.
• [dagit] The DAG rendering of composite ops with more than one input/output mapping has been fixed.
• [dagit] Selecting a source asset in Dagit no longer produces a GraphQL error
• [dagit] Viewing “Related Assets” for an asset run now shows the full set of assets included in the run, regardless of whether they were materialized successfully.
• [dagit] The Asset Lineage view has been simplified and lets you know if the view is being clipped and more distant upstream/downstream assets exist.
• Fixed erroneous experimental warnings being thrown when using with_resources alongside source assets.

Breaking Changes

• The arguments to the (internal) InputMapping and OutputMapping constructors have changed.

Community Contributions

• The ssh_resource can now accept configuration from environment variables. Thanks @cbini!
• Spelling corrections in migrations.md. Thanks @gogi2811!

New

• [dagit] nbconvert is now installed as an extra in Dagit.
• Multiple assets can be monitored for materialization using the multi_asset_sensor (experimental).
• Run status sensors can now monitor jobs in external repositories.
• The config argument of define_asset_job now works if the job contains partitioned assets.
• When configuring sqlite-based storages in dagster.yaml, you can now point to environment variables.
• When emitting RunRequests from sensors, you can now optionally supply an asset_selection argument, which accepts a list of AssetKeys to materialize from the larger job.
• [dagster-dbt] load_assets_from_dbt_project and load_assets_from_dbt_manifest now support the exlude parameter, allowing you to more precisely which resources to load from your dbt project (thanks @flvndh!)

Bugfixes

• Previously, types for multi-assets would display incorrectly in Dagit when specified. This has been fixed.
• In some circumstances, viewing nested asset paths in Dagit could lead to unexpected empty states. This was due to incorrect slicing of the asset list, and has been fixed.
• Fixed an issue in Dagit where the dialog used to wipe materializations displayed broken text for assets with long paths.
• [dagit] Fixed the Job page to change the latest run tag and the related assets to bucket repository-specific jobs. Previously, runs from jobs with the same name in different repositories would be intermingled.
• Previously, if you launched a backfill for a subset of a multi-asset (e.g. dbt assets), all assets would be executed on each run, instead of just the selected ones. This has been fixed.
• [dagster-dbt] Previously, if you configured a select parameter on your dbt_cli_resource , this would not get passed into the corresponding invocations of certain context.resources.dbt.x() commands. This has been fixed.

New

• Assets can now be materialized to storage conditionally by setting output_required=False. If this is set and no result is yielded from the asset, Dagster will not create an asset materialization event, the I/O manager will not be invoked, downstream assets will not be materialized, and asset sensors monitoring the asset will not trigger.
• JobDefinition.run_request_for_partition can now be used inside sensors that target multiple jobs (Thanks Metin Senturk!)
• The environment variable DAGSTER_GRPC_TIMEOUT_SECONDS now allows for overriding the default timeout for communications between host processes like dagit and the daemon and user code servers.
• Import time for the dagster module has been reduced, by approximately 50% in initial measurements.
• AssetIn now accepts a dagster_type argument, for specifying runtime checks on asset input values.
• [dagit] The column names on the Activity tab of the asset details page no longer reference the legacy term “Pipeline”.
• [dagster-snowflake] The execute_query method of the snowflake resource now accepts a use_pandas_result argument, which fetches the result of the query as a Pandas dataframe. (Thanks @swotai!)
• [dagster-shell] Made the execute and execute_script_file utilities in dagster_shell part of the public API (Thanks Fahad Khan!)
• [dagster-dbt] load_assets_from_dbt_project and load_assets_from_dbt_manifest now support the exclude parameter. (Thanks @flvndh!)

Bugfixes

• [dagit] Removed the x-frame-options response header from Dagit, allowing the Dagit UI to be rendered in an iframe.
• [fully-featured project example] Fixed the duckdb IO manager so the comment_stories step can load data successfully.
• [dagster-dbt] Previously, if a select parameter was configured on the dbt_cli_resource, it would not be passed into invocations of context.resources.dbt.run() (and other similar commands). This has been fixed.
• [dagster-ge] An incompatibility between dagster_ge_validation_factory and dagster 1.0 has been fixed.
• [dagstermill] Previously, updated arguments and properties to DagstermillExecutionContext were not exposed. This has since been fixed.

Documentation

• The integrations page on the docs site now has a section for links to community-hosted integrations. The first linked integration is @silentsokolov’s Vault integration.

New

• Failure now has an allow_retries argument, allowing a means to manually bypass retry policies.
• dagstermill.get_context and dagstermill.DagstermillExecutionContext have been updated to reflect stable dagster-1.0 APIs. pipeline/solid referencing arguments / properties will be removed in the next major version bump of dagstermill.
• TimeWindowPartitionsDefinition now exposes a get_cron_schedule method.

Bugfixes

• In some situations where an asset was materialized and that asset that depended on a partitioned asset, and that upstream partitioned asset wasn’t part of the run, the partition-related methods of InputContext returned incorrect values or failed erroneously. This was fixed.
• Schedules and sensors with the same names but in different repositories no longer affect each others idempotence checks.
• In some circumstances, reloading a repository in Dagit could lead to an error that would crash the page. This has been fixed.

Community Contributions

• @will-holley added an optional key argument to GCSFileManager methods to set the GCS blob key, thank you!
• Fix for sensors in fully featured example, thanks @pwachira!

Documentation

• New documentation for getting started with Dagster Cloud, including:
  • Serverless deployment documentation
  • Hybrid deployment documentation

New

• When the workpace is updated, a notification will appear in Dagit, and the Workspace tab will automatically refresh.

Bugfixes

• Restored the correct version mismatch warnings between dagster core and dagster integration libraries
• Field.__init__ has been typed, which resolves an error that pylance would raise about default_value
• Previously, dagster_type_materializer and dagster_type_loader expected functions to take a context argument from an internal dagster import. We’ve added DagsterTypeMaterializerContext and DagsterTypeLoaderContext so that functions annotated with these decorators can annotate their arguments properly.
• Previously, a single-output op with a return description would not pick up the description of the return. This has been rectified.

Community Contributions

• Fixed the dagster_slack documentation examples. Thanks @ssingh13-rms!

Documentation

• New documentation for Dagster Cloud environment variables.
• The full list of APIs removed in 1.0 has been added to the migration guide.

Bugfixes

• Fixed an issue where Dagster libraries would sometimes log warnings about mismatched versions despite having the correct version loaded.

Documentation

• The Dagster Cloud docs now live alongside all the other Dagster docs! Check them out by nagivating to Deployment > Cloud.

Major Changes

• A docs site overhaul! Along with tons of additional content, the existing pages have been significantly edited and reorganized to improve readability.
• All Dagster examplesare revamped with a consistent project layout, descriptive names, and more helpful README files.
• A new dagster project CLI contains commands for bootstrapping new Dagster projects and repositories
  • dagster project scaffold creates a folder structure with a single Dagster repository and other files such as workspace.yaml. This CLI enables you to quickly start building a new Dagster project with everything set up.
  • dagster project from-example downloads one of the Dagster examples. This CLI helps you to quickly bootstrap your project with an officially maintained example. You can find the available examples via dagster project list-examples.
  • Check out Create a New Project for more details.
• A default_executor_def argument has been added to the @repository decorator. If specified, this will be used for any jobs (asset or op) which do not explicitly set an executor_def.
• A default_logger_defs argument has been added to the @repository decorator, which works in the same way as default_executor_def.
• A new execute_job function presents a Python API for kicking off runs of your jobs.
• Run status sensors may now yield RunRequests, allowing you to kick off a job in response to the status of another job.
• When loading an upstream asset or op output as an input, you can now set custom loading behavior using the input_manager_key argument to AssetIn and In.
• In the UI, the global lineage graph has been brought back and reworked! The graph keeps assets in the same group visually clustered together, and the query bar allows you to visualize a custom slice of your asset graph.

Breaking Changes and Deprecations

Legacy API Removals

In 1.0.0, a large number of previously-deprecated APIs have been fully removed. A full list of breaking changes and deprecations, alongside instructions on how to migrate older code, can be found in MIGRATION.md. At a high level:

• The solid and pipeline APIs have been removed, along with references to them in extension libraries, arguments, and the CLI (deprecated in 0.13.0).
• The AssetGroup and build_asset_job APIs, and a host of deprecated arguments to asset-related functions, have been removed (deprecated in 0.15.0).
• The EventMetadata and EventMetadataEntryData APIs have been removed (deprecated in 0.15.0).

Deprecations

• dagster_type_materializer and DagsterTypeMaterializer have been marked experimental and will likely be removed within a 1.x release. Instead, use an IOManager.
• FileManager and FileHandle have been marked experimental and will likely be removed within a 1.x release.

Other Changes

• As of 1.0.0, Dagster no longer guarantees support for python 3.6. This is in line with PEP 494, which outlines that 3.6 has reached end of life.
• [planned] In an upcoming 1.x release, we plan to make a change that renders values supplied to configured in Dagit. Up through this point, values provided to configured have not been sent anywhere outside the process where they were used. This change will mean that, like other places you can supply configuration, configured is not a good place to put secrets: You should not include any values in configuration that you don't want to be stored in the Dagster database and displayed inside Dagit.
• fs_io_manager, s3_pickle_io_manager, and gcs_pickle_io_manager, and adls_pickle_io_manager no longer write out a file or object when handling an output with the None or Nothing type.
• The custom_path_fs_io_manager has been removed, as its functionality is entirely subsumed by the fs_io_manager, where a custom path can be specified via config.
• The default typing_type of a DagsterType is now typing.Any instead of None.

New since 0.15.8

• [dagster-databricks] When using the databricks_pyspark_step_launcher the events sent back to the host process are now compressed before sending, resulting in significantly better performance for steps which produce a large number of events.
• [dagster-dbt] If an error occurs in load_assets_from_dbt_project while loading your repository, the error message in Dagit will now display additional context from the dbt logs, instead of just DagsterDbtCliFatalRuntimeError.

Bugfixes

• Fixed a bug that causes Dagster to ignore the group_name argument to AssetsDefinition.from_graph when a key_prefix argument is also present.
• Fixed a bug which could cause GraphQL errors in Dagit when loading repositories that contained multiple assets created from the same graph.
• Ops and software-defined assets with the None return type annotation are now given the Nothing type instead of the Any type.
• Fixed a bug that caused AssetsDefinition.from_graph and from_op to fail when invoked on a configured op.
• The materialize function, which is not experimental, no longer emits an experimental warning.
• Fixed a bug where runs from different repositories would be intermingled when viewing the runs for a specific repository-scoped job/schedule/sensor.
• [dagster-dbt] A regression was introduced in 0.15.8 that would cause dbt logs to show up in json format in the UI. This has been fixed.
• [dagster-databricks] Previously, if you were using the databricks_pyspark_step_launcher, and the external step failed to start, a RESOURCE_DOES_NOT_EXIST error would be surfaced, without helpful context. Now, in most cases, the root error causing the step to fail will be surfaced instead.

Documentation

• New guide that walks through seamlessly transitioning code from development to production environments.
• New guide that demonstrates using Branch Deployments to test Dagster code in your cloud environment without impacting your production data.

New

• Software-defined asset config schemas are no longer restricted to dicts.
• The OpDefinition constructor now accept ins and outs arguments, to make direct construction easier.
• define_dagstermill_op accepts ins and outs in order to make direct construction easier.

Bugfixes

• Fixed a bug where default configuration was not applied when assets were selected for materialization in Dagit.
• Fixed a bug where RunRequests returned from run_status_sensors caused the sensor to error.
• When supplying config to define_asset_job, an error would occur when selecting most asset subsets. This has been fixed.
• Fixed an error introduced in 0.15.7 that would prevent viewing the execution plan for a job re-execution from 0.15.0 → 0.15.6
• [dagit] The Dagit server now returns 500 http status codes for GraphQL requests that encountered an unexpected server error.
• [dagit] Fixed a bug that made it impossible to kick off materializations of partitioned asset if the day_offset, hour_offset, or minute_offset parameters were set on the asset’s partitions definition.
• [dagster-k8s] Fixed a bug where overriding the Kubernetes command to use to run a Dagster job by setting the dagster-k8s/config didn’t actually override the command.
• [dagster-datahub] Pinned version of acryl-datahub to avoid build error.

Breaking Changes

• The constructor of JobDefinition objects now accept a config argument, and the preset_defs argument has been removed.

Deprecations

• DagsterPipelineRunMetadataValue has been renamed to DagsterRunMetadataValue. DagsterPipelineRunMetadataValue will be removed in 1.0.

Community Contributions

• Thanks to @hassen-io for fixing a broken link in the docs!

Documentation

• MetadataEntry static methods are now marked as deprecated in the docs.
• PartitionMappings are now included in the API reference.
• A dbt example and memoization example using legacy APIs have been removed from the docs site.

New

• DagsterRun now has a job_name property, which should be used instead of pipeline_name.
• TimeWindowPartitionsDefinition now has a get_partition_keys_in_range method which returns a sequence of all the partition keys between two partition keys.
• OpExecutionContext now has asset_partitions_def_for_output and asset_partitions_def_for_input methods.
• Dagster now errors immediately with an informative message when two AssetsDefinition objects with the same key are provided to the same repository.
• build_output_context now accepts a partition_key argument that can be used when testing the handle_output method of an IO manager.

Bugfixes

• Fixed a bug that made it impossible to load inputs using a DagsterTypeLoader if the InputDefinition had an asset_key set.
• Ops created with the @asset and @multi_asset decorators no longer have a top-level “assets” entry in their config schema. This entry was unused.
• In 0.15.6, a bug was introduced that made it impossible to load repositories if assets that had non-standard metadata attached to them were present. This has been fixed.
• [dagster-dbt] In some cases, using load_assets_from_dbt_manifest with a select parameter that included sources would result in an error. This has been fixed.
• [dagit] Fixed an error where a race condition of a sensor/schedule page load and the sensor/schedule removal caused a GraphQL exception to be raised.
• [dagit] The “Materialize” button no longer changes to “Rematerialize” in some scenarios
• [dagit] The live overlays on asset views, showing latest materialization and run info, now load faster
• [dagit] Typing whitespace into the launchpad Yaml editor no longer causes execution to fail to start
• [dagit] The explorer sidebar no longer displays “mode” label and description for jobs, since modes are deprecated.

Community Contributions

• An error will now be raised if a @repository decorated function expects parameters. Thanks @roeij!

Documentation

• The non-asset version of the Hacker News example, which lived inside examples/hacker_news/, has been removed, because it hadn’t received updates in a long time and had drifted from best practices. The asset version is still there and has an updated README. Check it out here

All Changes

https://github.com/dagster-io/dagster/compare/0.15.6...0.15.7 <details><summary>See All Contributors</summary>

• 57acdd5 - Correct check for pickle s3 io manager (#8834) by @ripplekhera
• 4114910 - [dagit] Always show “Materialize” instead of “Rematerialize” based on status (#8711) by @bengotow
• 972274c - [dagster-dbt] make group configurable for load_assets_from_dbt (#8863) by @OwenKephart
• 05fc596 - asset_partitions_def on InputContext and OutputContext (#8858) by @sryza
• 2d86c76 - [dagster-dbt] refactor the dbt asset integration (#8793) by @OwenKephart
• 150bc3e - PartitionMappings when non-partitioned assets depend on partitioned assets (#8866) by @sryza
• 70a7dbf - [dagster-dbt] seeds and snapshots are assets when using dbt build (#8794) by @OwenKephart
• e46b9a0 - Document valid names for asset keys (#8765) by @jamiedemaria
• 2357a02 - [docs] Dagster + dbt guide (#8714) by @OwenKephart
• dbaed58 - 0.15.6 changelog (#8876) by @yuhan
• 5bb50c0 - provide description for MAP ConfigType (#8824) by @Jiafi
• b1aa83a - Retrieve minimal set of asset records for assetsLatestInfo (#8835) by @bengotow
• b8493f3 - error when duplicate asset keys on a repository (#8874) by @sryza
• 084c66c - [docs] - Add Airflow Operator to Op Docs (#8875) by @clairelin135
• 2f15fbf - dagster-datahub Rest and Kafka Emitter integration (#8764) by @Jiafi
• 0988274 - Automation: versioned docs for 0.15.6 by @elementl-devtools
• 0e83834 - [1.0] move solid to dagster.legacy (#8843) by @dpeng817
• bc5e502 - Extract ECS task overrides (#8842) by @jmsanders
• e3ea175 - [graphql] tolerate empty runConfigData (#8886) by @alangenfeld
• 56c7023 - [dagit] Fix edge case where “ “ launchpad config is not coerced to an empty object (#8895) by @bengotow
• ee2e977 - Fix ScheduleRootQuery typo (#8903) by @johannkm
• 61c1c20 - unloadable shit (#8887) by @prha
• 711b323 - Change base image for OSS release builds (#8902) by @gibsondan
• c85e158 - change deprecation warnings to 1.0 (#8892) by @dpeng817
• cd779b1 - update README for hacker news assets example (#8904) by @sryza
• e657abd - [hacker news] add missing key prefix to activity analytics python assets (#8817) by @sryza
• 4da2a9e - [buildkite] Specify internal branch used for compatibility pipeline (#8881) by @smackesey
• 6c97c75 - [dagit] Remove “mode” label and description in explorer sidebar (#8907) by @bengotow
• 4cefd84 - remove the non-asset version of the hacker news example (#8883) by @sryza
• 23a9997 - Error when @repository-decorated function has arguments (#8913) by @roeij
• f787d6d - [docs] - Correct snippets for dbt (#8923) by @erinkcochran87
• 1961e51 - [bug] fix input loading regression (#8885) by @OwenKephart
• ff87738 - [docs] - graph backed assets doc fix (#8927) by @jamiedemaria
• 18f254d - silence system-originated experimental warning for PartitionMapping (#8931) by @sryza
• a2df1de - Add partition key to build_output_context, add documentation for partition_key on build_op_context (#8774) by @dpeng817
• 53287b9 - fix dimensions of screenshot on connecting ops tutorial page (#8908) by @sryza
• c00de5b - ttv: remove undocumented/legacy example - user in loop (#8934) by @yuhan
• 31f3283 - [docs] - Clean up graph-backed asset example, put under test (#8893) by @dpeng817
• 7c60a46 - [docs] - Fix garbled sentence in ops.mdx (#8935) by @schrockn
• c554461 - enable getting asset partitions def from op context (#8924) by @sryza
• 7c13e28 - Increase test_docker_monitoring timeout (#8906) by @johannkm
• 6365996 - PartitionsDefinition.get_partition_keys_in_range (#8933) by @sryza
• b58d711 - Move pipeline to dagster.legacy (#8888) by @dpeng817
• 7e11df2 - [dagit] Rename search open event (#8954) by @hellendag
• f3caeae - [dagit] Adjust shift-selection behavior in asset graphs (#8950) by @bengotow
• 65caf79 - [dagit] Clean up code around the graph sidebar (#8914) by @bengotow
• 5bd5c8b - add a job_name property to PipelineRun (#8928) by @sryza
• 9421f73 - remove partition entries from asset op config schema (#8951) by @sryza
• 000d37a - avoid pipelines in run status sensor doc snippets (#8929) by @sryza
• a9b25dd - [bug] Fix issue where 'invalid' asset metadata resulted in an error (#8947) by @OwenKephart
• 4dadcd4 - [dagster-dbt] fix tagged source asset loading (#8943) by @OwenKephart
• 065adbd - fix black in run status sensor docs example (#8974) by @sryza
• 164c585 - [known state] fix build_for_reexecution bug (#8975) by @alangenfeld
• cc70c88 - Document deprecation of MetadataEntry static constructors (#8984) by @smackesey
• eed9277 - changelog (#8986) by @jamiedemaria
• b283b8a - 0.15.7 by @elementl-devtools

</details>

New

• When an exception is wrapped by another exception and raised within an op, Dagit will now display the full chain of exceptions, instead of stopping after a single exception level.
• A default_logger_defs argument has been added to the @repository decorator. Check out the docs on specifying default loggers to learn more.
• AssetsDefinition.from_graph and AssetsDefinition.from_op now both accept a partition_mappings argument.
• AssetsDefinition.from_graph and AssetsDefinition.from_op now both accept a metadata_by_output_name argument.
• define_asset_job now accepts an executor_def argument.
• Removed package pin for gql in dagster-graphql.
• You can now apply a group name to assets produced with the @multi_asset decorator, either by supplying a group_name argument (which will apply to all of the output assets), or by setting the group_name argument on individual AssetOuts.
• InputContext and OutputContext now each have an asset_partitions_def property, which returns the PartitionsDefinition of the asset that’s being loaded or stored.
• build_schedule_from_partitioned_job now raises a more informative error when provided a non-partitioned asset job
• PartitionMapping, IdentityPartitionMapping, AllPartitionMapping, and LastPartitionMapping are exposed at the top-level dagster package. They're currently marked experimental.
• When a non-partitioned asset depends on a partitioned asset, you can now control which partitions of the upstream asset are used by the downstream asset, by supplying a PartitionMapping.
• You can now set PartitionMappings on AssetIn.
• [dagit] Made performance improvements to the loading of the partitions and backfill pages.
• [dagit] The Global Asset Graph is back by popular demand, and can be reached via a new “View global asset lineage ”link on asset group and asset catalog pages! The global graph keeps asset in the same group visually clustered together and the query bar allows you to visualize a custom slice of your asset graph.
• [dagit] Simplified the Content Security Policy and removed frame-ancestors restriction.
• [dagster-dbt] load_assets_from_dbt_project and load_assets_from_dbt_manifest now support a node_info_to_group_name_fn parameter, allowing you to customize which group Dagster will assign each dbt asset to.
• [dagster-dbt] When you supply a runtime_metadata_fn when loading dbt assets, this metadata is added to the default metadata that dagster-dbt generates, rather than replacing it entirely.
• [dagster-dbt] When you load dbt assets with use_build_command=True, seeds and snapshots will now be represented as Dagster assets. Previously, only models would be loaded as assets.

Bugfixes

• Fixed an issue where runs that were launched using the DockerRunLauncher would sometimes use Dagit’s Python environment as the entrypoint to launch the run, even if that environment did not exist in the container.
• Dagster no longer raises a “Duplicate definition found” error when a schedule definition targets a partitioned asset job.
• Silenced some erroneous warnings that arose when using software-defined assets.
• When returning multiple outputs as a tuple, empty list values no longer cause unexpected exceptions.
• [dagit] Fixed an issue with graph-backed assets causing a GraphQL error when graph inputs were type-annotated.
• [dagit] Fixed an issue where attempting to materialize graph-backed assets caused a graphql error.
• [dagit] Fixed an issue where partitions could not be selected when materializing partitioned assets with associated resources.
• [dagit] Attempting to materialize assets with required resources now only presents the launchpad modal if at least one resource defines a config schema.

Breaking Changes

• An op with a non-optional DynamicOutput will now error if no outputs are returned or yielded for that dynamic output.
• If an Output object is used to type annotate the return of an op, an Output object must be returned or an error will result.

Community Contributions

• Dagit now displays the path of the output handled by PickledObjectS3IOManager in run logs and Asset view. Thanks @danielgafni

Documentation

• The Hacker News example now uses stable 0.15+ asset APIs, instead of the deprecated 0.14.x asset APIs.
• Fixed the build command in the instructions for contributing docs changes.
• [dagster-dbt] The dagster-dbt integration guide now contains information on using dbt with Software-Defined Assets.

All Changes

https://github.com/dagster-io/dagster/compare/0.15.5...0.15.6

<details><summary>See All Contributors</summary>

• 583ce34 - Fold asset_defs submodule into main dagster structure (#8446) by @smackesey
• b4fa57e - Op that runs a kubernetes job (#8161) by @gibsondan
• a3a5ccb - add validation for graph backed asset graphs (#8754) by @OwenKephart
• 5d5dd71 - chore: mark snapshots as generated (#8758) by @rexledesma
• 4a9718d - [op] fix empty list output bug (#8763) by @alangenfeld
• 397ad03 - [dagster-dbt] Allow SDAs generated with load_assets_from_dbt* to be partitioned (#8725) by @OwenKephart
• b007a0c - docs: update config schema descriptions for default executors (#8757) by @rexledesma
• 58518d9 - Restore sensor daemon yielding when evaluating sensors synchronously (#8756) by @prha
• c449c0f - bypass bucketed queries for mysql versions that do not support it (#8753) by @prha
• b1ac8d6 - [dagster-dbt] Fix error that occurs when generating events for tests that depend on sources (#8775) by @OwenKephart
• 2a237a0 - Specifying executor docs examples (#8530) by @dpeng817
• eccdac2 - prevent multiple sensor evaluations from multithreaded race conditions (#8720) by @prha
• 519d77f - Fix config case for default executor (#8777) by @dpeng817
• daadb5a - Ensure graph inputs/outputs are included in all_dagster_types (#8736) by @smackesey
• a94210a - improve error for build_schedule_from_partitioned_job with non-partitioned asset job (#8776) by @sryza
• 0bab6fd - [dagit] Bring back the global asset graph as an “all asset groups” view (#8709) by @bengotow
• f6987da - fix source asset regression (#8784) by @smackesey
• 530c321 - fix issue with repos and partitioned scheduled asset jobs (#8779) by @sryza
• b3ba40c - 0.15.5 Changelog (#8781) by @prha
• 0e32054 - changelog (#8788) by @prha
• 1452280 - Option to hide daemon heartbeat timestamp in Dagit (#8785) by @johannkm
• 1e7691d - Fix bug with how resources are applied in materialize (#8790) by @dpeng817
• ccd1893 - Add default_logger_defs arg to repository (#8512) by @dpeng817
• e10b6f3 - update hackernews tests to use asset invocation and materialize_to_memory (#8592) by @dpeng817
• 5bdd6cf - Add MetaDataEntry.path to PickledObjectS3IOManager (#8732) by @danielgafni
• 6d19b1d - Automation: versioned docs for 0.15.5 by @elementl-devtools
• f459da0 - add define_asset_job to __all__ (#8791) by @sryza
• f1a4612 - eliminate incorrect SDA warnings (#8769) by @sryza
• 738e7eb - update hacker news assets example for post-0.15.0 APIs (#7904) by @sryza
• 5938830 - partition mappings on graph-backed assets (#8768) by @sryza
• b721c70 - Snowflake IO Manager handles pandas timestamps (#8760) by @jamiedemaria
• cb178b1 - Add Python 3.10 testing to BK and other image adjustments (#7700) by @smackesey
• f4415aa - Option to skip daemon heartbeats with no errors (#8670) by @johannkm
• 2d35b84 - Assorted type annotations (#8356) by @smackesey
• 3e1d539 - Bump urllib3 (#8808) by @dependabot[bot]
• de366db - Bump rsa (#8807) by @dependabot[bot]
• 6771e78 - Bump pyyaml (#8806) by @dependabot[bot]
• 44492db - Change default local grpc behavior to send back "dagster" as the entry point to use, rather than dagit's python environment (#8571) by @gibsondan
• 776d701 - move Metadata and Tags concept page under jobs section (#8813) by @sryza
• c142bd9 - updates to multi-assets docs page (#8814) by @sryza
• 3eb0afd - Update asset ID to contain repository location and name (#8762) by @clairelin135
• a05f9b1 - unpin gql (#8822) by @prha
• c89d01e - remove unused old partitions ui (#8796) by @prha
• 1539cfb - fix broken asset deps in hacker news example (#8809) by @sryza
• 7f5cd8e - [dagit] Bump TS version (#8704) by @hellendag
• 8da4644 - fix contributing docs (#8789) by @prha
• 1714bc9 - Better support for nested causes in dagit (#8823) by @gibsondan
• 641c707 - [easy] Fix docs link for RetryPolicy (#8830) by @gibsondan
• 04fb5c1 - Remove the "cronjobs" permission from the helm chart (#8827) by @gibsondan
• e7111f9 - fix gql resolver for graph-backed assets resources (#8825) by @smackesey
• 418952d - refactor run storage to enable backfill status queries (#8695) by @prha
• 3eae463 - refactor backfill / partition pages to stop run fetching (#8696) by @prha
• 47238c2 - add multi_or_in_process_executor to __all__ (#8831) by @smackesey
• a3ec60b - avoid apollo cache collision for partition/backfill status (#8841) by @prha
• f741443 - distinguish between [] and None for asset queries (#8838) by @prha
• 9906c4b - override batch loader to use asset records instead of legacy event materialization method (#8839) by @prha
• e523dad - [dagit] Add analytics.group (#8832) by @hellendag
• 52d4fdf - [dagster-io/ui] Fix disabled Button (#8844) by @hellendag
• 633d6b4 - Fix issue where partitioned assets with resources fail materialization in dagit (#8837) by @smackesey
• 652d12e - [dagit] Tweak analytics function sigs (#8851) by @hellendag
• 7cd7de8 - [asset-defs] allow multi assets to have group names (#8847) by @OwenKephart
• acf8c4d - Refactor op return checking code (#8755) by @dpeng817
• df5833e - [dagit] Remove frame-ancestors restriction (#8850) by @hellendag
• e5aca1b - adjust error messages (#8853) by @dpeng817
• fb89b1f - [dagit] Update CRA, simplify CSP (#8854) by @hellendag
• 50dbd6a - key_prefix for AssetsDefinition from_graph and from_op (#8859) by @jamiedemaria
• 105ad88 - easy: fix dagster pandas link (#8862) by @yuhan
• 1d33b3f - executor_definition on define_asset_job (#8856) by @sryza
• c9cc22b - include airflow_operator_to_op in apidoc (#8860) by @sryza
• 0fc21f5 - add metdata_by_output_name (#8861) by @OwenKephart
• 3916741 - [dagster-dbt] make group configurable for load_assets_from_dbt (#8863) by @OwenKephart
• 7eded6b - asset_partitions_def on InputContext and OutputContext (#8858) by @sryza
• 2dfcff7 - [dagster-dbt] refactor the dbt asset integration (#8793) by @OwenKephart
• 915948e - [dagster-dbt] seeds and snapshots are assets when using dbt build (#8794) by @OwenKephart
• e7a82d0 - PartitionMappings when non-partitioned assets depend on partitioned assets (#8866) by @sryza
• a555b22 - [docs] Dagster + dbt guide (#8714) by @OwenKephart
• 6e4dfcb - 0.15.6 by @elementl-devtools

</details>

New

• Added documentation and helm chart configuration for threaded sensor evaluations.
• Added documentation and helm chart configuration for tick retention policies.
• Added descriptions for default config schema. Fields like execution, loggers, ops, and resources are now documented.
• UnresolvedAssetJob objects can now be passed to run status sensors.
• [dagit] A new global asset lineage view, linked from the Asset Catalog and Asset Group pages, allows you to view a graph of assets in all loaded asset groups and filter by query selector and repo.
• [dagit] A new option on Asset Lineage pages allows you to choose how many layers of the upstream / downstream graph to display.
• [dagit] Dagit's DAG view now collapses large sets of edges between the same ops for improved readability and rendering performance.

Bugfixes

• Fixed a bug with materialize that would cause required resources to not be applied correctly.
• Fixed issue that caused repositories to fail to load when build_schedule_from_partitioned_job and define_asset_job were used together.
• Fixed a bug that caused auto run retries to always use the FROM_FAILURE strategy
• Previously, it was possible to construct Software-Defined Assets from graphs whose leaf ops were not mapped to assets. This is invalid, as these ops are not required for the production of any assets, and would cause confusing behavior or errors on execution. This will now result in an error at definition time, as intended.
• Fixed issue where the run monitoring daemon could mark completed runs as failed if they transitioned quickly between STARTING and SUCCESS status.
• Fixed stability issues with the sensor daemon introduced in 0.15.3 that caused the daemon to fail heartbeat checks if the sensor evaluation took too long.
• Fixed issues with the thread pool implementation of the sensor daemon where race conditions caused the sensor to fire more frequently than the minimum interval.
• Fixed an issue with storage implementations using MySQL server version 5.6 which caused SQL syntax exceptions to surface when rendering the Instance overview pages in Dagit.
• Fixed a bug with the default_executor_def argument on repository where asset jobs that defined executor config would result in errors.
• Fixed a bug where an erroneous exception would be raised if an empty list was returned for a list output of an op.
• [dagit] Clicking the "Materialize" button for assets with configurable resources will now present the asset launchpad.
• [dagit] If you have an asset group and no jobs, Dagit will display it by default rather than directing you to the asset catalog.
• [dagit] DAG renderings of software-defined assets now display only the last component of the asset's key for improved readability.
• [dagit] Fixes a regression where clicking on a source asset would trigger a GraphQL error.
• [dagit] Fixed issue where the “Unloadable” section on the sensors / schedules pages in Dagit were populated erroneously with loadable sensors and schedules
• [dagster-dbt] Fixed an issue where an exception would be raised when using the dbt build command with Software-Defined Assets if a test was defined on a source.

Deprecations

• Removed the deprecated dagster-daemon health-check CLI command

Community Contributions

• TimeWindow is now exported from the dagster package (Thanks @nvinhphuc!)
• Added a fix to allow customization of slack messages (Thanks @solarisa21!)
• [dagster-databricks] The databricks_pyspark_step_launcher now allows you to configure the following (Thanks @Phazure!):
  • the aws_attributes of the cluster that will be spun up for the step.
  • arbitrary environment variables to be copied over to databricks from the host machine, rather than requiring these variables to be stored as secrets.
  • job and cluster permissions, allowing users to view the completed runs through the databricks console, even if they’re kicked off by a service account.

Experimental

• [dagster-k8s] Added k8s_job_op to launch a Kubernetes Job with an arbitrary image and CLI command. This is in contrast with the k8s_job_executor, which runs each Dagster op in a Dagster job in its own k8s job. This op may be useful when you need to orchestrate a command that isn't a Dagster op (or isn't written in Python). Usage:

  from dagster_k8s import k8s_job_op
  
  my_k8s_op = k8s_job_op.configured({
   "image": "busybox",
   "command": ["/bin/sh", "-c"],
   "args": ["echo HELLO"],
   },
   name="my_k8s_op",
  )
  

• [dagster-dbt] The dbt asset-loading functions now support partitions_def and partition_key_to_vars_fn parameters, adding preliminary support for partitioned dbt assets. To learn more, check out the Github issue!

All Changes

https://github.com/dagster-io/dagster/compare/0.15.4...0.15.5

<details><summary>See All Contributors</summary>

• 5192aa0 - Remove unused check_heartbeats arg (#8673) by @johannkm
• 45e5019 - docs: use dagster brand colors for README (#8660) by @rexledesma
• a09234a - fix: use absolute url for README images (#8676) by @rexledesma
• 7a34e49 - Expose TimeWindow in dagster package (#8643) by @nvinhphuc
• 9842ba8 - hold shift to force asset config modal (#8668) by @smackesey
• d0ffb1e - add helm values for configuring instance sensor config (#8657) by @prha
• b335d19 - [dagit] If you have no jobs, prefer routing to asset group over asset catalog (#8613) by @bengotow
• 44bac29 - [dagit] Add control for graph depth on the Asset Lineage page, default to 5 (#8531) by @bengotow
• 593cfc4 - [assets] Fix issue with graph backed assets + partitions (#8682) by @OwenKephart
• eeb23e6 - [dagster-dbt] rework dagster dbt logging, cleanup (#8681) by @OwenKephart
• b6396aa - Yield run requests from run status sensors (#8635) by @jamiedemaria
• f53d5f4 - fix docs build (#8688) by @OwenKephart
• 62801c8 - apidoc for AssetSelection and AssetsDefinition (#8618) by @sryza
• 17b5233 - Materialize has resources arg, materialize_to_memory sets mem_io_manager for all io managers (#8659) by @dpeng817
• 44d95b1 - Load asset launchpad for assets with configurable resources (#8677) by @smackesey
• 01cfe05 - docs: update external links for github issues (#8661) by @rexledesma
• c1cb62b - docs: convert github issue templates to forms (#8663) by @rexledesma
• d338a0e - switch unloadable states to dedupe by selector id (#8656) by @prha
• d3678bc - 0.15.3 changelog (#8690) by @dpeng817
• e0d5dcb - Automation: versioned docs for 0.15.3 by @elementl-devtools
• dc91a2d - docs: remove extra cruft from issue form (#8701) by @rexledesma
• ca432f7 - docs: improve left nav for items with children DREL-359 (#8693) by @yuhan
• b88b33e - docs: fix prev/next pagination (#8697) by @yuhan
• 47118bb - docs: docs test should capture Next Image broken links and fix broken links (#8702) by @yuhan
• 293db00 - [dagit] Display last asset key component on DAG rather than truncated full path (#8692) by @bengotow
• 0d65304 - [dagit] Collapsed DAG rendering of multiple edges between the same ops (#8479) by @bengotow
• 7d5bcc8 - [dagit] Add Analytics context (#8674) by @hellendag
• 3e2fe7e - order the backfill partitions before creating (#8703) by @prha
• 0533455 - skip threadpool sensor daemon tests (#8717) by @prha
• 863ba80 - skip threaded sensor tests in py36 (#8726) by @prha
• dc0fbe8 - Allow run status sensors to support unresolved asset jobs (#8689) by @smackesey
• 979d644 - [dagster_databricks] - support configuration of job / cluster permissions (#8683) by @Phazure
• a9ff7ca - [dagster_databricks] support aws_attributes (#8684) by @Phazure
• 150a06b - [Job Log perf] Use rAF to call throttleSetNodes (#8735) by @salazarm
• 33f059f - [dagster_databricks] support arbitrary env variables (#8685) by @Phazure
• c72634c - minor changes to make dev_install (#8745) by @smackesey
• bc227bb - Remove deprecated daemon health-check cli (#8751) by @johannkm
• a57caf3 - Add reexecution strategy to auto run retries (#8718) by @johannkm
• d90fc4a - Fix for blocks_fn option (#8448) by @clairelin135
• 2d9a9e7 - Automation: versioned docs for 0.15.4 by @elementl-devtools
• d4735dc - Improve race condition in run monitor (#8729) by @brad-alexander
• 6f4b300 - Add get_daemon_statuses instance method (#8752) by @johannkm
• b974184 - 0.15.4 changelog here (#8766) by @prha
• bdca087 - add retention helm values (#8724) by @prha
• 823a5d8 - Op that runs a kubernetes job (#8161) by @gibsondan
• 62f41c4 - [op] fix empty list output bug (#8763) by @alangenfeld
• 4d88332 - [dagster-dbt] Allow SDAs generated with load_assets_from_dbt* to be partitioned (#8725) by @OwenKephart
• 874a368 - add validation for graph backed asset graphs (#8754) by @OwenKephart
• 38ae84b - docs: update config schema descriptions for default executors (#8757) by @rexledesma
• 0779ab7 - [dagster-dbt] Fix error that occurs when generating events for tests that depend on sources (#8775) by @OwenKephart
• 5f92ca1 - Restore sensor daemon yielding when evaluating sensors synchronously (#8756) by @prha
• 4176e42 - bypass bucketed queries for mysql versions that do not support it (#8753) by @prha
• fd1a3d9 - prevent multiple sensor evaluations from multithreaded race conditions (#8720) by @prha
• 4058bb5 - Fold asset_defs submodule into main dagster structure (#8446) by @smackesey
• 87e291e - Fix config case for default executor (#8777) by @dpeng817
• 1544643 - Ensure graph inputs/outputs are included in all_dagster_types (#8736) by @smackesey
• eeeb82c - [dagit] Bring back the global asset graph as an “all asset groups” view (#8709) by @bengotow
• e40122e - fix source asset regression (#8784) by @smackesey
• 2fe13df - improve error for build_schedule_from_partitioned_job with non-partitioned asset job (#8776) by @sryza
• a42146f - fix issue with repos and partitioned scheduled asset jobs (#8779) by @sryza
• 798e9c1 - 0.15.5 Changelog (#8781) by @prha
• 39fecd4 - Fix bug with how resources are applied in materialize (#8790) by @dpeng817
• 8f9f0d0 - 0.15.5 by @elementl-devtools

</details>

Bugfixes

• Reverted sensor threadpool changes from 0.15.3 to address daemon stability issues.

All Changes

https://github.com/dagster-io/dagster/compare/0.15.3...0.15.4

<details><summary>See All Contributors</summary>

• d0bc0b4 - Automation: versioned docs for 0.15.3 by @elementl-devtools
• 73ca70a - Revert "bump timeout for large sensor test (#8671)" by @johannkm
• c988915 - Revert "Add a threadpool to the sensor daemon (#8642)" by @johannkm
• 9441ce3 - 0.15.4 by @elementl-devtools

</details>

New

• When loading an upstream asset or op output as an input, you can now set custom loading behavior using the input_manager_key argument to AssetIn and In
• The list of objects returned by a repository can now contain nested lists.
• Added a data retention instance setting in dagster.yaml that enables the automatic removal of sensor/schedule ticks after a certain number of days.
• Added a sensor daemon setting in dagster.yaml that enables sensor evaluations to happen in a thread pool to increase throughput.
• materialize_to_memory and materialize now both have the partition_key argument.
• Output and DynamicOutput objects now work with deep equality checks:

Output(value=5, name="foo") == Output(value=5, name="foo") # evaluates to True

• RunRequest can now be returned from run status sensors
• Added resource_defs argument to AssetsDefinition.from_graph. Allows for specifying resources required by constituent ops directly on the asset.
• When adding a tag to the Run search filter in Dagit by clicking the hover menu on the tag, the tag will now be appended to the filter instead of replacing the entire filter state.

Bugfixes

• [dagster-dbt] An exception is now emitted if you attempt to invoke the library without having dbt-core installed. dbt-core is now also added as a dependency to the library.
• Asset group names can now contain reserved python keywords
• Fixed a run config parsing bug that was introduced in 0.15.1 that caused Dagit to interpret datetime strings as datetime objects and octal strings as integers.
• Runs that have failed to start are now represented in the Instance Timeline view on Dagit.
• Fixed an issue where the partition status was missing for partitioned jobs that had no runs.
• Fixed a bug where op/resource invocation would error when resources were required, no context was used in the body of the function, and no context was provided when invoking.
• [dagster-databricks] Fixed an issue where an exception related to the deprecated prior_attempts_count field when using the databricks_pyspark_step_launcher.
• [dagster-databricks] Polling information logged from the databricks_pyspark_step_launcher is now emitted at the DEBUG level instead of INFO.
• In the yaml editor in Dagit, the typeahead feature now correctly shows suggestions for nullable schema types.
• When editing asset configuration in Dagit, the “Scaffold config” button in the Dagit launchpad sometimes showed the scaffold dialog beneath the launchpad. This has been fixed.
• A recent change added execution timezones to some human-readable cron strings on schedules in Dagit. This was added incorrectly in some cases, and has now been fixed.
• In the Dagit launchpad, a config state containing only empty newlines could lead to an error that could break the editor. This has been fixed.
• Fixed issue that could cause partitioned graph-backed assets to attempt to load upstream inputs from the incorrect path when using the fs_io_manager (or other similar io managers).
• [dagster-dbt] Fixed issue where errors generated from issuing dbt cli commands would only show json-formatted output, rather than a parsed, human-readable output.
• [dagster-dbt] By default, dagster will invoke the dbt cli with a --log-format json flag. In some cases, this may cause dbt to report incorrect or misleading error messages. As a workaround, it is now possible to disable this behavior by setting the json_log_format configuration option on the dbt_cli_resource to False.
• materialize_to_memory erroneously allowed non-in-memory io managers to be used. Now, providing io managers to materialize_to_memory will result in an error, and mem_io_manager will be provided to all io manager keys.

All Changes

https://github.com/dagster-io/dagster/compare/0.15.2...0.15.3

<details><summary>See All Contributors</summary>

• [docs] - Update screenshots in tutorials by @erinkcochran87 in https://github.com/dagster-io/dagster/pull/8494
• GraphQL Fix indexing on deleted run by @clairelin135 in https://github.com/dagster-io/dagster/pull/8569
• [dagit] Do not load config, repo for assets in DAG views by @bengotow in https://github.com/dagster-io/dagster/pull/8537
• group-based asset dependency resolution by @sryza in https://github.com/dagster-io/dagster/pull/8517
• enable setting partitions_def and config at the same time by @sryza in https://github.com/dagster-io/dagster/pull/8581
• Add community meeting graph-backed asset example by @clairelin135 in https://github.com/dagster-io/dagster/pull/8523
• Add tests for dataframe annotations by @dpeng817 in https://github.com/dagster-io/dagster/pull/8538
• fix: add spacing to build context exception message by @rexledesma in https://github.com/dagster-io/dagster/pull/8589
• [docs] - Add section to testing page on invoking assets by @dpeng817 in https://github.com/dagster-io/dagster/pull/8306
• changelog for 0.15.1 by @sryza in https://github.com/dagster-io/dagster/pull/8582
• [dagit] Repair dialog ordering for confirmation/alert dialogs by @hellendag in https://github.com/dagster-io/dagster/pull/8570
• [dagit] Clean up tab focus on Firefox by @hellendag in https://github.com/dagster-io/dagster/pull/8599
• purge schedule/sensor ticks based on instance config settings by @prha in https://github.com/dagster-io/dagster/pull/8399
• Add cypress data attributes by @salazarm in https://github.com/dagster-io/dagster/pull/8608
• Implement output event equality checking by @dpeng817 in https://github.com/dagster-io/dagster/pull/8595
• Add partition_key arg to materialize_to_memory by @dpeng817 in https://github.com/dagster-io/dagster/pull/8591
• [dagit] Move Asset “Upstream Changed” computation to Python by @bengotow in https://github.com/dagster-io/dagster/pull/8535
• [dagit] Repair cron string regex for timezone by @hellendag in https://github.com/dagster-io/dagster/pull/8605
• docs: convert img html element to next image by @yuhan in https://github.com/dagster-io/dagster/pull/8598
• fix asset dep resolution bug when two assets in group have same name by @sryza in https://github.com/dagster-io/dagster/pull/8609
• refactor: simplify equality check for DynamicOutput by @rexledesma in https://github.com/dagster-io/dagster/pull/8611
• add docs for data retention config by @prha in https://github.com/dagster-io/dagster/pull/8614
• 0.15.2 changelog by @Ramshackle-Jamathon in https://github.com/dagster-io/dagster/pull/8624
• add resource_defs argument to AssetsDefinition.from_graph by @dpeng817 in https://github.com/dagster-io/dagster/pull/8615
• add instance method report_dagster_event by @prha in https://github.com/dagster-io/dagster/pull/8311
• docs: add section in bug report to add Dagster version by @rexledesma in https://github.com/dagster-io/dagster/pull/8631
• Github svg icons by @salazarm in https://github.com/dagster-io/dagster/pull/8633
• [dagit] Repair multi empty line config editor by @hellendag in https://github.com/dagster-io/dagster/pull/8634
• Allow op def / resource def invocation when resources are required, but not used in body of op/resource. by @dpeng817 in https://github.com/dagster-io/dagster/pull/8594
• [dagster-databricks] fix issue with prior_attempts_count by @OwenKephart in https://github.com/dagster-io/dagster/pull/8639
• use a custom yaml loader/dumper for loading and dumping run config, without surprising datetime parsing or octal string behavior by @gibsondan in https://github.com/dagster-io/dagster/pull/8621
• [dagit] Make "Add to filter" an append action by @hellendag in https://github.com/dagster-io/dagster/pull/8626
• GitHub icon by @salazarm in https://github.com/dagster-io/dagster/pull/8647
• Add shell_op job test as an example by @johannkm in https://github.com/dagster-io/dagster/pull/8117
• set start time for runs that have failed to start by @prha in https://github.com/dagster-io/dagster/pull/7972
• Add a threadpool to the sensor daemon by @prha in https://github.com/dagster-io/dagster/pull/8642
• SDA-related updates to resource and partition pages by @sryza in https://github.com/dagster-io/dagster/pull/8620
• [dagster-io/ui] Repair ConfigEditor typeahead for nullables by @hellendag in https://github.com/dagster-io/dagster/pull/8646
• Update text in asset catalog filter by @sryza in https://github.com/dagster-io/dagster/pull/8587
• fix: format files by @rexledesma in https://github.com/dagster-io/dagster/pull/8669
• fix(dagster-dbt): add check for dbt-core before executing CLI commands by @rexledesma in https://github.com/dagster-io/dagster/pull/8666
• Support nested lists in repository def by @jamiedemaria in https://github.com/dagster-io/dagster/pull/8649
• input managers by @jamiedemaria in https://github.com/dagster-io/dagster/pull/8170
• force partition status to return values for missing partitions by @prha in https://github.com/dagster-io/dagster/pull/8637
• bump timeout for large sensor test by @prha in https://github.com/dagster-io/dagster/pull/8671
• allow python keywords in asset group names by @jamiedemaria in https://github.com/dagster-io/dagster/pull/8655

</details>

Bugfixes

• Fixed an issue where asset dependency resolution would break when two assets in the same group had the same name

All Changes

https://github.com/dagster-io/dagster/compare/0.15.1...0.15.2

<details><summary>See All Contributors</summary>

• f78b5ee - Automation: versioned docs for 0.15.1 by @elementl-devtools
• d13e6b4 - fix asset dep resolution bug when two assets in group have same name (#8609) by @sryza
• 552cb44 - 0.15.2 by @elementl-devtools

</details>

New

• When Dagster loads an event from the event log of a type that it doesn’t recognize (for example, because it was created by a newer version of Dagster) it will now return a placeholder event rather than raising an exception.
• AssetsDefinition.from_graph() now accepts a group_name parameter. All assets created by from_graph are assigned to this group.
• You can define an asset from an op via a new utility method AssetsDefinition.from_op. Dagster will infer asset inputs and outputs from the ins/outs defined on the @op in the same way as @graphs.
• A default executor definition can be defined on a repository using the default_executor_def argument. The default executor definition will be used for all op/asset jobs that don’t explicitly define their own executor.
• JobDefinition.run_request_for_partition now accepts a tags argument (Thanks @jburnich!)
• In Dagit, the graph canvas now has a dotted background to help it stand out from the rest of the UI.
• @multi_asset now accepts a resource_defs argument. The provided resources can be either used on the context, or satisfy the io manager requirements of the outs on the asset.
• In Dagit, show execution timezone on cron strings, and use 12-hour or 24-hour time format depending on the user’s locale.
• In Dagit, when viewing a run and selecting a specific step in the Gantt chart, the compute log selection state will now update to that step as well.
• define_asset_job and to_job now can now accept a partitions_def argument and a config argument at the same time, as long as the value for the config argument is a hardcoded config dictionary (not a PartitionedConfig or ConfigMapping)

Bugfixes

• Fixed an issue where entering a string in the launchpad that is valid YAML but invalid JSON would render incorrectly in Dagit.
• Fixed an issue where steps using the k8s_job_executor and docker_executor would sometimes return the same event lines twice in the command-line output for the step.
• Fixed type annotations on the @op decorator (Thanks Milos Tomic!)
• Fixed an issue where job backfills were not displayed correctly on the Partition view in Dagit.
• UnresolvedAssetJobDefinition now supports the run_request_for_partition method.
• Fixed an issue in Dagit where the Instance Overview page would briefly flash a loading state while loading fresh data.

Breaking Changes

• Runs that were executed in newer versions of Dagster may produce errors when their event logs are loaded in older versions of Dagit, due to new event types that were recently added. Going forward, Dagit has been made more resilient to handling new events.

Deprecations

• Updated deprecation warnings to clarify that the deprecated metadata APIs will be removed in 0.16.0, not 0.15.0.

Experimental

• If two assets are in the same group and the upstream asset has a multi-segment asset key, the downstream asset doesn’t need to specify the full asset key when declaring its dependency on the upstream asset - just the last segment.

Documentation

• Added dedicated sections for op, graph, and job Concept docs in the sidenav
• Moved graph documentation from the jobs docs into its own page
• Added documentation for assigning asset groups and viewing them in Dagit
• Added apidoc for AssetOut and AssetIn
• Fixed a typo on the Run Configuration concept page (Thanks Wenshuai Hou!)
• Updated screenshots in the software-defined assets tutorial to match the new Dagit UI
• Fixed a typo in the Defining an asset section of the software-defined assets tutorial (Thanks Daniel Kim!)

All Contributors

https://github.com/dagster-io/dagster/compare/0.15.0...0.15.1

<details><summary>See All Contributors</summary>

• 67147f6 - [docs] - Asset grouping [CON-295] (#8375) by @erinkcochran87
• 0b1c33c - Run retries docs (#8367) by @johannkm
• 6bdeec7 - [dagit] Update the linking between asset groups + jobs (#8377) by @bengotow
• f1bda18 - [docs] - New Jobs & Graphs structure [CON-289] (#8035) by @erinkcochran87
• 6e9a5b1 - remove old asset lineage from docs (#8382) by @sryza
• 1ac7d04 - fix docs lint (#8387) by @gibsondan
• 6b597ff - Patch op definition resolution for assets in GQL (#8384) by @smackesey
• cf7b79f - Remove broken/unused examples from dbt guide (#8383) by @erinkcochran87
• a94ba57 - chore(graphql): remove description linter (#8376) by @rexledesma
• c497b70 - Make asset invocation error if resources conflict (#8390) by @dpeng817
• 5388559 - materialize_in_process -> materialize_to_memory (#8391) by @dpeng817
• c4da0da - apidoc for AssetOut and AssetIn (#8388) by @sryza
• f20939c - Fix problems from "remove old asset lineage from docs (#8382)" (#8389) by @sryza
• ee2dd3e - Changelog 0.15.0 (#8395) by @clairelin135
• ea3023c - [docs] - Update config docs with assets (#8370) by @smackesey
• 09b35ed - [dagit] Fix launch parameters when an asset is partitioned and a graph (#8393) by @bengotow
• 41ea0be - Automation: versioned docs for 0.15.0 by @elementl-devtools
• e7d5c2c - Migration guide 0.15.0 (#8392) by @smackesey
• 04471ad - fix type error (#8379) by @salazarm
• a93c447 - add tags in run_request_for_partition (#8344) by @jburnich
• 46aa277 - Add example asset groups (#8371) by @clairelin135
• 3381d24 - [docs] - Update types to contain assets (#8341) by @clairelin135
• 993ffae - [dagit] Change compute log op when selecting in Gantt (#8397) by @hellendag
• a790cd3 - docs: default to master version when vercel env is not prod (#8396) by @yuhan
• bf5d8be - docs: when gh discussion dont use Link and open in a new tab (#8402) by @yuhan
• d32d2e5 - Add redirect (#8412) by @erinkcochran87
• 390a70c - [ui] Add optional joined prop for buttons (#7491) by @benpankow
• a8598f2 - Fix deserialization of RunsFilter (#8415) by @johannkm
• 06f9a6e - docs: fix slack hooks example (#8403) by @rexledesma
• c4c68ad - Handle errors with an empty stack trace in dagit error viewer (#8427) by @gibsondan
• 200b87f - [dagster-io/ui] Move ConfigEditor to ui (#8413) by @hellendag
• 514f2e0 - [dagster-io/ui] Release v1.0.5 (#8433) by @hellendag
• 2f1f0e5 - [ui] Ensure JoinedButtons works with nested Buttons (#8419) by @benpankow
• c33f1c1 - bump target deprecation versions (#8386) by @sryza
• dc6f99a - [docs] - Add SDA upgrade guide to sidenav (#8437) by @erinkcochran87
• 5721e2f - swap string run config parser from JSON to YAML (#8436) by @gibsondan
• 473fe82 - [dagit] Send YAML insead of parsed JSON (#8443) by @hellendag
• dc10e4d - fix some grammar (#8435) by @prha
• 57d0d70 - add types-proto pin (#8447) by @gibsondan
• 3b26aae - [docs] - Clarify the relationship between ops and assets on op concept page (#8409) by @sryza
• 70e2512 - [dagit] Repair loading state on Instance Overview (#8450) by @hellendag
• 20a5c9f - fix typo (#8451) by @clairelin135
• ffbdb0a - Update runtime type checks in resource definition invocation (#8432) by @gibsondan
• af50d0c - remove dagster io repo mention (#8459) by @yuhan
• 154e77f - Update docs/content/concepts/configuration/configured.mdx (#8464) by @wenhoujx
• 3bb4e92 - fix(dagster/decorators): op_decorator documentation update (#8179) by @mict0
• 3dc36d0 - [dagit] “Canvas” dots on the DAGs 🎨 (#8455) by @bengotow
• 196f6e4 - [dbt-python-assets] demo repository (#8460) by @OwenKephart
• 642196b - [dagit] Fix UI issues around graph backed assets (#8458) by @bengotow
• 65730eb - [dagit] Repair empty config case in Launchpad (#8483) by @hellendag
• d1c75c5 - [dagster-io/ui] MaybeTooltip (#8445) by @hellendag
• afc44ef - add backfills for jobs (#8496) by @prha
• 1c94d14 - fix lint error in types concepts page (#8495) by @sryza
• ec8b6d1 - post-0.15.0 docs fixups (#8470) by @sryza
• bbf601d - [hopefully easy] annotations of resource_defs become mapping (#8501) by @dpeng817
• 18d8c82 - remove unsupported python version for python dbt example (#8503) by @sryza
• 3beed66 - Accept a group_name in AssetsDefinition.from_graph() (#8406) by @shalabhc
• bfb0ea4 - Clarify points in direct operation section (#8497) by @erinkcochran87
• fb489ea - [dagit] Remove code for a disabled layout feature (#8478) by @bengotow
• 49f7e2b - [dagit] Loosen the state machine expectations around run steps (#8480) by @bengotow
• 73cdf31 - Add constant for default io manager key (#8506) by @dpeng817
• 906aa43 - annotations of logger_defs become mapping (#8502) by @dpeng817
• 2e8f365 - [dagit] Add timezone and 24h format to humanCronString (#8520) by @hellendag
• 26f7fec - Fix module structure on deploying docs examples (#8511) by @dpeng817
• 283d643 - Make default_executor_def on repo work on jobs (#8490) by @dpeng817
• 8ba8735 - refactor stephandler API to support the same yield+write pattern as other executors (#8357) by @gibsondan
• b5db3ef - graphql test case for a memoization job (#8442) by @gibsondan
• 71b6d4f - [dagit] Collapse assets in Run header if many of them (#8533) by @hellendag
• 830cfba - Fallback to a placeholder event on a new event type that we don't recognize instead of raising an exception (#8542) by @gibsondan
• c2d5da3 - solid to op in graph error (#8539) by @sryza
• 483c276 - new typed events for step process start/end and resource init/teardown (#8368) by @gibsondan
• 2e15db4 - Use status column for reading runs from sql storage (#8528) by @johannkm
• b480254 - [ui] Add ability to supply TokenizingField with raw tokens (#8444) by @benpankow
• c578e58 - resources_init mypy (#8536) by @dpeng817
• aca896d - fix executor test (#8566) by @gibsondan
• 0446281 - AssetsDefinition.from_op (#8508) by @clairelin135
• 3373106 - Add resource_defs arg to multi_assets (#8529) by @dpeng817
• c695fe1 - Add run_request_for_partition to UnresolvedAssetJobDefinition (#8441) by @smackesey
• 14f32bc - [docs] - Update screenshots in tutorials (#8494) by @erinkcochran87
• eb0b3c1 - group-based asset dependency resolution (#8517) by @sryza
• 534a619 - enable setting partitions_def and config at the same time (#8581) by @sryza
• fa7acc5 - changelog for 0.15.1 (#8582) by @sryza
• d66a5fe - 0.15.1 by @elementl-devtools

</details>

Major Changes

• Software-defined assets are now marked fully stable and are ready for prime time - we recommend using them whenever your goal using Dagster is to build and maintain data assets.
• You can now organize software defined assets into groups by providing a group_name on your asset definition. These assets will be grouped together in Dagit.
• Software-defined assets now accept configuration, similar to ops. E.g.

  from dagster import asset
  @asset(config_schema={"iterations": int})
  def my_asset(context):
      for i in range(context.op_config["iterations"]):
          ...
  

• Asset definitions can now be created from graphs via AssetsDefinition.from_graph:

  @graph(out={"asset_one": GraphOut(), "asset_two": GraphOut()})
  def my_graph(input_asset):
      ...
  graph_asset = AssetsDefinition.from_graph(my_graph)
  

• execute_in_process and GraphDefinition.to_job now both accept an input_values argument, so you can pass arbitrary Python objects to the root inputs of your graphs and jobs.
• Ops that return Outputs and DynamicOutputs now work well with Python type annotations. You no longer need to sacrifice static type checking just because you want to include metadata on an output. E.g.

  from dagster import Output, op
  @op
  def my_op() -> Output[int]:
      return Output(5, metadata={"a": "b"})
  

• You can now automatically re-execute runs from failure. This is analogous to op-level retries, except at the job level.
• You can now supply arbitrary structured metadata on jobs, which will be displayed in Dagit.
• The partitions and backfills pages in Dagit have been redesigned to be faster and show the status of all partitions, instead of just the last 30 or so.
• The left navigation pane in Dagit is now grouped by repository, which makes it easier to work with when you have large numbers of jobs, especially when jobs in different repositories have the same name.
• The Asset Details page for a software-defined asset now includes a Lineage tab, which makes it easy to see all the assets that are upstream or downstream of an asset.

Breaking Changes and Deprecations

Software-defined assets

This release marks the official transition of software-defined assets from experimental to stable. We made some final changes to incorporate feedback and make the APIs as consistent as possible:

• Support for adding tags to asset materializations, which was previously marked as experimental, has been removed.
• Some of the properties of the previously-experimental AssetsDefinition class have been renamed. group_names is now group_names_by_key, asset_keys_by_input_name is now keys_by_input_name, and asset_keys_by_output_name is now keys_by_output_name, asset_key is now key, and asset_keys is now keys.
• Removes previously experimental IO manager fs_asset_io_manager in favor of merging its functionality with fs_io_manager. fs_io_manager is now the default IO manager for asset jobs, and will store asset outputs in a directory named with the asset key. Similarly, removed adls2_pickle_asset_io_manager, gcs_pickle_asset_io_manager , and s3_pickle_asset_io_manager. Instead, adls2_pickle_io_manager, gcs_pickle_io_manager, and s3_pickle_io_manager now support software-defined assets.
• (deprecation) The namespace argument on the @asset decorator and AssetIn has been deprecated. Users should use key_prefix instead.
• (deprecation) AssetGroup has been deprecated. Users should instead place assets directly on repositories, optionally attaching resources using with_resources. Asset jobs should be defined using define_assets_job (replacing AssetGroup.build_job), and arbitrary sets of assets can be materialized using the standalone function materialize (replacing AssetGroup.materialize).
• (deprecation) The outs property of the previously-experimental @multi_asset decorator now prefers a dictionary whose values are AssetOut objects instead of a dictionary whose values are Out objects. The latter still works, but is deprecated.
• The previously-experimental property on OpExecutionContext called output_asset_partition_key is now deprecated in favor of asset_partition_key_for_output

Event records

• The get_event_records method on DagsterInstance now requires a non-None argument event_records_filter. Passing a None value for the event_records_filter argument will now raise an exception where previously it generated a deprecation warning.
• Removed methods events_for_asset_key and get_asset_events, which have been deprecated since 0.12.0.

Extension libraries

• [dagster-dbt] (breaks previously-experimental API) When using the load_assets_from_dbt_project or load_assets_from_dbt_manifest , the AssetKeys generated for dbt sources are now the union of the source name and the table name, and the AssetKeys generated for models are now the union of the configured schema name for a given model (if any), and the model name. To revert to the old behavior: dbt_assets = load_assets_from_dbt_project(..., node_info_to_asset_key=lambda node_info: AssetKey(node_info["name"]).
• [dagster-k8s] In the Dagster Helm chart, user code deployment configuration (like secrets, configmaps, or volumes) is now automatically included in any runs launched from that code. Previously, this behavior was opt-in. In most cases, this will not be a breaking change, but in less common cases where a user code deployment was running in a different kubernetes namespace or using a different service account, this could result in missing secrets or configmaps in a launched run that previously worked. You can return to the previous behavior where config on the user code deployment was not applied to any runs by setting the includeConfigInLaunchedRuns.enabled field to false for the user code deployment. See the Kubernetes Deployment docs for more details.
• [dagster-snowflake] dagster-snowflake has dropped support for python 3.6. The library it is currently built on, snowflake-connector-python, dropped 3.6 support in their recent 2.7.5 release.

Other

• The prior_attempts_count parameter is now removed from step-launching APIs. This parameter was not being used, as the information it held was stored elsewhere in all cases. It can safely be removed from invocations without changing behavior.
• The FileCache class has been removed.
• Previously, when schedules/sensors targeted jobs with the same name as other jobs in the repo, the jobs on the sensor/schedule would silently overwrite the other jobs. Now, this will cause an error.

New since 0.14.20

• A new define_asset_job function allows you to define a selection of assets that should be executed together. The selection can be a simple string, or an AssetSelection object. This selection will be resolved into a set of assets once placed on the repository.

  from dagster import repository, define_asset_job, AssetSelection
      
  string_selection_job = define_asset_job(
      name="foo_job", selection="*foo"
  )
  object_selection_job = define_asset_job(
      name="bar_job", selection=AssetSelection.groups("some_group")
  )
  @repository
  def my_repo():
      return [
          *my_list_of_assets,
          string_selection_job,
          object_selection_job,
      ]
  

• [dagster-dbt] Assets loaded with load_assets_from_dbt_project and load_assets_from_dbt_manifest will now be sorted into groups based on the subdirectory of the project that each model resides in.

• @asset and @multi_asset are no longer considered experimental.

• Adds new utility methods load_assets_from_modules, assets_from_current_module, assets_from_package_module, and assets_from_package_name to fetch and return a list of assets from within the specified python modules.

• Resources and io managers can now be provided directly on assets and source assets.

  from dagster import asset, SourceAsset, resource, io_manager
        
  @resource
  def foo_resource():
      pass
      
  @asset(resource_defs={"foo": foo_resource})
  def the_resource(context):
      foo = context.resources.foo
      
  @io_manager
  def the_manager():
      ...
      
  @asset(io_manager_def=the_manager)
  def the_asset():
      ...
  

  Note that assets provided to a job must not have conflicting resource for the same key. For a given job, all resource definitions must match by reference equality for a given key.

• A materialize_to_memory method which will load the materializations of a provided list of assets into memory:

  from dagster import asset, materialize_to_memory
        
  @asset
  def the_asset():
      return 5
      
  result = materialize_to_memory([the_asset])
  output = result.output_for_node("the_asset")
  

• A with_resources method, which allows resources to be added to multiple assets / source assets at once:

  from dagster import asset, with_resources, resource
      
  @asset(required_resource_keys={"foo"})
  def requires_foo(context):
      ...
      
  @asset(required_resource_keys={"foo"})
  def also_requires_foo(context):
      ...
      
  @resource
  def foo_resource():
      ...
      
  requires_foo, also_requires_foo = with_resources(
      [requires_foo, also_requires_foo], 
      {"foo": foo_resource},
  ) 
  

• You can now include asset definitions directly on repositories. A default_executor_def property has been added to the repository, which will be used on any materializations of assets provided directly to the repository.

  from dagster import asset, repository, multiprocess_executor
  @asset
  def my_asset():
    ...
  @repository(default_executor_def=multiprocess_executor)
  def repo():
      return [my_asset]
  

• The run_storage, event_log_storage, and schedule_storage configuration sections of the dagster.yaml can now be replaced by a unified storage configuration section. This should avoid duplicate configuration blocks with your dagster.yaml. For example, instead of:

  # dagster.yaml
  run_storage:
  module: dagster_postgres.run_storage
  class: PostgresRunStorage
  config:
      postgres_url: { PG_DB_CONN_STRING }
  event_log_storage:
  module: dagster_postgres.event_log
  class: PostgresEventLogStorage
  config:
      postgres_url: { PG_DB_CONN_STRING }
  schedule_storage:
  module: dagster_postgres.schedule_storage
  class: PostgresScheduleStorage
  config:
      postgres_url: { PG_DB_CONN_STRING }
  

  You can now write:

  storage:
    postgres:
      postgres_url: { PG_DB_CONN_STRING }
  

• All assets where a group_name is not provided are now part of a group called default.

• The group_name parameter value for @asset is now restricted to only allow letters, numbers and underscore.

• You can now set policies to automatically retry Job runs. This is analogous to op-level retries, except at the job level. By default the retries pick up from failure, meaning only failed ops and their dependents are executed.

• [dagit] The new repository-grouped left navigation is fully launched, and is no longer behind a feature flag.

• [dagit] The left navigation can now be collapsed even when the viewport window is wide. Previously, the navigation was collapsible only for small viewports, but kept in a fixed, visible state for wide viewports. This visible/collapsed state for wide viewports is now tracked in localStorage, so your preference will persist across sessions.

• [dagit] Queued runs can now be terminated from the Run page.

• [dagit] The log filter on a Run page now shows counts for each filter type, and the filters have higher contrast and a switch to indicate when they are on or off.

• [dagit] The partitions and backfill pages have been redesigned to focus on easily viewing the last run state by partition. These redesigned pages were previously gated behind a feature flag — they are now loaded by default.

• [dagster-k8s] Overriding labels in the K8sRunLauncher will now apply to both the Kubernetes job and the Kubernetes pod created for each run, instead of just the Kubernetes pod.

Bugfixes

• [dagster-dbt] In some cases, if Dagster attempted to rematerialize a dbt asset, but dbt failed to start execution, asset materialization events would still be emitted. This has been fixed.
• [dagit] On the Instance Overview page, the popover showing details of overlapping batches of runs is now scrollable.
• [dagit] When viewing Instance Overview, reloading a repository via controls in the left navigation could lead to an error that would crash the page due to a bug in client-side cache state. This has been fixed.
• [dagit] When scrolling through a list of runs, scrolling would sometimes get stuck on certain tags, specifically those with content overflowing the width of the tag. This has been fixed.
• [dagit] While viewing a job page, the left navigation item corresponding to that job will be highlighted, and the navigation pane will scroll to bring it into view.
• [dagit] Fixed a bug where the “Scaffold config” button was always enabled.

Community Contributions

• You can now provide dagster-mlflow configuration parameters as environment variables, thanks @chasleslr!

Documentation

• Added a guide that helps users who are familiar with ops and graphs understand how and when to use software-defined assets.
• Updated and reorganized docs to document software-defined assets changes since 0.14.0.
• The Deploying in Docker example now includes an example of using the docker_executor to run each step of a job in a different Docker container.
• Descriptions for the top-level fields of Dagit GraphQL queries, mutations, and subscriptions have been added.

All Changes

https://github.com/dagster-io/dagster/compare/0.14.0...0.15.0

<details><summary>See All Contributors</summary>

• fcba344 - add fallback error state for snapshot types (#6671) by @prha
• 43835eb - Getting Started and Basic Tutorial fixes (#6673) by @jmsanders
• 5db89f5 - run timeline blank state (#6674) by @prha
• 7b0b436 - [multiproc] default to spawn (#6676) by @alangenfeld
• c71062f - Asset Observations Doc (#6630) by @clairelin135
• d2117a4 - Restore missing repo.py file (#6677) by @gibsondan
• 85aff95 - [dagster-aws] Fix bug when logging records from pyspark_step_launcher (#6680) by @OwenKephart
• 360f7b2 - Automation: versioned docs for 0.14.0 by @elementl-devtools
• a4fd446 - Changelog (#6667) by @clairelin135
• 67b0426 - Fix formatting on schedules feature in changelog (#6686) by @gibsondan
• fb38c93 - Pin grpcio-health-checking (#6685) by @johannkm
• eb65568 - fix changelog page in docs (#6694) by @yuhan
• 1f79613 - redirect ecs page (#6695) by @yuhan
• 93b0bf9 - Remove crons from our example dockerfiles (#6645) by @gibsondan
• 787c7ca - pin markupsafe (#6713) by @alangenfeld
• 95333ec - add cursor label to tick timeline (#6712) by @prha
• 3583858 - whitelist_for_serdes WorkerStatus enum (#6716) by @johannkm
• e81374a - docstrings for AssetGroup and AssetGroup.build_job (#6696) by @dpeng817
• b65900c - 0.14.1 changelog (#6720) by @alangenfeld
• fe61ba5 - Remove bad invariant from back when sensor status was a fixed status, not a default status (#6711) by @gibsondan
• be11e4a - Automation: versioned docs for 0.14.1 by @elementl-devtools
• 9eba0d0 - Add a check in run execution that the run actually exists, just like in step execution (#6689) by @gibsondan
• 19b106c - add back mutation tests for sensor/schedules with default status (#6721) by @prha
• 47e8533 - Export build_reconstructable_target to match cragified docs (#6317) by @prha
• 46c3e0b - add build_run_status_sensor_context (#6543) by @jamiedemaria
• f55ef00 - Rename metadata (remove "Event") in graphql/dagit (#6656) by @smackesey
• 8cba839 - Speed up get_assets_for_run_id when there are a lot of events (#6735) by @gibsondan
• ef56abe - undefined (#6742) by @alangenfeld
• fe88147 - Fix invariant check in databricks job/pipeline path (#6718) by @iswariyam
• cded56d - make Content Gap an issue label instead of part of title (#6745) by @sryza
• 229578c - 0.14.0 migration.md (#6715) by @clairelin135
• 091702a - Move bollinger example to AssetGroup and add README (#6749) by @sryza
• d7d1351 - move software-defined assets toy to asset collection (#6643) by @sryza
• 640aa5d - rename conditional branching example to nyt and add README (#6757) by @sryza
• b2b2672 - [docs infra] remove old versions from versioned_navigation (#6704) by @yuhan
• 0c55d96 - fix typo in ops rst (#6758) by @yuhan
• e83b11c - update error messages for ExternalAssetDependency/ExternalAssetDependedBy (#6748) by @OwenKephart
• 38b8283 - [Docs] Upgrade Next.js to 12.1.0 (#6722) by @keyz
• e602e45 - [docs infra] fix changelog's On This Page (#6706) by @yuhan
• da26522 - fix(helm): remove registry field (#6744) by @rexledesma
• 2b6ded6 - docs(helm): update chart metadata (#6778) by @rexledesma
• e53dec4 - respect default_value in loadable input check (#6771) by @alangenfeld
• f470ea9 - display non-sda assets in asset catalog (#6779) by @prha
• b26ba51 - replace EventMetadata in docs (#6774) by @smackesey
• a360bbd - Migrate user event docs to use log_event, various docs cleanups (#6772) by @dpeng817
• 5cec1c8 - [dagster-dbt] fix issues with dbt asset integration (#6777) by @OwenKephart
• 0ed9ad7 - Filter custom events from stats_for_run query (#6781) by @johannkm
• 90faed0 - [Dagit] Fix missing op name in the asset sidebar (#6765) by @bengotow
• 83643b2 - add validate_run_config to api docs (#6782) by @prha
• 472c4dc - add api doc for op_def, graph_def (#6776) by @prha
• 64095bd - [examples] dbt -> dbt-core in dbt_example (#6786) by @OwenKephart
• 5f3966f - [dagit] Show full asset paths on details pages, tweak filter interactions (#6764) by @bengotow
• 7495fea - Add small toggles to Asset Details graphs, consolidate localStorage-backed state (#6762) by @bengotow
• f700471 - [dagster-shell] Fix deadlock with large output (#6435) (#6490) by @kbd
• e574200 - [dagster-dbt] pin some stuff (#6791) by @OwenKephart
• 50eda7a - deflake test_retry_policy (#6644) by @alangenfeld
• f17a1fa - 0.14.2 changelog (#6789) by @sryza
• d90d1dd - add run_request_for_partition for job definition (#6790) by @prha
• 4d89075 - Add a "log framework error" hook to execute_step invocations, similar to the one in execute_run (#6690) by @gibsondan
• 82c80c3 - fix up reexecution api example docs (#6787) by @prha
• 7b3dde3 - reorganize dagster.check module (#6750) by @smackesey
• 1a3f5c1 - Adjustments/condensing of dagster.check (#6751) by @smackesey
• b4a1715 - Automation: versioned docs for 0.14.2 by @elementl-devtools
• 308429b - fix: use run status to determine success for an in-process result (#6784) by @rexledesma
• 47b9a68 - namedtuple to NamedTuple (1st batch) (#6752) by @smackesey
• 44f4f2a - corrected export message from "~/dagster_home" to ~"/dagster_home" (#6327) by @Proteusiq
• a1acd3f - docs: remove duplicate entries under integrations (#6724) by @kahnwong
• 032f442 - namedtuple to NamedTuple (2nd batch) (#6753) by @smackesey
• e0c66db - namedtuple to NamedTuple (3rd batch) (#6767) by @smackesey
• 6327300 - namedtuple to NamedTuple (4th batch) (#6785) by @smackesey
• 1f5a7ec - [docs infra] speed up vercel build with ISR (#6773) by @yuhan
• ce7441c - [docs infra] no s3 when pulling versioned content (#6705) by @yuhan
• ee07c9c - Still write cursors even when a sensor tick fails (#6807) by @gibsondan
• ef3ad62 - Direct invocation of assets for unit testing (#6761) by @jamiedemaria
• 6098bf0 - Partition docs updates (#6814) by @jamiedemaria
• 779e27f - fix pandera link on integrations page (#6813) by @smackesey
• 6ac6349 - [docs] regenerate yarn.lock (#6815) by @alangenfeld
• ea19544 - [events] remove message field (#6769) by @alangenfeld
• c157260 - [dagster-postgres] improve connection retry (#6831) by @alangenfeld
• ca94d8d - set ErrorSource when retry request limit exceeded (#6806) by @alangenfeld
• 4dbfa80 - Update black, isort (#6714) by @smackesey
• 9dd063a - Exclude materializing run from materialization warning run count (#6692) by @clairelin135
• 63556df - rename PipelineRunsFilter => RunsFilter (#6811) by @prha
• c1a6018 - assorted function type annotations (batch 1) (#6800) by @smackesey
• ab08410 - restore old metadata name deprecation warnings (#6688) by @smackesey
• a3ecc86 - Also override cpu/memory in containerOverrides (#6836) by @jmsanders
• f330e26 - move toys from build_assets_job to AssetGroup (#6801) by @sryza
• 3408263 - Simplify IWorkspace API (#6821) by @gibsondan
• 0158c07 - [dagster-aws] Add secretsmanager resources (#6802) by @benpankow
• d98aa5c - chore(daemon): retrieve daemon heartbeats in batch (#6843) by @rexledesma
• 5b10719 - Revert isort Makefile config to use git ls-files (#6845) by @smackesey
• 6092f3e - fix: query heartbeats db once (#6854) by @rexledesma
• 42d2d72 - [examples] Modern Data Stack + SDA Example (#6862) by @OwenKephart
• f0323af - add table api doc page (#6840) by @smackesey
• de1b5ef - [dagit] Avoid copying definition-time tags into launchpad state (#6851) by @bengotow
• 7269df9 - [dagit] Replace black tag tooltips with as-needed, selectable ones (#6844) by @bengotow
• 4d4d6a7 - default to fs_asset_io_manager in AssetGroup (#6859) by @sryza
• e9834b4 - move slack apidoc example to op/job (#6855) by @sryza
• 0ac9e1c - Remove @abstractproperty (#6867) by @johannkm
• fe4a530 - gate bucketing queries on sqlalchemy version (#6841) by @prha
• 8f29bf9 - Don't carry forward all container definitions (#6850) by @jmsanders
• 8b7b27d - when multiple repos have the same asset, have graphql return the non-source one (#6856) by @sryza
• 3067d32 - [dagit] Add a helper for building instance/runs paths, resolved #6847 (#6848) by @bengotow
• e6fb91b - [dagster-fivetranAdd Fivetran Resync Op (#6868) by @dwallace0723
• 9b08239 - [dagit] Use a custom hook to manage query polling (#6805) by @bengotow
• cbc48e8 - enable cross-job asset partitions (#6865) by @sryza
• c808d3b - [dagit] Add a gradient behind the Gantt chart step selector now by @souterjk
• e33302a - remove gitignore setting from isort (#6872) by @smackesey
• 7601eab - Random formatting fixes (#6888) by @dpeng817
• 8467f5c - AssetGroup.from_modules (#6884) by @sryza
• 6299284 - isort dagster-aws conftest (#6892) by @sryza
• c14ac93 - fix expansion so its shell-neutral (#6894) by @smackesey
• 6b57eb6 - build_assets_job -> AssetGroup in SDA guide (#6879) by @sryza
• 4b27d8b - Changelog (#6908) by @dpeng817
• c577ec5 - add entries to 0.14.3 changelog (#6909) by @sryza
• d894501 - Rename build_run_status_sensor_context in changelog (#6904) by @gibsondan
• e4fabe9 - make EventLogEntry.message change safe to downgrade (#6912) by @alangenfeld
• f8319ea - For those whome detest wordwrap in vscode (#6887) by @dpeng817
• e3e2143 - Solid error message migration (#6883) by @dpeng817
• 919fb31 - Error when amultiple asset groups are used in one repo, when a job is passed in with the asset group reserved name (#6891) by @dpeng817
• 498b261 - assorted sensor and schedule fixes (#6759) by @jamiedemaria
• 7f35164 - [examples] For modern_data_stack_assets, more detailed README, helper script (#6914) by @OwenKephart
• 28fe827 - fix bug with subset selection (#6921) by @OwenKephart
• 7fa5308 - enable vixie-style cron strings (#6810) by @prha
• 66bdcff - Automation: versioned docs for 0.14.3 by @elementl-devtools
• 66f5ce6 - In run status sensors, snapshot the status in a PipelineRunReaction rather than pulling it from the PipelineRun object (#6931) by @gibsondan
• 7ea0c61 - [dagster-airbyte][examples] fix bug with empty prefix (#6932) by @OwenKephart
• c52b495 - RFC: resolve k8s job and ops tag entirely using k8s_model_from_dict (#6205) by @gibsondan
• 096a17d - revert stray dagster[lint] for BK mypy (#6936) by @smackesey
• bfcc59b - Migrate reference to solid in error message (#6935) by @dpeng817
• 8b5a4ca - dagit-debug fix in-mem daemon heartbeats (#6943) by @alangenfeld
• 9168e73 - [docs] details on mapping dynamic outputs (#6140) by @alangenfeld
• 66ae1c0 - [docs] op retries concepts section (#6818) by @alangenfeld
• 4145d10 - fix makefile issue with check_black/check_isort (#6944) by @smackesey
• cf3f6d2 - change query to be compatible with sqlalchemy 1.3 (#6930) by @prha
• c4fe9a6 - add a blank state message for the asset table (#6792) by @prha
• 81fdc76 - fix black by @alangenfeld
• ca7aedf - Remove update_doc_snapshot from root Makefile (#6837) by @smackesey
• d1d6003 - namedtuple to NamedTuple (5th batch) (#6917) by @smackesey
• 3c2a0b5 - function type annotations (batch 2) (#6933) by @smackesey
• cddc63c - when building AssetGroups from modules, gather lists of assets (#6967) by @sryza
• 5bfadb9 - remove legacy version of hacker news schedule (#6797) by @sryza
• c193ccd - ungate run batching (#6966) by @prha
• 803063e - Use AssetGroup in hacker news demo (#6965) by @sryza
• 4262be1 - AssetGroup.from_current_module (#6959) by @sryza
• 3f5aeb7 - [Dagit] Nits and polish based on user feedback (#6799) by @bengotow
• 9dc310c - add support for filtering ticks by status (#6919) by @prha
• 2910338 - [dagit] Only warn about the backfill daemon when backfills are present (#6846) by @bengotow
• ed210ae - switch arg name for metadata value class (#6675) by @smackesey
• 3df7e14 - [dagit] Use a virtualized list in <SuggestWIP /> to fix launchpad partition picker perf (#6852) by @bengotow
• 63f6048 - Fix missing partition bars in large partition data-sets (#6877) by @souterjk
• 07f2386 - Quote selection to avoid grabbing downstreams with the same name prefix when viewing upstream graph, fix react table unhappy log (#6907) by @souterjk
• 37be307 - make all optional params explicit (#6934) by @smackesey
• 2649808 - Adds pez popover as requested in #6470 (#6871) by @souterjk
• e703ac6 - fix pylint for modern data stack example (#6975) by @smackesey
• 73ec3a2 - dev_install tweaks (#6970) by @smackesey
• 9e1fa57 - partitioned assets toy (#6960) by @sryza
• a242d6b - Update create-new-project.mdx (#6898) by @frcode
• e0dc13b - Add short-circuit in run start time if run failed to start (#6991) by @gibsondan
• 02679a1 - Allow PathMetadataValue to accept os.PathLike input (#6920) by @abkfenris
• 68b2746 - Raise run_task errors (#6993) by @jmsanders
• fae347c - [dagit] React.FunctionComponent -> React.FC (#7002) by @hellendag
• b86b0f1 - Fix partitioned asset jobs with double downstream non-partitioned (#6986) by @sryza
• 7542130 - add ipython_genutils as dagit, dagstermill dep (#7016) by @smackesey
• 64033df - Set up graphql mutation for setting a sensor cursor value (#6922) by @prha
• 02524aa - [dagit] Fix warning icon tests, add one for present-but-stopped schedule (#7026) by @bengotow
• 06f6fa8 - Stop building duplicate downstream asset dependencies (#7001) (#7012) by @bengotow
• 5d5d2b0 - [dagit] Add error handling to Asset "Rematerialize Partitions" modal (#7011) by @bengotow
• 07ce468 - [dagit] Fix tooltips in Gantt chart, use monospace font and middle truncation (#7025) by @bengotow
• 16b7b4d - [dagit] Fix right Asset Graph panel missing if saved "explorer" state is absent (#7013) by @bengotow
• 1dc6cf5 - Fix dagster k8s typo (#7028) by @johannkm
• 1cb6d56 - Add sequence and mapping check functions (#7006) by @smackesey
• b0a0dc2 - change backcompat pin to nbconvert (#7027) by @smackesey
• 57f44c0 - [dagit] Hide the asset group job on the instance overview (#7019) by @bengotow
• 3d22aef - [dagit] When materializing assets, link to run instead of showing it (#7014) by @bengotow
• feb1c5e - Assorted typing additions and adjustments (#7007) by @smackesey
• ce4b5ae - remove unneeded deps from dagit tox file (#7022) by @gibsondan
• daa1188 - Switch sensor/schedule view to show tick list instead of run list (#6861) by @prha
• 336ee6b - update AssetGraph dedup logic (#7021) by @smackesey
• b2df1b0 - Make parse_run_config_input more defensive (#7003) by @gibsondan
• 809665e - Expose metadata on AssetNode (and source assets) directly (#6900) by @smackesey
• 94c8e7f - adds source asset metadata to dagit (#7015) by @smackesey
• 1244427 - Adllow telemetry URL to be overridden with env var (#6946) by @dpeng817
• 79827ef - Remove space from dbt example notebook path (#7038) by @smackesey
• 52b1527 - add tick batching to schedule storage (#6995) by @prha
• 6e7cd25 - add batch tick loader for repository-scoped schedule/sensor ticks (#6997) by @prha
• b60c070 - K8s executor filter down to dagster events (#7000) by @johannkm
• 8e6bccd - Allow the use of environment variables in k8s_job_executor (#7031) by @johannkm
• c7bfb9c - [bug] hacky fix for step launcher behavior (#6866) by @OwenKephart
• 1327a6d - make resource config available when loading from assets outside job (#7029) by @sryza
• a88e658 - Rename step jobs 'dagster-step' in k8s and docker (#6982) by @johannkm
• 756527a - minor improvements to check (#7036) by @smackesey
• 28d74c3 - test raising exceptions in AssetGroup in deterministic order (#6945) by @jamiedemaria
• 07c3887 - SDA docs fixes (#6905) by @jamiedemaria
• d9b79ec - skip test_giant_external_repository_streaming_grpc (#7049) by @alangenfeld
• 43cf866 - Fault tolerance docs (#5432) by @johannkm
• 396da6c - Silence experimental warnings triggered by internal code (#6873) by @jamiedemaria
• 8ad1195 - K8s docs ignore tag and show executor (#7053) by @johannkm
• aa5712e - Assorted fixes to the hackernews assets demo (#7030) by @sryza
• b2fe162 - Support the same Secrets config syntax as ECS (#6985) by @jmsanders
• 8045d57 - fix the sqlite gating logic (#7058) by @prha
• 8d0319a - [dagster-airbyte] [dagster-fivetran] cleanup + tests (#7056) by @OwenKephart
• 07e3403 - Reuse ECS task definitions (#7009) by @jmsanders
• 6bbfce2 - Update last_materialization_timestamp to also update on observation (#6885) by @clairelin135
• d906c85 - Optimize Latest Runs Query (#7066) by @clairelin135
• 8a611ae - [dagit] Remove default InstanceWarning mocks injecting randomness (#7054) by @bengotow
• 7a78835 - fix tag batching with tag-based filters (#7055) by @prha
• 75b48ea - Fix #6512 (#6969) by @petehunt
• ee146ef - K8s check run health test (#7070) by @johannkm
• 0f61c12 - hn local repo that uses duckdb as warehouse (#6968) by @sryza
• b3e776b - fix bug in fetch_assets.get_asset_nodes_by_asset_key (#7063) by @sryza
• 9941cbc - Clear out 'dependsOn' when creating a default ECS task definition (#6929) by @gibsondan
• f6baac7 - [dagit] Render status page navigation when factory floor loading (#7079) by @benpankow
• adcf93f - run mypy on dagster-graphql (#7075) by @prha
• 2e2029d - fix logic for turning schedule ticks into hour boundaries when there's an offset (#7071) by @gibsondan
• 3df4da6 - Fixes #7082 (#7086) by @smackesey
• 61c43eb - Fix failing check.path test on windows (#7077) by @smackesey
• 15b5ad9 - [dagit] refactor graphql execute methods (#7065) by @alangenfeld
• 47f8fbc - dagster-snowflake drop 3.6 support (#7092) by @alangenfeld
• 862fe7e - [dagit] Fix Gantt chart layout hang on certain graph structures #6942 (#7085) by @bengotow
• 69f5e3b - Log failed compute log to event log rather than buried in a logging output (#7093) by @gibsondan
• e282d44 - [dagit] Disallow use of gql from graphql.macro (#7042) by @bengotow
• 72807a1 - [dagit] Add sensors/schedules to the asset details page (#7041) by @bengotow
• 7bcdd05 - [dagit] Keep user-entered tags when changing presets / partitions (#7068) by @bengotow
• 17313fe - Add gitpod.yml (#7088) by @petehunt
• f32802e - [dagit] Lower factory floor timeline fidelity to seconds from ms (#7091) by @benpankow
• d07a108 - [dagit] Show job failures on asset graph, fix in-progress runs on global graph (#7067) by @bengotow
• 5765c38 - Optionally use sidecars (#7087) by @jmsanders
• aa1dc4a - Fix kwarg case with invocation (#7095) by @dpeng817
• 3f6458c - Return an empty secrets list for task definitions (#7096) by @jmsanders
• eff26c4 - Fixup ECS secrets docs (#7097) by @jmsanders
• 8c3cfd2 - 0.14.4 changelog (#7099) by @yuhan
• ffd8318 - refetch the backfill before updating, to avoid clobbering cancels (#7094) by @prha
• 0c72a16 - Automation: versioned docs for 0.14.4 by @elementl-devtools
• 1c7f030 - fix perms in dagit (#7115) by @prha
• 24290ad - Remove hard failure if missing inputs (#7116) by @dpeng817
• 5c2d0bf - Fix internal code metadata warnings (#7112) by @smackesey
• 40fdc1e - 0.14.5 Changelog (#7119) by @jmsanders
• e1209fa - Automation: versioned docs for 0.14.5 by @elementl-devtools
• b0a885f - batch run fetches in sensor daemon (#6950) by @prha
• 8063405 - gcs asset io manager (#7081) by @jamiedemaria
• 62883d7 - Don't give up if the terminate() call raises an exception during force-termination (#7110) by @gibsondan
• 8b3ad9f - [dagit] Add optional close button to Alert component (#7107) by @benpankow
• 5efccff - Add dagit test to make sure Permission flags stay in sync (#7117) by @prha
• 8aaa2e6 - expose compute_kind tag value on gql AssetNode (#7120) by @smackesey
• dfeff6c - replace internal uses of MetadataEntry static api (#7126) by @smackesey
• 4812f93 - dynamic out bug fix (#6780) by @alangenfeld
• 821d3e0 - Create oss-issues.yml (#7132) by @yuhan
• cd90176 - fix docs search: js_render shouldnt be true (#7146) by @yuhan
• 3509c98 - mypy version bump (#7149) by @smackesey
• 98e99c8 - docs(sda): fix typos and edit wording (#7136) by @rexledesma
• 3ee5790 - add permissions guard for schedule mutations (#7133) by @prha
• b17183e - enable saving run key for ticks that skip runs because of run key idempotence (#7130) by @prha
• 291fbf5 - add tick tooltip info for skipped ticks due to run key idempotence (#7131) by @prha
• bfb8978 - Rename the InstanceMigratedError to be less pushy about migrating (#7141) by @gibsondan
• f426230 - [postgres] share db engine with watcher_thread (#7143) by @alangenfeld
• 01294b4 - Add Map to config type API docs (#7154) by @gibsondan
• 6f10ab0 - Pass on PythonError handling that should be using PythonErrorFragment (#7152) by @gibsondan
• 2cffa49 - [oss telemetry] Get rid of telemetry version (#6760) by @dpeng817
• 35f5f7e - only show warning when force termination checkbox is checked (#7157) by @prha
• ff8d594 - rename several internal submodules (#7134) by @smackesey
• dc994c9 - Remove another cron from example dockerfile (#7155) by @gibsondan
• 0f75e1a - Be more willing to return serialized errors in schedule/sensor/grpc partition methods (#7164) by @gibsondan
• 4f91896 - docs: fix spelling (#7166) by @antquinonez
• b986894 - [multiprocess] tweak preload modules logic (#7142) by @alangenfeld
• 0c7f490 - feat(dagster-k8s): add run id label to run/step workers (#7167) by @rexledesma
• bcc1fd7 - docs(logging): remove example about capturing from the root logger (#7168) by @rexledesma
• 070610d - docs: fix typo in job/graph concepts page (#7172) by @antquinonez
• a3b44e7 - [docs] fix dynamic graph examples (#7174) by @alangenfeld
• e8f1745 - Fix inclusion of tests in our published packages (#7169) by @smackesey
• 0fc9ba4 - RFC: add partition tags to partitioned config (#7111) by @prha
• 2bb2da5 - add gcs pickle asset manager to api docs (#7177) by @jamiedemaria
• 1fff21f - [dagster-databricks] better retry support for step launchers (#7171) by @OwenKephart
• c1d674d - fix lint/mypy failures (#7183) by @prha
• 0c1fea5 - Automation: versioned docs for 0.14.6 by @elementl-devtools
• a892e0d - 0.14.6 changelog (#7184) by @jamiedemaria
• efa9363 - update docs on Dict type (#7179) by @jamiedemaria
• 7c69b8e - docs: use word "op" instead of "solid" (#7195) by @joe-hdai
• e642465 - refactor: rename is_failure to is_failure_or_canceled (#7200) by @rexledesma
• 8378c0c - Time window partitions with custom minute, hour, day offsets (#7125) by @jamiedemaria
• d0188e4 - Fixup ECS docs (#7193) by @jmsanders
• fd7189a - Update version placeholder "dev" to "0+dev" (#7175) by @smackesey
• 7abf706 - fix: prevent runs filter from accepting an empty list of run ids (#7217) by @rexledesma
• 3800687 - refactor: remove screenshots (#7225) by @rexledesma
• a7989b7 - refactor: remove scala modules (#7226) by @rexledesma
• 9444b2f - get MetadataValue.dagster_run to show up in apidoc (#7231) by @sryza
• 1fc1bbc - Azure asset IO Manager (#7201) by @jamiedemaria
• ff9596f - PartitionedConfig testing helpers (#7216) by @jamiedemaria
• e66faf9 - [postgres] dont dispose engine in watch thread (#7241) by @alangenfeld
• 404a7c1 - feat(helm): add configuration for Dagster telemetry (#7190) by @rexledesma
• 847889d - add some type annotations for code loading (#7244) by @sryza
• 7024d7c - feat: add alert failure event (#7238) by @rexledesma
• 81360ce - [dagit] Remove the Asset Node context menu (#7151) by @bengotow
• 6550409 - Only consider repos defining the asset with a corresponding op for multiple repo notice (#7186) by @bengotow
• 6ecb496 - [dagit] Fix Gantt chart layout hang on certain graphs, round 2 #6942 (#7188) by @bengotow
• 8f271ef - [dagit] If re-executing from failure fails, default to from failure again #5019 (#7189) by @bengotow
• f064389 - [dagit] Ignore source assets for “Materialize All” button (#7196) by @bengotow
• 4588b22 - [dagit] Load “Launch Asset partitions” modal when it’s opened + every time (#7207) by @bengotow
• b9284ce - [dagit] Explicitly add the dagster/step_selection tag to asset runs (#7213) by @bengotow
• dd6f57e - Remove redundant k8s launcher event (#7250) by @johannkm
• 0831538 - Allow asset observations in IO Managers (#6653) by @clairelin135
• 023adc4 - update docs for adls2 asset io manager (#7252) by @jamiedemaria
• 42b65dd - 0.14.7 changelog (#7256) by @OwenKephart
• ca369c4 - Update CHANGES.md by @prha
• 14204d5 - Automation: versioned docs for 0.14.7 by @elementl-devtools
• 99d40dd - fix typecheck for In.dagster_type (#7242) by @smackesey
• b61db9d - [dagit] RunTimeline: Fix overlap with ongoing run (#7245) by @hellendag
• c997aa3 - [dagster-io/ui] Remove WIP suffixes (#7229) by @hellendag
• 5c7ec2e - [dagit] Modify "Open in Launchpad" from Run (#7263) by @hellendag
• 60059f9 - RFC: Add register_run_asset event (#7098) by @clairelin135
• 8503e6a - Add table to keep track of schedule data migrations 1/5 (#7182) by @prha
• ae93b45 - add schema for instigators table, keyed by selector 2/5 (#7185) by @prha
• 6262a51 - double-write selector_id to ticks, jobs, instigators 3/5 (#7191) by @prha
• 8b8ce0d - add tick selector index migration (#7198) by @prha
• 2e6eabd - Read instigator state / ticks off of selector_id instead of origin id 5/5 (#7268) by @prha
• 4115505 - [dagster-io/ui] Declaration files for TS, Rollup for built JS (#7220) by @hellendag
• 96d2a25 - Ensure telemetry logging dir is created before using logger (#7192) by @dpeng817
• ec6402d - cross-repo assets toy (#7210) by @sryza
• da2205e - update apidoc for non_argument_deps (#7271) by @clairelin135
• 79235ab - enable loaded_input metadata entries to display in Dagit (#7255) by @clairelin135
• 38cbd24 - [@dagster-io/ui] Cleanup for publish (#7270) by @hellendag
• de5f244 - Remove empty k8s_job_task file (#7286) by @johannkm
• 298d8b1 - Add MenuLink shim for internal compatibility (#7297) by @hellendag
• ce69cc1 - Avoid mentioning pipelines in Dagit empty state (#7288) by @sryza
• 052bf22 - fix invariant in batch tick loader (#7280) by @prha
• e965190 - [dagster-airbyte] fix airbyte materializations without streamStats (#7281) by @OwenKephart
• 34ed951 - point top-level workspace to latest toys (#6876) by @sryza
• 36b189f - [dagster-io/ui] Eliminate RR dependency (#7285) by @hellendag
• 21c8a12 - fix selector injection for k8s-celery test (#7307) by @prha
• ef95e46 - remove experimental warning for mysql storage (#7290) by @prha
• 4996767 - make some edits to changelog (#7264) by @prha
• 0692f86 - [@dagster-io/ui] Clean up deps (#7302) by @hellendag
• f7f2b0f - [dagit] Fix config refresh when reloading repo (#7325) by @hellendag
• 84550f6 - [@dagster-io/ui] Prepare for 1.0.3 (#7326) by @hellendag
• 504c65f - Add run_id as execute_in_process arg (#7317) by @dpeng817
• 8dca877 - Fixes to root input manager memoization and no context provided (#7316) by @dpeng817
• 2e900d5 - add apple m1 instructions to contributor guide (#7279) by @jamiedemaria
• 1860b3a - Fix celery docker executor failing on None metadata (#7330) by @jamiedemaria
• e073b6f - replace old celery docs site (#7331) by @jamiedemaria
• 48f07e9 - [dagster-dbt] fix issue w/ conflicting dbt op names for assets (#7311) by @OwenKephart
• e80494f - docs infra: remove github/slack bot from docs site (#7315) by @yuhan
• 98c41b4 - [dagit] Fix CodeMirror show-hint error (#7334) by @hellendag
• 7551d4f - 0.14.8 Release Notes (#7336) by @clairelin135
• 8c85ad3 - add email icon (#7346) by @salazarm
• 4ebba64 - [dagit] Don't show instigation switch for assets (#7347) by @hellendag
• 4f2b613 - [dagit] Add "Copy config" button to "View tags and config" dialog (#7348) by @hellendag
• f1441de - [dagit] Allow toggling repos in bulk (#7344) by @hellendag
• 8e7d7d6 - [dagit] Store whitespace state in localStorage (#7350) by @hellendag
• a7bf12e - [dagit] Adjust details in Run header (#7345) by @hellendag
• 3039616 - [dagit] Add feature flag to collapse the left Jobs navigation by default (#7204) by @bengotow
• 793aeda - Automation: versioned docs for 0.14.8 by @elementl-devtools
• 4cb70cc - update README - include github discussion (#7339) by @yuhan
• a515c58 - [@dagster-io/eslint-config] Create shared config (#7328) by @hellendag
• ff2f632 - [@dagster-io/eslint-config] Fix dependencies (#7354) by @hellendag
• d344eed - Add Slack icon (#7355) by @salazarm
• ca8a616 - support container kwargs for celery-docker (#7335) by @jamiedemaria
• c444420 - [dagster-dbt] Add /runs method to DbtCloudResourceV2 (#7258) by @kstennettlull
• e0dc0a4 - [dagster-dbt] add "build" to DBT_RUN_RESULTS_COMMANDS (#7362) by @OwenKephart
• 4a0adb3 - [dagit] Show timestamps for “failed to start” runs, other metadata tweaks (#7359) by @bengotow
• 60840ac - Fix contention bug in windows (#7353) by @dpeng817
• bdd4dd7 - fix dagster-ge library (#7369) by @prha
• 3d82da9 - have fs io manager record path metadata entry (#7261) by @sryza
• bdb1192 - AssetGroup.materialize_in_process (#7260) by @sryza
• 7ecd4b3 - Add authenticator field to conn_args (#6983) by @swotai
• d52463c - Make shell ops inherit env vars from outside environment (#6513) by @kbd
• 4e9ee2e - Allow jobs to be passed in lazily to repository definitions (#7382) by @gibsondan
• fac5a5f - [dagit] Make the “Latest Run” tag on the Job page auto-refresh (#7360) by @bengotow
• 6d5eae5 - [dagit] Move data fetching for the asset graph into hooks (#7363) by @bengotow
• 7472843 - autodiscover assets at module scope (#7247) by @sryza
• 00bde2b - [hacker news] pin duckdb (#7390) by @alangenfeld
• 38625ae - Let InProcessRepositoryLocationOrigin take in multiple repos (#7386) by @gibsondan
• 7acd90e - [dagster-fivetran] Fix format strings (#6436) by @mgasner
• 96a6762 - [dagster-airbyte] Handle 204 reponse in Airbyte Library (#7209) by @HAMZA310
• 4f77c97 - run materializations (#7394) by @prha
• 72e5b85 - [dagit] Fix default Dagit page in single repo, multiple jobs case (#7368) by @bengotow
• 104e780 - Add grpc server timeout config to instance settings (#7387) by @gibsondan
• 0ba3b4e - infer resource descriptions from docstrings (#7309) by @sryza
• 3f5011b - [dagit] Async layout of large asset graphs (#7366) by @bengotow
• bdb8126 - [dagit] Align terminology and data structures in asset / op graph layout (#7372) by @bengotow
• 925ddbc - [dagit] Move asset-graph out of workspace dir, non-graph code to /assets (#7373) by @bengotow
• f23a00b - Update AssetKeyTable.last_run_id upon materialization planned (#7319) by @clairelin135
• 1600323 - asset tutorial (#7269) by @sryza
• 9f1d04e - Add a field to the now-somewhat-poorly-named RepositoryPythonOrigin that can add 'deployment target' information within a repo location (#6925) by @gibsondan
• 9f8e68f - Add EcsContainerContext (just with secrets for now) (#7176) by @gibsondan
• 1a3a76e - Add K8sContainerContext for configuring k8s pods from the repository location (#7203) by @gibsondan
• 71661a7 - Fixed typo in deploying-with-helm.mdx (#7416) by @LeoHuckvale
• ec89e79 - [dagit] Use react-scripts@5.0.2 (#7393) by @hellendag
• 94f7984 - [dagit] Add arrow key and double click interactions to asset graph #6407 (#7374) by @bengotow
• 8120421 - [dagit] Allow colon in run tag value (#7409) by @hellendag
• e2de0b7 - Re-execute run instance method (#7417) by @johannkm
• afe0492 - skip adls test (#7420) by @jamiedemaria
• 95aea0a - add open launch pad to run details page (#7304) by @salazarm
• 0e46052 - Fix prerequisite typos in docs (#7424) by @johannkm
• 9f9db4e - [@dagster-io/eslint-config] Enable object-shorthand (#7423) by @hellendag
• 6d1cf77 - Filter out asset_materialization_planned event logs client-side (#7397) by @clairelin135
• 8e75a98 - Make user repo filters per deployment (#7376) by @salazarm
• cb642cb - [postgres] urlquote usernames as well (#7430) by @alangenfeld
• e615623 - [toys] squelch warnings (#7419) by @alangenfeld
• f38e1c3 - docs for gRPC sever timeout flag (#7400) by @gibsondan
• 7a9e8c9 - Fix master and add DagsterEventType import (#7435) by @clairelin135
• 6cf75eb - avoid extra experimental warning when asset has partitions def (#7431) by @sryza
• 90ac1b5 - Add option to disable max_concurrent_runs (#7327) by @johannkm
• a5c2854 - RFC: Fetch Cross-Repo Asset Deps (#7259) by @clairelin135
• 7aef716 - make k8s container context volumes and volume mounts use snake case, not camel case (#7437) by @gibsondan
• b09c8e1 - move expensive tests out of core_tests (#7436) by @gibsondan
• 3b55c4e - allow AssetGroups to have assets with different partitions defs (#7388) by @sryza
• be2469b - Update key for saving data in useExecutionSessionStorage (#7333) by @salazarm
• bb27b2d - asset partitioned io manager (#7413) by @sryza
• 0bc57e9 - Unbreak stopping unloadable schedules/sensors (#7446) by @gibsondan
• e0b991b - [dagit] Check for asset group job prefix to support multiple groups per repo (#7418) by @bengotow
• 5a27fcd - changelog for 0.14.9 (#7450) by @sryza
• 7ad594b - update isort comment formatting (#7448) by @smackesey
• a0a321d - bump black to 22.3.0 (#7449) by @smackesey
• 1bdb802 - Automation: versioned docs for 0.14.9 by @elementl-devtools
• 12b87c3 - Typo in the create a new project doc (#7441) by @ascrookes
• 2e8136d - Fix issue with blank telemetry instance id file (#7433) by @dpeng817
• 1ec9318 - Show repo selector in unloadable schedules/sensors instead of repo location metadata (#7444) by @gibsondan
• a0cc868 - Split up screenshot specs (#7460) by @sryza
• 9ad8f54 - resource dependencies for root input managers (#7459) by @sryza
• 920c759 - avoid framework code triggering metadata_entries warning (#7412) by @sryza
• a744f84 - allow combining memoization with step selection (#6431) by @sryza
• 53b4f09 - refactor: remove local storage file that was committed (#7440) by @rexledesma
• 74dafde - [dagster-graphql] remove gevent (#7402) by @alangenfeld
• 2a9b06f - [dagit] Apply middle truncation to long asset nodes, hide run info (#7187) by @bengotow
• 7f8f9d8 - [dagit] Use CodeMirror for all config displays (#7467) by @hellendag
• f22619d - Add container_context to gRPC server and Helm chart (#7236) by @gibsondan
• d1a1e44 - Add asset metadata to one of the dagster_test asset groups (#7462) by @bengotow
• 2c54844 - Fix back-compat for new helm chart param for container context (#7476) by @gibsondan
• 39c853a - [graph] use the correct input definition for type checks (#7453) by @alangenfeld
• b94eeb0 - [dagit] Repair async graph layout for path-prefix (#7480) by @hellendag
• e781ce6 - Add docs to k8s docs about includeConfigInLaunchedRuns (#7477) by @gibsondan
• 98cdc50 - Acquire leases when using ADLS (#7410) by @jamiedemaria
• 83bfccf - Update Materialization Warning (#7265) by @clairelin135
• ad6c38a - [dagit] Fix Suggest (#7489) by @hellendag
• cca4169 - Make the scheduler/sensor daemons key the schedules that it inspects by selector ID, not origin ID (#7492) by @gibsondan
• a34a9e6 - Add right of search bar prop to top nav (#7500) by @salazarm
• d57d83b - add flag argument to backfill all partitions in launch backfill graphql mutation (#7499) by @prha
• 725e6d9 - [dagit] Fix job header “View assets” link, add missing graphQL types (#7493) by @bengotow
• 4572857 - fix sqlite gating logic for batch tick queries (#7505) by @prha
• 017bd49 - [dagster-census] Census Integration Library (#7249) by @dehume
• 768c1b3 - [dagit] Show a better error on invalid run filters (#7494) by @hellendag
• 13a420a - Add log message to grpc health check (#7506) by @johannkm
• 8fc5c61 - refactor: remove duplicate solids in event log test (#7511) by @johannkm
• a4618b4 - correct workspaces icon (#7504) by @salazarm
• d7ff7bc - [dagit] More cleanup of query callsites performing background polling (#7508) by @bengotow
• f5bdb5e - [dagit] Avoid resetting hidden repos on each page load #7519 (#7520) by @bengotow
• 4181eba - [dagit] Allow disabling keyboard shortcuts (#7515) by @hellendag
• eab1d7d - Update pylint and associated config (#6517) by @smackesey
• b258c47 - 0.14.10 Changelog (#7521) by @johannkm
• 36f51f4 - fix asset partitions runtime check (#7525) by @smackesey
• fc57d23 - [dagit] Tiny fix for “asset graph search, dropdown stops showing when ">" character is used” (#7514) by @bengotow
• 078d828 - [dagit] Move yarn analyze to packages/app (#7523) by @hellendag
• 45d9c92 - [dagit] Clean up lodash imports (#7526) by @hellendag
• 1f1e233 - [dagit] Display repositories in a stable order (#7518) by @bengotow
• 81e7cba - Automation: versioned docs for 0.14.10 by @elementl-devtools
• 460e482 - [hooks] fix resource reqs in nested graphs (#7529) by @alangenfeld
• 610745b - [instigator] dont throw in start/stop if already in desired state (#7483) by @alangenfeld
• 772afa6 - [dagit] Fix InstigationTick "Skipped" message if zero runs (#7527) by @hellendag
• d9b523d - Get rid of mode in JobDefinition constructor (#7497) by @dpeng817
• 45db8cd - Fix issue where we were calling the partition function n times, where n is the number of partitions returned by the function (#7539) by @gibsondan
• 3f1f0fd - [bugfix] Check that oldData is defined before trying to migrate (#7537) by @salazarm
• 697e701 - Assorted tox updates (#6971) by @smackesey
• 1052405 - 0.14.11 changelog (#7543) by @gibsondan
• f160f7d - Automation: versioned docs for 0.14.11 by @elementl-devtools
• 19dff51 - Reexecute run mutation using just parent run (#7510) by @johannkm
• 942c995 - Updated changelog for 0.14.12 (#7545) by @gibsondan
• 5bf224a - Revert "[bugfix] Check that oldData is defined before trying to migrate (#7537)" and "Update key for saving data in useExecutionSessionStorage (#7333)" (#7551) by @gibsondan
• ade2be8 - [graphql] subscription handle_graphql_errors (#7482) by @alangenfeld
• 501d053 - Automation: versioned docs for 0.14.12 by @elementl-devtools
• 812f0b9 - [dagit] Add bulk re-execution dialog to Runs page (#7451) by @hellendag
• 3bb37d3 - Downgrade tox pip for Python 3.6 (#7547) by @smackesey
• 7599886 - annotate dagster.builtins.Nothing with correct type (#7546) by @smackesey
• 1756b97 - Pylint: Add encoding to open() (#7548) by @johannkm
• e9f0ebc - asset op tags (#7472) by @sryza
• 65b8cbd - [dagit] Show asset “compute kind” in the global asset graph (#7516) by @bengotow
• 91ced9c - [dagit] Hide __asset_group partition sets from search bar (#7556) by @bengotow
• 7452ec3 - Add typing-extensions >=3.10 as dependency. (#7555) by @smackesey
• a141905 - validate tags in partition sets (to turn dicts into strings) (#7554) by @gibsondan
• 5480b04 - [dagit] Feature flag for disabling WebSocket usage (#7557) by @hellendag
• 2f7ccad - [dagit] Add opNames field to AssetNode (#7565) by @hellendag
• c31de97 - add DEFAULT_PYTHON_VERSIONS env var for BK (#7553) by @smackesey
• 922d786 - add DEFAULT_PYTHON_VERSIONS env var for BK (#7553) by @smackesey
• fdb2274 - [1/n] Interop Stack: Use AssetJobInfo at runtime instead of OutputDefinition (#7473) by @OwenKephart
• 4d44a79 - [2/n] Interop Stack: Use AssetJobInfo to construct ExternalAssetNodes (#7475) by @OwenKephart
• 65f15cd - [dagit] Introduce sticky table headers (#7568) by @hellendag
• 36e7e6a - [dagit] Truncate global search result list (#7569) by @hellendag
• 38c650e - Test test_create_reexecuted_run_from_failure (#7566) by @johannkm
• bb87a40 - Tag behavior option for reexecute method (#7574) by @johannkm
• 6e12f83 - add setuptools as dep (#7576) by @smackesey
• b87ef89 - fix errors (#7577) by @smackesey
• b49aadc - Register a new task definition for IAM changes (#7564) by @jmsanders
• 69546e9 - Improve typing of schedule/sensor and hooks (#7560) by @smackesey
• 7ef7384 - All steps option for rexecute method (#7575) by @johannkm
• e9af45b - [docs] fix multi asset sensor example (#7583) by @alangenfeld
• 5926b2c - Create runs during event log tests (#7502) by @johannkm
• 22e0267 - [dagit] Refactor ExecutionSessionStorage (#7559) by @hellendag
• 4516740 - fix(helm): support numeric type image tags (#7579) by @jrouly
• fc4358e - [dagit] Add bulk re-execution of all steps (#7591) by @hellendag
• 0b25e62 - ResourceDefinition.none_resource instead of no_step_launcher (#7381) by @sryza
• db0bf2b - Remove unused imports from event log tests (#7592) by @johannkm
• 81a09ff - [dagit] Fix asset graph navigation edge cases (#6955, #7208) (#7532) by @bengotow
• 2a3595b - fullbuild in branch name for full BK build (#7595) by @smackesey
• 5a85d2c - document partitioned assets (#7466) by @sryza
• 61a6861 - fix secretsmanager example doc (#7593) by @prha
• f8f87f6 - Add ConfigSchema type alias (#7596) by @smackesey
• 8ecaadf - lint dagster_tests (#7594) by @alangenfeld
• 7ed2dbd - add dialect-specific update logic (#7572) by @prha
• d0d4d5d - fix volumes example for docker run launcher (#7607) by @gibsondan
• 0620355 - [tests] set platform on docker build (#7603) by @alangenfeld
• ed8323c - Use explicitly set env vars rather than autoenvvar prefixes for the grpc command (#7611) by @gibsondan
• 0a1616b - Fixes for running the user-code-deployment helm chart in a different namespace than the system namespace (#7597) by @gibsondan
• e947de9 - standardize additional_message kwarg in check methods (#7605) by @smackesey
• 0cbc38b - [3/n] Interop Stack: Remove asset key from inputs/outputs (#7550) by @OwenKephart
• cc2060a - fix(helm): allow max concurrent runs to be 0 (#7618) by @rexledesma
• 6d27f54 - [4/n] Interop Stack: node_def on AssetsDefinition can be graph (#7573) by @OwenKephart
• 3bcb757 - Asset key to node handle mapping (#7599) by @clairelin135
• 00fa234 - [dagster-aws] update emr pyspark step launcher (#7604) by @OwenKephart
• 00d8e1c - Make dagster-user-deployments service account set up a role and role binding by default, like the main dagster chart service account (#7622) by @gibsondan
• 4cb262c - Add resources to K8sContainerContext (#7619) by @gibsondan
• d71cb6f - [dagit] Add "shared key path" outlines, experimental asset graph flag (#7608) by @bengotow
• ee4cb0e - [@dagster-io/ui] Separate tsconfig for build (#7584) by @hellendag
• 98fe635 - Fixup XXXSource docs (#7629) by @gibsondan
• febd18f - Changelog 0.14.13 (#7631) by @dpeng817
• ba9d867 - AssetGroup.prefixed (#7395) by @sryza
• b8be100 - AssetsDefinition.from_graph (#7620) by @clairelin135
• b9be0d4 - [Testing] Support for resolvers in ApolloTestProvider (#7612) by @salazarm
• 9345ebc - [dagit] Show default value for config (#7598) by @hellendag
• aa37ece - [docker] improve test time (#7589) by @alangenfeld
• b9b9d97 - Update mypy config and bump version (#7625) by @smackesey
• 67547d0 - [dagster-ge] drop 3.6 (#7638) by @alangenfeld
• b12af8c - Automation: versioned docs for 0.14.13 by @elementl-devtools
• 6e52818 - Unify all alembic scripts into single directory, creating single history (#7411) by @prha
• 8ab7bfb - enable mypy checking on dagit (#7626) by @smackesey
• 5870039 - [dagit] Add dummy graphName field to AssetNode (#7637) by @hellendag
• 3630d97 - [RFC] lighter weight pull request template (#7538) by @alangenfeld
• cc07c17 - [dagit] AssetView: Don't show historical view message if it's not (#7639) by @hellendag
• a83f387 - allow flexible types for SourceAsset key arg (#7633) by @sryza
• 1cfcac3 - add graphql fields to quickly query partition status (#7614) by @prha
• b0da1bf - add option to GraphQueryInput to apply changes without hitting enter (#7635) by @prha
• 8f80e59 - Refactors the PartitionGraph component to be divorced from the actual graphql query (#7616) by @prha
• 6003fdb - Adds a new partition page behind a feature flag (#7617) by @prha
• a953214 - enable adding asset groups together (#7634) by @sryza
• 8fbd524 - Fix mypy config for dagster main package (#7644) by @smackesey
• 596515d - allow multiple asset groups on a repository (#7649) by @sryza
• f7e85ff - refactor: remove airline demo (#7653) by @rexledesma
• 4c65d82 - to_source_assets (#7643) by @sryza
• d543f82 - adding a Terminate Run method for in-progress execution to the Python GraphQL client (#7443) by @Javier162380
• 87c0f27 - fix apidoc for fs io manager (#7671) by @sryza
• 90b4739 - Fix celery docker jobs that involve writing directly to command-line output (#7665) by @gibsondan
• f46fc50 - Fix docs lib links (#7680) by @johannkm
• 9941f61 - Fix configmaps that need to be manually set when running the user deployments helm chart in a different namespace (#7660) by @gibsondan
• 37d9629 - Add all repos to hacker_news __init__.py (#7677) by @gibsondan
• dbbeadf - remove primitive types from apidoc (#7674) by @sryza
• 7b54c30 - Add py.typed to dagster and all extension libs (#7561) by @smackesey
• c6ff832 - [eslint-config] Bump dependencies (#7681) by @hellendag
• faad5ea - [dagster-airbyte] terminate airbyte sync w python process (#7687) by @OwenKephart
• c99184c - [dagster-dbt] remove annoying color formatting strings from log output (#7688) by @OwenKephart
• d79bb7b - Pull out PostgresEventWatcher into its own module and parameterize it more (#7666) by @gibsondan
• 0422087 - graph and job apidoc fixes (#7673) by @sryza
• 7bdffdb - docs: fix example for multiprocess run config (#7693) by @rexledesma
• 2db9bd1 - finish dagster-fivetran types (#7563) by @smackesey
• 636decb - finish dagster-airbyte types (#7562) by @smackesey
• e9dd228 - chore(graphql): add linter to ensure docstrings on graphene graphql objects (#7669) by @rexledesma
• acf8f9f - docs(graphql): add docstrings to mutations.py (#7691) by @rexledesma
• 1bb7112 - Move EventSpecificData out of TYPE_CHECKING (#7697) by @smackesey
• 441f5ba - Add ability to set mapping key on op/solid invocation context (#7364) by @dpeng817
• 671cea6 - Make Output generic (#7202) by @dpeng817
• c55e3f4 - update fake adls resource to work with leases (#7587) by @jamiedemaria
• de0eae9 - add core partition status storage query (#7662) by @prha
• 439b4d5 - docs: specify the correct mysql event log module (#7703) by @rexledesma
• 8566057 - Revert "Make Output generic (#7202)" (#7715) by @dpeng817
• 11f9496 - Add bool metadata type (#7694) by @clairelin135
• 52ff71f - Add platform arg and other improvements to dagster-docker CLI (#7698) by @smackesey
• d4293b8 - Add a guard around dagit's JSON parsing of incoming requests (#7714) by @gibsondan
• d32fcff - Switch new partition page to use more efficient partition status query (#7663) by @prha
• bc2e62d - [docs] retry policy invocation example (#7716) by @alangenfeld
• 756540e - docs: fix example with lazy loading repository (#7704) by @rexledesma
• 2a579a6 - update slack api call in op hook docs (#7712) by @jamiedemaria
• 2f828aa - Improve Buildkite test version specification API (#7699) by @smackesey
• 64c6d85 - use ">" for multi-component selection in AssetGroup.build_job (#7661) by @sryza
• d6af7e1 - finish dagster-pagerduty types (#7668) by @smackesey
• 857079d - enable providing asset_key to build_output_context (#7696) by @sryza
• 7504407 - [dagit] Support special cron strings (#7717) by @hellendag
• 5fac185 - solid -> op in dagster-slack (#7684) by @sryza
• 6e467c5 - Typing additions (#7039) by @smackesey
• 84cc1a4 - [instance] rm get_addresses_for_step_output_versions (#7601) by @alangenfeld
• e90d8d9 - Add run tags for repository/location names (#6893) by @prha
• c81a825 - Surface retried error in Dagit (#7692) by @johannkm
• 504aa08 - RFC: Keep firing hooks even if a framework exception is raised mid-execution (#7652) by @gibsondan
• 948499e - combine validate_tags and check_tags to JSONify nested tags (#7720) by @gibsondan
• ae55df0 - test dynamic_output output for node (#7718) by @clairelin135
• 5e32caa - [postgres] dont use urlquote_plus (#7723) by @alangenfeld
• d9cc191 - [docs] fix jobs from graphs example (#7722) by @alangenfeld
• 4c57949 - Make output generic (#7719) by @dpeng817
• 2155bf5 - Missed changes for Dagit surface retry error (#7731) by @johannkm
• ed04dab - Change head_bucket to list_objects. (#7485) by @trevenrawr
• fbb1abb - RFC: include schema in default dbt asset keys (#7645) by @sryza
• 59a9915 - populate op names and graph name for ExternalAssetNode (#7721) by @OwenKephart
• 69c57d3 - add extra sensor state fetch to minimize chance of clobber state (#7738) by @prha
• 97ed20f - Workaround for mypy cache bug (#7732) by @smackesey
• 8cc4bdc - fix run list for new partitions view (#7742) by @prha
• 81a0421 - 0.14.14 changelog (#7743) by @prha
• ee3081f - Make Output and DynamicOutput no longer be tuple types (#7740) by @dpeng817
• de65762 - [dagit] Use a CSP header instead of meta tag (#7727) by @hellendag
• 0260b0f - Add a test with various wait/act use cases (#7728) by @hellendag
• da15d8d - Add job/op equivalents to execution context (#7734) by @johannkm
• 30e7ced - Rename ReexecutionPolicy -> ReexecutionStrategy (#7746) by @johannkm
• 9beeff4 - [dagit] Update tsconfig to es2022, update browserslist (#7749) by @hellendag
• 2930507 - [dagit] Use Array flat (#7754) by @hellendag
• a4ffde0 - remove usable_as_dagster_type from integrations (#7682) by @sryza
• 41f705c - Automation: versioned docs for 0.14.14 by @elementl-devtools
• 1d35e58 - enable returning lists of run requests from sensor/schedule evaluation functions (#7755) by @prha
• db8f725 - remove usable_as_dagster_type from types concept page (#7758) by @sryza
• f305b65 - [easy] fix missing space in error message for util function (#7761) by @gibsondan
• f85b25d - [dagit] Add more security headers (#7764) by @hellendag
• 89efb37 - Additional asset graph improvements (#7707) by @bengotow
• 9f73f3f - [dagit] Add Reload All button to the asset catalog, remove repo filter (#7708) by @bengotow
• 6e50207 - [dagit] Better indentation guide in our CodeMirrors (#7765) by @hellendag
• afe23af - Allow python objects to be passed as top-level inputs to job (#7032) by @dpeng817
• 9366a7e - Add a static method on DefaultRunLauncher that launches a run without a workspace (#7777) by @gibsondan
• 68a99d6 - Revert "Allow python objects to be passed as top-level inputs to job (#7032)" (#7779) by @dpeng817
• cba9144 - Allow repositories to contain asset definitions and source assets for the same asset key (#7781) by @sryza
• 8e25adc - fix error message when can't find upstream asset for dep (#7648) by @sryza
• 8e05a7d - Allow TextInput to have a type of number (#7778) by @salazarm
• 1b08360 - fix error with inferring description from empty docstring (#7788) by @sryza
• 8096b17 - [dagit] Add hover actions to run Tags (#7751) by @hellendag
• 42e00e1 - Fixes to the apidocs page (#7729) by @sryza
• 9cf7cde - [dagit] Show asset nodes on Asset Details (#7689) by @hellendag
• 7055f90 - [dagit] Make log filter input clearable (#7805) by @hellendag
• 10d9fc5 - [eslint-config] Add recommended jest lint (#7807) by @hellendag
• 64cf6a2 - pin xmltodict to fix aws test failures, until moto 3.1.9 (#7806) by @prha
• 5f1e9fb - update s3 sensor docs to use context.cursor instead of context.last_run_key (#7748) by @prha
• 8cfa136 - Add namespaces / prefixes to the hacker news assets demo (#7782) by @sryza
• 742ba2b - toposort frozenset workaround (#7793) by @smackesey
• 4462e81 - Increase executor event log poll interval (#7803) by @johannkm
• abd59f6 - Add a few __init__.py in test packages (#7809) by @smackesey
• b4115ef - remove TableSchema experimental warnings (#7799) by @smackesey
• 1f0d9fb - snowflake io manager (#7726) by @sryza
• ccb2fd1 - Change the fake repo that simulates an import error while loading dagster code (#7816) by @gibsondan
• e6c5bb0 - remove jinja / nbconvert pin (#7819) by @prha
• 9ca7fdd - [dagit] Use /instead of> for displaying asset paths (#7818) by @bengotow
• b0e3ca2 - fix snowflake io manager tests on python 3.7 (#7824) by @sryza
• 35a198b - link to examples in concepts pages (#7753) by @jamiedemaria
• 0540339 - [docs] job execution (#7776) by @alangenfeld
• 14947c2 - [dagit] Clean up my todos (#7810) by @hellendag
• 99259a7 - [dagit] Fix global search for repeated item keys (#7827) by @hellendag
• cd7de4d - sys.meta_path import mapping layer (#7040) by @smackesey
• 629a689 - Allow passing posargs to tox commands (#7822) by @smackesey
• 2572d66 - Rename check > _check (#7808) by @smackesey
• 62b2ffb - context.partition_time_window (#7795) by @sryza
• 7abbdb6 - take out airline demo from make dev_install (#7839) by @sryza
• 27ca0c0 - [dagit] Make schedule/sensor tags more prominent in RunTags (#7834) by @hellendag
• 106dca2 - add BoolMetadataValue to __all__ in dagster/__init__.py (#7838) by @sryza
• 42e35af - [dagit server] guard on async websocket send (#7833) by @alangenfeld
• 03c8691 - [top-level inputs 1/2] Move InvalidSubsetError try-catch up a level (#7780) by @dpeng817
• 8cb4101 - docs design update (#7531) by @yuhan
• bb75f3b - Remove test-connection pod (#7842) by @gibsondan
• d60c3c8 - remove validate_asset_key_string (#7811) by @sryza
• 3125227 - partitioned assets from graphs (#7837) by @sryza
• 15e9eb5 - fallback logic for resolvers (#7766) by @OwenKephart
• d099e43 - migrate concept docs code snippet paths to graph/job/op (#6775) by @sryza
• 0356320 - [rfc] generic + returnable dynamic outputs (#7744) by @dpeng817
• 923342e - [dagster-dbt] dbt build + AssetObservations from tests (#7783) by @OwenKephart
• 77dff37 - [dagit] New “folder grid” asset view behind experimental flag (#7767) by @bengotow
• 7207a6e - Handle backcompat asset observation (#7831) by @clairelin135
• 2238f8b - [dagit] Remove search item key (#7830) by @hellendag
• 0ba6b2a - use generic Output type annotations in hacker news (#7600) by @sryza
• 5e8f310 - changelog 0.14.15 (#7850) by @yuhan
• 773cf30 - [dagit] add log-level flag (#7853) by @alangenfeld
• b71f150 - [changelog][skip ci] dagit line and autofmt (#7857) by @alangenfeld
• 61afd9a - [dagit] tweak websocket disable (#7860) by @alangenfeld
• 21237cf - Limit pods per run with k8s_job_executor (#7846) by @johannkm
• d917df5 - additions to the 0.14.15 changelog (#7856) by @sryza
• c0f86a3 - fix asset key prefixes in MDS assets example (#7855) by @sryza
• 042d3d2 - [helm] dagit.logLevel (#7862) by @alangenfeld
• f06689b - warn for instance.get_event_records calls without an event type filter (#7848) by @prha
• 8455843 - [top-level inputs 2/2] top level inputs on job and execute_in_process (#7786) by @dpeng817
• e362fad - Automation: versioned docs for 0.14.15 by @elementl-devtools
• 632cc79 - rm duplicate changelog entry from navigation (#7873) by @gibsondan
• b5d52ea - Fix error message assertions in Helm tests (#7858) by @johannkm
• 6732a87 - Toxfile adjustments (adding mypy) (#7867) by @smackesey
• 993a3aa - [resources on repos 1/n] Reorganize job/pipeline/graph duplication error logic in repository (#7817) by @dpeng817
• a4d79e3 - docs: fix vercel build (#7876) by @yuhan
• bfbfa62 - enable @asset-decorated functions to accept kwargs (#7871) by @sryza
• 21b6a5e - fix type error (#7887) by @smackesey
• 78d4b57 - dagster-dbt types (#7878) by @smackesey
• 272b250 - Add health check for ECS tasks (#7695) by @jamiedemaria
• 91b6c0f - [cloud dagit] Add optional nav tabs parameter for settings root page (#7892) by @benpankow
• ea27cdc - feat(helm): support custom configmap for workspace (#7882) by @johannkm
• 9e786a1 - add data migration for run repo tags (#7815) by @prha
• c71b117 - make docker_compose_cm a contextmanager rather than a fixture (#7905) by @gibsondan
• 727c059 - [@dagster-io/eslint-config] v1.0.4 (#7913) by @hellendag
• 5897837 - docs: fix flash white box (#7877) by @yuhan
• fe59b68 - InputContext.upstream_output is missing key when source asset (#7919) by @sryza
• 80153e0 - remove handle_schema_errors that wrap db errors with schema outdated exceptions (#7886) by @prha
• c95cfd1 - Enable asset partitioning support for @multi_asset defined assets (#7908) by @aroig
• 5e22b7b - update multiple dynamic outs example (#7917) by @jamiedemaria
• 6e15dfb - dagster-databricks mypy (#7879) by @smackesey
• 252f285 - [1/n] Subsetting Stack: use subset_selector for AssetGoup.build_job (#7796) by @OwenKephart
• 06149b6 - Don't call it a 'framework error' when a run worker unexpectedly restarts (#7885) by @gibsondan
• 17fa439 - chore(helm): remove local from compute log manager options (#7924) by @rexledesma
• f6d63d1 - Executor and dagster-docker typing (#7881) by @smackesey
• a7e0b69 - remove markupsafe pin (#7898) by @bollwyvl
• 1564c9f - [2/n] Subsetting Stack: AssetsDefinition subsetting (#7797) by @OwenKephart
• 3b5c135 - fix docstring (#7948) by @OwenKephart
• d0524c2 - RFC: Better error handling when an ExternalRepository grpc call fails (#7929) by @gibsondan
• f27ee8b - Use correct configured link in resources (#7949) by @gibsondan
• 445194b - add CLI command to print the storage schema version (#7910) by @prha
• 2d219f4 - [dagit] Present location reload errors after “Workspace > Reload All” (#7907) by @bengotow
• 943ab6c - [dagit] Asset details: Graph/op links (#7655) by @hellendag
• 448ca76 - Fix detection of parent process death (#7914) by @aroig
• 15887bc - Merge identical get_location and get_repository_location methods (#7952) by @gibsondan
• 0d90a3c - add missing check to AssetsDefinition.from_graph (#7960) by @sryza
• f7b14c4 - Fix issue where azure IO manager was sometimes failing on recursive deletes (#7956) by @gibsondan
• 780f771 - make input defs deterministic (#7957) by @clairelin135
• b8f3816 - [events] can_load mitigation (#7955) by @alangenfeld
• 2cbf3cd - fix slack image link on Getting Started (#7964) by @yuhan
• 1bd83e4 - Typing for automation package (#7812) by @smackesey
• 205d811 - make StepLauncher and friends public (#7945) by @smackesey
• fb025dc - feat(helm): bump minimum supported kubernetes version to 0.19 (#7925) by @rexledesma
• 422fa71 - Asset Subselection (#7835) by @clairelin135
• d6b4eaf - various fixes for new partition page (#7950) by @prha
• 0424134 - [dagster-airbyte] don't cancel completed syncs (#7888) by @OwenKephart
• 2a56fa2 - [dagster-dbt] small tweaks (#7967) by @OwenKephart
• e168466 - Error rather than warn when an event log query comes in that tries to use an int cursor on an event log DB that needs a RunShardedEventsCursor (#7970) by @gibsondan
• c7b2387 - Still make cursor editable if it is not set (#7969) by @gibsondan
• 738799d - [daemon] tweak per-thread instance/workspace setup (#7965) by @alangenfeld
• b66a915 - Revert "feat(helm): bump minimum supported kubernetes version to 0.19 (#7925)" (#7968) by @rexledesma
• 50ac89f - 0.14.16 Changelog (#7976) by @johannkm
• abc35c0 - fix reexecution (#7980) by @clairelin135
• 129a8d6 - Automation: versioned docs for 0.14.16 by @elementl-devtools
• b49cfe4 - [dagit] Fix SearchBootstrapQuery to be lazy (#7981) by @hellendag
• 7a72d66 - split graphql query for the backfill page (#7986) by @prha
• a8d3526 - Remove can_terminate from run launchers and run coordinators, use a status check instead in graphql (#7983) by @gibsondan
• 6afdaf9 - style(helm): remove unnecessary pylint suppressions (#7923) by @rexledesma
• 8f375c4 - provide job_timeout_in_seconds in dataproc resource config (#7941) by @3cham
• 605d274 - [dagit] Replace opName with opNames everywhere (#7977) by @hellendag
• d910d23 - Set default hour of day and minute of hour in build_schedule_from_partitioned_job based on the offsets of the parittioned job (#7954) by @gibsondan
• 4293e7c - WIP: Don't fetch every single run in the InstanceBackfillsQuery (#7985) by @gibsondan
• cbd7da9 - [dagit] Show assets on run pages in more scenarios (#7874) by @bengotow
• 5ae7d32 - [dagit] Do not display the partition_set, step_selection tags on asset group runs (#7953) by @bengotow
• f51d7e6 - [dagit] WebSocketProvider: Time out and fall back to disabled (#7978) by @hellendag
• 515e194 - Coerce asset_key on AssetIn (#8009) by @aroig
• d19ec31 - Fix issue where we added the run ID repeatedly to the runs for the tick during scheduler failure recovery (#8003) by @gibsondan
• 58230bf - Load fewer backfills on each page (#7998) by @gibsondan
• f625b6d - add loading indicator for job backfills (#8004) by @prha
• abac74a - Remove duplicate count_resume_run_attempts method (#7915) by @johannkm
• d9133bb - Make asset related checks more robust (#8008) by @aroig
• c4d844d - Fix relative path for pytest fixture in backcompat tests (#7999) by @dpeng817
• 60e8774 - feat(helm): allow name override for Dagit deployment (#8005) by @rexledesma
• be20db5 - Add a __main__.py to dagster/daemon (#8012) by @gibsondan
• 07210b9 - dagster-buildkite refactor (#7813) by @smackesey
• d5d7068 - Fix unsightly scrollbar in custom alert (#8018) by @gibsondan
• 1bda005 - create get_records_for_run for event log storage, using an opaque string cursor (#7973) by @prha
• 7cc7681 - Fix error when dagit passes in a non-dict string to run config valiation (#8020) by @gibsondan
• bf78a28 - [dagster-io/ui] Fix Dialog header/footer alignment (#7975) by @hellendag
• 8c6a8fe - [dagit] Disable unloadables if no permissions (#8021) by @hellendag
• 71802bd - use mkdir -p in migration guide by @johannkm
• aebfe43 - [dagit] Sectioned left nav (#8017) by @hellendag
• e385dec - Handle InvalidSubsetError in launch run results (#8014) by @gibsondan
• 08ef634 - Use dagster-postgres in alembic migration guide (#7992) by @johannkm
• 0b50cc2 - Generate buildkite headers inside dagster-buildkite (#7987) by @smackesey
• 3ac0dd2 - bug fix: should allow re-executing all dynamic steps generated from mapper step (#7979) by @yuhan
• 654a748 - refactor: has_index for migrations (#8034) by @johannkm
• 08e37e3 - In-progress indicator for graph-backed assets (#8015) by @clairelin135
• b1ab74b - resource defs on asset def (#7918) by @dpeng817
• 9d98be1 - Load all code artifacts on grpc server startup, not just pipelines/jobs (#8040) by @gibsondan
• 4062bca - Accept string values for non_argument_deps (#8023) by @shalabhc
• 49aa991 - add example Python incompatibilities on Py 3.6 (#8036) by @smackesey
• 31e1f3d - Add missing deps for docs-snippets and skip 36 in BK (#8048) by @smackesey
• e7fca20 - docs: update readme with social links (#8046) by @rexledesma
• 4e1a53f - OpExecutionContext.output_asset_key (#7961) by @sryza
• 10b9225 - Add missing method to SnowflakePandasTypeHandler (#8051) by @gibsondan
• 10d8e65 - [dagit] Flip section arrow in left nav (#8056) by @hellendag
• b5604ce - [dagster-io/ui] Make disabled+checked Switch state more obvious (#8033) by @hellendag
• 9122968 - [dagit] Enable sectioned left nav by default (#8039) by @hellendag
• 407e698 - [3/n] Subsetting Stack: dbt assets can be subset (#7798) by @OwenKephart
• 84ccc32 - Raise error upon incomplete graph-backed asset subset (#8041) by @clairelin135
• ac6007c - throw error for empty asset key (#8069) by @smackesey
• 71a85ba - [dagit] Use dialog for multiple schedules/sensors in left nav (#8065) by @hellendag
• f8ff6f6 - remove dynamic mapping and collect section from jobs concepts page (#7844) by @sryza
• 307b363 - grammar (#7984) by @dwinston
• 71872c3 - Pass in engine rather than connection in SnowflakePandasTypeHandler (#8070) by @gibsondan
• b6caeaf - [dagit] Launchpad: Show disabled "Scaffold" and "Remove" buttons instead of hiding them (#8066) by @hellendag
• 220e74d - chore(buildkite): mention user in private channel (#8071) by @rexledesma
• d7129a5 - [dagster-dbt] allow for static manifest.json-based selection (#8047) by @OwenKephart
• 03b24cd - [asset-resources 2/n][rfc] io manager defs directly on asset defs (#7920) by @dpeng817
• 1e411ad - adding a FromSourceAsset StepInputSource (#7942) by @OwenKephart
• 284e912 - Add protobuf pin to dagster (#8078) by @gibsondan
• 0e0bf21 - Skip flaky snowflake+pandas tests (#8081) by @gibsondan
• e557f0e - docs: revamp the README (#8052) by @rexledesma
• 28f9930 - 0.14.17 changelog (#8083) by @smackesey
• 8e8ee85 - Fix upstream context handling in fs_asset_io_manager (#8007) by @aroig
• 3e68731 - KnownState.ready_outputs (#8016) by @alangenfeld
• d072b41 - Automation: versioned docs for 0.14.17 by @elementl-devtools
• aad66c0 - chore: mark FileCache for deprecation in 0.15.0 (#7922) by @rexledesma
• d980112 - remove step stats query (#8089) by @clairelin135

Note truncated.

New

• [dagster-aws] Added an env_vars field to the EcsRunLauncher that allows you to configure environment variables in the ECS task for launched runs.
• [dagster-k8s] The env_vars field on K8sRunLauncher and k8s_job_executor can now except input of the form ENV_VAR_NAME=ENV_VAR_VALUE, and will set the value of ENV_VAR_NAME to ENV_VAR_VALUE. Previously, it only accepted input of the form ENV_VAR_NAME, and the environment variable had to be available in the pod launching the job.
• [dagster-k8s] setting ‘includeConfigInLaunchedRuns’ on a user code deployment will now also include any image pull secrets from the user code deployment in the pod for the launched runs.

Bugfixes

• A recent change had made it so that, when IOManager.load_input was called to load an asset that was not being materialized as part of the run, the provided context would not include the metadata for that asset. context.upstream_output.metadata now correctly returns the metadata on the upstream asset.
• Fixed an issue where using generic type aliases introduced in Python 3.9 (like list[str]) as the type of an input would raise an exception.
• [dagster-k8s] Fixed an issue where upgrading the Helm chart version without upgrading your user code deployment version would result in an “Received unexpected config entry "scheme" at path root:postgres_db" error.

All Changes

https://github.com/dagster-io/dagster/compare/0.14.19...0.14.20

<details><summary>See All Contributors</summary>

• b6cf5d2 - 0.14.20rc0 by @elementl-devtools
• d421a1b - Fixes for asset graphql tests to make them easier to generalize and call with other graphql context fixtures (#8199) by @gibsondan
• 58fa8ca - bring back source asset metadata (#8195) by @sryza
• 561b81a - fix is_subclass bug where issubclass(list[str], DagsterType) throws a surprising exception (#8287) by @gibsondan
• de28811 - Add max_retries to user editable tags (#8285) by @johannkm
• 37d0062 - Derive origin from pipeline run instead of the arg to ExecuteRunArgs (#8156) by @gibsondan
• cb800cc - add an "environment" key to EcsRunLauncher / EcsContainerContext that sets env vars (#8243) by @gibsondan
• ed1b9b0 - Fix upgrade story for new scheme field in postgres (#8220) by @gibsondan
• 455fb97 - Add image pull secrets to k8s container context (#8221) by @gibsondan
• d1319e2 - extract event connection to top-level graphql query (#8077) by @prha
• 5e2badf - Create unified storage configuration for dagster.yaml (#7283) by @prha
• 601a9cb - [dagster-dbt] Fix bug where a dbt invocation that did not successfully start could emit materialization events. (#8293) by @OwenKephart
• aa5d8ca - Move config editor to core so we can reuse in admin portal (#8237) by @salazarm
• 8f9294a - Make dagster/reexecution_strategy tag editable (#8286) by @johannkm
• 2a8d755 - only return config field for solids (#8278) by @clairelin135
• 5f13fcd - clean up create_pg_connection (#8165) by @alangenfeld
• b6c65e4 - 0.14.20 Changelog (#8301) by @johannkm
• a44bb67 - 0.14.20 by @elementl-devtools

</details>

New

• Metadata can now be added to jobs (via the metadata parameter) and viewed in dagit. You can use it to track code owners, link to docs, or add other useful information.
• In the Dagit launchpad, the panel below the config editor now shows more detailed information about the state of the config, including error state and whether the config requires further scaffolding or the removal of extra config.
• FileCache is now marked for deprecation in 0.15.0.
• In Dagit, the asset catalog now shows the last materialization for each asset and links to the latest run.
• Assets can now have a config_schema. If you attempt to materialize an asset with a config schema in Dagit, you'll be able to enter the required config via a modal.

Bugfixes

• [helm] Fixed an issue where string floats and integers were not properly templated as image tags.
• [dagster-k8s] Fixed an issue when using the k8s_job_executor where ops with long names sometimes failed to create a pod due to a validation error with the label names automatically generated by Dagster.
• [dagster-aws] Fixed an issue where ECS tasks with large container contexts would sometimes fail to launch because their request to the ECS RunTask API was too large.

Breaking Changes

• fs_asset_io_manager has been removed in favor of merging its functionality with fs_io_manager. fs_io_manager is now the default IO manager for asset jobs, and will store asset outputs in a directory named with the asset key.

Community Contributions

• Fixed a bug that broke the k8s_job_executor’s max_conccurent configuration. Thanks @fahadkh!
• Fixed a bug that caused the fs_io_manager to incorrectly handle assets associated with upstream assets. Thanks @aroig!

Documentation

• [helm] Add documentation for code server image pull secrets in the main chart.
• The Dagster README has been revamped with documentation and community links.

All Changes

https://github.com/dagster-io/dagster/compare/0.14.17...0.14.19

<details><summary>See All Contributors</summary>

• 9d14055 - Automation: versioned docs for 0.14.17 by @elementl-devtools
• d52d4cb - Revert "throw error for empty asset key (#8069)" (#8093) by @gibsondan
• fab2e6e - [dagit] Launchpad: Show disabled "Scaffold" and "Remove" buttons instead of hiding them (#8066) by @hellendag
• b763dab - chore(buildkite): mention user in private channel (#8071) by @rexledesma
• 3d9ac9d - [asset-resources 2/n][rfc] io manager defs directly on asset defs (#7920) by @dpeng817
• b31b849 - docs: revamp the README (#8052) by @rexledesma
• 82a7a0c - 0.14.17 changelog (#8083) by @smackesey
• 9a14f45 - Fix upstream context handling in fs_asset_io_manager (#8007) by @aroig
• a558f1b - KnownState.ready_outputs (#8016) by @alangenfeld
• 5e6ba12 - chore: mark FileCache for deprecation in 0.15.0 (#7922) by @rexledesma
• c1fae94 - remove step stats query (#8089) by @clairelin135
• 1e06847 - migrate from deprecated sqlalchemy methods (#7864) by @alangenfeld
• c43efa9 - fix typos (#8090) by @OwenKephart
• 798eedc - Add skip for flaky grpc server test (#8097) by @gibsondan
• 2d3275c - chore(buildkite): ignore notifications on canceled builds (#8099) by @rexledesma
• beba51d - [dagit] Break apart LaunchpadSessionContainer (#8101) by @hellendag
• c53fe10 - [dagit] Small changes to the asset catalog (#7993) by @bengotow
• fa58bbf - [dagit] Track partition set sort order for Launchpad (#8104) by @hellendag
• 170fff0 - Lint rule to make sure queries requiring QueryVariables have them (#8102) by @salazarm
• adf8de6 - Make dagster-images build work when it is called outside of a git repo (#8088) by @gibsondan
• e2e5dfa - Fix configuration schema for k8s executor (#8107) by @fahadkh
• c42fa90 - [dagit] Clean up some yarn peer deps (#8108) by @hellendag
• 1108dfd - Add latestRun resolver to AssetsLatestInfo (#8072) by @clairelin135
• d3b00eb - Handle op outputs in default asset IO manager (#8074) by @clairelin135
• 6fde698 - [dagit] Add last materialization, latest run columns to the asset table (#7996) by @bengotow
• c36c26c - fix(mypy): refine types (#8129) by @rexledesma
• ad3a0f9 - fix(helm): allow numeric quoted strings as image tags (#8120) by @rexledesma
• fbc8477 - docs: add a button to copy/paste code snippets (#8106) by @yuhan
• 0f7417c - [asset-resources 4/n][rfc] Refactor resource requirement checking code (#7947) by @dpeng817
• 6855ea4 - fix(helm): use templated comment (#8137) by @rexledesma
• 9318254 - docs(helm): add doc hint for dagster-user-deployments.imagePullSecrets (#8112) by @ceefour
• 8bba00e - Added group_name to asset (#8110) by @shalabhc
• e254b76 - Fix invocation on ops that use generic dynamic outputs (#8133) by @dpeng817
• 6b6a7b7 - [dagit] Show more information for last run on Schedules/Sensors (#8130) by @hellendag
• 0240beb - KnownState parent run info (#8030) by @alangenfeld
• 7fb6ddb - add metadata to jobs (#7849) by @jamiedemaria
• 3901e47 - fix dagit-debug (#8148) by @alangenfeld
• 92f6fcd - add kwargs to pipeline snapshot from storage (#8149) by @jamiedemaria
• efe448e - Fix label sanitization for strings that end in a period (#8151) by @gibsondan
• c05b98d - [dagit] Fix duplicate styled-components (#8144) by @hellendag
• 02a17a5 - Basic asset config (#7590) by @smackesey
• 357022a - chore(helm): christen the code server service port name as grpc (#8142) by @calebfornari
• e5de6a3 - feat(helm): allow the postgresql scheme to be configurable (#8126) by @peay
• 10dc973 - Avoid launching ECS runs with large overrides (#8152) by @jmsanders
• 45e5a3e - [dagit] Remove clear-site-data header (#8134) by @hellendag
• b5c625e - fix(helm): use pre-2022 bitnami repository (#8166) by @rexledesma
• 3b69d5c - Asset config gql resolver (#8163) by @smackesey
• 163bc40 - Merge branch 'release-0.14.18' of https://github.com/dagster-io/dagster into release-0.14.18 by @OwenKephart
• 36ff292 - changes (#8169) by @OwenKephart
• 85fadac - [dagit] asset Config (#8154) by @smackesey
• bf4aaf3 - 0.14.18 by @elementl-devtools
• 612d82e - 0.14.18rc5 by @elementl-devtools
• e84ed05 - 0.14.18rc6 by @elementl-devtools
• bda8c50 - [easy] 0.14.18 => 0.14.19 in changelog (#8180) by @gibsondan
• 8a028c2 - 0.14.19 by @elementl-devtools

</details>