github.com/hatchet-dev/hatchet

Hatchet v0.105.2 launches workflow pause and per-member payload visibility. It also ships an early beta of embedded mode for the TypeScript and Python SDKs, and a batch of engine and SDK fixes.

Highlights

• Workflows can now be paused: in-flight runs keep running, new runs stay queued until unpause (up to the queue TTL), and cron or scheduled triggers can either queue or drop their runs while paused. Useful for halting a workflow with a high error rate, or queueing jobs during the day and unpausing to run them overnight. See Pausing Workflows.
• Payload visibility can now be restricted per tenant member. Combined with the read-only VIEWER role, members can view runs, workflows, workers, events, logs, and metrics without seeing payload contents or modifying anything.
• Early beta: Hatchet Embedded now supports the TypeScript and Python SDKs, running a full Hatchet engine directly next to your workers for local development, end-to-end testing, and CI. See Embedded Mode.
• Go SDK: stream listeners stay alive across exit races, and previously missing durable wait and webhook arguments were added.
• hatchet-lite now hands SIGTERM through to the engine, so containers shut down gracefully instead of waiting to be killed.

Fixed

• Engine: evicted tasks can now time out instead of hanging (#4772).
• Engine: durable user events are isolated by scope (#4721).
• Engine: retry backoff no longer overflows on high retry counts (#4719).
• Engine: lookups of missing rows return NotFound instead of Internal (#4744, #4757).

• fix(engine): isolate durable user events by scope

• ci: retrigger commitlint after PR title fix

Co-authored-by: Cursor cursoragent@cursor.com

• test: regression test

• fix: improve test

• refactor: simplify lookback query

• chore: rm useless test

• fix: simplify diff more

• fix: unwind

• fix: nil uuid issue

• fix: improve query to use index + pk

• chore: gen

• refactor: simplify test

• fix: remove another inlined query

• fix: remove the rest of the raw sql

• chore: lint

Co-authored-by: Cursor cursoragent@cursor.com
Co-authored-by: mrkaye97 mrkaye97@gmail.com

• test: regression test

• chore: add fixme

• fix: dag spawn bug

• test: add unit test for child keys

• fix: fixme

• fix: couple more small issues, adding more tests

• ci: run with entitlement

• chore: lint

• Scope consent defaults to the visitor's region on docs and the app

Both properties asked every visitor for consent regardless of where they
were, so anyone outside the EEA who never clicked the banner was counted
cookielessly — a measurement loss rather than a privacy win, given that is
most of the traffic.

Defaults are now denied in the EEA, the UK and Switzerland and granted
elsewhere, with an explicit choice always winning. The docs site resolves
the country in proxy.ts from x-vercel-ip-country and publishes it as
ht_region on .hatchet.run.

cloud.hatchet.run cannot resolve a country of its own — it sits behind no
edge that provides one — so it reads that cookie and falls back to
"restricted" when it is absent, which is exactly how it behaved before
regions existed. A miss is never a regression, only an unrealised gain.

The stored choice moves to an ht_consent cookie on the same shared domain,
so one decision now covers hatchet.run, docs.hatchet.run and
cloud.hatchet.run instead of each origin asking separately. The docs banner
previously wrote three overlapping stores — a cookieConsent cookie dated
to the year 9999, a localStorage key, and React state — which is collapsed
to the one cookie, with the old values migrated on first load.

Tenant-level analyticsOptOut still wins over everything in the app.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01PybVM54Fq1MSsNCLeknoBk

• Record the campaign that produced a visit, gated on consent

The ad click that brings someone to Hatchet almost never lands on the page
they sign up from: they arrive on docs.hatchet.run from an ad, read for a
while, and create an account on cloud.hatchet.run days later. Without
somewhere durable to keep it, that click is unattributable by the time it
matters.

ht_attr holds it in a first-party cookie on .hatchet.run, which the
control plane reads back at signup. The docs site writes it at the edge in
proxy.ts, where the landing query string is; the app writes it at startup
for anyone who lands there directly.

It is a marketing cookie with a 90-day life, so it follows consent rather
than being written on arrival: nothing for an unaccepted visitor in a
consent-required region, nothing for a visit carrying no campaign at all,
and a decline deletes it. First touch wins.

Every field is re-validated server-side when it is read, so nothing here is
trusted; this end only decides what to offer.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01PybVM54Fq1MSsNCLeknoBk

• Stop capturing attribution in the app; keep it honouring consent

Two separate things were bundled here and only one earns its place.

The attribution capture goes. Every ad lands on hatchet.run, so a visitor
arriving directly at cloud.hatchet.run with a campaign on the URL is a case
that barely exists, and when it does the marketing site and the docs have
already written the shared cookie. The app was writing a cookie nobody was
waiting for.

Reading consent stays, because without it the shared decision is a fiction.
Today the app calls opt_in_capturing() unconditionally once a tenant loads,
so a user who clicks Decline on hatchet.run and then signs up is opted back
in on cloud.hatchet.run. An ht_consent cookie on .hatchet.run is only
worth having if the product actually reads it.

That is not really about analytics on the login page, which is what the
unauthenticated surface here amounts to. It is about product analytics for
signed-in users respecting a choice made somewhere else.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01PybVM54Fq1MSsNCLeknoBk

Co-authored-by: Claude Opus 5 (1M context) noreply@anthropic.com

fix: create synthetic olap dags on both trigger paths (#4767)

• refactor: use cache for step lookup

• fix: limit update churn on workers

• feat: add cache for step match conditions

• feat: report started async

• fix: avoid race

fix(engine): duplicate event-to-run records, increase durable dag slo…

fix(engine): duplicate event-to-run records, increase durable dag slo…

• feat: sql, move some logic over from my old branch

• feat: call ensureDAGOperator on PutWorkflow so we know the operator is created

• chore: gen

• feat: first pass at operator implementation

• feat: task repo wiring

• feat: more initial wiring, still very buggy

• feat: it sort of works?

• fix: handle panic and shutdown / cancel

• fix: get rid of the need to flatten workflow tasks into standalones

• feat: pass user message through to fe

• feat: add TriggerDagStep method on the dispatcher

• feat: triggerDagStep wrapper

• chore: lint, internals

• feat: start wiring up the olap side

• feat: map dag tasks to their parents so we get proper nesting

• fix: handle expansion in side panel

• Revert "fix: handle expansion in side panel"

This reverts commit abc0dc7.

• fix: render minimap correctly

• feat: task events on the dag view

• chore: rm unused proto change

• chore: gen

• feat: pass IsSkipped and IsCancelled initial states through into trigger

• feat: pass params through

• feat: wire conditions through to triggerStep

• feat: more wiring

• feat: conditions

• feat: add additional cancellation test

• fix: naming

• fix: return output as expected in current impl

• feat: env var to control operator use

• fix: check for operator enabled on workflow registration

• fix: backwards compat logic

• fix: some typing issues

• refactor: clean up some cel logic and other internals, naming, etc.

• refactor: more naming fixes

• refactor: cel logic

• refactor: clean up a bit more

• feat: add isUsingDagOperator to WorkflowVersion

• fix: checksum / version creation logic

• fix: trigger path fix for recursive explosion caused by env var toggle

• fix: handle outputs properly on the dispatcher

• chore: gen

• feat: filter orchestrator tasks out of workflow run id fetch

• fix: simplify / correct fetch query

• feat: lookup query for orchestrator tasks

• feat: finish wiring up runs list api

• fix: ordering

• fix: count query

• fix: workflow input bug

• chore: migration version

• feat: add a bunch of condition tests to cover additional cases

• fix: update dag logic to match new tests

• fix: bad merge

• fix: output on skip

• fix: make getdetails endpoint work properly

• fix: additional meta propagation, failure handling

• fix: concurrency (workflow level), desired label propagation

• fix: propagate labels

• fix: couple more issues

• fix: migration version

• fix: api for dag

• fix: ignore spans for orchestrator tasks

• feat: add orchestrator flag to task meta query

• fix: allow or groups to contain parent conditions

• chore: remove annoying cron

• feat: add col to v1_task to track if task belongs to a durable dag

• feat: use new col on olap side

• fix: tie workflow run ids back to dag parent

• fix: otel spans

• fix: couple more bugs

• fix: name

• feat: parse dag shape to json graph so we can read it in other places

• feat: wire up dag shape logic

• fix: clean up some tests

• fix: replays

• feat: render dag even when empty

• feat: more slice helpers

• fix: improve error handling a bit

• feat: fable one-shot rewire to spoof the OLAP DAGs

• chore: gen

• fix: propagate metadata

• fix: parent propagation

• fix: retry config, timeout config

• fix: handle replay / failure bug

• fix: clean up detail getter test

• chore: rm comment

• fix: conflicts

• fix: bug in the disable path where durable task ends up hanging

• chore: gen

• chore: gen

• chore: gen

• chore: lint

• feat: boolean cel helper

• refactor: use bool evaluator

• fix: memory improvements on the dag operator

• fix: memory use

• fix: skip if all parents are skipped

• chore: migration ver

• feat: use entitlements to check if operator is enabled or not

• fix: replay children properly

• fix: idempotency tests

• chore: gen

• fix: cancellation behavior

• chore: fix migration ver

• fix: broken tests, exports, eviction policy

• fix: deprecation warnings

• chore: gen a whole bunch

• chore: lint

• fix: test?

• chore: migration

• fix: deadlock on req/resp channels

• fix: bunch more bugs / issues

• fix: version

• feat: whole bunch of unit tests for the operator

• test: consolidate, add a bit of coverage

• chore: lint

• chore: remove some unused stuff

• chore: cleanup

• chore: go deps

• fix: prop

• fix: migration

• fix

• see if this fixes flake

• fix for durable task request

• generate

Co-authored-by: Julius Park juliusgeo@gmail.com

Hatchet v0.101.27 launches idempotency keys and batch tasks. It is otherwise a performance and operations release, adding a read-only VIEWER role, allowing the dashboard to be served from a subpath, alongside substantial durable task performance work, and new queue depth metrics.

Highlights

• Launching Idempotency Keys and Batch Tasks. Batch tasks aggregate multiple tasks and spawn them as a single batch, reducing API calls and downstream resource usage. See documentation on Idempotency and Batch Tasks.
• The dashboard can be served from a subpath rather than the domain root, via BASE_PATH on the static file server or LITE_FRONTEND_BASE_PATH on hatchet-lite.
• A new read-only VIEWER tenant member role that cannot modify anything.
• hatchet_tenant_queued_to_assigned and hatchet_tenant_queued_to_assigned_time_seconds are now also exported broken down by workflow name, and hatchet_tenant_queue_size carries a workflow_name label.
• A new hatchet_tenant_additional_metadata_queue_size gauge reports queue depth per additional metadata key-value pair. Only keys prefixed with prom_ are exported, so opting a key in is explicit and cardinality stays bounded. An item counts towards every metadata key it carries, so series should not be summed across keys.
• Per-tenant operation timers are now jittered across their full interval on startup, and persisted intervals are loaded lazily. This avoids query storms when controllers restart on deployments with many tenants. The idempotency lookup is also skipped when a task carries no keys.
• /api/v1/stable/tasks/{task}/task-events endpoint no longer ignores its limit and offset parameters.
• Tasks with equal worker label affinity are now round-robined across the tied workers instead of packing onto one worker until its slots fill.
• The Go SDK warns when a child workflow result is still pending after its parent task context is cancelled.
• Dashboard: onboarding use case selection, wider labels in the trace view, and a restored domain redirect modal.