NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #4442 most downloaded on PyPI
Python client for Taskcluster
Last release 13 days ago
21 Sep 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
12 years old
491 releases · first in 2014
▶ [minor] bug 1599247 The auth service's gcpCredentials endpoint now supports any number of GCP projects configured in gcp_credentials_allowed_project
▶ [minor] bug 1599247
The auth service's gcpCredentials endpoint now supports any number of GCP projects configured in gcp_credentials_allowed_projects, rather than being limited to a single project. Each configured project keeps its own credentials and allowedServiceAccounts allow-list, and requests are scoped to the project named in the request path.
▶ [patch] bug 2071937
Generic Worker on Windows no longer interpolates payload.osGroups into a PowerShell -Command string when adding or removing the task user from OS groups. Membership is updated with NetLocalGroupAddMembers / NetLocalGroupDelMembers, so group names cannot break out of a PowerShell single-quoted literal (including via Unicode quotation marks U+2018–U+201B).
▶ [patch]
On Windows, the worker now replaces the whole security descriptor of the files it secures instead of just dropping ACL inheritance on it. The ownership is now also changed to BUILTIN\Administrators
▶ [minor]
index.findTasksAtIndex no longer returns expired indices
▶ [patch] #9127
Dashboard shows pending and claimed task counts for task queues that worker-manager does not own (hardware workers, for example).
These queues were previously missing from the dashboard's task totals entirely.
Discovering them requires the queue:list-task-queues scope, plus queue:pending-count:<taskQueueId> and queue:claimed-count:<taskQueueId> for each queue.
▶ [patch] #9172
In the python client, importing taskcluster.helper before anything from taskcluster.aio no longer leaves taskcluster.aio without any of its clients
▶ [patch]
Indexing a task no longer silently does nothing when an expired entry with a higher rank still exists at the same index path
▶ [patch] #9065
The worker pools list now shows a "Claimed Tasks" column alongside "Pending Tasks". The claimed counts already came back from the batched taskQueueCounts request the page makes, so this adds no extra API calls.
▶ [patch] #9065
UI: Dashboard and Worker Manager load pending/claimed counts in batched requests. Stopping capacity is no longer shown.
Public deployments must grant both queue:pending-count:* and queue:claimed-count:* to the anonymous role for these counts to appear.
▶ [patch]
UI: hardened task artifact log links with proper encoding.
▶ [patch] bug 2072160
Worker-manager improves error handling when duplicate static worker is created.
▶ [patch]
yarn db:new and yarn db:renumber don't require you to provide either ADMIN_DB_URL or USERNAME_PREFIX anymore
▶ Additional changes not described here: #9119, #9127.
One column per quarter.
▶ [patch] #7447 Generic-worker no longer chowns read-only content that is mounted as the task user.
▶ [patch] #7447
Generic-worker no longer chowns read-only content that is mounted as the task
user.
▶ [patch] bug 2071940
Use LSA to store the next task user's password instead of storing it as plaintext in the registry
▶ [patch] #9156
Worker-pool errors from failed Azure ARM deployments now include the nested, actionable ARM error in the error description. This makes errors like "image X was not found in " visible in the UI and API.
▶ [MAJOR] bug 2060945
A writable directory cache will now refuse being mounted at a directory that already exists.
▶ [MAJOR] bug 2060945
Moving a directory into place now refuses a destination that already exists on
every platform: on Windows it no longer replaces an existing file, and on POSIX
systems it no longer replaces an existing empty directory.
▶ [patch] #9007
Generic Worker no longer panics when the Queue rejects a createArtifact call with a 4xx response. The task is resolved as exception/malformed-payload, or as exception/resource-unavailable for 408 and 429. logs.live and logs.backing must now match ^[\x20-\x7e]+$, and a non-empty artifact name must match the same character set. An empty artifact name is still allowed and means "derive from path"; if that path contains other characters, the Queue still rejects the artifact at upload time.
▶ [patch] #9058
GitHub tasks no longer fail when concurrent pull request events race with automatic cancellation.
▶ [patch] bug 2072198
Mounting a cache on Windows no longer fails when the cache contains a file that is hardlinked more than once inside the cache itself
▶ [MAJOR] The deprecated Auth service Azure Credentials API methods have been removed: azureAccounts , azureTables , azureTableSAS , azureContainers ,…
▶ [MAJOR]
The deprecated Auth service Azure Credentials API methods have been removed: azureAccounts,
azureTables, azureTableSAS, azureContainers, and azureContainerSAS. No known Taskcluster
component uses these methods.
The auth.azure_accounts Helm property is no longer allowed, and the corresponding
AZURE_ACCOUNTS environment variable is no longer used. Deployers must remove
auth.azure_accounts from their Helm values before upgrading.
▶ [MAJOR] bug 2069456
When uploading artifacts, Generic Worker multiuser engine will now create
temporary files as the worker user (root/LocalSystem) and stream content
into them as the task user.
The internal generic-worker copy-to-temp-file command has been replaced with
generic-worker cat-file
▶ [MAJOR] bug 2069332
When uploading an optional artifact, if it's not readable (or encounters any
unreadable file for directory artifacts), the task will now fail instead of
silently omitting that file.
▶ [minor] #9065
The Queue service now exposes taskQueueCountsBatch to fetch pending and
claimed task counts for multiple task queues in one request.
▶ [patch] Bumps uv to v0.12.8 for the in-tree ci and python docker images, the taskgraph decision image to v24.2.3, the git for windows version to v2.
▶ [patch]
Bumps uv to v0.12.8 for the in-tree ci and python docker images, the taskgraph decision image to v24.2.3, the git for windows version to v2.55.0, and the nvm version used during releases to v0.40.7.
▶ [patch]
Upgrades to Node.js v24.20.0.
▶ [patch]
Upgrades to go1.27.1 and golangci-lint v2.13.2.
Release notes here.
▶ [patch]
Upgrades to rust v1.98.0.
▶ [patch]
Generic Worker no longer deadlocks on shutdown or when interrupted with Ctrl+C / SIGINT while a task is running. Shutdown waits on task completions until no tasks remain, instead of blocking on a wait group that only advanced when those completions were processed.
▶ [patch] bug 2069456
On Windows multiuser workers, command environment read by generic-worker from the task directory will now refuse to follow links.
▶ [patch] bug 2069456
On Windows multiuser workers, command scripts written by generic-worker into the task directory will now refuse to follow links.
▶ [minor] bug 1917274
The github service publishes a new exchange/taskcluster-github/v1/taskcluster-yml-update
message when a push changes a repository's .taskcluster.yml. The ordinary push
message is still published as well, so existing consumers are unaffected.
The payload names the organization, the repository, the ref that was pushed to, and
the webhook delivery id, and nothing else. It deliberately does not carry the file's
contents. A consumer can therefore act on a push in one repository from inside
another, treating the ref as a value to compare against rather than one to pass on.
▶ [patch] bug 2066797
Changes the pull-request policy to public_restricted and isolates trusted and untrusted task graphs. External pull requests run at level 1 with separate caches, without secrets or generic-worker CI, and rebuild Docker images instead of sharing an image index. Collaborators can trigger the full level-3 graph with /taskcluster run.
▶ [patch] #9093
UI Scopes pages (ViewScope and ScopesetExpander) switch from GraphQL to direct REST service calls.
▶ [patch] #9074
UI WMViewWorkers and WMViewWorkerPools pages switches to use direct REST API calls
▶ Additional change not described here: #9117.
▶ [MAJOR] bug 2060854 The web-server now validates REGISTERED_CLIENTS at startup. Client registrations with unknown properties, invalid property types
▶ [MAJOR] bug 2060854
The web-server now validates REGISTERED_CLIENTS at startup. Client registrations with unknown properties, invalid property types, or requirePkce: true when responseType is not code must be corrected before upgrading.
Startup validation also rejects a maxExpires that fromNow cannot parse or that does not resolve to a future date (such as '', 0 seconds or -1 year, which would have issued already-expired credentials), and non-unique clientId.
Each redirectUri must now be an absolute http: or https: URL.
▶ [MAJOR] bug 2065117
Workers will now refuse to hand a task ownership of a hardlinked file that
belongs to anyone but the previous owner of the tree being chowned.
▶ [MAJOR] bug 2059762
Workers will now refuse to operate caches / mounts if they have to resolve a
junction on windows or a symlink on linux/macos (unless the parent of the
symlink is only writable by root).
▶ [patch] bug 2058249
Fix a bug where uploading logs and writing CoT artifacts was following symlinks
▶ [patch] #8943
Generic Worker no longer garbage collects a file cache that a running task is still using in capacity > 1 cases. Relatedly, when a cached download no longer matches a task's required SHA256, the stale entry is now dropped from the cache table immediately (with its deletion deferred until any tasks still using it finish) instead of being served to the task again.
▶ [patch] #8944
Generic Worker no longer leaks disk space when cache files remain on disk without a cache table entry. Garbage collection and worker startup now delete anything in the caches directory that the worker does not know about, so a failed deletion (or a leftover from a crash) is retried instead of occupying space forever.
▶ [patch]
Generic-worker will once again report errors if internally ran commands fail
▶ [patch] bug 2058254
Generic worker on windows won't follow junctions anymore when changing
ownership/rights/deleting cached files.
▶ [MAJOR] bug 2060854
The web-server OAuth authorization-code exchange now requires the requesting client_id to match the client that received the code.
Existing authorization-code clients must include their registered client_id when exchanging codes.
Clients can also use PKCE with the S256 challenge method, and deployments can require PKCE for individual registered clients.
▶ [minor] #9056
Generic worker will now resolve a task as exception if it read the content of an
optional artifact but then failed to upload it
▶ [patch] bug 2060854
Third-party OAuth2 clients registered with more than one redirectUri are now granted CORS access from every registered origin. Previously only the first entry's origin was allowed, so calls to /login/oauth/token and /login/oauth/credentials from any other registered origin were blocked by the browser.
▶ [patch] #9066
Fixes UI regression in react-codemirror2 where editing text in any textarea would be very slow.
▶ [patch] bug 2064002
Notifications through task routes now validate the name of the template used just like the rest API
▶ [patch] #8751
The GitHub service now creates a build record for every unique taskGroupId defined in
.taskcluster.yml, so checks and statuses are reported for all task groups, not just
the first task's group.
▶ [patch] #8992
UI ClaimedTasks and PendingTasks pages switch from GraphQL to direct service calls
▶ [patch] #9011
UI Denylist page switches from GraphQL to direct service calls
▶ [patch] #9023
UI Hooks page switches to use direct REST API calls
The View Hook page showed a Next Scheduled Date which is now removed from UI.
This was done as GraphQL invoked an outdated REST endpoint getHookStatus for which we do not have any alternate endpoint or way to get this information.
▶ [patch] #9006
UI Task Index page switches to use decorator for api call. Removed the now-unused indexedTask, namespaces, and taskNamespace GraphQL queries and their resolvers/loaders from web-server, since the UI no longer uses them. Other GraphQL queries against Task (e.g. latestArtifacts, still used by the Interactive Connect page) are untouched.
▶ [patch] #9060
UI ViewProvisioners and ViewWorkerTypes pages switch from GraphQL to direct service calls
▶ [patch] #9072
UI ViewWorker and ViewWorkers pages switch from GraphQL to direct service calls
▶ [patch] #9045
UI WMEditWorkerPool and WMLaunchConfigs pages switch from GraphQL to direct service calls
▶ [patch] #9063
UI WMViewErrorCenter and WMViewErrors pages switch from GraphQL to direct service calls
▶ [MAJOR] bug 2062161 Removed the deprecated queue.declareProvisioner and queue.declareWorkerType methods.
▶ [patch]
Upgrades to go1.27.0 and golangci-lint v2.13.1.
Release notes here.
▶ [patch] bug 2062170
Pass taskcluster proxy credentials through environment variables instead of named parameters
▶ [MAJOR]
Removed all mentions of the unused queue actions feature.
The actions property is removed from the responses of
queue.listProvisioners, queue.getProvisioner, queue.getWorkerType,
queue.getWorker, and workerManager.worker, from the corresponding GraphQL
types, and from the web UI.
▶ [MAJOR] bug 2062161
Removed the deprecated queue.declareProvisioner and
queue.declareWorkerType methods.
▶ [patch]
Fixed unsafe handling of inherited JavaScript properties when rendering .taskcluster.yml files through v0 parameters and v1 as_slugid() labels.
▶ [patch] bug 2065461
Prevent the notify service from fetching files and remote URLs referenced in
the content of the emails it sends. This also stops the plain text part of
those emails from being mangled by the HTML processing.
▶ [patch] bug 2062161
queue.declareWorker now returns a 400 instead of a 500 when passed an empty body
▶ [patch] #8981
UI Clients page switches from GraphQL to direct service calls
▶ [patch]
Upgrades to yarn 4.18.0
▶ [MAJOR] bug 2064901 Github hooks triggered from .taskcluster.yml now require the github:trigger-hook:<name> scope instead of hooks:trigger-hook:<nam
▶ [MAJOR] bug 2064901
Github hooks triggered from .taskcluster.yml now require the
github:trigger-hook:<name> scope instead of hooks:trigger-hook:<name>.
▶ [minor] #8378
The auth service can now hand out repository scoped tokens through
/api/auth/v1/github/<app>/<owner>/repo-token. See the route documentation for more
information.
▶ [patch] bug 2062167
Fixed a bug where multiple interactive sessions on a worker with capacity greater than 1 would override eachother's secrets.
▶ [patch]
Upgrade json-e to 4.8.4.
▶ [patch] Upgrades to Node.js v24.19.0.
▶ [patch]
Upgrades to Node.js v24.19.0.
▶ [patch]
Upgrades to go1.26.6.
Release notes here.
▶ [patch] #9008
Generic worker now purges a task's writable directory caches when the worker
kills the task's commands (cancellation, max runtime, OOM), instead of keeping
a potentially corrupt cache
▶ [minor] bug 2064373
Slack notifications no longer show link previews by default. You can re-enable
that through the unfurlLinks / unfurlMedia fields on the slack API endpoint
or with task.extra.notify.slackUnfurlLinks / slackUnfurlMedia for route
based notifications.
▶ [patch]
Properly report errors from d2g when a docker image has an invalid name rather
than letting docker fail on it and reporting those errors.
▶ [patch]
The JS clients now throw an error if authorizedScopes is passed as anything
but an array (or null) instead of outright ignoring it in that case.
▶ [patch]
finishArtifact now reports a 400 instead of a 500 when called on a non object artifact
▶ [MAJOR] #8990 Generic worker has ipv6 enabled again for d2g tasks if it's enabled on the default bridge. Due to how docker handles ipv6 on networks
▶ [MAJOR] #8990
Generic worker has ipv6 enabled again for d2g tasks if it's enabled on the
default bridge. Due to how docker handles ipv6 on networks that are necessary
for capacity > 1, this raises the minimum docker version supported by generic
worker to 27.
▶ [patch]
Fix the UI crashing on a fresh OIDC signin
▶ [patch] #8994
UI Cache Purges page switches from GraphQL to direct service calls
▶ [patch] #8927
UI Create Task page switches to use decorator for api call
▶ [MAJOR] #8990 Generic worker has ipv6 enabled again for d2g tasks if it's enabled on the default bridge. Due to how docker handles ipv6 on networks that are necessary for capacity > 1, this raises the minimum docker version supported by generic worker to 27.
▶ [patch] Fix the UI crashing on a fresh OIDC signin
▶ [patch] #8994 UI Cache Purges page switches from GraphQL to direct service calls
▶ [patch] #8927 UI Create Task page switches to use decorator for api call
<details> <summary>5 Dependabot updates</summary>
</details>
…is discarded and no task is created. This is a breaking change: previously triggerSchema was only enforced on the API and webhook paths, and pulse mes…
▶ [patch] #8947
Worker-manager refuses to re-register worker if it is no longer in Running state or has expired.
▶ [patch] #8662
Github: display artifact size in check results
▶ [MAJOR] #8867
Pulse-triggered hooks now validate matching pulse message payloads against the hook's triggerSchema before creating a task.
If a pulse message matches the hook's bindings but fails triggerSchema validation, the message is discarded and no task is created.
This is a breaking change: previously triggerSchema was only enforced on the API and webhook paths, and pulse messages fired the hook regardless of their payload.
Validation is unconditional, including for hooks that did not set a triggerSchema: the default schema only accepts an empty payload, so such a hook will no longer fire on pulse messages that carry a payload.
Before upgrading, review all hooks with pulse bindings and make sure each triggerSchema accepts the pulse payloads that should still create tasks. Once deployed, "Debug Bindings" feature will be available in the UI.
▶ [minor] bug 2056618
Fixes a possible SSRF in the github and web-server services, which download task artifacts. Both
now resolve artifacts through the queue's artifact API and refuse to fetch a reference artifact,
whose URL is supplied by the task; s3, link, and object artifacts, whose URLs the queue
derives itself, are unaffected. The JS client exposes this as downloadManagedArtifact.
Two visible consequences:
customCheckRun.textArtifactName or annotationsArtifactName must be a stored artifact. A reference now produces an explanatory comment on the commit instead of being fetched.live.log, which is a reference to the livelog server by design. It falls back to live_backing.log, so resolved tasks are unaffected.▶ [minor] #8867
The hook page now has a Debug bindings button that opens a Pulse-binding debugger drawer.
It watches the Pulse messages arriving on the hook's saved bindings and shows, per message, whether the payload passes the hook's triggerSchema or is .
This makes it easy to see why a Pulse-triggered hook is silently not firing after triggerSchema validation was introduced, without reading server logs.
▶ [patch] #8711
The /tasks and /tasks/groups pages now show up to 20 recently viewed tasks/task groups
with task name, queue, age and status information instead of 5 bare identifiers.
▶ [patch] #8978
UI Roles page switches from GraphQL to direct service calls
▶ [patch] #8901
UI Secrets page switches from GraphQL to direct service calls
▶ [patch] #8947 Worker-manager refuses to re-register worker if it is no longer in Running state or has expired.
▶ [patch] #8662 Github: display artifact size in check results
▶ [MAJOR] #8867
Pulse-triggered hooks now validate matching pulse message payloads against the hook's triggerSchema before creating a task.
If a pulse message matches the hook's bindings but fails triggerSchema validation, the message is discarded and no task is created.
This is a breaking change: previously triggerSchema was only enforced on the API and webhook paths, and pulse messages fired the hook regardless of their payload.
Validation is unconditional, including for hooks that did not set a triggerSchema: the default schema only accepts an empty payload, so such a hook will no longer fire on pulse messages that carry a payload.
Before upgrading, review all hooks with pulse bindings and make sure each triggerSchema accepts the pulse payloads that should still create tasks. Once deployed, "Debug Bindings" feature will be available in the UI.
▶ [minor] bug 2056618
Fixes a possible SSRF in the github and web-server services, which download task artifacts. Both
now resolve artifacts through the queue's artifact API and refuse to fetch a reference artifact,
whose URL is supplied by the task; s3, link, and object artifacts, whose URLs the queue
derives itself, are unaffected. The JS client exposes this as downloadManagedArtifact.
Two visible consequences:
customCheckRun.textArtifactName or annotationsArtifactName must be a stored artifact. A reference now produces an explanatory comment on the commit instead of being fetched.live.log, which is a reference to the livelog server by design. It falls back to live_backing.log, so resolved tasks are unaffected.▶ [minor] #8867
The hook page now has a Debug bindings button that opens a Pulse-binding debugger drawer.
It watches the Pulse messages arriving on the hook's saved bindings and shows, per message, whether the payload passes the hook's triggerSchema or is .
This makes it easy to see why a Pulse-triggered hook is silently not firing after triggerSchema validation was introduced, without reading server logs.
▶ [patch] #8711 The /tasks and /tasks/groups pages now show up to 20 recently viewed tasks/task groups with task name, queue, age and status information instead of 5 bare identifiers.
▶ [patch] #8978 UI Roles page switches from GraphQL to direct service calls
▶ [patch] #8901 UI Secrets page switches from GraphQL to direct service calls
<details> <summary>22 Dependabot updates</summary>
</details>
▶ [patch] Generic worker now continues trying to garbage collect caches even if one removal fails for any reason
▶ [patch]
Generic worker now continues trying to garbage collect caches even if one
removal fails for any reason
▶ [patch] #8942
Fix workers panicking if a task that's not resolved yet would exhaust enough
disk space for the worker to not meet their minimum disk space required to
claim new tasks.
▶ [patch]
Generic worker will try evicting writable cache directories again when garbage
collecting. This was regressed in v100.0.0
▶ [patch]
Workers will now reliably clean task directories / users again. The cleanup
behavior was regressed in v100.0.0
▶ [minor] The Helm chart now supports annotations on Taskcluster workload pods. Use the global podAnnotations map for every Deployment and CronJob pod
▶ [minor]
The Helm chart now supports annotations on Taskcluster workload pods. Use the global podAnnotations map for every Deployment and CronJob pod. A service process's podAnnotations map is merged with the global map for that workload, with process values taking precedence for matching keys.
▶ [patch]
Fixed a potential panic in websocktunnel when a worker sent a malformed ACK
▶ [patch]
Websocktunnel will now close sessions when a client sends an invalid packet instead of silently ignoring it
▶ [patch] bug 2057491
Fixed a bug in the auth service's scope resolver where the scopes :*, ::*,
:a*, :as*, :ass*, :assu*, :assum* and :assume* were expanded as if
they were *
▶ [MAJOR] #8869
The client-rust AsyncWriterFactory now needs to be Send and the get_writer method now takes an optional content_length parameter that indicates what size the writer should expect. The get_writer method is also now only called when the initial request succeeded and the response stream is about to be pulled.
▶ [patch] #7590
Add UI to view audit history for client, roles, hooks, secrets
▶ [patch]
Fix a potential panic in the interactive feature when a window resize was sent with fewer bytes than expected
▶ [patch] bug 2056592
Fixed a bug in the auth service where a concurrent modification conflict while
creating, updating, or deleting a role was silently reported as success
▶ [patch]
Fixed a bug in the auth service where purging an expired client recorded the
deletion in the audit history as created instead of expired
▶ [patch] bug 2056597
Fixed a worker panic when declaring a directory artifact as an absolute path
▶ [patch]
The github YAML debug route now properly reports templating issues as a 400 instead of a 500
▶ [patch] #8533
Refreshed the Azure IMDS attested-data test fixture (services/worker-manager/test/fixtures/azure_signature_good.json), whose leaf certificate expired on 2026-07-28.
The new document is signed under the post-2025 Microsoft TLS RSA Root G2 hierarchy, so the Microsoft TLS G2 RSA CA OCSP 02 and 04 intermediates are now bundled in the worker-manager Azure CA store.
▶ [patch] #8912
UI adds withTaskclusterClient decorator to create rest client removing the need to using props
for rest client creation
▶ Additional change not described here: bug 2055774.
▶ [patch] #8890 Generic Worker FreeBSD build has been fixed.
▶ [patch] #8890
Generic Worker FreeBSD build has been fixed.
▶ [patch] #8890 Generic Worker FreeBSD build has been fixed.
<details> <summary>4 Dependabot updates</summary>
</details>
▶ [patch] Remove support for google analytics ( GA_TRACKING_ID doesn't do anything anymore) in the UI as it's been unsupported and untested for years.
▶ [patch]
Remove support for google analytics (GA_TRACKING_ID doesn't do anything
anymore) in the UI as it's been unsupported and untested for years.
▶ [patch] #8664
When a worker-manager checkWorker call times out, the scanner now aborts the in-flight provider API call and logs a warning instead of reporting an error, since the worker is re-checked on the next loop. Other checkWorker failures are still reported as errors.
▶ [patch]
The Google provider in worker-manager now logs a single transient GCP compute 5xx at notice rather than warning level.
▶ [patch] bug 2053178
Fix a way for PRs from forks to bypass the public_restricted policy by
declaring version: 0 in their own taskcluster.yml
▶ [patch] bug 2053380
Fix login CSRF in the github login flow by validating the oauth state parameter
▶ [patch] #8171
Task creation in the UI now calls the Queue API directly instead of routing
through GraphQL, fixing 413 errors when creating tasks with large payloads
(up to the Queue API's 10MB limit).
▶ [minor]
Switch the UI build system to vite. Switch tests to vitest
▶ Additional change not described here: #4007.
▶ [minor] #8815 The worker-manager worker scanner can now check workers with bounded concurrency instead of strictly one at a time. It now checks up t
▶ [minor] #8815
The worker-manager worker scanner can now check workers with bounded concurrency instead of strictly one at a time.
It now checks up to 2 workers at once by default; set the WORKER_SCANNER_CONCURRENCY environment variable to tune this.
The per-provider CloudAPI rate limiter still bounds the actual cloud API call rate, so this only controls how much of that rate budget
each scan loop uses. Applies to both the Azure scanner and the GCP/AWS scanner.
▶ [MAJOR]
Drop support for application/graphql as the request content type on
/graphql. Use application/json instead as is specified in the graphql over
http spec.
▶ [minor] #8660
Relax the required node version for taskcluster-client from 24.17.0 to ^24
▶ [patch] #8009
Fixed a bug where the github status API was marked resolved as success before any re-ran tasks were resolved
▶ [patch] #4585
Generic worker interactive shells now set TERM to xterm-256color instead of
hterm-256color which fixes some whitespace quirks on copy
▶ [patch] #7773
The github service will now respect secondary rate limits and retry those
requests instead of just commenting that they failed.
▶ [MAJOR]
The @taskcluster/client-web package is now published as a native ES module.
Any consumer that was importing the package via CommonJS require() must now
import it as an ES module.
▶ [patch] #4959
Removed the outdated Makefile and the lint.sh/test.sh helper scripts in
client-py. Call uv directly instead: uv run pytest to test, uv run ruff check to lint and uv run ruff format to format
▶ Additional change not described here: #8761.
▶ [patch] Bump node version to 24.17.0 security release.
▶ [patch] Bump node version to 24.17.0 security release.
▶ [minor] #8161
The Azure provider now detects if resources will be cascade-deleted with the VM (deleteOption: 'Delete').
When they provably cascade, deprovisioning skips the per-resource GET/delete walk and just calls VM delete + 404 confirm,
cutting the redundant Azure API calls.
Detection is best-effort and fails open: any uncertainty (multi-NIC/IP, Detach, missing fields, probe error/timeout, or
tracked resources that are not VM-owned) falls back to the existing resource-by-resource deletion.
New log type: azure-teardown-mode and worker_manager_azure_teardown_total metric record whether each teardown used the fast or slow path.
▶ [patch] #6561 On Windows, generic-worker now transfers ownership of mounts in addition to giving the task user full control.
▶ [patch] Prevent a worker preemption to be reported twice for the same run when the worker had time to report it itself
▶ [patch] #8372 Switch linting from eslint to biome on all services/libraries
<details> <summary>4 Dependabot updates</summary>
</details>
▶ [minor] The worker-manager Azure provider now exposes ARM deployment creation failures and failed deployment operations as a Prometheus counter for
▶ [minor] The worker-manager Azure provider now exposes ARM deployment creation failures and failed deployment operations as a Prometheus counter for observability.
New Prometheus metric:
worker_manager_azure_arm_deployment_errors_total (counter) - incremented once for each ARM deployment creation failure or failed deployment operation, labeled by providerId, workerPoolId, workerGroup, errorKind, errorCode, statusCode, provisioningState, provisioningOperation, targetResourceType, vmSize, and priority.This lets deployers chart Azure ARM deployment creation failures and failed operations by worker pool, region, Azure error code, and VM size in Prometheus/Grafana.
▶ [patch] #8721 The UI and references nginx servers now support Brotli compression in addition to gzip, allowing browser requests that prefer Brotli to receive compressed static assets, schemas, and references.
▶ [minor] bug 2045069
Generic-worker: skip gzip compression for artifacts with extensions matching .pkg or .wasm.
▶ [patch]
Fake taskcluster clients now properly raise an error when used in a production NODE_ENV
▶ [patch] #8758 The hooks service now properly forwards network errors from failures to contact the queue service instead of masking it with some serialization error
▶ [patch] #8198
The API reference pages now show correct example commands for curl and the
taskcluster CLI. The curl examples previously showed an invalid
Authorization: Bearer header (Taskcluster uses Hawk authentication); they
now explain that taskcluster-proxy should be used instead. The taskcluster CLI
examples previously showed path arguments as named flags (--taskId value)
when the CLI actually requires positional arguments; they are now shown
correctly.
▶ Additional change not described here: #8595.
<details> <summary>9 Dependabot updates</summary>
</details>
▶ [patch] Upgrades to go1.26.4.
▶ [patch] Upgrades to go1.26.4.
Release notes here.
▶ [minor] #8347
HTTP responses from Taskcluster API services are now gzip-compressed when the client sends Accept-Encoding: gzip. Compression is applied at the @taskcluster/lib-api router level with a 1 KB threshold, so small payloads are sent uncompressed.
▶ [minor] #8716
Removed the sift dependency from the web-server. The GraphQL filter: JSON argument was only ever used for simple case-insensitive substring search, so it has been replaced with a typed searchTerm: String argument on the list queries that support search (clients, roles, secrets, worker pools, denylist addresses). Task-action filtering is now applied server-side, and the unused filter argument has been removed from the remaining GraphQL queries.
▶ [minor] bug 2045069
Generic-worker: skip gzip compression for artifacts with extensions matching .aab, .apk, .jar, .xpi.
▶ [patch] bug 2042324 Enforce oauth2 access token lifetimes when issuing Taskcluster credentials
▶ [patch] #8688 Fix the changelog page so it doesn't ignore URL parameters anymore
▶ [patch] #8689
Fixed on-defined route notifications not firing
▶ [patch]
Bump node version to 24.16.0
<details> <summary>9 Dependabot updates</summary>
</details>
▶ [patch] Bump local-dev nginx image from 1.29.4 to 1.30.1 (CVE-2026-42945).
▶ [minor] #8574
The Azure provider in worker-manager now submits VM beginDelete inline from removeWorker, removing one scanner cycle of latency before cloud-side deletion. Brings Azure to parity with GCP and AWS; the worker-scanner remains the fallback and verifier.
▶ [patch] bug 2042324 Enforce a 10 minutes lifetime to provided oauth2 authorization tokens
▶ [patch] Fix the changelog page filters to actually reflect what's in the URL.
▶ [patch] bug 2042324 Invalidate oauth2 authorization codes immediately on exchange
▶ [patch] #8658
UI: fixes Queue Workers table appearing unsorted after navigating away and back to the page.
▶ [patch] Use the current version for taskcluster in API code examples instead of a hardcoded '93'.
▶ [patch] Bump local-dev nginx image from 1.29.4 to 1.30.1 (CVE-2026-42945).
▶ [patch]
Drop yarn minify from our commands. It's been broken for 3 years without
anyone complaining. To minify lockfiles, use the builtin yarn dedupe instead.
▶ Additional changes not described here: #8377, #8372.
<details> <summary>9 Dependabot updates</summary>
</details>
…versions, resolving multiple critical security vulnerabilities.
▶ [patch]
Replaces the deprecated sentry-api package with a small Sentry API client in the Auth service and upgrades transitive form-data dependencies to patched versions, resolving multiple critical security vulnerabilities.
▶ [patch] Display YAML parser errors in the github taskcluster yml debugger
▶ [patch] Display errors when fetching error stats for worker pools fails
▶ [patch] #8625
Explicitly ensure the GitHub service can fetch the well-defined artifacts used
as integration points. Previously the static/taskcluster/github client
attempted to assume the hook role for hook-created tasks, which it does not
have scopes to do.
▶ [patch] #5431 Fix the worker view's recent tasks table rendering duplicate rows when the same task has been run more than once on the worker.
▶ [patch] #8631
GitHub check runs now report started_at based on the time the task was actually claimed by a worker, rather than when the task was first defined. Previously, the GitHub Checks UI showed elapsed time from when a task was first scheduled, making queued/pending time appear as running time.
▶ Additional change not described here: #6866.
<details> <summary>10 Dependabot updates</summary>
</details>
▶ [patch] #8613 Migrates the taskcluster shell client's Homebrew release pipeline from the deprecated GoReleaser brews field to homebrew_casks. brew i…
▶ [patch] Upgrades to go1.26.3 and golangci-lint v2.12.2.
Release notes here.
▶ [minor] #8586
Fixes a bug in worker-manager where the worker-scanner could incorrectly mark a running worker as over capacity and tell it to terminate, even when it was the sole running worker in a pool with minCapacity >= 1. Caused by offset-based pagination yielding duplicate rows when other workers were inserted mid-scan, leading to conflicting termination decisions for the same worker. The provisioner had the same root cause, silently over-counting existingCapacity on pools with more than 1000 non-stopped workers and under-provisioning them. Both call sites now use keyset pagination via a new function get_non_stopped_workers_with_launch_config_scanner_after in DB version 0125. The most visible symptom was excessive worker churn on small pools.
▶ [patch] Fixed the diff viewer in the scopes/client editing pages
▶ [patch] #8613
Migrates the taskcluster shell client's Homebrew release pipeline from the deprecated GoReleaser brews field to homebrew_casks. brew install taskcluster/tap/taskcluster still works, and existing formula installs auto-migrate on the next brew update.
▶ [patch] Removed the interactive shell viewer for docker-worker in the UI
▶ [patch] Adds scripts to automate Node.js and Go version upgrades.
<details> <summary>12 Dependabot updates</summary>
</details>
▶ [patch] #7388 Generic Worker (FreeBSD): taskcluster-proxy now cross-compiles for freebsd/amd64 and freebsd/arm64 again. The new connection-verificat
▶ [patch] #7388
Generic Worker (FreeBSD): taskcluster-proxy now cross-compiles for freebsd/amd64 and freebsd/arm64 again. The new connection-verification feature (--allowed-user / --allowed-network) only has darwin, linux, and windows implementations; on FreeBSD the proxy refuses to start if either flag is set. FreeBSD support for taskcluster-proxy is experimental.
This release also contains the changes for v100, which had a broken release. Here's v100's changelog:
▶ [MAJOR] #8437
Removed docker-worker from the monorepo. Docker-worker has been decommissioned across Taskcluster deployments and is no longer released. The d2g translation layer remains, so generic-worker continues to accept the legacy docker-worker payload format on Linux and the docker-worker:* scope namespace is unchanged. Existing tasks using the docker-worker payload format continue to run unchanged on generic-worker.
Notes for deployers:
docker-worker worker-runner implementation has been removed; deployments must run generic-worker (or a third-party worker that uses the Queue's worker protocol). worker-runner's --help no longer lists docker-worker.docker-worker entry has been removed from the task-creator UI's TASK_PAYLOAD_SCHEMAS map. Deployments that set SITE_SPECIFIC.tutorial_worker_schema to docker-worker should change it to a generic-worker schema key (e.g. generic-multi-posix on Linux, generic-multi-win on Windows). Deployments that did not set this variable now default to generic-multi-posix instead of docker-worker.workers/docker-worker/ source tree is gone; deployments that built the docker-worker image themselves from this monorepo must source it from a docker-worker fork instead.workers/docker-worker/schemas/v1/payload.yml to tools/d2g/schemas/docker-worker/v1/payload.yml. The published service-schema URL (schemas/docker-worker/v1/payload.json) is unchanged, so consumers fetching the schema from a running deployment are unaffected.▶ [patch] #8569
Fix Azure worker registration in regions whose Azure IMDS attested-data leaf certificates have rotated to the new Microsoft TLS RSA Root G2 hierarchy (uksouth as of 2026-04-29; other regions follow as their leaves renew). The G2 root is bundled in worker-manager's azure CA store, so addIntermediateCert succeeds for the dynamically fetched Microsoft TLS G2 RSA CA OCSP NN intermediates and registerWorker returns 200 again.
▶ [minor] #7388
Generic Worker now supports running multiple tasks concurrently via the new capacity configuration option.
Configuration:
capacity (uint8, default: 1, max: 255) - the number of tasks the worker will claim and execute in parallel.capacity is 1, behavior is unchanged from previous releases.capacity > 1, each task slot is allocated a block of 4 ports offset from the configured base ports (livelogPortBase, interactivePort, taskclusterProxyPort). Deployers must ensure these base ports are spaced far enough apart to avoid overlapping ranges. The worker validates this at startup and exits with an error if ranges collide.Engine support:
headlessTasks is enabled. Non-headless multiuser mode (which reboots between tasks) is restricted to capacity = 1.Task isolation:
tasksDir, its own set of dynamically allocated ports for LiveLog, Interactive, and TaskclusterProxy, and (in multiuser mode) its own OS user./proc/net/tcp on Linux, lsof on macOS, and GetExtendedTcpTable on Windows. Note: in insecure mode, all tasks run as the same OS user, so UID-based connection verification is not possible; insecure mode with capacity > 1 does not provide credential isolation between concurrent tasks.Constraints:
runTaskAsCurrentUser and runAsAdministrator task features are not supported when capacity > 1 and will return a MalformedPayloadError if requested.numberOfTasksToRun is respected across all concurrent slots - the worker will not start more tasks than the configured total.▶ [MAJOR] #3521
The taskcluster proxy no longer inserts a Content-Type: application/json requests missing a content type and having a non empty body
▶ [minor] #8537 The web UI now displays artifact sizes in the task-runs and indexed-task artifact lists, when reported by the worker. Workers that do not report a size show a blank size column.
▶ [patch]
Replaced locally-redefined Win32 constants in workers/generic-worker with their equivalents from golang.org/x/sys/windows and syscall. No behavior change.
▶ Additional changes not described here: #6822, #7722.
<details> <summary>11 Dependabot updates</summary>
</details>
▶ [patch] #8525 Fix two related races that could leave a Taskcluster task in an inconsistent state when a Pulse publish failed during task creation or
▶ [patch] #8525 Fix two related races that could leave a Taskcluster task in an inconsistent state when a Pulse publish failed during task creation or a run state transition:
pending in tasks.runs without a corresponding queue_pending_tasks row, making the task invisible to workers and to the "pending tasks" UI/API counts. Transitions of a run to pending (schedule_task, rerun_task, resolve_task, check_task_claim) now commit the queue_pending_tasks row in the same database transaction as the tasks.runs update.tasks without a corresponding queue_task_deadlines row, leaving it untracked by the deadline resolver. createTask now inserts both rows atomically inside create_task_atomic.In the queue's HTTP API handlers (createTask, scheduleTask, rerunTask, reportException), Pulse taskPending / taskException / taskDefined publishes that follow these now-atomic DB commits are best-effort: the database is the source of truth, so a Pulse publish failure no longer fails the operation. Consumers that need exact-once notification should treat Pulse as advisory and read queue.task(taskId).status for the authoritative state. Background resolvers (claim-resolver, worker-removed-resolver, dependency-resolver) preserve their pre-existing at-least-once publish semantics by continuing to fail the handler on Pulse error so redelivery re-attempts the publish.
▶ [patch] Update sha2 dependency in the rust client
▶ Additional changes not described here: bug 2035431, #5457.
▶ [minor] #8243 Add optional Kubernetes Gateway API support (Gateway, HTTPRoute, HealthCheckPolicy) as an alternative to the existing Ingress resource
▶ [minor] #8243
Add optional Kubernetes Gateway API support (Gateway, HTTPRoute, HealthCheckPolicy) as an alternative to the existing Ingress resource. These new resources are only rendered when ingressType: gateway is set in Helm values, so existing Ingress-based deployments are unaffected and no new CRDs or skipResourceTypes entries are required.
To adopt Gateway API for traffic routing, set ingressType: gateway along with gatewayClassName, and for GKE regional external ALBs, gatewayStaticIpName and gcpManagedCertName. Both Ingress and Gateway API resources will be rendered side-by-side, letting you migrate at your own pace; add ingress to skipResourceTypes once the Gateway setup is validated to stop rendering the legacy Ingress.
See the Gateway API section of the dev deployment docs for setup instructions.
▶ [patch] #8526
Fixed the Azure provider's deprovisionResource wasting a worker-scanner cycle per resource when the backing VM/NIC/IP/disk had already been removed out-of-band (e.g. ARM cascade-delete via deleteOption: 'Delete', Spot preemption). Previously the pre-flight GET was skipped whenever the worker still had a stored id, so the scanner fired a no-op beginDelete first and only discovered the resource was gone on the following cycle. The helper now always performs the pre-flight GET, so a missing resource is marked deleted immediately and the reap chain continues in a single cycle, shortening the STOPPING tail for affected Azure pools.
▶ [patch]
The default sendDeadline for the pulse publisher has been raised from 12 seconds to 30 seconds. Under load, RabbitMQ blocking and client reconnects could consume most of the 12-second budget before a single publish-confirm round-trip completed, causing cascading PulsePublisher.sendDeadline exceeded errors. The new default gives more headroom while still remaining below typical HTTP proxy timeouts. Services can override this per-publisher via the sendDeadline option to exchanges.publisher().
▶ [patch] bug 2028956 Worker Manager's Azure registration flow now restricts intermediate certificate downloads to trusted certificate distribution endpoints and records rejected certificate URLs in service logs.
▶ [patch] bug 2032277
worker-runner now tightens the permissions of its configuration file (typically runner.yml / worker-runner.json) to be readable only by its owner before reading it, and logs a warning if the file was previously group- or world-readable. This closes an exposure where a task running on a worker using the static provider could read the staticSecret out of a loosely-permissioned runner config and impersonate the worker via registerWorker. Worker deployers using the static provider should update their provisioning so the runner config is created with mode 0600 (or the equivalent owner-only ACL on Windows) from the start.
▶ [patch] #8534 Fix a 500 raised from hooks.triggerHook when a hook's task template evaluates to nothing. The endpoint now correctly replies with an empty object in that case.
▶ [patch] #8529
Fix a bug in queue.createTask where idempotent retries could insert multiple
rows into the queue_task_deadlines table for a single task. Once those
duplicates became visible, several deadline-resolver instances could pick up
the same task concurrently, the first cancelled it, and the others crashed
because they assumed they were the only one working on the cancellation of said
task. A new unique constraint on task_id now prevents duplicate deadline
rows, and the migration deduplicates any existing stale rows.
▶ Additional changes not described here: #3684, #8540.
<details> <summary>3 Dependabot updates</summary>
</details>
▶ [patch] Upgrades to Node.js v24.15.0 and yarn 4.14.1
▶ [patch] Upgrades to Node.js v24.15.0 and yarn 4.14.1
▶ [patch] #8517
Fixed Azure provider workers getting stuck in STOPPING indefinitely when their backing Azure resources (VM, NIC, IP, disks, or ARM deployment) were deleted out-of-band (e.g. Spot preemption, ARM cascade delete). The deprovisionResource helper now treats a 404 from beginDelete the same way as a 404 from get: mark the resource as deleted and let the reap path continue.
▶ [patch] #8270
Fixed a permission error on startup where nginx could not open its default error log
at /var/lib/nginx/logs/error.log when the container runs as a non-root user (UID 1000).
▶ [patch] #7802 The pending tasks and claimed tasks pages now correctly display errors (such as insufficient scopes) instead of showing a blank page when the worker pool is also absent from worker-manager.
▶ [patch] #8325 Fixed a flaky test in the worker-manager launch config selector suite by increasing the statistical sample size from 100 to 500 draws.
▶ [patch] Yarn will not execute the postinstall scripts from third-party packages when installing the project. This change helps to reduce supply-chain risks by preventing potentially malicious scripts from running automatically.
Note that you also have the ability to disable scripts on a per-package basis using dependenciesMeta, or to re-enable a specific script by combining enableScripts and dependenciesMeta.
<details> <summary>2 Dependabot updates</summary>
</details>
…setTag() calls, update import style, and remove deprecated autoSessionTracking option.
▶ [patch] Upgrades to go1.26.2 [SECURITY].
▶ [minor] #8502
A new Prometheus histogram metric iterate_duration_seconds is now emitted by
all background iteration loops (provisioner, worker-scanner, queue resolvers,
etc.) via lib-iterate.
The metric is registered as a global builtin, meaning it automatically propagates to whichever Prometheus registry a process exposes — no per-service configuration is required. It is a no-op in deployments without Prometheus configured.
▶ [minor] #8497
The worker-manager Azure provider now captures ARM throttling signals for observability.
When Azure returns rate-limit headers (x-ms-ratelimit-remaining-subscription-reads, -writes, -deletes,
x-ms-ratelimit-remaining-resource, Retry-After), the provider records them as Prometheus metrics and,
on HTTP 429 responses, emits a structured azureThrottled warning log with the full header payload.
New Prometheus metrics:
worker_manager_azure_throttle_total (counter) — incremented on every 429, labeled by providerId and operationTypeworker_manager_azure_ratelimit_remaining (gauge) — tracks the most recently observed remaining-quota value, labeled by providerId and limitTypeThe error handler in CloudAPI's enqueue path now respects Retry-After headers from Azure, using the server-specified delay (capped at 120 seconds) instead of the previous fixed backoff for 429 responses.
Rate-limit headers are observed on both Track 2 SDK calls (via a pipeline policy) and Track 1 REST polling calls (in handleOperation), covering all Azure API interactions made by the provider.
▶ [patch] #8470
The worker-manager scanner now guards against overlapping scan loops and adds a per-worker timeout to checkWorker calls. Previously, when a scan exceeded maxIterationTime (common with large Azure worker pools), lib-iterate would start a new iteration while the old one was still running, resetting shared state and causing crashes in provider checkWorker methods. The scanner now detects loop overlap to prevent silent state corruption, and times out individual checkWorker calls after 60 seconds so a single hung cloud API call cannot abort the entire scan.
▶ [minor] #8430
The GitHub service now supports triggering Taskcluster hooks directly from
.taskcluster.yml. Add a hooks array to your config to trigger one or more
hooks on push, pull request, or other events:
hooks:
- name: taskgraph/decision
context:
project: myproject
Each hook is triggered via the hooks.triggerHook API with a payload
containing event, now, taskcluster_root_url, tasks_for, taskId, and
the user-defined context. hooks and tasks may be used together in the same
.taskcluster.yml. autoCancelPreviousChecks applies to hooks the same way it
does to tasks.
▶ [patch] The claim resolver no longer stalls between batches, only backing off when the queue is drained. It also now properly uses its configured batch size instead of being hardcoded to 32.
▶ [patch] The deadline resolver no longer stalls between batches, only backing off when the queue is drained.
▶ [patch] The dependency resolver no longer stalls for 5 seconds between every batch of 32 resolved tasks. It now only backs off when the queue is drained, which dramatically improves scheduling latency when many tasks are resolved at once.
▶ [patch]
Update @sentry/node from v6 to v10. Migrate Sentry SDK usage to v10 APIs: replace removed configureScope() with direct setTag() calls, update import style, and remove deprecated autoSessionTracking option.
▶ Additional change not described here: #7838.
<details> <summary>12 Dependabot updates</summary>
</details>
▶ [patch] #8446 Worker Manager no longer removes Azure workers that are actively running tasks when an ARM deployment operation URL expires or the dep
▶ [patch] #8446
Worker Manager no longer removes Azure workers that are actively running tasks when an ARM deployment operation URL expires or the deployment shows a failed state. Previously, if Azure cleaned up a deployment record or the operation tracking URL expired before worker-manager marked provisioning as complete, the worker would be removed and any in-flight tasks resolved as exception/worker-shutdown, even though the worker was healthy.
▶ [patch]
Fixed scope resolver performance tests that were silently not running due to an async
suite callback in mocha (which does not await suite callbacks). The tests using real-world
role and client fixture data are now registered and executed correctly. Also removed a stale
docker_posix.json entry from the unreferenced schemas list in lib-references.
<details> <summary>12 Dependabot updates</summary>
</details>
Release v99.0.1 was sacrificed to the release Gods. Here were its changelogs:
Release v99.0.1 was sacrificed to the release Gods. Here were its changelogs:
▶ [patch] #6898
Azure workers that report a failed provisioning state (e.g., OSProvisioningClientError) but are actually running (PowerState/running) are no longer immediately terminated. Instead, they are allowed to attempt registration, with the existing terminateAfter timeout serving as a safety net for truly broken workers.
▶ [patch] Web Server: OAuth2 token scopes are now intersected with the registered client's allowed scopes in addition to the user's scopes, preventing a tampered consent form submission from requesting scopes beyond what the client was registered for. A warning is logged when a scope mismatch is detected.
▶ [patch] #8410
Generic Worker: Fix panic "close of closed channel" in Command.Kill() when multiple abort paths (e.g., reclaim failure and graceful termination) race to kill a task's processes.
▶ Additional changes not described here: #8013, #8013, #8013.
▶ [patch] D2G: limits concurrent docker cp artifact extractions to 10 to reduce RAM usage and avoid overwhelming the Docker daemon.
▶ [patch]
D2G: limits concurrent docker cp artifact extractions to 10 to reduce RAM usage and avoid overwhelming the Docker daemon.
▶ [patch]
D2G: performance improvements to docker run for d2g tasks. Adds --pull=never to skip redundant registry checks (image is already loaded), --log-driver=none to eliminate duplicate log writes, and parallelizes artifact extraction from stopped containers.
▶ [MAJOR]
The taskcluster/websocktunnel Docker image tags now include a v prefix (e.g., v99.0.0 instead of 99.0.0), matching the convention used by all other Taskcluster Docker images. A duplicate task definition in the release tooling was silently overriding the correct tag format since v36.0.0. If you reference websocktunnel images by tag, update your configurations to use the v-prefixed format.
<details> <summary>5 Dependabot updates</summary>
</details>
▶ [patch] Upgrades to Node.js 24.14.1 [SECURITY].
▶ [patch] Upgrades to Node.js 24.14.1 [SECURITY].
▶ [patch] #8388
The GitHub service now streams backing log artifacts instead of downloading them entirely into memory. Previously, tasks with very large logs (e.g. ~98MB) caused the status handler to crash with an out-of-memory error, leaving GitHub check runs stuck as in_progress indefinitely.
▶ Additional change not described here: #3665.
<details> <summary>8 Dependabot updates</summary>
</details>
▶ [MAJOR] #6689 Breaking change: Generic Worker now evaluates absolute paths inside mounts (properties directory and file) and artifacts (property pat…
▶ [patch] Upgrades @octokit/core (v3 to v6), @octokit/rest (v18 to v21), and @octokit/auth-app (v4 to v6) to resolve peer dependency warnings.
▶ [patch] Upgrades taskgraph decision image to v20.0.0
▶ [MAJOR] #6689
Breaking change: Generic Worker now evaluates absolute paths inside mounts
(properties directory and file) and artifacts (property path) correctly.
Previously Generic Worker would effectively strip leading path separators and
treat them as relative paths inside the task directory. For example, /tmp
would be resolved as the relative path tmp from inside the task directory.
Although this is technically a bug fix, it does change the behaviour of Generic Worker when absolute paths are specified in task payloads. We have examined production tasks on both the Community and Firefox CI deployments of taskcluster, and are reasonably confident that this change should not have adverse effects on existing tasks. However we are bumping the major version number of the taskcluster release, in recognition of the backward incompatibility.
▶ [minor]
Generic Worker: Replaced the deprecated github.com/mholt/archiver/v3 library with github.com/mholt/archives, and added support for new archive and compression formats in task payload mounts.
New decompression formats for FileMount: br (Brotli), lz (Lzip), mz (MinLZ), sz (Snappy/S2), zz (Zlib).
New archive formats for ReadOnlyDirectory and WritableDirectoryCache: 7z, tar, tar.br, tar.lz, tar.mz, tar.sz, tar.zz.
Artifact uploads now skip gzip compression for files with extensions matching the newly supported compressed formats (.br, .lz, .lz4, .mz, .sz, .zz).
▶ [patch]
Fix a panic in taskcluster task status and taskcluster task artifacts when the task has no runs (e.g., is in the unscheduled state). These commands now return a clear error message instead of crashing.
▶ Additional change not described here: #7901.
<details> <summary>9 Dependabot updates</summary>
</details>
▶ [patch] Upgrades to go1.26.1 [SECURITY], Node.js v24.14.0, yarn v4.13.0, and golangci-lint v2.11.2.
▶ [patch] Upgrades to go1.26.1 [SECURITY], Node.js v24.14.0, yarn v4.13.0, and golangci-lint v2.11.2.
▶ [minor] #8287
Add ability to provide taskId in the payload to triggerHook and triggerHookWithToken. The hook service will look for taskId in the payload and use it to set the task ID if provided.
▶ [patch] #8357 Fix third-party login flow with extra window not being closed.
▶ [patch] #8341 Fixes issues with local dev env where tc-admin image was built for single architecture. tc-admin:5.2.0 publishes multi-arch image
▶ [patch] #8300 Task log profiler returns empty profile when log doesn't have correct time markers instead of throwing an error.
▶ [patch] The generic-worker now notifies task status listeners outside of the internal status lock, reducing the risk of lock contention or deadlock when listeners query task status during callbacks.
Internal artifact upload error handling was also refactored into a dedicated classifier helper without changing runtime behavior.
<details> <summary>7 Dependabot updates</summary>
</details>
No changes
No changes
▶ [patch] D2G: avoids unnecessary I/O of copying cached docker image to task user's directory.
▶ [patch] D2G: avoids unnecessary I/O of copying cached docker image to task user's directory.
▶ [patch] #8326
Generic Worker: when running with worker-runner, the worker now checks with Worker Manager before shutting down due to idle timeout. If Worker Manager says the worker is still needed (e.g., to satisfy minCapacity), the idle timer resets instead of shutting down. Workers not running with worker-runner are unaffected.
▶ [patch] #8328
Worker Manager: the worker scanner now uses a dedicated target capacity formula for termination decisions based on pending tasks, claimed tasks, and minCapacity/maxCapacity. Previously, the provisioning formula was reused, which inflated the target by existing worker counts, so idle workers were never terminated even when minCapacity was lowered to 0.
▶ [patch] #8323 Fixes queue.listTaskGroup endpoint that in some cases didn't return full list of tasks in the given group.
<details> <summary>5 Dependabot updates</summary>
</details>
▶ [minor] #7652 Generic Worker & Livelog: fix livelog temporary streaming files not being cleaned up. The livelog process creates a temp directory per
▶ [minor] #7652
Generic Worker & Livelog: fix livelog temporary streaming files not being cleaned up. The livelog process creates a temp directory per stream, but since the generic worker kills it with SIGKILL, the process never has a chance to clean up. Fixed by having the generic worker create a dedicated temp directory for each livelog process (via the new LIVELOG_TEMP_DIR env var) and removing it after the process is killed.
▶ [patch] #8318 Generic Worker & Livelog: fix intermittent "address already in use" error on livelog ports. When a livelog process failed to start, an orphaned goroutine would keep polling the port and later send a duplicate PUT request to the next task's livelog process, causing its GET server to fail binding. Fixed by cancelling the goroutine on early process exit, killing orphaned livelog processes on connection failure, and fixing the livelog binary's duplicate PUT request guard which was checked but never set.
▶ [minor] #8101
Queue artifact creation (createArtifact) now accepts an optional contentLength field for S3 and object artifacts. When provided, the artifact size in bytes is stored and returned in listArtifacts, artifactInfo, and related endpoints.
<details> <summary>1 Dependabot updates</summary>
</details>
▶ [patch] #8314 Azure double-checks if vm is gone by calling virtualMachines.get after instanceView returns 404. This is to prevent situations when wo
▶ [patch] #8314 Azure double-checks if vm is gone by calling virtualMachines.get after instanceView returns 404. This is to prevent situations when worker is being removed based on one failed instanceView call.
▶ [patch] Adds Weight Distribution Playground for Launch Configurations in documentation to see how initialWeight value change affect distribution
▶ [patch] #8311 Fixes UI issue for LaunchConfigs not showing location when ARM deployment template was used
<details> <summary>2 Dependabot updates</summary>
</details>
▶ [patch] #7472 The queue service's workerRemovedResolver now also listens for workerStopped events from worker-manager, in addition to workerRemoved
▶ [patch] #7472
The queue service's workerRemovedResolver now also listens for workerStopped events from worker-manager, in addition to workerRemoved events. This ensures claimed tasks are resolved as exception/worker-shutdown as early as possible when a worker disappears. Both events are handled idempotently, so receiving both for the same worker is safe.
▶ [minor] #7472 The queue service now listens for workerRemoved events from worker-manager and immediately resolves any tasks claimed by that worker a
▶ [minor] #7472
The queue service now listens for workerRemoved events from worker-manager and immediately resolves any tasks claimed by that worker as exception/worker-shutdown, triggering an automatic retry.
Previously, when a worker disappeared (due to VM preemption, crash, or manual termination), its claimed tasks would wait up to 20 minutes for the claim to expire before being retried.
This new workerRemovedResolver background process runs alongside the existing claim-resolver and requires no configuration changes.
▶ [patch] #8300 Task log profiler optimizations for parsing large task logs, more memory and cpu efficient.
<details> <summary>2 Dependabot updates</summary>
</details>
▶ [patch] #8083 Generic Worker (windows): waits for the User Profile Service (ProfSvc) to be running before performing profile operations on first boo
▶ [patch] #8083
Generic Worker (windows): waits for the User Profile Service (ProfSvc) to be running before performing profile operations on first boot, and fixes a bug where LoadUserProfile retry logic for "device not ready" errors never actually retried due to an incorrect error type assertion.
▶ [patch] #8292 Claim-resolver and deadline-resolver expose metrics.
▶ [patch] #8292 Claim-resolver and deadline-resolver expose metrics.
▶ [minor] #7147 Worker-manager now decides which workers should be kept during the worker-scanner loop, surfaced via the shouldWorkerTerminate API. De
▶ [patch] Upgrades to go1.26.0
Release notes here.
▶ [minor] #7147
Worker-manager now decides which workers should be kept during the worker-scanner loop, surfaced via the shouldWorkerTerminate API.
Decision is being made based on several policies:
▶ [patch] #8289 Generic Worker: fixes credential expiration during high-volume artifact uploads by narrowing the scope of the queue client lock so that credential refresh is no longer blocked by in-flight HTTP calls.
▶ [patch] #8291
Generic Worker: handles SIGTERM during task execution by triggering graceful termination, ensuring preempted tasks are properly resolved as exception/worker-shutdown instead of exception/claim-expired.
▶ [minor] #8035
The index service no longer adds a bewit (time-limited auth token) to redirect URLs for public artifacts. Artifacts are considered public if the anonymous role has the necessary scopes to get them. The index service caches the scopes associated to the anonymous role and refreshes them from the auth service every 5 minutes. Additionally, for public artifacts, the index service now resolves the final artifact URL server-side by calling the queue's latestArtifact endpoint, reducing the redirect chain from two hops (Index → Queue → storage) to one (Index → storage).
▶ [patch] #8070
Show a warning banner in the UI when live updates are disabled due to missing web:read-pulse scope, instead of silently showing stale data.
▶ [patch] #8292
Add missing queue_exception_tasks metric to the claim-resolver and deadline-resolver.
▶ [patch] #8261
Fix webpack-dev-server crash (ECONNRESET) when proxying WebSocket /subscription endpoint during local UI development.
▶ Additional change not described here: #8271.
<details> <summary>6 Dependabot updates</summary>
</details>
▶ [minor] #7374 Add task profiler REST API endpoints:
▶ [minor] #7374 Add task profiler REST API endpoints:
GET /api/web-server/v1/task-group/<taskGroupId>/profile — returns Firefox Profiler JSON for a task group timelineGET /api/web-server/v1/task/<taskId>/profile — returns Firefox Profiler JSON for a task's logProfiles are shareable via profiler.firefox.com/from-url/<encoded-url>. External tools like Treeherder can link directly.
UI adds "Open in Profiler" speed-dial actions on task group and task views, and a dedicated /task/groups/:id/profiler route.
v96.3.0 had issues uploading GitHub artifacts due to a GitHub incident, so this release is just retrying that one.
No changes
v96.3.0 had issues uploading GitHub artifacts due to a GitHub incident, so this release is just retrying that one.
▶ [patch] Bumps uv to v0.10.0 and taskgraph decision task image to v18.1.0.
▶ [patch]
Bumps uv to v0.10.0 and taskgraph decision task image to v18.1.0.
▶ [patch] #8247 Fix azure checkWorker() call during provisioning.
▶ [patch] bug 2015057 Fix typo in object service artifact upload.
▶ Additional change not described here: bug 2015056.
▶ [patch] Upgrades to go1.25.7 [SECURITY].
▶ [patch] Adds securityContext.runAsNonRoot: true and securityContext.allowPrivilegeEscalation: false to all k8s Deployments and CronJobs. Containers
▶ [patch]
Adds securityContext.runAsNonRoot: true and securityContext.allowPrivilegeEscalation: false to all k8s Deployments and CronJobs. Containers are all now run as non-root, node user (UID/GID 1000).
▶ [minor] #8247
Worker-manager includes workerGroup label for some metrics (location/zone/region): existing_capacity, stopping_capacity, requested_capacity
▶ [patch] #8245 Adds missing providerId label to all worker-manager exposed metrics.
▶ [patch] github: add the task id to the task status header
<details> <summary>10 Dependabot updates</summary>
</details>
▶ [minor] #8012 Generic Worker (windows): adds task.payload.features.hideCmdWindow [default: false] to hide the cmd.exe window that appears during tas
▶ [minor] #8012
Generic Worker (windows): adds task.payload.features.hideCmdWindow [default: false] to hide the cmd.exe window that appears during task execution. This may be useful if the cmd.exe window gets in the way of GUI applications while running tasks. Please note: if your task needs to allocate new consoles (with AllocConsole(), for example), it will not be able to if you set this to true.
▶ [patch]
Client (shell): taskcluster task log command falls back to use live_backing.log if live.log is unavailable.
<details> <summary>7 Dependabot updates</summary>
</details>
▶ [MAJOR] #7147 Breaking changes:
▶ [patch] Upgrades to go1.25.6 [SECURITY].
Read release notes here.
▶ [patch] Include session storage cleanup script in web server.
▶ [patch] bug 2006698 Worker manager incorrectly identified zombie workers even when they were active.
▶ [MAJOR] #7147 Breaking changes:
deploymentId and checkForNewDeploymentEverySecs are no longer supported and must not be usedNew features:
shouldWorkerTerminate API endpoint in Worker Manager allows workers running with worker runner (--with-worker-runner) to query whether they should terminateworker-manager:should-worker-terminate:<workerPoolId>/<workerGroup>/<workerId> required to call this endpoint<details> <summary>4 Dependabot updates</summary>
</details>
▶ [patch] Upgrades to Node.js v24.13.0 [SECURITY].
▶ [patch] Upgrades to Node.js v24.13.0 [SECURITY].
Can read more about this release here.
▶ [patch] Fixes TypeError in web-server when no oauth2 clients are configured.
▶ [patch] Upgrades goreleaser tool to latest version to fix broken release. Release v95.1.2 did not publish, so its changelog will be effective for v9
▶ [patch] Upgrades goreleaser tool to latest version to fix broken release. Release v95.1.2 did not publish, so its changelog will be effective for v95.1.3.
▶ [patch] bug 2006698 Fixes worker-manager killing zombie workers when queue data is missing. For long running tasks that took longer than queueInacti
▶ [patch] bug 2006698 Fixes worker-manager killing zombie workers when queue data is missing. For long running tasks that took longer than queueInactivityTimeout to finish was a high risk of getting claim-expired because w-m incorrectly assumed worker is not doing anything (missing queue information)
▶ [minor] Queue service allows changing the priority of unresolved tasks. Two new endpoints are being introduced:
▶ [minor] Queue service allows changing the priority of unresolved tasks. Two new endpoints are being introduced:
This is the implementation of the RFC190
▶ [patch] Notify Service: upgrades to use @aws-sdk/client-sesv2 with the latest version of nodemailer to resolve some security vulnerabilities.
▶ [patch]
Notify Service: upgrades to use @aws-sdk/client-sesv2 with the latest version of nodemailer to resolve some security vulnerabilities.
▶ [patch] Upgrades CI Decision task to use Taskgraph v18.0.3.
▶ [patch] Upgrades to Node.js v24.12.0.
▶ [patch] #8176 Worker-manager ensures workers are spawned for single launch config worker pools when adjusted weight might drop below zero with remaining capacity.
▶ [patch] Changed the garbage collector to clean caches after docker resources when d2g is enabled
▶ [patch] Switch back web-server queues to classic, as those are only used for short-lived task group updates in the UI and don't require same durability and Raft consensus algorithm.
<details> <summary>9 Dependabot updates</summary>
</details>
▶ [patch] #8083 Generic Worker (windows): adds retries to the win32 LoadUserProfile call to help prevent The device is not ready worker errors.
▶ [patch] #8083
Generic Worker (windows): adds retries to the win32 LoadUserProfile call to help prevent The device is not ready worker errors.
▶ [patch] Generic Worker (with worker-runner): properly unregisters a worker when it exits due to internal error or non-current deployment ID. Followup to #8165.
▶ [patch] #8023 Generic Worker: limits concurrent artifact uploads to 10 at a time. Followup to #8032.
▶ [patch] #8153 Generic Worker (d2g): anonymous volumes from the task container are no longer removed at the end of a task run in order to resolve the
▶ [patch] #8153 Generic Worker (d2g): anonymous volumes from the task container are no longer removed at the end of a task run in order to resolve the task sooner. The garbage collector was updated to take care of these instead.
▶ [patch]
Generic Worker: moves the AbortFeature to be the first to initialize and start up for each task. This change allows the worker to be ready for spot terminations as early as possible, so that it doesn't get preempted while it's setting up other task features, not knowing the cloud provider wants the worker to shutdown.
This will help tasks resolve properly as worker shutdown in the case where a spot termination request comes in as soon as the worker starts up. Beforehand, when this situation would happen, the task would commonly resolve as claim expired because the worker didn't have enough time to react to the preemption notice.
▶ [patch] Worker-runner will now properly unregister a worker when it exits because of an idle timeout
▶ [MAJOR] #8074 Pulse consumer expects core Taskcluster classic RabbitMQ queues to be deleted before being auto-recreated as quorum queues.
▶ [MAJOR] #8074 Pulse consumer expects core Taskcluster classic RabbitMQ queues to be deleted before being auto-recreated as quorum queues.
The API reference documentation now includes code examples for every endpoint in multiple languages:
▶ [minor] #8049
The API reference documentation now includes code examples for every endpoint in multiple languages:
Each example demonstrates how to:
Examples are generated dynamically in the browser from API metadata, keeping the documentation lean while providing comprehensive, up-to-date code samples. Examples include syntax highlighting and can be copied to clipboard directly from the documentation.
▶ [patch] #8074 Fixes pulse consumer issue where services would assert the queue exists as a quorum queue and wouldn't fall back to classic type as a backwards compatibility followup solution to #8156.
edit: This should've been a breaking change due to the fact that old, classic queues cannot be converted to quorum type in place. You must delete the…
▶ [minor] #8156 Pulse library declares non-ephemeral core Taskcluster queues as quorum queues to prepare for upgrading to RabbitMQ v4+.
edit: This should've been a breaking change due to the fact that old, classic queues cannot be converted to quorum type in place. You must delete the old classic queue and let the pulse consumer recreate it as a quorum queue. Please upgrade to at least v95.0.0.
build(deps): bump jws from 3.2.2 to 3.2.3
<details> <summary>1 Dependabot updates</summary>
</details>
▶ [patch] Client (python): upgrades many dependencies to latest minor/patch releases using uv lock --upgrade.
▶ [patch]
Client (python): upgrades many dependencies to latest minor/patch releases using uv lock --upgrade.
▶ [patch] Upgrades to Node.js v24.11.1 and rust v1.91.1. Additionally upgrades yarn to 4.12.0.
Replaces backoff crate with backon due to https://rustsec.org/advisories/RUSTSEC-2025-0012
▶ [patch] Upgrades to go1.25.5 [SECURITY].
See more here.
▶ [patch] #8115
Generic Worker (windows): reverts #8030 to use CREATE_NEW_CONSOLE over CREATE_NO_WINDOW so that child processes can call AllocConsole() to create new consoles.
▶ [minor] #8093 Github webhook endpoint returns 200 instead of 400 for unsupported events. 200 means we received and processed webhook, even if we don't actually support such event at the moment. 400 is only for validation issues.
<details> <summary>8 Dependabot updates</summary>
</details>
Your coding agent can read these notes before it upgrades. Set up the MCP server →