github.com/odvcencio/gotreesitter

Correctness-and-evidence release. Query ranges, literals, missing-node patterns, property metadata, highlight inheritance, lazy tree finalization, DFA EOF seeking, grammar imports, and generated C metadata now match their locked contracts more closely. Locked incremental and work-count receipts authenticate the exercised behavior, while durable corpus roots, split-grammar layouts, and bounded floors make real-corpus checks reproducible. The authenticated production receipt at 2c702656 measures public Parser.Parse at 4.886056x C by equal-fixture geomean, 5.517602x C by fixed-suite sum, and 5.648204x C on the worst fixture against the locked static -O2 C oracle.

Fixed

• Deferred result-compatibility finalization stays lazy while trees are owned by parser retry/selection code, then synchronizes every public read that can observe normalized nodes or diagnostics, including pooled tree values.
• Query byte and point ranges now match the locked C runtime for half-open boundaries, zero-width nodes at the range start, reversed range updates, and zero-valued unbounded-end sentinels.
• DFA token-source seeks clamp past-EOF offsets before integer narrowing and preserve exact EOF coordinates across both skip APIs, including 32-bit builds.
• Query string literals now decode control, quote, and backslash escapes through execution and reject unescaped newlines like the locked C query parser.
• Grammar imports now decode C string and Unicode escapes without losing the reversible question-mark spelling shared with grammargen; refreshed Agda and Dhall blobs expose their Unicode symbols correctly. Generated C now uses the ABI-appropriate lexer-mode layout, emits flattened parse-action offsets, and validates complete ABI-15 supertype metadata before emission. Lowercase keyword leaves are classified from parser-reachable ownership like tree-sitter.
• Query MISSING patterns now test missing nodes, and inert #is?/#is-not? properties are available through public metadata accessors. Descendant range walks now match upstream behavior for reversed ranges and zero-width missing children.
• Highlight queries now resolve supported built-in inheritance chains across registration order and same-name replacements without duplicating cyclic queries. Incompatible locked grammar/query pairs remain fail-closed.
• Incremental parses that accept a full-span ERROR tree under a wider merge policy may retry once with the corresponding fresh-parse policy and adopt only a strictly better result. Runtime and profile diagnostics report the retry attempt, selection, cap, cause, and whether old-tree reuse was active.
• Token-invariant single-leaf edits stay outside accepted-error retry routing, avoiding a whole-tree error scan on the one-token validation path.

Tooling

• Bank an authenticated quiet-host production receipt against the locked static -O2 C oracle. At the v0.39.0 production-code baseline 2c702656, public Parser.Parse measures 4.886056x C by equal-fixture geomean and 5.517602x C by fixed-suite sum of medians, with a 5.648204x worst fixture.
• Add a bounded, build-tagged parser trace that separates lookup cells from execution-time cell reconstruction and retains whole-parse aggregates after its chronological event prefix fills. Scanner checkpoints bind their cached state to the current event token span and remain distinct from unavailable state after relexing. Collision keys have explicit memory caps; reaching a cap exposes unaudited counts, marks the audit incomplete, and blocks claims that require complete collision evidence. A base-pinned content manifest and fail-closed paired receipt identify which production, compact, and locked-C observations can actually be compared. Observer equality and untagged assembly tests keep the trace diagnostic-only.
• Add the four-fixture authenticated Go/static-C work-count board with direct counters at their exact hook boundaries, Go-only representation rows marked incomparable, and missing mandatory instrumentation reported separately from out-of-band work-ratio audit findings.
• Add a versioned, locked incremental admission matrix that separates identity, leaf-validation, real-code GLR, recovery, and stateful-scanner behavior using runtime evidence. It rejects full-parse fallback, authenticates both edit directions against fresh Go and C trees, and atomically publishes a machine-readable closure receipt only after every row passes.
• Real-corpus grammar parity can use a durable configurable corpus root and split-grammar corpus layouts without silently losing colliding basenames. Eligible-sample caps now apply on every generated-result path, committed floors reject over-cap rows, and the aggressive runner and floor share the same 30-sample limit.

Incremental-correctness, full-parse-efficiency, and benchmark-hardening release. Incremental parsing now preserves fresh-parse selection across GLR reuse, score, cull, and retry edges; terminal materialization stops and multiline edits report accurately; evidence-gated arena and merge policies reduce full-parse cost; and authenticated static-C, fleet, and forest measurements are stricter.

Performance

• The exact locked Odin grammar now caps first-pass arena preallocation for large ASCII token-sparse sources using a complete structural-density scan. This cuts arena allocation by 72% and full-parse time by 6% on the locked 6.2 MB Odin test-vector witness with a byte-identical tree. Non-ASCII input fails open, while custom, same-name, stale-blob, and other fleet grammars retain the baseline policy.
• GLR boundary merging now rejects candidates with unequal cumulative scores before recovery-cost and graph-equivalence work. The order-balanced canonical real-Go benchmark improved by 4.3% geomean with unchanged parser work, tree identity, arena use, and stack maxima.

Fixed

• Incremental parsing now preserves the configured bounded GLR width and rejects reused leaves whose stored parser state conflicts with the current shift. This prevents stale leaf context and over-aggressive two-stack pruning from changing selected trees on token-class and recovery edits.
• An incremental parse whose full-parse retry produces no strictly better tree now keeps its first-pass result instead of replacing it with a quality-tied fresh tree. This stops spurious incremental_parse_full_retry reporting on grammars whose intended trees contain ERROR productions and legitimately use the full GLR width.
• Reused subtrees now credit their cumulative dynamic precedence to the GLR stack score, so score-sensitive merge, cull, and result-selection decisions in an incremental parse match a fresh parse of the same structure.
• The GLR stack-cull trigger no longer depends on the arena class: incremental parses keep the same cull slack window as fresh parses, which previously pruned disambiguating forks early and changed selected trees on the early-newline canonical witness.
• A parse whose result materialization stops on a terminal condition (for example the memory budget tripping while the tree is being built) after the parser loop already accepted now reports that condition through Tree.ParseStopReason instead of accepted. Previously such a parse could return a sentinel full-span ERROR root labeled as a successful parse.
• Multiline tree edits now keep node byte and point ranges aligned with the C runtime across insertions, deletions, and replacements.
• Rewriter edits now reject reversed and out-of-source byte ranges instead of panicking while applying them.

Tooling

• Report-mode fleet reduction now preserves closed-vocabulary no_static_c_oracle, no_corpus, and no_corpus_files shards as fatal closure findings in the combined artifact. Certification remains fail-closed, and report mode still rejects untyped, contradictory, or mixed oracle evidence.
• Add a diagnostic-only, authenticated Go/static-C GLR work-count contract for the locked real-Go query_compile fixture. A separate ordinary untagged Go child performs admission before tagged Go and fully static C diagnostic children report saturating direct action/pop/selected-tree counters and explicitly labeled representation proxies. Go counters attribute each parseInternal attempt to a logical retry rung, resolved cap mode, parser loop, and finalization; accept_actions is explicitly an action count, and aggregate counters must equal attempts plus the outside-attempt residual. Frozen retry-active and straight-LR witnesses pin the attribution semantics; a Go-only v3 supplement now records a bounded, attempt-local convergence frontier across reduction selection, post-reduce packing, boundary merge and cull, pending work, terminal acceptance, packed-root expansion, and final selection. It retains the first 256 events plus first rejection evidence, uses attempt-local decision IDs for target/candidate pairs, records scanner checkpoint identity at the current token election, detects partial merge mutations from exact semantic GSS writes, separates saturation from truncation, serializes no pointer identities, and leaves the shared v2 Go/static-C counter semantics unchanged. Authenticated v4 receipts bind both manifests and fail closed on malformed convergence payloads. Authoritative receipts require a clean Git source identity; compile from sealed private Go and C input snapshots; bind sanitized build/runtime environments; independently verify fixture, grammar, GLR-regime, span, and deep-tree identities; contain the complete cold static-C admission plus all repository, compiler, linker, identity, and linkage-verifier descendants in wall-bounded process groups; and publish atomically only after all rechecks pass.
• The static C oracle now recognizes locked grammar entry points declared with either C's empty () or (void) parameter spelling, restoring artifact construction for SCSS while rejecting genuinely parameterized near misses.
• The authenticated fleet scoreboard now times fresh full parses against a per-language, fully static executable built from the locked upstream runtime and grammar sources. Every selected file requires matching static/cgo deep tree digests, and reduction fails closed on missing or mixed oracle identity, dynamic linkage, source/flag drift, or legacy incremental axes. Deep dumps and cgo admissions use iterative cursors with independent wall bounds; C failures retain bounded Go evidence without fabricating ratios. Each file measures one whole Go block and one whole static-C block, alternating their order across file ordinals; compiler/linker absolute paths and executable hashes are part of the serialized protocol. The reducer preserves complete, typed C-oracle failures as authenticated closure failures while rejecting untyped, generic, or incomplete evidence. Admission, parser, transport, digest, measurement, and protocol failures now have a closed serialized status vocabulary. Content-keyed static executables are atomically installed with build-key/artifact-hash manifests only after a post-link recheck of the captured compiler, linker, pinned source trees, and every compiled source hash; cache hits repeat that check before use, and unstable inputs are recaptured once before failing. Shards execute a reverified private artifact snapshot, and per-language wall/RSS stops kill the entire child process group. The budget and status tools read both schema generations while keeping v1 historical ratchets separate from v2 full-only hard-gate verdicts.
• Forest-routing performance screens now require fresh order-balanced confirmation before promotion. Immutable content-addressed trial, run-config, cohort, and index receipts bind the selected head to the recorded host fingerprint, image, and single-CPU resource configuration. Failed or drifting attempts remain unpublished: the runner executes the recorded image digest, verifies the created container identity, and reauthenticates every corpus read by manifest size and SHA-256 before timing. The reducer requires locked-C coverage for every routed path, emits explicit A+B or A-B-B-A plans, pools reverse-order evidence, and keeps possible C-oracle corrections review-required. Repeated trials release every tree on success and negative paths, complete the full corpus before the next sweep, and remain isolated to one language per container.

Full-parse benchmark-integrity, forest-certification, and GLR-performance release. Publication now uses one locked static C oracle and authenticated, forking real-Go fixtures; authenticated C-first fleet evidence gates automatic forest routes; general multi-stack work is reduced; and high-level highlight and tag parsing can be bounded. The 206-grammar curated structural-parity milestone remains banked.

Added

• Highlighter and tagger construction now accept parser timeout options, and their byte-oriented incremental APIs have strict variants that return the partial tree with ErrParseStoppedEarly while skipping query execution.

Changed

• ParseForestExperimental now reports only a tree produced by the experimental forest parser. A forest decline returns nil, false with ForestDeclineInfo diagnostics instead of silently substituting a production-parser result, so callers can measure and certify forest routing without mistaking fallback work for a forest success.

Tooling

• Canonical real-Go benchmark admission now ratchets each fixture's multi-stack runtime regime and required syntax coverage across Go, cgo, and static-C preflights. Publication samples fail closed if a fixture drifts back toward a straight-LR control workload.
• The authenticated real-corpus source for Git rebase fixtures now uses the grammar repository's committed highlight corpus, so performance and forest audit manifests cover all 206 languages.
• One-language forest audit shards now revalidate only their selected corpus checkout and files while retaining the complete manifest identity, avoiding a repeated fleet-wide authentication pass for every isolated container.
• Forest eligibility sweeps now share an authenticated, revision-pinned corpus manifest between production and C-oracle lanes. Per-language Docker runs verify source checkout identity and exact file hashes, compare complete trees including anonymous children, points, flags, and fields, and emit strict resumable result shards for deterministic fleet reduction. Generated corpus files may be untracked only beneath the lock-declared .gts-extracted/<language> directory; tracked changes and untracked files elsewhere still fail authentication. The production lane separately times and verifies the actual forest-enabled automatic route on every file, so promotion requires exact routed parity and a net wall-time improvement after production fallbacks, not merely a fast forest attempt.
• C-first forest screening now terminally records no_forest_coverage when a complete authenticated C-oracle shard declines every file without timing out. The reducer skips the potentially expensive production lane for that non-promotable class while leaving missing and timeout-ambiguous evidence incomplete.
• Forest manifests, real-corpus benchmarks, and corpus inventory now share one file-selection policy for lock matchers, registry extensions, and canonical extensionless filenames such as go.mod. This lets authenticated manifests cover every lock entry that has an eligible source while keeping explicit lock matchers authoritative.

Performance

• Automatic forest routing now covers the exact checked-in AWK, KDL, and Uxntal grammar artifacts after authenticated corpus gates found zero forest/C-oracle divergence on accepted forest files, zero routed/production divergence on every file, and an aggregate route wall-time win. The opt-in is attached through blob-identity runtime profiles, so same-name custom and adapted grammars remain on the conservative production path.

• Multi-stack DFA token elections now scan each unique active parser state once and reuse that result while scoring candidates, instead of rescanning the state for every candidate. The authenticated real-Go matrix avoided 80-84% of those repeated scans and improved full parse by 3.5-9.8% across four fixtures, with unchanged parser shape, arena bytes, allocations, and exact 25/25 strict Go parity.

• GLR merge, hashing, shape, equivalence, and recovery-trace helpers now pass stack descriptors by pointer instead of repeatedly copying the 104-byte values. The authenticated real-Go matrix improved by 3.54% geomean, with every fixture improved or statistically unchanged and identical arena, token, stack, iteration, node, depth, and normalization counters.

• Fresh full parses now share the parser's existing no-error-payload proof with GSS merge and C-recovery cost selection, avoiding recursive graph and subtree walks until an ERROR, MISSING, or inherited error is actually constructed. Paired runs of a 148 KiB clean Java witness improved by 43-47% with unchanged full-span acceptance; isolated Go, Python, and Swift corpus parity remained exact, and incremental/reuse parses retain the conservative path.

• Conflict-reduction frontiers now reuse one fixed-table lookup when reading and updating the forked and seen flags for a reduction key. Paired runs of a 707 KiB clean Dart witness improved by a further 6-13%, with fresh and incremental C parity unchanged.

• Automatic forest dispatch for the exact built-in JavaScript grammar now limits the speculative forest phase to 128 MiB while preserving the caller's full budget for the production fallback and for explicit forest parsing. A 20,784-file locked-corpus comparison preserved every tree, byte range, span, error, and stop outcome while reducing aggregate parse time by 3.09%, allocated bytes by 2.96%, and peak RSS by 19.02%. Caller-provided, modified, and same-name grammars retain the existing full-budget behavior.

Fixed

• Accept pinned C-oracle checkouts whose raw tracked bytes match the locked commit even when an upstream .gitattributes rule makes Git report a fresh clone as modified, while continuing to reject real byte, mode, and untracked changes.
• Real-Go benchmark fixture admission now drains its arena-pool state after validation, and the arena GC-retention regression establishes its own clean pool boundary instead of measuring memory retained by earlier tests.
• C-oracle forest audits now compare wide syntax nodes with a linear tree cursor, avoiding quadratic indexed-child walks while retaining exact fields, spans, flags, anonymous children, and child order.
• Forest audit timeouts now stop the parser synchronously instead of leaving abandoned parse goroutines to contend with later files in the same shard.
• Go/C benchmark admission now uses the same locked upstream runtime and Go grammar as structural parity, fingerprints the -O2 C artifact, and times immutable, clean, forking real-Go fixtures with symmetric tree lifecycles. The generated 500-function source remains a straight-LR regression control; its former 1.895x headline and 29% materialization decomposition are withdrawn because the C lane used a different grammar and the source never exercised the multi-stack path. A checked-in strict receipt driver now reproduces the pinned-core Go-C-C-Go schedule; both C transports reject dirty pinned-source caches, and the static lane snapshots each input once before identity, parity, and timing checks. The first complete publication receipt establishes the corrected full-parse baseline at 5.481673x C by equal-fixture geomean and 6.313799x C for the fixed-suite sum of medians, with per-fixture ratios from 4.639849x to 6.513909x.

Parser recovery, recurring-work, grammar-contract, and browser-runtime release. C-recovery elections and retained memo invalidation reduce fixed overhead; retry selection and generated-language provenance are stricter; and the browser runtime gains persistent incremental documents plus reproducible selected- language bundles for Go and TinyGo.

This release supersedes v0.35.0. That tag was published from incomplete ancestry; its browser-runtime changes have been reconciled here with every change on the current main line. The v0.35.0 tag remains immutable so existing Go module downloads continue to identify one source revision.

Added

• The browser runtime now supports persistent UTF-16 documents through open, update, close, and queryDocument. Updates compute a surrogate-safe minimal edit, reuse the prior parse tree, and run highlights, tags, and bounded queries over the same retained tree while leaving the existing stateless parse, query, and highlight APIs intact.
• cmd/wasmassets now emits reproducible single-language browser bundles for either the Go or TinyGo WebAssembly compiler. Bundles contain the external grammar blob, highlight and optional tags queries, the matching compiler bootstrap, and a manifest with compiler identity and SHA-256 digests; the runtime build is restricted to the selected grammar's tables, registry, and scanner support instead of embedding the full grammar fleet.

Performance

• Linearized C-recovery strategy-1 elections with reusable cursor and dedupe scratch. Against exact current main on the pinned quiet core, KDL recovery improved 19.26% with 29.39% fewer bytes and 30.83% fewer allocations, while the tiny-clean control remained statistically unchanged.
• Reused C-recovery node memos now use generation invalidation instead of clearing a retained 16K-entry cache on every parse. On the pinned quiet host, recovery-primed KDL tiny-clean parses fell from 31.98 microseconds to 13.46 microseconds (57.93%), while alternating error/clean parses improved by 3.06%, with unchanged bytes and allocations per operation.
• The exact built-in Meson grammar now skips the redundant accepted-error retry ladder only for sources of at least 2 KiB. A locked 1,549-file corpus certification preserved complete and structural trees across all 28 eligible error-bearing files and cut their aggregate one-pass parse time by 78.04% (1.410s to 0.310s). Smaller inputs keep the generic retry ladder, including seven witnesses where retrying changes the selected tree.
• The exact built-in Enforce grammar now reuses a certified complete accepted-error widened result instead of repeating it with recovery enabled on sources of at least 128 KiB. Locked full-tree, structural, and semantic runtime checks remained exact; count-10 runs cut playerbase time by 28.01% and itembase time by 7.31%, with corresponding allocation reductions and an unchanged clean control.

Fixed

• Browser runtime results are now assembled as explicit JavaScript objects and arrays, avoiding TinyGo's primitive-only syscall/js.ValueOf path while preserving the richer structured-tree and bounded-query wire formats.
• The real-corpus Docker runner now forwards REAL_CORPUS_ONLY, allowing a reproducible single-language run without switching to a different wrapper.
• HTML range normalization no longer extends already-closed child elements across trailing trivia to an enclosing end tag; genuinely unclosed recovered element chains retain their C-compatible range extension.
• Full-parse retry selection now preserves an accepted error tree when a later retry stops early, instead of replacing it with a farther provisional tree.
• Grammargen-owned Go, Regex, and Swift blobs now share one registry provenance contract, and ts2go's Go regeneration hint uses the safe grammargen emit go command without LR splitting.
• Fleet scoreboard reduction now canonicalizes hard-gate finding order and records clean reducer provenance separately from immutable measurement provenance, allowing later reducer fixes to authenticate historical shards without weakening tamper checks.

Forest-routing performance and compatibility-hygiene release. Automatic dispatch now avoids five language paths that consistently discarded their forest result, while confirmed-dead C, C++, and Rust compatibility walks are removed after full-corpus verification.

Changed

• Automatic forest dispatch no longer speculates through Beancount by default. The exact four-file clean corpus produced no forest return, so every parse paid for a discarded forest before production. In matched automatic-parser scans, removing that retry cut the 347 KiB witness from 30.255x to 2.130x C and the corpus aggregate from 27.621x to 2.088x, eliminating the only ratio above 10x while returning the same accepted, full-span, error-free trees. Explicit forest experiments and Beancount's certified recovery policy remain available.
• Automatic forest dispatch no longer speculates through Org or Vimdoc by default. Representative clean witnesses produced no forest return and returned the exact production tree after declining at EOF. Production-only routing cut fresh parses by about 97%; on reused parsers it also removed the repeated 97% Vimdoc penalty, while Org's bounded decline memo had already made warm parses production-like. Explicit forest experiments and both certified recovery policies remain available.
• Automatic forest dispatch no longer speculates through Fish or Racket by default. Two locked clean witnesses per language produced no forest return and returned the exact production tree after declining at EOF. Removing the discarded attempt cut fresh Fish parses by 90-95% and fresh Racket parses by 94-95%; reused 234-248 KiB witnesses improved by 94-96%, while smaller warm parses were already protected by the bounded decline memo. Explicit forest experiments and both certified recovery policies remain available.

Removed

• Five confirmed-dead C/C++ post-parse compatibility passes, found by a full-corpus census (c: ~974 files from git/git; cpp: ~71 files from fmt; each parsed both clean and truncated to the first 55% of every second file, through the production C token-source backend, not the generic DFA lexer): pointer-assignment precedence rewriting (a full-tree postorder walk on every c/cpp parse); collapsed-keyword-children restoration (a second full-tree walk on every c/cpp parse, covering the null/type-qualifier/ storage-class/noexcept/lambda-default-capture rule families); both preprocessor-directive-shape sub-rewrites (whitespace-separated function-macro reshape and directive-range extension); the declaration-bounds and variadic-ellipsis handlers from the fused declaration/variadic walk, plus their now-exclusive comment-scan helper cluster; and the top-level-item-wrapper collapse (also checked for any error/recovery path that could construct a visible _top_level_item node before cutting; none found, including on the truncated/error-inducing corpus phase). Zero rewrites were observed across every corpus file in both phases for all five passes. The fused walk itself remains — its builtin primitive-type-identifier promotion and preprocessor newline-span extension handlers are confirmed live and unaffected. normalizeCTranslationUnitRoot and the typedef-struct error-recovery branch remain untouched pending their own repros. Net ~575 lines removed from parser_result_c.go (~944 including pruned dead-pass-only tests); verified byte-identical (S-expression and span hashes) across the entire c and cpp corpora (1,045 files), and both the root package's production-backed BenchmarkCPPConditionClauseAmbiguityDFA and the grammars package's BenchmarkParse_C show a statistically significant ~13-27% drop in parse time and 26-50% drop in allocations per parse from dropping the two full-tree walks.
• Rust's collapsed named-leaf-children compat pass (parser_result_rust_recovery.go), a full-tree walk gated on the source containing true, false, .., or ; — a gate that opens on essentially every real Rust file. A full-corpus re-verification (all 37,127 .rs files under the rust corpus, parsed both clean and truncated to 55% on every second file — 55,691 parses total) recorded 130,018 gate fires and zero rewrites. The companion candidate, Rust's dot-range-expressions walk, was re-verified the same way and found live (3,030 real rewrites on the same corpus, the simplest case being a bare .. full-range slice index such as s[..]) and is therefore kept untouched. Net 228 lines removed; verified byte-identical (S-expression and full node-span dumps) on 30 real Rust files spanning size and content (macro-heavy, ..-heavy), with the Rust parity suite, including the dot-range-motivated weird-expressions fixture, unaffected.

Recurring parser and forest performance, recovery allocation, compatibility, and lifecycle-hygiene release. Warm parsers avoid repeated stable forest declines and oversized runtime-record copies, missing-shift recovery reuses parser-state chains, exact bundled C# blobs skip redundant post-parse work, arena and browser-WASM lifetimes are tightened, and confirmed-dead compatibility code is removed.

Performance

• Missing-token recovery now materializes each parser-state chain once per attempt and reuses pointer-free state buffers across candidate simulations, instead of rebuilding and allocating the same deep chain for each fallback. On the pinned 163 KB C++ recovery witness this reduced full-parse time by 28.4%, allocated bytes by 79.9%, and allocation count by 41.1%, while preserving the accepted full-span error tree and parser-runtime counters.

Changed

• Automatic forest dispatch now remembers stable semantic declines for a small, bounded set of unchanged sources and routes warm recurring full parses directly to the production parser after exact source verification. Explicit forest experiments remain uncached; resource-, timeout-, cancellation-, and work-cap-driven declines are never remembered. On the pinned Make witnesses this removed repeated discarded forest construction, cutting both a clean 18 KiB parse and the 129 KiB error-bearing parse by about 90% while preserving the returned production trees and runtime status.
• Automatic forest dispatch no longer speculates through CSV by default. An all-23-file corpus census produced no forest fast-path returns: the two largest files exhausted the forest budget and the other 21 conservatively declined at EOF before repeating the parse in production. Explicit forest experiments and CSV's certified recovery policy remain available.
• Internal retry and tree-selection decisions now inspect each tree's stored parse-runtime record in place instead of repeatedly copying the 2,928-byte public snapshot. Tree.ParseRuntime() remains a value API with its live final-child-counter overlay unchanged. In pinned recurring benchmarks this reduced the five-byte KDL floor by 11.2% and Java's registered token-source path by 14.9%, with unchanged bytes and allocations per operation; the standard full/incremental benchmark trio remained neutral.
• Tiny fresh full parses now reserve a source-scaled logical range from the existing physical entry-scratch slab, avoiding repeated clearing of unused stack entries while preserving incremental and large-source reservations.
• The exact bundled C# grammar now advertises native result compatibility for notnull constraints, Unicode identifier spans, scoped-lambda statements and blocks, and LINQ query expressions, allowing the runtime to skip the five corresponding post-parse passes for that certified blob. The implementations remain quarantined conservative fallbacks for legacy blobs, grammargen output, caller-built languages, and overrides unless those artifacts explicitly carry the relevant append-only capability bits. Runtime-profile attachment remains pinned to the exact blob SHA; attaching scanner support by name alone does not certify native result shapes. The skip is backed by the 1,700-file C# corpus sweep, the original motivating fixtures, an 84-program LINQ battery, explicit capability round-trip and identity gates, and direct embedded-grammar Unicode, scoped-lambda, and LINQ regressions.

Fixed

• Oversized full arenas rejected by Release now clear every matching stale checkout reference from the pool's unused backing slots instead of remaining reachable after rejection. Ordinary checkout and successful repooling remain unchanged.
• The browser runtime now releases every parse tree returned by both the runtime and grammargen WASM bridges. Runtime queries stream through a 500-match cursor limit instead of materializing an unbounded result before slicing it, and structured trees report truncated only when the 20,000-node payload limit actually omits a node rather than when a tree exactly fills it. Empty sources now return stable empty results from both bridges instead of dereferencing a nil root; unexpected nil parse results and language handles now fail safely at the browser boundary.
• Blob-loaded browser languages now retain any registered token-source factory for parsing, queries, and highlighting, matching the certified registry path used outside WASM. Grammar-subset builds now attach those factories regardless of file-init order and no longer revive Go's intentionally disabled scanner path. Reloads publish a language and highlighter together, clear stale highlighters when no query is supplied, and leave the prior pair intact when a replacement query is invalid.

Removed

• Four confirmed-dead post-parse compatibility passes, found by a full-corpus census (every source file under go ~11.4k, java ~9.2k, ruby ~3.5k, and haskell ~2.2k real-world corpora, each parsed both clean and truncated to the first 55% of every second file): Go's dot-leaf walk, a second full-tree DFS on the canonical parse lane gated only on the source containing ., whose sole job was synthesizing the anonymous . child under an already-childless dot import-alias node — the DFA emits that child directly on every real parse, so the walk visited every node in the tree without ever performing its one rewrite; Java's entire compat block (primitive-type token collapse, dotted-assignment-declaration reshape, and recovered-program-root retagging — parser_result_java.go in full); Ruby's then-span start walk; and three of Haskell's seven compat passes (collapsed named-leaf children, let-bound local-binds start, and quasiquote start). Zero rewrites were observed across every corpus file in both phases despite substantial visit counts (Java's primitive-type check alone matched 82,504 times). Haskell's remaining four passes — including normalizeHaskellRootImportField, which rewrites on nearly every parse — and Ruby's top-level module-bounds fixup, which fires on truncated input, are unaffected and untouched. Net ~886 lines removed; verified byte-identical (S-expression and span hashes) on real Go, Java, Ruby, and Haskell samples, and a benchmark variant of the canonical Go parse benchmark with a single added period (the synthetic canonical benchmark source contains no . bytes and so never exercised the removed walk's gate either way) shows a statistically significant ~37% drop in allocations per parse from dropping the walk.

Browser query/structured-tree, compatibility cleanup, clean-parse recovery performance, and Bash parity-coverage release. The runtime WASM target now exposes structured parsing and queries with both UTF-8 and UTF-16 spans. Six dead JavaScript/TypeScript rewrites are gone, clean parses avoid recursively re-summing C-recovery subtrees, and Bash's committed real-corpus floor is backed by an executable witness.

Added

• The browser-focused WASM runtime now parses JavaScript strings through the UTF-16 parser entry point and exposes structured JSON trees and bounded query results for languages loaded through loadBlob. Tree nodes and query captures carry both canonical UTF-8 byte offsets and JavaScript UTF-16 code-unit offsets; node- and match-count limits report when results were truncated.
• A dedicated bash real-corpus parity witness (grammargen/bash_parity_test.go), mirroring the existing Python witness. Previously the bash entry in the real_corpus_parity_floors.json v3 floor file (25 eligible / 9 no-error / 6 S-expression / 6 deep) was phantom: the generic real-corpus loop skips any grammar without a jsonPath/path, and bash had neither and no dedicated test, so nothing exercised that floor. The new witness grammargen-compiles the locked tree-sitter-bash grammar and reproduces the floor exactly, skipping (not failing) when the corpus is not seeded locally. A companion reducer test pins echo ${x} and echo ${#x} as working controls and echo ${x:-y} as a self-healing known-defect witness for the underlying grammargen expansion-suffix table defect.

Removed

• Six confirmed-dead post-parse rewrite passes from the fused JavaScript/ TypeScript/TSX compat walk: statement-keyword (if/while) leaf retype, empty_statement semicolon retype, existential_type collapse, call- precedence reshape, and unary- and binary-precedence rotation, plus their exclusively-owned helpers and an already-unreachable standalone fallback path from an earlier compat-tier sunset. A census over roughly 23 MB of real JavaScript/TypeScript/TSX (including undici.js and TypeScript's own checker.ts, parser.ts, and utilities.ts), the original regression corpus that added these passes, and independent adversarial precedence chains found zero rewrites from any of the six; later grammargen table fixes already produce the correct tree shape directly, so the passes had become dead weight on every JS/TS/TSX parse. The compat pipeline's two remaining live fixups (top-level object-literal reinterpretation and trailing- continue-comment reattachment) and the memory-budget stop-polling on the surviving walk are unaffected. Net ~1,100 lines removed; verified byte- identical (S-expression and spans) on real JS, TS, and TSX samples.

Performance

• Clean full parses now make C-recovery condense summaries demand-driven. Before any error-bearing payload exists, condense charges only the exact open-recovery costs instead of recursively re-summing clean subtrees; visible node counts are evaluated only by the unequal-cost comparison branch that consumes them. On a 4,096-entry generated Go composite witness this reduces a full accepted parse from 18.72 s to 93 ms and one-shot maximum RSS from 1,388,988 KiB to 498,240 KiB, with the same full, error-free S-expression. The exact 726,532-byte Go manifest witness now completes in 0.39 s with full span and no error. The standard full/incremental/no-edit benchmark trio and KDL recovery benchmark retain unchanged allocations and show no candidate regression.

Fixed

• Missing extra shifts, including the C-family zero-width missing-token case, and alias-prefixed recovered-suffix resyncs now mark error-bearing content before the next condense pass. This keeps the clean-subtree proof exact without adding node metadata, parser caches, or language-specific fast paths.

Memory containment, Python parity, and authenticated fleet-reporting release. Failed forest attempts now apply the parser's runtime heap and system memory guard, and discarded forest GSS slab batches no longer remain live behind the retention cap. Python real-corpus S-expression and deep parity return to 25/25 after removal of a misfiring compatibility fold. Fleet reducers can publish valid failing scoreboards while certification remains blocking.

Changed

• The authenticated performance-shard reducer now distinguishes reporting from certification. report mode publishes a recomputed PASS or FAIL fleet board without turning valid failure evidence into a reducer error; the default certify mode publishes the same artifact before blocking on a combined FAIL. Exact stored shard gates may be PASS or FAIL, while missing, stale, or malformed evidence still fails closed.
• The Docker parity wrapper accepts a fixed --hostname and records it in run metadata so one-language containers on the same physical benchmark host can produce a consistent authenticated host identity.
• The tier-scan guide now describes full-corpus tier publication as a staged release gate instead of claiming the 33 GB scan runs for every release. The committed tier board remains explicitly unreleased until a fresh full scan is intentionally published.

Performance

• Forest parsing now applies the parser's existing runtime heap and system memory guard in addition to its node-arena budget, covering GSS slabs and alternative indexes before a failed forest attempt falls back to production parsing. On the default-budget JavaScript Poppler witness, this moved the forest decline from byte 1,147,865 to roughly byte 360,000, cut combined elapsed time from 5.313 s to 2.610 s, total allocation from 2.450 GB to 1.289 GB, and maximum RSS from 1,961,948 KiB to 1,012,296 KiB while preserving exact stopped-tree hashes. Final-diff successful-forest B-C-C-B timing remained neutral (+1.4%, p=0.142), with a small measured allocation cost (+0.28% B/op and +0.01% allocs/op). This bounds a failed attempt; Poppler still reports the ordinary 512 MiB production fallback stop and is not claimed to complete within that policy.

Fixed

• Removed foldPythonTrailingSelfCallIntoNestedFunction, a Python compat-normalization heuristic that spuriously folded a same-named trailing call into a preceding nested function's block when that function's body ended in a dangling ; before a dedent. Raw parser results already matched the reference; only the post-parse fold diverged. Python real-corpus S-expression and deep parity both improve from 20/25 to 25/25.
• The pooled forest GSS slab now clears outer batch references discarded by its 32 MiB retention cap. On the 3,447,275-byte JavaScript Poppler witness under the default 512 MiB parser budget, this reduced one-GC live heap from 1,475,142,360 to 608,167,688 bytes and eliminated all 866,975,744 bytes of hidden tail references while preserving the 33,488,896-byte warm prefix and identical parse output. Peak RSS remained effectively unchanged because the batches are still allocated before release; a recurring successful-forest benchmark was neutral (2.627 ms to 2.636 ms, p=0.947, n=20) with unchanged bytes and allocations.

Recurring-parser performance and fleet-measurement integrity release. Reused parsers now invalidate the 16,384-entry clean-zero front cache by epoch, reducing recurring one-byte KDL and JSON wall time by 33.10% and 35.67% with unchanged allocation counts while the primary benchmark trio remains neutral. Certified runtime profiles retain the required D, Groovy, and C# retry policies. The real-corpus tooling now distinguishes clean, error-bearing, and stopped parses and can reduce revision-pinned one-language checkpoints into a single authenticated fleet report without rerunning parsers.

Changed

• The real-corpus performance scan now supports resumable one-language shard campaigns with a blocking merge-only reducer. New scoreboards record their repository revision and clean-source state; reduction requires exactly one quiet, unexcluded, hard-gate-clean shard per authenticated lock language at one revision, host/runtime identity, and measurement configuration, then recomputes the fleet aggregates, clean/error split, coverage, and hard gate before emitting authoritative JSON and Markdown.
• The real-corpus Go/C performance scoreboard now classifies each full parse as clean, error-bearing, or stopped outside the timed path, and reports per-language clean/error counts, timing totals, ratios, stopped subsets, and error share. Existing coverage and zero-cliff gates remain unchanged.
• Large D and Groovy accepted-error parses now retain their certified initial stack ceilings through exact-blob runtime profiles, and C# skips its redundant first same-stack merge retry through the same fail-closed profile mechanism. Caller-adapted grammars, incremental fallbacks, and explicit diagnostic overrides retain the conservative retry ladder.

Performance

• Reused parsers now invalidate the pointer-free clean-zero front cache by advancing its epoch instead of clearing all 16,384 entries between parses. On recurring one-byte KDL and JSON witnesses this reduces wall time by 33.10% and 35.67%, respectively, with unchanged allocation counts; the materialized full-parse, single-byte incremental, and no-edit benchmark trio remains neutral.

Fixed

• real_corpus_inventory --require-corpus-sources now rejects pinned corpus checkouts that contain no benchmark-eligible regular files matching the language's source policy. Inventory and benchmarks share traversal and subdirectory validation, so invalid paths and scan failures are reported instead of allowing an empty language sweep to appear complete.

Recurring-parser performance and compatibility-cleanup release. Repeated small parses now pay for the current input rather than stale pooled capacity, forest parsing returns its token-source resources promptly, and the common small forest indexes stay inline. On the recurring C# and CSS witnesses, wall time falls 34–35% and allocated bytes fall 81–82%; across a selected six-language family, geomean wall time falls 3.18% and bytes fall 6.81%. The primary Go benchmark trio remains neutral while full-parse bytes fall 18.68%.

Performance

• Forest parsing now acquires the parser's reusable DFA token source and closes it at the parse boundary. This removes the recurring scanner buffer and source/lexer allocations while preserving external-scanner checkpoints in the result arena before the source is returned.
• gssForestIndex and forestAlternativeIndex keep their common small sets in inline storage and allocate spill space only when needed. Insertion order, lookup identity, and cache reset semantics are unchanged.
• Full parses size their initial GLR entry reservation from the current source length. A large prior parse can no longer make every later tiny parse clear a retained 65,536-entry slab; incremental reuse keeps its established capacity.
• Visible alias targets are precomputed once per parser, removing repeated language-table scans during result normalization without treating hidden aliases as visible terminal leaves.

Added

• Recurring tiny-input benchmarks for JSON, C#, CSS, Java, and the DFA parser expose warm-pool lifecycle and fixed-overhead regressions directly.

Changed

• Trailing-span compat shims for Caddy, Comment, Fortran, Nim, Pug, and RST moved onto a single data-driven trailingSpanRules table (normalizeResultTrailingSpanCompatibility) instead of six separate runLanguageResultCompatibility switch arms and four hand-written wrapper functions (normalizeNimTopLevelCallEnd, normalizeCommentTrailingExtraTrivia, normalizeRSTTopLevelSectionEnd, normalizeFortranStatementLineBreaks). Each row names the language, the shared primitive it drives (extend the sole top-level child across a trailing line break, trim a trailing invisible extra-trivia child at the root, shrink a top-level child's end off trailing whitespace, or extend a statement across the line break before its next sibling), and that primitive's node-kind parameters, so adding another language to any of these four span shapes is a table row, not a new function. Fortran's statement-vs-sibling pass is generalized from a hardcoded program/program_statement walk into extendChildLineBreakBeforeNextSibling, parameterized the same way. The four wrapper functions being retired were already thin call-throughs into shared primitives from an earlier consolidation, so this pass nets a modest line increase (the switch/wrapper boilerplate shrinks, but the new table and its per-row rationale comments are larger than the code they replace) in exchange for one auditable, greppable rule set instead of six scattered dispatch sites.
• The go.mod, Dart, and C repetition-conflict compat-tier helpers (gomodRepetitionShiftConflictChoice, dartRepetitionShiftConflictChoice, cRepetitionShiftConflictChoice) are retired in favor of certified, blob-SHA-pinned ConflictPolicies rows in grammars/runtime_profiles.go. C's rule recurs at thousands of table rows (reduce-symbol identity alone, not table position), so it is the first profile to use two new sentinel values, ConflictPolicyAnyState/ConflictPolicyAnyLookahead, matching every state/lookahead instead of one exact row. Dot's equivalent helper is retired outright rather than migrated: it was already dead code in the shipped dispatch path (dot never opted out of the engine-wide C repetition-skip fold, which already folds it with a flat parse stack), and reviving it as a live policy grew the LR parse-stack depth O(n) with statement count for no fork-count benefit. C#'s helper is left in place: it depends on the literal source text of a contextual keyword (scoped), which ConflictPolicies' state/lookahead-symbol matching cannot express.

Containment closure and measurement-honesty release. The runtime memory budget is now path-uniform across every public parse construction: the parse loop, the Go compat walk, and the JS/TS fused compat walk all poll the same budget and surface the same stop reason. On the quiet host, a bare Parse of the Poppler witness (3.4 MB ambiguous JavaScript) stops bounded at ~1.78 GiB peak RSS under the default 512 MiB budget — a path that previously escaped accounting entirely — and completes clean under a 2 GiB budget. Error-recovery throughput improves ~18% on recovery-heavy workloads, and the perf ledger tooling learns to separate clean-parse from error-recovery throughput so tail-language ratchet rows stop conflating the two.

Performance

• The C-recovery per-subtree error-cost/visible-count memo (cNodeErrorCost/cNodeVisibleSubtreeCount) is now a fixed-capacity, pointer-keyed 2-way set-associative cache instead of a map[*Node]cNodeMemoEntry: warm CPU profiles of error-bearing parses on fleet-tail languages (kdl, uxntal) showed runtime.mapaccess2_fast64 as the single hottest leaf, driven almost entirely by these two lookups. Every decision made by the recovery cost-competition machinery (cRecoverStrategy1Election, cHandleError, cCondenseAndResume, etc.) is unchanged — a cache miss simply falls back to the same full recompute as before. The cache starts small (matching the old map's practical per-parse footprint) and grows to its full working-set size only the first time a parse actually enters C error handling, so clean parses of recovery-capable grammars are unaffected (measured neutral-to-positive on the canonical Go workload). A synthetic KDL truncated/garbage-suffix recovery benchmark (BenchmarkKDLRecoveryGarbageSuffix) improves ~18% (83.2ms to 68.2ms median, p=0.002, n=6).

Added

• A pointer-light tree measurement rig: benchmarks and a structure-of-arrays prototype (pointer_light_measurement_test.go, pointer_light_soa_test.go) that measure bytes-per-node, GC scan cost, and walk throughput for the current pointer-rich node layout against a contiguous index-based layout, plus a constructed-versus-final node census. These are the standing gate instruments for the frozen-tree store investigation.
• parse_gap_report and parse_gap_correlate now split each language's Go/C ratio by corpus-file policy: every sample is classified clean (Go tree has no ERROR nodes and did not stop early) or error-bearing, and the per-language ledger reports clean_ratio/error_ratio plus clean_file_count/error_file_count/error_file_share alongside the existing combined ratio. This keeps an error-dense tail-language corpus from making clean-parse throughput look artificially slow in ratchet decisions.

Fixed

• The Go compat-normalization walk now honors the parse memory budget: it is skipped when the parse already carries a budget stop, polls the runtime budget at its existing walk stride, and surfaces the stop reason on the final tree via a sticky trip flag. Previously the walk ran budget-blind at result finalization and could balloon on recovered trees after the parse loop had stopped cleanly. Clean-parse trees are byte-identical. The JS/TS fused walk has the same blind spot (no stop polling at all) and is tracked separately.
• The JS/TS fused compat walk (and its unary/binary candidate-index rebuild) now carries the same containment as the Go compat walk above: it is skipped when the parse already carries a budget stop, polls timeout/cancellation/memory-budget at the same coarse, ~1024-node stride, and surfaces the stop reason via the same sticky trip flag. Previously this walk polled nothing at all — no timeout, cancellation, or memory-budget check of any kind — and could run to completion budget-blind regardless of tree size. Clean-parse JS/TS trees are byte-identical; no measurable regression on the canonical Go benchmark.

Containment and canonical-parse-lever release. Memory-budget enforcement is now layered (volume-triggered polling, in-merge checks, and an absolute hard ceiling) so runaway parses stop instead of ballooning, while certified bounded-overshoot witnesses still complete. Two independent hot-path levers land together: single-stack raw-shape elision and supertype hidden-choice collapse. Combined same-host receipt on the canonical Go workload: full parse 12.25 ms to 10.91 ms — 2.14x to 1.89x the C runtime measured in the same session — with allocations unchanged (9 per full parse, zero on both incremental lanes). The compat tier continues shrinking, the field-map generation ceiling is lifted (Bash and Dart now carry real-corpus floor rows), and parity floors are reproducible against lock-pinned corpora.

Added

• Memory-budget containment is now layered: volume-triggered polling forces a real budget check whenever tracked arena growth exceeds 64 MiB since the last check (bypassing the iteration-count poll mask), the GLR stack-merge survivor loop polls the budget mid-grind, and a decoupled absolute hard ceiling (GOT_PARSE_MEMORY_HARD_CEILING_MB, default 2048, 0 = off) stops runaway growth regardless of soft-budget overshoot tolerance. A bare-Parse giant-table witness now stops with ParseStopMemoryBudget at 2.6-4x budget instead of ballooning; the Poppler witness still completes full-span under its certified 2 GiB budget.

• A hard zero-cliff gate for nightly fleet perf sweeps, with a hard-gate-only mode on the perf-scan budget checker; the scheduled perf-scan gate is disabled in favor of the nightly hard gate.

• Runtime profiles for ASM (bounded stack retries), Haxe, Odin, and SCSS.

• Dedicated non-terminal alias-map parity coverage: derivation gates for Go, Swift, and Caddy mirroring the Lua gate, plus live-parse regression tests for each language's alias behaviors.

• BENCH.md: the canonical performance-claims page, including the first pinned quiet-host receipt for the corrected full-parse benchmark and a same-host C-baseline calibration (full parse 2.14x C on the canonical workload; incremental lanes orders of magnitude faster than the cgo binding path).

• docs/compat-tier.md documenting the C-faithful result-normalization tier and its retirement policy.

• A reproducible real-corpus floors workflow: corpora seeded at grammars/languages.lock SHAs (scripts/seed_real_corpus_from_lock.sh plus a committed seed manifest), opt-in ratchet regeneration, and floor artifacts captured in the mounted workspace.

Changed

• Collapsed-named-leaf compat adapters for Kotlin, Hack, Dart, and Elixir moved onto the data-driven resultCollapsedNamedLeafRules table (previously data-driven for Ruby and Apex only): Kotlin's identifier -> simple_identifier, Hack's true/false/null literal wrappers, Dart's super/this, and Elixir's nil are now table rows instead of hand-written adapter functions. The table gained a bySource column so a row can pick the source-text-verified matcher (normalizeCollapsedNamedLeafChildrenBySource, needed when the collapsed span must be confirmed before a child is attached) instead of the plain structural one. Hack's dedicated compat file and switch arm are retired entirely (all three of its rules were table-eligible); Dart and Elixir keep their compat functions for unrelated rewrites but lose the adapter that only fired these rules. Net ~37 lines of per-language adapter code retired in favor of ~7 declarative table rows. Haskell's wildcard -> "_" was evaluated for the same migration but left in place: the anonymous token name "_" collides with the special query-wildcard sentinel in Language.symbolByNameAndNamed/SymbolByName (both short-circuit to (0, true) for name == "_"), so migrating it through the shared table resolves the child to Symbol(0) (EOF) instead of the real anonymous _ token; OCaml, HCL, and Rust were left unmigrated too (OCaml's and most of HCL's rules need multi-candidate source disambiguation the one-parent/one-child schema doesn't represent, and HCL/Rust also fold their collapse checks into a single perf-tuned tree walk that per-rule table entries would fragment).
• Real-corpus parity floors regenerated against lock-pinned corpora: 55 grammars, 851/1026 deep parity, including first-ever bash and dart rows; the docker wrapper's default skip list shrinks to OCaml only.
• GLR replay stacks use interned structural nodes, and GSS prefix aggregate caching and scratch retention are tightened.
• Certified full-parse retry passes are bounded, and redundant certified retries are skipped.

Fixed

• grammargen field maps no longer emit one entry run per production: entries deduplicate by ProductionID (compaction fingerprints include the field set, so shared IDs always carry identical fields). This removes 60-87% orphaned entries from the shipped grammargen blobs (go.bin 673 to 267 entries, swift.bin 2904 to 384) and lifts the uint16 field-map ceiling that made Bash (65,536) and Dart (65,538) generation- fatal; both now generate and carry real-corpus floor rows. A regression test pins one-reachable-run-per-ID through the real compaction path.
• The Swift certified retry profile is re-pinned to the regenerated blob SHA (fail-closed certification behaved as designed).

Removed

• Eight retired Python compat-normalization helpers and their orphaned tests (test-only since the combined single-pass source-flags path), and six test-only compat wrappers plus one dead Go range normalizer, with all remaining test coverage redirected to the live variants.

Performance

• grammargen's hidden-choice passthrough table no longer excludes supertype symbols whose alternatives are all neutral-unary; Go's _statement and _simple_statement wrappers now collapse in the zero-allocation unary reduce path (1,500 wrapper nodes eliminated per canonical-workload parse). Only two bits change in the regenerated go.bin — parse tables, field maps, alias and supertype query tables are byte-identical, and supertype query predicates match by concrete descendant so observable trees and captures are unchanged. Canonical quiet-host full parse improves ~3.4%.

• Raw-shape capture and content hashing are elided while a parse has only ever had a single GLR stack and has not entered error recovery; capture resumes permanently at the first fork or recovery event. Shape-dependent tie-breaks are unaffected: elided prefix nodes are only ever compared to themselves (structural pointer-sharing), evidenced by forced-descent and recovery differential tests. Canonical quiet-host full parse improves ~4.8% with allocations unchanged (9/0/0 preserved).

• gssNode layout compacted to a 64-byte budget on 64-bit targets (pointer-backed extra links with uint8 count/cap, uint32 depth, aggVisValid bool), enforced by a size-budget test and a compile-time uint8 guard; transient GSS slabs are recycled after linear demotion with address-keyed caches invalidated and fingerprinted spine memoization preserved. Canonical quiet-host lanes are timing-neutral with allocations unchanged (9/0/0).

• Contiguous recovery cost calculation and recovery stack allocation are optimized; tree error state is cached and compat walk frames reused.

Large-tree memory follow-up to v0.26.0. Exceptionally large completed full parses can now release arena storage retained by discarded GLR alternatives before returning to the caller. This patch does not change the public API.

Changed

• Accepted fresh UTF-8 DFA full parses with unique arena ownership are copied into a right-sized arena when the retained arena is at least 512 MiB, the projected reclaim is at least 256 MiB and 30%, and the parser memory budget leaves enough headroom for both arenas during the copy.
• Compaction runs after retry selection, result normalization, and recovery resolution. Forest, incremental, included-range, borrowed-arena, deferred compatibility/checkpoint, and lazy final-child results remain unchanged.
• Final-tree cloning now preserves arena-backed field metadata and avoids empty external-scanner checkpoint lookups.

Performance

• On the exact 3,447,275-byte JavaScript Poppler witness under a hard 2 GiB container, retained heap after GC fell from 862,803,056 to 409,862,040 bytes (-431.96 MiB, -52.50%) while preserving accepted error-free EOF output and exact Go/C S-expression and deep parity.
• The controlled full-parse, one-byte incremental, and no-edit incremental benchmark trio was statistically unchanged. The Poppler macro probe's elapsed time increased 7.98% and peak RSS increased 2.30%, so this release makes no full-parse latency or peak-RSS improvement claim.

Parser-memory, registry-lifecycle, and build-hygiene release following v0.25.0. It shrinks common node state, removes a per-parse ranking memo, and stops returned trees from retaining parser-only shape overflow. This minor release adds an exported diagnostic field; callers using positional ArenaBreakdown literals must update them.

Added

• ArenaBreakdown.NodeFieldMetadataBytesAllocated reports storage used by arena-backed node field metadata.

Changed

• Node field IDs and field-source slice headers now live in bounded arena sidecars. Accessors preserve the previous shallow-copy and shared-backing semantics across parsing, normalization, cloning, and tree mutation.
• Documentation-only pull requests use an explicit CI scope gate so required checks resolve without running compile, race, parity, or performance suites.
• Language-authoring documentation now reflects forest fallback/recovery and the hard parse-action group overflow check.

Fixed

• Extension grammar generation is now synchronized and memoized, including failures, so concurrent first access cannot race or regenerate repeatedly.
• ParseFilePooled replaces a cached parser pool when a same-name registry update supplies a different language instance.
• Parser-only raw-shape references and excess slab storage are reclaimed after final tree materialization, including both forest result paths. Parse-time arena accounting remains intact, and a bounded warm prefix is retained for reuse.

Removed

• Removed unused internal forwarding helpers from GLR stack-entry comparison and performance-scan summarization.

Performance

• Arena-backed field metadata shrinks Node from 144 to 104 bytes and removes 245,966,616 bytes from the exact Poppler arena allocation while preserving exact structural parity.
• Current-arena node error ranks are cached inline instead of in an arena-wide map. The pinned full-parse benchmark improved from 7.813 ms to 6.750 ms and from 100 to 30 allocations per operation; incremental and query baselines were unchanged.
• Bounded raw-shape reclamation reduces the hard-2-GiB Poppler probe's retained post-GC heap by exactly 192 MiB with exact deep C parity. The controlled primary benchmark was statistically unchanged; peak RSS is not claimed as improved.

Performance, memory, and runtime-hygiene release following v0.24.1. It makes pending-parent field metadata compact and exact, removes retired zero-only telemetry, and narrows redundant Java retry passes behind an exact-blob profile. This minor release intentionally includes the exported diagnostic telemetry removals listed below. It also re-certifies the exact Poppler witness inside a hard 2 GiB envelope without claiming that JavaScript's throughput tail is closed.

Changed

• Pending-parent child entries now pack their full 16-bit field ID and field source beside the payload kind. Fielded parents use one 16-byte entry per child instead of a second sidecar entry, and materialization no longer reconstructs direct fields from grammar tables.

Fixed

• Stack dedupe and GSS link merging now treat pending-parent hashes as coarse prefilters and recursively verify packed fields, field sources, and nested pending descendants. Missing arena context and excessive depth fail closed instead of allowing a hash collision to collapse distinct alternatives.

Removed

• Removed the retired direct no_alias reduction-attribution lane from ParseRuntime, ArenaBreakdown, PerfCounters, and the Java/Python and parse-gap reports. The path has had no production producer since reductions moved to all_visible or scratch_no_alias; every exposed value was permanently zero.
• Removed two unexported transient-materialization wrappers used only by tests; tests now call the stop-aware implementations directly.

Performance

• Java's exact built-in grammar profile keeps the initial 14-stack ceiling on large fresh parses whose first result accepts at EOF with an error. The cap-16 same-stack merge retry remains intact; only two proven-redundant cap-64 passes are suppressed, while overrides and incremental paths retain the conservative generic ladder.
• The exact 3,447,275-byte JavaScript Poppler witness now has a current-main receipt for no-error, S-expression, and deep C parity plus a 1,708,712 KiB hard-RSS run. Its full parse remains 3.50x C, so JavaScript stays pending on throughput and retained-node work.

Performance-contract and repository-hygiene follow-up to v0.24.0. This patch corrects the canonical full-parse benchmark before the long-tail optimization campaign continues, banks focused Caddy and Kotlin wins with fail-closed certification, and deletes superseded conflict and profiling machinery. It does not change the v0.24.0 Poppler memory claim or declare the remaining fleet performance tail closed.

Fixed

• Caddy's SHA-pinned recovered string-literal repetition row now follows C's deterministic reduce after active recovery ends, preventing a quadratic GLR fork/refold cliff while preserving exact C tree parity on the witness.
• cmd/ts2go now accepts non-terminal aliases from the grammar's alias-symbol range instead of incorrectly rejecting every alias ID above SymbolCount.
• BenchmarkGoParseFullDFA now exercises the public Parser.Parse path and a fully materialized tree. The former implementation silently enabled the no-tree diagnostic and was mislabeled as a full parse.
• ParseNoResultCompatibilityBenchmarkOnly no longer implicitly enables the no-tree path. Its result is materialized, so parse_gap_report can separate no-tree parser-core cost from the broader no-compat diagnostic. Some large-input diagnostic materialization strategies still key off this mode, so it is not yet a pure compatibility-only A/B.

Changed

• Added the explicitly diagnostic BenchmarkGoParseCoreDFA lane and withdrew the older generated-Go full-parse headline pending a pinned quiet-host rerun of the corrected public benchmark.
• External-scanner full-parse retry suppression now uses explicit certified language-profile metadata instead of parser-core language-name checks. Python and Dart retain their existing behavior, and Kotlin now treats the first retry ladder's selected tree as authoritative. Built-in policies are pinned to the exact checked-in blob SHA-256; caller-constructed, adapted, and override languages retain the conservative generic retry path.

Removed

• Fourteen retired language-specific repetition/conflict dispatch helpers and their dead Java, JavaScript, and TypeScript implementation closure. The production C-faithful global repetition fold remains the sole active path.
• The superseded Python compatibility profiler, temporary C# wave-2 profiler, and an unreferenced perf-recording helper, removing 137 lines of obsolete diagnostic surface in favor of parse_gap_report, the retained shape harness, and standard Go profiles.

Performance

• The 687-byte Caddy security-header witness now allocates about 5.4 MB/op and completes in milliseconds in the loaded-host smoke probe, down from roughly 2.07 GB/op and seconds before the certified row policy. The exact CGo deep parity gate and bounded runtime regression both pass; timing is not ratcheted until a quiet-host sample is available.
• Kotlin's pinned eight-file performance set drops from 1.423 seconds to 790 milliseconds in a one-CPU Docker A/B after removing the redundant second external-scanner retry ladder. All eight selected trees retain identical S-expression hashes, stop reasons, EOF spans, and error states.

JavaScript large-file parity and parser memory-economy release. With an explicit 2 GiB parser budget, the 3,447,275-byte Poppler witness now reaches exact EOF with no error and exact structural parity. Its allocation and arena-capacity reductions reproduced in a same-day base/head audit, and a separate single-pass runtime gate completes inside a hard 2 GiB container. JavaScript's broader focused gate is 25/25 no-error, S-expression, and deep parity. The shipped 512 MiB Poppler budget gap remains open and tracked in the performance ledger.

Added

• Version-aware zero-width external-token relex probes for stateless scanners, including transactional token-source rollback when a speculative relex is rejected.
• Parse-runtime memory attribution for arena, scratch, GSS, runtime heap, and runtime sys budget stops, plus detailed arena/raw-shape/transient counters in parse-gap reports.
• An opt-in GOT_TRANSIENT_REDUCE_CHECKPOINT_MB path that materializes the live linear stack and reuses transient slabs once a configured threshold is crossed.

Changed

• JavaScript's precise external lex-state table is now enabled for ordinary scanner arbitration. Faithful C-recovery competition remains explicitly default-opted-out because it still regresses clean large-file throughput.
• Resolved single-path GSS stacks demote back to contiguous entries, and GSS, transient-child, and transient-parent overflow slabs use bounded growth.
• Node, rawShape, and rawShapeChild layouts remove alignment waste and pack raw-shape edge metadata, shrinking the records from 152 to 144 bytes, 32 to 24 bytes, and 24 to 16 bytes respectively.

Fixed

• JavaScript automatic-semicolon arbitration now probes same-line comments in the pinned C-scanner order, so the zero-width ASI precedes a trailing comment extra and the comment remains owned by the surrounding statement list.
• The JavaScript block-comment probe uses a labeled loop break; adjacent block comments can no longer consume the following token during speculative ASI scanning.
• Transient checkpoint recycling now follows raw-shape-only sidecar edges and retargets them to arena clones before slab addresses are reused, preventing later ambiguity decisions from observing overwritten reductions.
• Transient checkpoints defer when pending-parent compaction is active; pending payloads can retain nodes outside the semantic/raw-shape graph and must not observe recycled transient slabs.

Performance

• A same-day #221/#222 audit reproduced the Poppler memory gains: 4.150 GB/op fell to 3.329 GB/op (-19.8%) and arena capacity fell from 1.422 GB to 1.282 GB (-9.8%) while exact deep parity stayed green. Go wall time moved -2.1% in that sequential sample. An earlier sample recorded 11.754 s and a 1.63x Go/C ratio, but later loaded-host samples ranged from 2.92x to 3.46x; v0.24.0 therefore does not ratchet Poppler timing until a pinned quiet-host sweep replaces the variable measurements.
• A prebuilt single-pass Poppler runtime probe accepts the exact 3,447,275-byte witness with the explicit 2 GiB parser budget under a hard 2 GiB cgroup at 1,729,836 KiB maximum RSS. The separate Go/C deep-parity oracle remains in its 8 GiB envelope because it retains both giant trees for comparison.
• Memory-budget diagnostics are embedded in Parser so attribution adds no steady-state parser-core allocation. The v0.24.0 978 B/5 allocs sample used the then-mislabeled no-tree benchmark and is not a full-parse allocation claim; both incremental lanes remained at zero allocation.

Generator throughput and certification follow-up. This cut banks the shared, deterministic blob encoder and the bounded recursive-extra construction that landed immediately after the exhaustive-parity release. It also removes the C# generation cliff caused by repeatedly rebuilding the same skipped-extra lex-preemption analysis. The release does not claim that Crystal's broader real-corpus parity tail is closed; the newly visible floors remain explicit.

Added

• Crystal-specific regression coverage proving that the bounded LALR item-set path preserves heredoc interpolation and completes the locked generation pipeline at 14,471 states with 1/1 exact parity.
• Direct-C Crystal visibility in the grammargen harness. The current measured floor is 16/20 no-error and 11/20 tree parity, while the aggressive corpus remains 10/26 exact and tracked for follow-up.

Changed

• Runtime, grammargen, and cmd/ts2go blob production now share one deterministic encoder, including stable ordering for map-bearing trailer data and preservation of large-state GOTO metadata.
• Recursive heredoc extras are constructed with bounded on-the-fly LALR item-set core merging instead of merge-history expansion. Ruby's locked pipeline completes at 11,915 states with exact parity, and unrelated grammars stay on the legacy construction path.

Fixed

• Skip-extra lex-preemption results are memoized by lookahead during lex-mode construction. On the pinned C# witness, generation plus the real-corpus test now completes in 39.38 seconds where the exact base still times out after five minutes; CGo parity remains 20/20 with zero divergences and the existing 24/25 no-error, 20/25 deep-parity corpus floor is preserved.

Exhaustive parity closure release. The curated structural matrix is now 206/206 pass with no known-degraded skips: the stale-skip ratchet landed first, then the final Norg alias-target divergence was fixed and its exemption removed. This cut also banks the parser, recovery, scanner, and Wave 3 measurement work that landed after v0.22.5. It does not claim that every measured grammar is near-C on performance; the remaining JavaScript, Scala, and other cliffs stay explicit in the perf ledger rather than being hidden by the release milestone.

Added

• An exhaustive-parity assertion that reparses every remaining known-degraded language and fails CI when an exemption has gone stale.
• CI wiring and ratchets for a zero-entry known-degraded structural list, so a future exemption requires an explicit, reviewable policy change.

Changed

• Wave 3 perf metadata and budgets were refreshed for CMake, Java, Kotlin, Lua, C#, Rust, Swift, Crystal, Scala, TypeScript, and JavaScript.
• JavaScript's Poppler memory cliff and Scala's incomplete largest-corpus run are recorded as attribution evidence instead of being presented as ordinary green measurements.

Fixed

• COBOL EXEC CICS error normalization now trims recovered procedure_division/program_definition spans to the last material child when C stops before trailing trivia, while preserving C's zero-width EOF recovery shape. A cgo-backed adversarial fixture now guards the recovered error signal and byte spans against the C oracle.
• Nested terminal aliases now collapse through metadata-mediated chains, and punctuation-prefixed literals in word-declaring grammars retain the same leaf structure as tree-sitter C.
• Visible alias targets preserve distinct-named children, closing Norg's final four structural divergences and removing the last exhaustive-parity skip.
• External-scanner tokens now retain whether they were shifted exclusively as extras, preventing literal trailing whitespace inside Python string content from selecting an after-whitespace lex mode and suppressing an immediate escape token.
• Doxygen and JSDoc recovered trees now match the C oracle through scoped result normalizers.
• HCL external scanner symbols are bound positionally, preserving scanner ABI behavior when generated symbol names differ from the shipped language.

Scoped held-out perf-ratchet release. This release follows the fleet coverage cut with the exclusion machinery and ledger language needed to keep Wave 3 honest: Groovy is now budgeted under a named scoped basis, while D and F# remain explicit held-outs because their current large-file witnesses fail in the C-reference side or hit the harness RSS watchdog before a stable Go-vs-C ratio can be ratcheted.

Added

• perf-scan file exclusion support for reproducible scoped reruns, including GTS_PERF_SCAN_EXCLUDE_PATHS, persisted scoreboard config, and status reporting for measurement-basis exclusions.
• perf_scan_status coverage accounting for scoped held-out budget rows, so budgeted-with-caveat languages are visible separately from ordinary green rows and hard held-outs.
• Groovy Wave-3 scoped held-out budget row excluding groovy/subprojects/performance/src/files/pleac11_15.groovy, with observed full-parse and no-edit ratios recorded from the Docker perf sweep.

Changed

• The perf-ratchet policy now treats expansion of measurement_basis.exclude_paths as budget loosening that requires the same RCA evidence as loosening a ratio threshold.
• F# and D Wave-3 ledger entries now carry the exact C-reference timeout/RSS witnesses found by the scoped reruns instead of presenting them as ordinary unmeasured gaps.

Wave-3 perf-fleet coverage and ongoing correctness checks. This release line extends perf-scan measurement coverage: the Go-vs-C full-parse ratio ratchet now covers 203/206 grammars (up from the targeted subset measured at the v0.22.0 checkpoint) via batches 1–7 plus a fleet gap-close sweep, while keeping the three held-out grammars explicit in the ledger. It does not yet claim universal near-C throughput: the ratchet records where every grammar stands, including held-out rows and known cliffs, and optimizing that tail plus a memory-blowup class in a few grammars remains tracked for the next waves.

Added

• Wave-3 fleet perf ratchet: Go-vs-C full-parse ratio budgets extended to 203/206 grammars (batches 1–7 plus a fleet gap-close sweep), turning sparse fleet coverage into a per-language scoreboard with perf_scan_status coverage reporting and CI budget validation.
• Auto-triggered scoped CGo parity on parser_result_* and recovery-path PRs, so masking-normalization or recovery changes surface byte-exact verification against the tree-sitter v0.25.0 oracle instead of relying only on smoke coverage.
• Runtime memory budget with unified materialization stop checks, bounding worst-case parse memory (with a small-source fast path).
• perf-scan harness hardening: explicit C-reference failure budget, active-file tracking in partial fragments, and a parent-side RSS watchdog.
• Non-terminal alias maps carried in ts2go blobs as part of the parallel correctness work, preserving parent-context aliasing across the blob boundary.
• Grammargen now derives Language.NonTerminalAliasMap, with Lua parity, shipped-blob inventory coverage, and synthetic edge-case tests for self-recursive alias wrappers, singleton aliases, terminal aliases, and deterministic row ordering.

Changed

• COBOL promoted to Tier III and marked parity-clean after the EXEC CICS parity fix below, retaining the 25/25 real-corpus and 20/20 direct C-oracle parity gates for this line.
• CGo parity tests now run inside Docker; grammar race lanes split by test range / package group for CI throughput.
• Groovy and D large-file GLR retry paths now use scoped stack ceilings to contain Go-side RSS cliffs; both remain explicitly tracked in the perf ledger until their exact Go-vs-C rows are ratchetable.

Fixed

• COBOL if-header EXEC CICS error parity aligned with the C parser.

Cobol frontier and perf-ratchet release. This release completes the focused Wave 4 Cobol frontier work and adds the first checked-in Wave 3 perf budget ratchet. It does not claim universal near-C performance; the ratchet and CI validator make future claims auditable.

Added

• cgo_harness/cmd/perf_scan_budget and the initial 20-language perf-ratio budget seed, with a fast CI validation job wired into the aggregate build.

Changed

• Cobol precise ELS is elected by default, and recovered BBANK roots plus recovery-frontier shapes are normalized against the C oracle.
• Race tests and cgo harness cleanup paths are split and tag-gated so CI stays more responsive while preserving the correctness lanes.

Fixed

• Cobol zero-width skipped tokens are absorbed into open ERROR regions so recovered trees keep the C-oracle error carriers instead of timing out or dropping them.

C++ recovery parity fold. This release fixes the focused C++ malformed-class recovery gap without enabling cpp C-recovery globally.

Added

• Registry-backed REPRO_GO_BACKEND=registry mode for C tree-dump diagnostics.

Fixed

• Malformed class bodies followed by void A::b() {} now normalize to the C-oracle recovered function_definition shape.
• Added scoped cpp result-compatibility coverage for recovered class_specifier, retagged namespace_identifier, and nested extra ERROR(identifier) shapes.

Cobol oracle cleanup and CI/dead-code maintenance release.

Changed

• CI build gating is split into faster parallel jobs.
• GLR debug/stat scaffolding, scanner/staticcheck leftovers, cgo_harness dead helpers, and untagged C-benchmark helpers were removed or gated.

Fixed

• Cobol fixed-format trailing trivia spans now match the C oracle while preserving free-format long-line content.
• Added focused Cobol cgo regression coverage and validated Cobol against the 80-case C oracle and strict real-corpus gates.

Roadmap checkpoint release after the v0.21 engine cut. This release lands the first four campaign buckets after v0.21.0: the runtime/recovery foundation, the external lex-state election ledger, broad precise ExternalLexStates coverage, and the Cobol large-table/recovery cleanup. It does not claim that all grammar tiers are parity-clean yet; the remaining tier-IV rows stay visible and classified for the next campaign waves. It also does not claim universal near-C parser performance across the registry yet; v0.22.0 publishes the perf-scan scaffold and targeted wins while the broader wave-3 ratchet remains tracked for the next release line.

Added

• cgo_harness/perf_scan: a nightly Go-vs-C scoring harness and CI proposal for tracking full-parse and incremental parser performance without mixing correctness and performance gates.
• External lex-state election ledger for all 206 tracked grammars (cgo_harness/tier_scan/external_lex_elections.{md,json}), including the default-elected, staged, missing-ELS, and no-scanner categories used by the C-recovery rollout.
• Precise ExternalLexStates tables for a broad set of external-scanner grammars, with JavaScript and Cobol kept staged behind their explicit opt-in policies until their defaults are separately certified.
• Release-tier scan documentation and generation support. Tier publication is staged in-tree, but the zero-IV release block is intentionally not enabled for this checkpoint release.

Changed

• Runtime conflict handling now centralizes C's repetition-skip dispatch rule behind the shared conflict-policy path, leaving only languages with measured recovery-shape hazards on explicit opt-outs or scoped handlers.
• External scanner symbol binding now uses positional table metadata instead of brittle name-only assumptions, reducing stale-table failure modes across generated blobs.
• C# namespace recovery and forest/retry paths avoid redundant large-span work, keeping the boundedness tests focused on parser behavior instead of retry overhead.
• Arena overflow slab growth is capped so pathological large parses fail or recover under an explicit memory budget rather than growing unbounded slabs.

Fixed

• Generated parse tables no longer silently truncate action or GOTO indexes that exceed uint16. The runtime now supports large state targets, and the generator reports actionable boundary errors for unsupported table shapes.
• Cobol fixed-format compatibility now preserves program headers and recovered paragraph structure across large-table parses. Focused Docker gates measured Cobol at 25/25 real-corpus parity and 20/20 direct C-oracle parity for this release line.
• Cobol recovered paragraph-header normalization now preserves unrelated parent HasError state, so a clean recovered header no longer masks a separate retained error in the same procedure_division subtree.
• C recovery table validation now checks both action and GOTO bounds before accepting the recovery path, preventing large-table grammars from taking a silently invalid C-recovery route.
• Recovery cycle diagnostics now include non-materializing acyclicity checks around transient recovery children, guarding against parent-link cycles in debug sweeps.

Performance

• GLR merge-equivalence and recovery-path cache work removes the largest repetition-boundary cliffs found after v0.21.0 while keeping correctness gates separate from benchmark gates.
• Bash retry overhead, C# namespace recovery retries, and TypeScript fourslash arena pressure are reduced by targeted fast-path and cap fixes.
• The perf harness is present for nightly scoring and follow-up CI wiring, but this release does not enable a universal near-C performance gate.

The engine release. The generalized GLR parser core — a C-faithful error recovery engine plus a GSS-forest fast path — replaces the v0.20.x per-language recovery approximations as the default runtime. Error recovery for 123 elected grammars now reproduces C tree-sitter's decisions (strategy-1 election order, per-stack-version error-mode lexing identity, error-cost model, and condense scheduling were each verified decision-for-decision against an instrumented tree-sitter v0.25.0 oracle). The campaign branch's full correctness backlog — 42 known failing tests at its peak — is zero in this release, and the root test suite dropped from ~340s to ~62s along the way.

Changed

• Error-tree shapes for elected languages now match C tree-sitter in many previously divergent constructs: PHP static named functions, the authzed recovery family, Angular non-null assertions, FIDL versioned layout modifiers, Julia trailing-comma assignment tuples, doxygen comment blocks, and Go range-with-function-literal bodies, among others. Downstreams that pinned the old (non-C) error shapes for these languages will see diffs on upgrade — the new shapes are the C-oracle-verified ones.
• Go grammar: real automatic-semicolon insertion via an external scanner (_automatic_semicolon), replacing the grammar-level approximation. Fixes the byte-strict ASI parity gap; spurious-HasError files on a large real repo walk dropped 436 → 17.
• The strategy-1 recovery election was rewritten to C's order and cost basis (merged-version m0 basis, finished-tree-first cost checks, missing-token versions created before the recover pass, shiftable originals preserved as their own paths, record-time dedup), and ordinary reduces no longer dissolve extra ERROR carriers (C keeps every popped subtree).
• Recovery-time lookahead identity: the engine now lexes error-mode lookaheads per stack version like C (LexModes[0] at ERROR_STATE), including an engine-side error-mode relex for custom token sources, with the capability forwarded through included-range wrappers.

Fixed

• Conflict-policy metadata inference no longer vetoes hand-written repeat-boundary conflict resolvers for embedded languages. The veto — intended only for grammargen-generated grammars — silently disabled the C#, Java, C, Rust, TypeScript, PHP, Python (and more) resolvers; C# designer-style files forked ~32 GLR stacks per statement and exhausted the arena (2064 stacks); they now parse with 1 stack well under the 500ms test budget.
• Forest engine: hidden-symbol dedup starvation and a cap-eviction tie livelock no longer force dead-end declines on valid input (bash/CMake/JSON repeat-heavy shapes; python module_repeat1 worklist blowup); the EOF-recovery competition probe no longer declines ordinary clean input; and Parser.SetTimeoutMicros is now enforced inside the forest path (previously unenforceable for forest-dispatched languages).
• Forest cap-eviction comparisons no longer walk full subtrees per comparison (O(bytes²) on repetitive inputs): raw-shape content fingerprints, dirty-keyed resident caches, and exact per-node error-rank memoization take C# designer-style n=300 from ~1.9s to ~282ms in the forest path.
• Root spans no longer shrink when a hidden childless leaf vanishes during invisible-root-child flattening (doxygen whole-block comment shapes).
• The authzed hand-written lexer emits _whitespace extra tokens like C's generated lexer instead of silently skipping horizontal whitespace, fixing a 1-byte recovery-anchor divergence.
• The doxygen whole-block-comment ERROR normalizer no longer collapses recovered structure (highlight queries produce results again); a recovered-structure guard scopes the collapse to genuinely empty ERROR trees.
• Trailing-trivia trimming is gated on symbol visibility again (a generalization had dropped the guard, breaking field-mapping preservation).
• Skipped-real-gap recovery: a stray token dropped by the lexer mid-production (Julia f(a,b) = c[d] tuple assignments) no longer corrupts the enclosing production or kills the stack; C's nested shape is restored.
• Seven grammars that regressed from parity-clean during campaign development (cmake, git_config, git_rebase, regex, ruby, tsx, twig) were re-measured: six healed by the engine fixes and are locked with oracle-verified regression-pin tests; regex's remaining hidden-child divergence is pinned with a self-healing skip and tracked.
• CI: the full -race suite now actually runs for non-draft PRs without panicking Go's default per-package timeout (-timeout 35m -p 1, 60m job budget); wall-clock boundedness contracts skip under -race (instrumentation-slowdown measurement, not parser boundedness); ./grammargen runs as a non-blocking visibility step until its enumerated pre-existing backlog (stale markdown blob, two Dart parity gaps) is burned down.

Added

• docs/authoring-languages.md — adding a language without forking: grammar.json → grammargen → blob → Register/RegisterExtension/taproot, the wantsForest opt-in, generator budgets, and blob provenance discipline.
• docs/external-scanners.md — when a grammar needs an external scanner and the Go porting contract (emit extras, C-EOF behavior, error-mode lexing, token-source responsibilities), with Pawn's five externals as the worked case study.
• Oracle-verified clean-regression pin tests for seven grammars (grammars/clean_regression_pins_test.go).

Performance

• Full-parse memory (Go grammar contract benchmark): −91% B/op, −80% allocs/op vs the pre-campaign baseline.
• Incremental single-byte edit: 0 B/op, 0 allocs/op (baseline was 176 B / 3 allocs) at CPU parity (~1.4μs on CI hardware; ~70× faster than native C on the same workload). The external-scanner leaf-fastpath bailout introduced by ASI was replaced with a pooled verification source, and per-parse lexer/closure allocations were pooled away.
• Full-parse CPU is microarchitecture-dependent vs the pre-campaign baseline: −20% on modern desktop cores, +17% on 2-core CI-runner hardware (the engine parses Go via the production path instead of the retired forest dispatch, plus real ASI lexing at ~7-9%). The CI perf contract was rebased to the v0.21.0 engine; the runner-side delta is accepted and tracked for reclamation.
• No-edit reparse: ~7.5ns, 0 allocs (within the CI gate threshold vs baseline).
• Root test suite wall-clock: ~340s → ~62s (the two C# boundedness tests no longer burn 100-200s each before failing).

Patch release recovering C# large-namespace method/type declarations and the Swift ternary/conditional operator, both via post-parse source recovery passes, plus a CI stability fix for the new C# recovery test under -race.

Fixed

• Large C# files whose class body is shredded by a cumulative GLR failure (e.g. Newtonsoft.Json's JsonTextReader.cs / JsonReader.cs) now recover their method_declaration nodes instead of yielding only a comment-filled namespace shell. Follow-up to #115/#116: the source-based type/method reconstruction was gated off above 4096 bytes, so nothing rebuilt the members of a large collapsed class. Namespace recovery now falls back, when the child-based pass surfaces no method, to a per-member bounded source reconstruction — the type shell's header is reparsed for its modifiers/name/base list, and each member is recovered on its own (a method via signature-shell + lenient block, other members by a single small wrapped reparse), skipping any that still won't parse. Each reparse is a single small snippet capped by size and count and honors the parser timeout, so the anti-OOM guarantees from #64/#98/#106 are preserved and the whole-file 4096-byte gate is unchanged. JsonTextReader.cs now recovers 68 methods (was 0) and JsonReader.cs 41 (was 0). Thanks @richardwooding (#136, #138).
• Swift ternary/conditional operator (cond ? a : b) now recovers instead of dropping ? a : b into an ERROR node in every position. The runtime Swift blob never fired the ternary_expression reduction, so any function containing a ternary lost its whole parse (collapsing to _modifierless_function_declaration_no_body). A post-parse recovery pass reconstructs the ternary_expression — reparsing the source with each ? if_true : if_false tail blanked so the condition parses in place, then splicing a synthesised node with the upstream condition/if_true/if_false layout. The rewrite is accepted only when the result is error-free and byte-faithful, so non-ternary code is never affected. Thanks @richardwooding (#135, #137).

CI

• TestCSharpLargeShreddedNamespaceRecoversMethods now skips under go test -race: the per-member bounded recovery reparses each class member as its own small GLR parse, which normally finishes well inside the parser's timeout budget, but race-detector instrumentation slows the same work enough to trip the parser's internal wall-clock timeout. Non-race coverage keeps the full recovery assertions; mirrors the existing Scala realworld-recovery -race skip.

Adds consumer-controllable forest parsing.

Added

• Downstream consumers that generate a parser table with grammargen can opt their own grammar into the GSS-forest GLR fast path without forking, via three surfaces: the Language.WantsForest field (gob-serialized into blobs), the grammargen.Grammar.WantsForest flag, and a declarative "gotreesitter": { "wantsForest": true } object in grammar.json (read by ImportGrammarJSON, mirrored back by ExportGrammarJSON only when set, so standard grammars' output is unchanged). Built-in languages keep their curated, byte-range parity-certified forest defaults; consumer opt-in is at the consumer's responsibility, with the forest's decline→production fallback still preventing hard failures on declined inputs. ExtendGrammar inherits the flag from its base grammar (#134).

Patch release for parser timeout propagation and targeted language recovery scanner fixes merged after v0.20.6.

Fixed

• Parser timeout and cancellation budgets now flow through the parser loop, recovery reparses, result compatibility/finalization, and Go normalization, so strict parses stop consistently instead of continuing unbounded work after the primary parse (#114, #128).
• F# external-scanner keyword dedent fallback now guards empty indentation stacks for then, and, with, else, elif, and end, preventing scanner panics on malformed or edge-case indentation (#129, #130).
• Swift if … else if … chains no longer collapse the enclosing function to _modifierless_function_declaration_no_body (#131). The trailing-closure ambiguity recovery now follows the whole if/else-if chain — the chained if keyword is swallowed into an ERROR node, so it is discovered by scanning from the body's matching close brace — and requires a byte-faithful reparse so a partially-bracketed chain (which silently truncates without an ERROR node) is rejected rather than accepted (#132).

Patch release for parser recovery correctness, grammargen parity, and the forest/performance workstream merged after v0.20.5.

Added

• Strict parse variants return ErrParseStoppedEarly for timeout, cancellation, token-source EOF, and parser safety-limit partial trees while preserving the returned tree for diagnostics.
• NodeAtByte and NamedNodeAtByte helpers on Tree and Node for editor offset lookup without hand-written tree walks.
• One-pass code-understanding helpers for common definition spans, call references, heritage edges, and enclosing-definition lookup.
• Benchmarks comparing the one-pass code-understanding helpers against the tags-query path for both parse-plus-inspect and already-parsed trees.
• grammars.LoadLanguage(name, blob) attaches registered external scanners and external lex-state tables when loading raw grammar blobs.
• Language.Size() reports approximate decoded table and lookup-cache bytes for diagnostics and cache policy decisions.

Fixed

• JavaScript, TypeScript, and TSX automatic-semicolon scanning now preserves standalone block statements before simple assignments such as {a}b=c, matching the C parser on minified bundle shapes (#111).
• Large Go files with wide table-driven literals now have an opt-in Cobra regression gate so release validation catches parser stack overflows like the ParserPool.Parse crash reported against command_test.go and completions_test.go (#110).
• Recovered result trees now strip self-references and ancestor back-edges before parent-link wiring, while keeping children, fieldIDs, and fieldSources aligned when a cyclic edge is removed (#121).
• Go and Go module recovered parses keep the grammar source_file root when child nodes contain parse errors, matching the root-shape behavior already used for SQL and Swift (#112).
• grammargen now treats explicit precedence wrappers around finite string choices, such as prec.right(choice("=", "+=", "-=")), as reducible nonterminals instead of overlapping named lexer tokens. This restores wrapper nodes and lets LR precedence resolve the intended conflict (#122).
• Swift functions that iterate a for…in loop over a range (0..<n, 0...n) or a call expression (stride(from:to:by:)) no longer silently collapse to _modifierless_function_declaration_no_body with the loop body spilled out as file-level siblings. As with the if/while case, the loop body brace was being consumed as a trailing closure of the iterable; recovery now re-parses the affected for…in headers with synthetic parentheses around the iterable and maps the result back to byte-faithful original coordinates. Because this misparse produced no ERROR node, the recovery pass now runs whenever the detection walk finds a collapsed header rather than only on errored trees (#123).

Testing

• Added TestGoCobraLargeFileParseRegression, gated by GTS_COBRA_REGRESSION_ROOT, for exact large-file release validation without making normal test runs network- or corpus-dependent.

Changed

• grammargen no longer imports the grammars registry. The grammar.js importer (ImportGrammarJS) previously pulled in grammars for the embedded JavaScript language, which transitively bundled all ~200 grammar blobs (~22MB) into every consumer that merely defined a grammar via the DSL — including taproot and all downstream DSLs. The JS language is now injected via SetJSGrammarProvider; blank-import grammargen/grammarjs (or cmds that need -js) to register it. Net effect: grammargen, taproot, and anything that only defines/loads a grammar are now grammar-registry-free.

Added

• taproot/walk: a grammar-free core of the taproot harness. It loads a tree-sitter Language from a pre-generated blob (LanguageFromBlob) and navigates the CST (Walker, ParseFromBlob, ParseWithLanguage) depending only on the gotreesitter runtime — not grammargen or the grammars registry. DSLs that embed a generated grammar blob can now parse/highlight without linking the ~200-grammar registry (~22 MB). The grammargen-backed build-from-DSL fallbacks remain in taproot, which re-exports walk.Walker so existing taproot.Walker/Parse callers are unaffected.

Fixed

• C# files whose namespace body does not parse cleanly no longer collapse into a single top-level ERROR node with zero recoverable declarations. Namespace recovery now falls back to a best-effort namespace_declaration built from the existing sub-parse, surfacing the type declarations (and the members that parsed) instead of discarding the whole file. C# brace matching used during recovery is now trivia-aware, so braces inside char literals, strings and comments no longer truncate a recovered declaration's span (#115).
• Swift functions whose if/while condition contains a comparison operator (< / > / ==, etc.) no longer collapse into an ERROR tree with no recoverable function_declaration. The body brace was being consumed as a trailing closure of the condition's last operand; recovery now re-parses the affected conditions with synthetic parentheses to remove the ambiguity and maps the result back to byte-faithful original coordinates (#118).
• Go: normalizeGoDotLeafChildren now walks dotted-selector chains with an iterative DFS instead of recursion, removing a stack-depth risk on very long selector chains.

Changed

• Removed dead unexported code (#117).

Testing

• Banked a (skipped) regression guard, TestJavaScriptBlockThenAssignmentParsesClean, for the JavaScript block-then-simple-assignment GLR collapse ({a}b=c, #111). The root cause is the JSX-attribute-continuation ASI heuristic in the JS scanner; the fix is still pending (targeted for the C-oracle-verified parity line). Remove the t.Skip to validate once fixed.