Last release 2 days ago
26 Aug 2026
Ships on a steady schedule
a new release about every 9 days
Nearly every release is documented
notes for 59 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
6 months old
640 releases · first in 2026
Release timeline
640 releases since 2026Releases
- v0.51.016 Aug 2026
Release notes
Open source →Added
-
Add
grammars.ParseFilePooledStrict. It rejects a partial tree and returns
the parser stop error. -
Add
Tagger.TagStrict. It returns tags only after a complete parse and
preserves parser errors.
Changed
-
Bound pending fork-stack retention. Keep a bounded reserve during parsing,
and drop oversized storage at parse and parser-pool boundaries. -
Separate C-runtime recovery walk, convergence, and fallback state. Reset
these states across parse, retry, snippet, pool, and iteration boundaries. -
Provision dense reach marks when generalized LR (GLR) slabs grow. Owned nodes
no longer use the pointer-map fallback.
Fixed
- Clear stale C-runtime recovery state after a clean condense. A clean suffix
no longer inherits an active recovery-cost gate.
Tests
- Align the Application Binary Interface version 14 (ABI-14) lexer-mode probe
with generated end-of-file behavior. The probe now checks the emitted layout
without injecting an end-of-file state.
Continuous integration (CI)
- The project merged pull request (PR) #732.
Its CI-only change separates query-fleet smoke tests from regular shards and
is not part of this release.
Open work
The following items remain open:
-
Issue #454 tracks
downstream field reports. -
Issue #576 tracks
Swift recovery divergence. -
Issue #586 tracks
shared GLR error-cost bounds. -
Issue #728 tracks
external-scanner incremental reuse.
Additional notes
Open source →Added
-
Add
grammars.ParseFilePooledStrict. It rejects a partial tree and returns the parser stop error. -
Add
Tagger.TagStrict. It returns tags only after a complete parse and preserves parser errors.
Changed
-
Bound pending fork-stack retention. Keep a bounded reserve during parsing, and drop oversized storage at parse and parser-pool boundaries.
-
Separate C-runtime recovery walk, convergence, and fallback state. Reset these states across parse, retry, snippet, pool, and iteration boundaries.
-
Provision dense reach marks when generalized LR (GLR) slabs grow. Owned nodes no longer use the pointer-map fallback.
Fixed
- Clear stale C-runtime recovery state after a clean condense. A clean suffix no longer inherits an active recovery-cost gate.
Tests
- Align the Application Binary Interface version 14 (ABI-14) lexer-mode probe with generated end-of-file behavior. The probe now checks the emitted layout without injecting an end-of-file state.
Continuous integration (CI)
- The project merged pull request (PR) #732. Its CI-only change separates query-fleet smoke tests from regular shards and is not part of this release.
Open work
The following items remain open:
-
Issue #454 tracks downstream field reports.
-
Issue #576 tracks Swift recovery divergence.
-
Issue #586 tracks shared GLR error-cost bounds.
-
The Swift #586 compact-parser receipt records a NO-GO recovery-cost and token-producer blocker. Keep the issue open.
-
Issue #728 tracks external-scanner incremental reuse.
Additional notes
Open source →Added
-
Add
grammars.ParseFilePooledStrict. It rejects a partial tree and returns the parser stop error. -
Add
Tagger.TagStrict. It returns tags only after a complete parse and preserves parser errors.
Changed
-
Bound pending fork-stack retention. Keep a bounded reserve during parsing, and drop oversized storage at parse and parser-pool boundaries.
-
Separate C-runtime recovery walk, convergence, and fallback state. Reset these states across parse, retry, snippet, pool, and iteration boundaries.
-
Provision dense reach marks when generalized LR (GLR) slabs grow. Owned nodes no longer use the pointer-map fallback.
Fixed
- Clear stale C-runtime recovery state after a clean condense. A clean suffix no longer inherits an active recovery-cost gate.
Tests
- Align the Application Binary Interface version 14 (ABI-14) lexer-mode probe with generated end-of-file behavior. The probe now checks the emitted layout without injecting an end-of-file state.
Continuous integration (CI)
- The project merged pull request (PR) #732. Its CI-only change separates query-fleet smoke tests from regular shards and is not part of this release.
Open work
The following items remain open:
-
Issue #454 tracks downstream field reports.
-
Issue #576 tracks Swift recovery divergence.
-
Issue #586 tracks shared GLR error-cost bounds.
-
Issue #728 tracks external-scanner incremental reuse.
-
- v0.50.2-0.20260816132621-8c17e0e942e016 Aug 2026pre-release
Nothing published for this version
- v0.50.2-0.20260816120041-298fa050001716 Aug 2026pre-release
Nothing published for this version
- v0.50.2-0.20260816072012-556ba43b2ada16 Aug 2026pre-release
Nothing published for this version
- v0.50.2-0.20260816051803-ed5dd9cdd5e416 Aug 2026pre-release
Nothing published for this version
- v0.50.2-0.20260816033615-899783970a9316 Aug 2026pre-release
Nothing published for this version
- v0.50.2-0.20260815010114-bd1fa3cc8ded15 Aug 2026pre-release
Nothing published for this version
- v0.50.114 Aug 2026
Release notes
Open source →Fixed
-
Restore single-language
grammar_subsetbuilds. Shared lexer helpers and
Python-derived scanner state now compile with each grammar that uses them.
Derivative-only builds do not register Python scanner metadata. -
Add a blocking subset build sweep. Continuous integration now builds all
206 registered grammars with their individualgrammar_subsettags.
Additional notes2 sources agree
Open source →Fixed
-
Restore single-language
grammar_subsetbuilds. Shared lexer helpers and Python-derived scanner state now compile with each grammar that uses them. Derivative-only builds do not register Python scanner metadata. -
Add a blocking subset build sweep. Continuous integration now builds all 206 registered grammars with their individual
grammar_subsettags.
-
- v0.50.1-0.20260814213657-4e586e92e37614 Aug 2026pre-release
Nothing published for this version
- v0.50.014 Aug 2026
Release notes
Open source →Added
-
TestOutlineOracleDifferential(cgo_harness/outline_differential_test.go)
runs each language's resolved tags query through both the pure-Go query
engine and the official C tree-sitter runtime, then diffs the two capture
streams. It hard-asserts capture parity for the core nine outline
languages (go,python,javascript,typescript,tsx,rust,
java,c,cpp) and logs a census for every other language with a
resolvable tags query. -
TestOutlineCoverageWitnesses(grammars/outline_coverage_witness_test.go)
pins 30 languages against the realOutlinerpipeline. Each case names an
exact symbol its resolved tags query must produce, so a pattern that
compiles but never fires now fails the test instead of hiding behind a
non-empty query string. -
File-outline tags-query coverage rose from a 30-language floor to 84 of
206 registered languages: 83 with a real-corpus fixture and 1 with a
smoke-sample fixture (grammars/testdata/outline_census/baseline.json).
TestInferredTagsQueryCoverage(grammars/registry_test.go) now enforces
84 as the floor. Seedocs/outline.mdfor the full coverage tiers and the
file outline API. -
OutlineSymbol.Ownernow resolves on everyOutlineTreecall. A rule
attached throughWithOutlineOwnerRulesmatches a symbol byNodeType,
reads itsOwnerField, and descends through the rule'sUnwrapnode
types until it reaches exactly oneNameTypesterminal. Any other
outcome — an absent field, or a walk that reaches zero or more than one
terminal — leavesOwnerempty and counts one
OutlineReport.OwnerRuleMisses; aNodeTypeno attached rule names
touches neither field. -
grammars.OutlineOwnerRules(entry)gates the shipped owner-rule table by
symbol and field presence in each language's own compiled grammar, the
same wayResolveTagsQuery's inference table is gated, and composes
directly withgotreesitter.WithOutlineOwnerRules. The shipped Go rule
resolves all four receiver shapes — value, pointer, generic value, and
generic pointer — to the receiver's base type name. Seedocs/outline.md
for the full resolution contract and a worked example.
Removed
- Native root-extra folding and reduction now own Elixir comment placement
and map entry grouping. Parsing drops the hidden
_newline_before_commentscanner token without a post-parse pass.
Reduction groups map keyword pairs and wraps update and arrow entries in
the map grammar'sbinary_operatornode. This removes the Elixir
result-compatibility dispatcher arm.
Removed
- Native scheduling and reduction now own the Enforce
const intformal
parameter shape. Parsing classifiesconstas a
formal_parameter_modifierandintastype_intdirectly. It keeps
the parameter's own name and default value instead of losing them to a
misread type/name pair. This removes the Enforce result-compatibility
dispatcher arm.
Fixed
-
TypeScript and TSX no longer split a signed right-shift operator into two
generic closers.splitCompactCloseAngleTokennarrows a>>token to a
single>so nested closers such asArray<Array<string>>parse. Java
gated that split behind an "unclosed<precedes this run" check, because
>>is also a shift operator there; TypeScript and TSX carried no such
gate. Any>>whose next byte was one of( ) [ ] { } , . ; : ?was torn
apart, sox = a >> (b)failed to parse whilex = a >> bsucceeded. The
angle-depth gate now covers TypeScript and TSX, and runs only when a>>
symbol is actually active in the state: with no shift alternative to
protect, narrowing to>remains the only way to make progress. Found
while migrating the GoSX browser runtime to TypeScript, on
indices[i] = (src[i >> 3] >> (i & 7)) & 1;.
on,off,yes,no) at lex time. The grammar's word token pattern can
absorb leading whitespace before a keyword, and the keyword re-lex
previously required its match to start at byte zero, so it missed that
case and left the value as a generic string token. DFA keyword promotion
now skips the leading run first, the same way tree-sitter's own generated
keyword lexer does. This retires the Hyprlang dispatcher arm and fixes a
related bug: a trailing space after the keyword no longer produces an
incorrect boolean node. -
Seven of the file outline's core nine languages carried tags-query rows
that reference a child node type the grammar never produces at that
position:javascript,typescript,tsx,java,c,cpp, and
rust. The C query compiler rejects such a row as an "Impossible
pattern" and drops the entire multi-pattern query with it. The pure-Go
query engine has no matching check, so it compiled the same row and
silently matched nothing. The dead rows are now removed or corrected.
TestOutlineOracleDifferential's core-nine tier reports capture-stream
parity for all nine languages, and the fix changed no Go outline output —
confirmed by unchanged golden fixtures.
Removed
- The Bash assignment-wrapper and if-condition-field repairs. Native
reduction already builds the C-shapedvariable_assignmentswrapper for
two or more consecutive assignments. It already sets thecondition
field on an if-statement's condition tokens too. The Bash compatibility
pass no longer splices that wrapper's children into the enclosing node or
rebuilds the field afterward.
tree-sitter-bash's own corpus (test/corpus/literals.txt) pins the same
wrapper for two top-level assignments.
Production, compact, forest, and incremental routes match the raw parse
exactly, and the isolated C-oracle comparison matches for every case.
One unrelated Bash subpass remains live. - The FIDL result-compatibility dispatcher arm is retired. Native recovery
already builds the C-equivalent error shape for a versioned-layout-modifier
declaration whose modifier keyword carries a stray(name=value)argument
list. Production, compact, forest, and incremental routes produce the same
tree, and an isolated C-oracle parity check confirms the shape. - The HLSL subscript-assignment declarator member of the result-compatibility
dispatcher arm is retired.structured_binding_declaratorcarries a
negative dynamic precedence in the grammar. Native parsing already elects
the C-equivalent subscript-assignment expression forName[index] = value;
without a post-parse pass. Production, compact, forest, and incremental
routes stay exact. The negative-number cast and unorm-buffer members of the
HLSL arm remain live.
Additional notes2 sources agree
Open source →Added
-
TestOutlineOracleDifferential(cgo_harness/outline_differential_test.go) runs each language's resolved tags query through both the pure-Go query engine and the official C tree-sitter runtime, then diffs the two capture streams. It hard-asserts capture parity for the core nine outline languages (go,python,javascript,typescript,tsx,rust,java,c,cpp) and logs a census for every other language with a resolvable tags query. -
TestOutlineCoverageWitnesses(grammars/outline_coverage_witness_test.go) pins 30 languages against the realOutlinerpipeline. Each case names an exact symbol its resolved tags query must produce, so a pattern that compiles but never fires now fails the test instead of hiding behind a non-empty query string. -
File-outline tags-query coverage rose from a 30-language floor to 84 of 206 registered languages: 83 with a real-corpus fixture and 1 with a smoke-sample fixture (
grammars/testdata/outline_census/baseline.json).TestInferredTagsQueryCoverage(grammars/registry_test.go) now enforces 84 as the floor. Seedocs/outline.mdfor the full coverage tiers and the file outline API. -
OutlineSymbol.Ownernow resolves on everyOutlineTreecall. A rule attached throughWithOutlineOwnerRulesmatches a symbol byNodeType, reads itsOwnerField, and descends through the rule'sUnwrapnode types until it reaches exactly oneNameTypesterminal. Any other outcome — an absent field, or a walk that reaches zero or more than one terminal — leavesOwnerempty and counts oneOutlineReport.OwnerRuleMisses; aNodeTypeno attached rule names touches neither field. -
grammars.OutlineOwnerRules(entry)gates the shipped owner-rule table by symbol and field presence in each language's own compiled grammar, the same wayResolveTagsQuery's inference table is gated, and composes directly withgotreesitter.WithOutlineOwnerRules. The shipped Go rule resolves all four receiver shapes — value, pointer, generic value, and generic pointer — to the receiver's base type name. Seedocs/outline.mdfor the full resolution contract and a worked example.
Removed
- Native root-extra folding and reduction now own Elixir comment placement
and map entry grouping. Parsing drops the hidden
_newline_before_commentscanner token without a post-parse pass. Reduction groups map keyword pairs and wraps update and arrow entries in the map grammar'sbinary_operatornode. This removes the Elixir result-compatibility dispatcher arm.
Removed
- Native scheduling and reduction now own the Enforce
const intformal parameter shape. Parsing classifiesconstas aformal_parameter_modifierandintastype_intdirectly. It keeps the parameter's own name and default value instead of losing them to a misread type/name pair. This removes the Enforce result-compatibility dispatcher arm.
Fixed
-
TypeScript and TSX no longer split a signed right-shift operator into two generic closers.
splitCompactCloseAngleTokennarrows a>>token to a single>so nested closers such asArray<Array<string>>parse. Java gated that split behind an "unclosed<precedes this run" check, because>>is also a shift operator there; TypeScript and TSX carried no such gate. Any>>whose next byte was one of( ) [ ] { } , . ; : ?was torn apart, sox = a >> (b)failed to parse whilex = a >> bsucceeded. The angle-depth gate now covers TypeScript and TSX, and runs only when a>>symbol is actually active in the state: with no shift alternative to protect, narrowing to>remains the only way to make progress. Found while migrating the GoSX browser runtime to TypeScript, onindices[i] = (src[i >> 3] >> (i & 7)) & 1;.on,off,yes,no) at lex time. The grammar's word token pattern can absorb leading whitespace before a keyword, and the keyword re-lex previously required its match to start at byte zero, so it missed that case and left the value as a generic string token. DFA keyword promotion now skips the leading run first, the same way tree-sitter's own generated keyword lexer does. This retires the Hyprlang dispatcher arm and fixes a related bug: a trailing space after the keyword no longer produces an incorrect boolean node. -
Seven of the file outline's core nine languages carried tags-query rows that reference a child node type the grammar never produces at that position:
javascript,typescript,tsx,java,c,cpp, andrust. The C query compiler rejects such a row as an "Impossible pattern" and drops the entire multi-pattern query with it. The pure-Go query engine has no matching check, so it compiled the same row and silently matched nothing. The dead rows are now removed or corrected.TestOutlineOracleDifferential's core-nine tier reports capture-stream parity for all nine languages, and the fix changed no Go outline output — confirmed by unchanged golden fixtures.
Removed
- The Bash assignment-wrapper and if-condition-field repairs. Native
reduction already builds the C-shaped
variable_assignmentswrapper for two or more consecutive assignments. It already sets theconditionfield on an if-statement's condition tokens too. The Bash compatibility pass no longer splices that wrapper's children into the enclosing node or rebuilds the field afterward. tree-sitter-bash's own corpus (test/corpus/literals.txt) pins the same wrapper for two top-level assignments. Production, compact, forest, and incremental routes match the raw parse exactly, and the isolated C-oracle comparison matches for every case. One unrelated Bash subpass remains live. - The FIDL result-compatibility dispatcher arm is retired. Native recovery
already builds the C-equivalent error shape for a versioned-layout-modifier
declaration whose modifier keyword carries a stray
(name=value)argument list. Production, compact, forest, and incremental routes produce the same tree, and an isolated C-oracle parity check confirms the shape. - The HLSL subscript-assignment declarator member of the result-compatibility
dispatcher arm is retired.
structured_binding_declaratorcarries a negative dynamic precedence in the grammar. Native parsing already elects the C-equivalent subscript-assignment expression forName[index] = value;without a post-parse pass. Production, compact, forest, and incremental routes stay exact. The negative-number cast and unorm-buffer members of the HLSL arm remain live.
-
- v0.49.1-0.20260814193439-9fd59953f9f314 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260814164056-e79ea7a15ea314 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260814124753-25a50aaafd8614 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260814034523-fb8728a556d914 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260813230901-2ca9d370e56f13 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260813211645-aa8c172fd00913 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260812045839-1d8da9f88bdd12 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260812040033-56e4e0913de112 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260812024711-a5f139249cbd12 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260812020523-cf38f2b9537c12 Aug 2026pre-release
Nothing published for this version
- v0.49.1-0.20260812003028-658153fc060012 Aug 2026pre-release
Nothing published for this version
- v0.49.011 Aug 2026
Release notes
Open source →Removed
- One member of the Go result-compatibility arm is retired: the member
that widened the root span across a trailing end-of-file newline.
extendNodeToTrailingWhitespaceruns unconditionally after the
compatibility pass and accepts a superset of the byte set the Go member
tested, so the Go member could never reach a span the later pass did not
already reach.
A census recorded 35,244 gate entries and zero rewrites.
The four pinned canonical Go deep-tree digests are byte-identical, and
the exhaustive C-oracle fresh and incremental parity sweep stays green.
Added
-
FactProgramcompiles selected definition, call, heritage, and import work
into a dense 16-bit instruction stream. One program can process compatible
trees with one traversal while the individual extraction APIs remain stable.
The 20-seed combined benchmark reduced Go tree inspection time by 71.19%
for definitions and calls. The all-facts path reduced time by 84.14%, bytes
by 1.88%, and allocations by 49.41%. Parse-plus-extraction stayed unchanged. -
Opt-in Lean 4 support now provides a grammar blob, scanner, highlights,
outline tags, and focused corpus tests ingrammars/lean. The default
206-language registry remains unchanged. -
The V10 fleet harness now runs bounded Google Cloud spot workers across all
registered languages. It enforces wall, memory, disk, and automatic deletion
limits. Accepted epoch20260808T202958Z-v10-full-5003ffbacompleted all
1,435 measurements for 206 languages. -
scripts/run_randomized_benchmarks.shnow runs one process for each shuffle
seed and randomizes benchmark order. The standard comparison uses 20 seeds,
GOMAXPROCS=1, a 750 millisecond benchmark time, and memory reporting. -
Merge-event census instrumentation now records merge decisions and refusal
gates on the production and C-oracle paths. Its first constructed receipt
reports 15 Go merges against 191 C merges across 104 sources. It records no
source where Go merges more often than C. -
A regression guard covers issue #660.
It checks anonymous comma nodes in Python imports and subscripts on both
parser routes. -
The included-ranges route now has committed test coverage.
Parser.SetIncludedRangeshad no test for any language, and injection
uses that call for every injected child.
A Markdown document with two or more Go fences reaches it in production.
Four new gates cover the route: a root-symbol gate, a positive control
that proves the Go arm still rewrites the tree there, a C-oracle table
that pins the measured root of both parsers across four range
geometries, and a deletion guard.
The route is not at parity with C, and the new tests do not claim it is.
The root span matches C only when the first range starts at byte 0 and
the last ends at end of file, which is a shape an injection child never
receives.
One pinned geometry publishes anERRORroot where C publishes
source_file.
That case is an open defect in included-range clipping, recorded so the
fix moves the pin. -
The Go result-compatibility arm's three members are now registered as
named census subpasses, so a census receipt names the member that
rewrote the tree instead of only the arm.
Census receipts are recorded on the production route only; the default
candidate route leavesParseRuntime().NormalizationPassesnil.
Fixed
-
C enum lists with three or more enumerators no longer publish
ERRORor
MISSINGnodes. A clean forest result can replace a recovered tree only
after it covers the full source and contains no recovery nodes. This fixes
issue #667. -
Node.HasErrorOrMissingreports both recovery node forms. The
grammargen parse -strictcommand now rejects either form. -
JavaScript, TypeScript, and TSX scanners now bind external results through
each language's positional symbol table. Regenerated blobs no longer mistype
shifted external symbols. -
The full-parse retry selector no longer releases an incumbent when a
candidate aliases it. This restores selected roots across retry and compact
fallback paths. -
The accepted-error retry ladder now honors explicit stack and merge caps.
It also keeps bounded pass counts and configured wall budgets across retries. -
Kotlin published an
ERRORroot instead ofsource_filewhen a parse used
Parser.SetIncludedRangeswith more than one range and the parse entered
recovery.
The recovered-root normalization that owns this result was removed on
2026-08-02 as dead code.
The census behind that removal measured the fresh, over-64-KiB, incremental
and pinned routes.
It never measured the included-ranges route, and the member is live there.
The member is restored.
On the committed witness,testdata/included_ranges/kotlin_work_queue_test.kt
with three ranges, the root returns tosource_file, which is the kind the
locked Kotlin C reference runtime publishes for the same input.
Production reaches this route through injection.
The member retags the root.
Its downstream consequence is not always toward the reference runtime: on one
measured file the retag lets a later stage flatten a clean
class_declarationinto root-level members.
The route now has committed test coverage for Kotlin, in the root package and
in the C-parity lane. -
The C-recovery missing-token search (
cHandleError/
cDoAllPotentialReductions,parser_recover_c.go) cloned the whole GSS
stack without a work limit.
A 4-byte erlang input and a 56-byte jsdoc input drove heap use past 2 GB
in seconds.
Two new loop ceilings now bound the search directly.
cRecoverMaxReductionCandidateAttemptscaps candidate attempts within one
cDoAllPotentialReductionscall.
cRecoverMaxMissingTokenTrialscaps total trials across one
cHandleErrorsearch.
cRecoverMaxReductionCandidateAttemptsis the active mechanism.
It is what stops every known witness and every corpus file measured so
far.
cRecoverMaxMissingTokenTrialshas not fired once on any of them.
It stays in place as an unexercised backstop for input this codebase has
not sampled yet, not as a mechanism this fix currently relies on.
A newParser.budgetScratchpointer feeds GSS-scratch allocation into
the existing 512 MB soft memory budget.
The check already in the main parse loop now covers the C-recovery
candidate search too.Measured through the shipped regression test on the
erlang_pfx_017_71b
witness (testdata/recovery_memory_bound_witnesses/): heap growth after
this fix is 144.1 MB on the production route and 138.9 MB on the compact
route, in well under half a second.
Before this fix, the same input reached 695 MB and 52.0 seconds.
Both post-fix numbers are the real, reproducible figures.
An earlier draft of this fix reported smaller ones.parser_memory_budget_runtime.go'sruntimeMemoryHardCeilingEnabled
function keeps its exact prior behavior.
This fix adds a comment there recording why an earlier draft's
source-length-independent hard ceiling was tried and dropped.
It reopened issue #454's determinism symptom class on sub-64 KiB input.
It also cost 3.6-9x more time on ordinary small parses, with no
offsetting protection over the two loop ceilings above. -
The GSS-forest link cap could silently drop the widest hidden-symbol
alternative when it tied a narrower one on score and error cost.
The cap kept the earlier arrival by default in every tie.
json5's flat-array grammar shape hits this tie constantly on ordinary
input.
Other forest-default languages hit it rarely or not at all on their
current tables.
The cap now keeps the wider alternative when two links tie on the same
symbol and the same end byte.
A narrower-tie or cross-symbol tie keeps the prior behavior unchanged.
Parser.ForestCapTieStats()is a new method.
It reports how often the tie fires and how often the fix changes the
outcome.
SetGOT_FOREST_CAP_TIE_DUMP=1to also record a bounded per-decision
receipt list. -
Corrected the root-cause comment on the javascript declared-conflict
election witness test.
grammargen/lr.goalready retains thelabeled_statement/_property_name
GLR fork that the real tree-sitter-javascript grammar declares.
That fix landed 2026-03-16.
The shippedjavascript.binblob predates the fix by two weeks.
Nothing resynced the blob afterward, so the raw parse still diverges from
the C oracle today.
A new unit test pins the retention rule directly against the generator.
Regressions now surface without a blob rebuild. -
The Swift optional-binding vs trailing-closure fix (#542) added a
shift/reduce precedence branch togrammargen/lr.go.
It ran before the declared-conflict retention check.
It also matched ordinary undeclared conflicts in other grammars and
picked the wrong side.
JavaScript'supdate_expressionvsbinary_expressionconflict is one
example.
The branch now runs last, after declared-conflict retention and the
ordinary precedence ladder get a chance to resolve the conflict first.
The Swift case from #542 still resolves correctly.
A new test now guards javascript and typescript regeneration against the
C reference parser on real source files.
Changed
-
Parser stop checks now skip inactive callbacks and keep the common callback
direct. Result materialization reads the wall clock every 64 checkpoints.
Cancellation and sticky stop checks still run at every checkpoint. -
GLR recovery now computes C-compatible error cost and visible counts in one
tree walk. Memo indexing uses pointer-bit folds and checks the primary way
first. Graph-structured stack (GSS) nodes store clean-zero merge results
without a larger node layout. Extra-link mutations invalidate the result. -
C-recovery promotes an error stack to the graph-structured stack before
reduction forks. Deep recovery branches now share their immutable prefix.
The Swift recovery witness reduced time by 9.96%, bytes by 59.65%, and mean
peak resident memory by 22.09%. The 20-seed combined suite reduced KDL
recovery time by 1.20%, bytes by 13.14%, and allocations by 1.58%.
Other parser timings stayed neutral. -
The randomized benchmark suite now accepts an exact recovery corpus file and
language. The 20-seed comparison against the release boundary reduced the
timing geomean by 1.77%. Elixir recovery improved by 15.21%, KDL recovery by
9.12%, full parse by 1.16%, and incremental no-edit by 6.51%.
FactProgramparse and extraction improved by 1.23%.
The parser-core control stayed neutral. No timing, byte, or allocation metric
had a significant regression. -
The guarded parser-core bytecode experiment now supports
REDUCE_CHAINand
REDUCE_SHIFT. The corridor remains off by default. Each superinstruction
also requires its own experiment gate. -
Synthetic-root replay now hashes frames and gap cursors, memoizes gap tokens
and advance transitions, reuses scratch, and pools external token sources.
Paged advance and close streams bound retained memo storage.
Advance memoization cut the hard Elixir target latency by 22.62 percent.
Close paging cut bytes per operation by 5.15 percent and allocations by
60.48 percent. Its latency remained neutral across 20 balanced pairs. All
16 combined-suite latency rows also remained neutral. -
Exact C and V runtime profiles now avoid certified duplicate retry work.
Other grammars keep the conservative retry ladder. -
Performance counters now expose maximum resident memory, replay closure
distribution, memo capacity skips, and parser stop attribution. -
The compact fresh-path route now skips two tail steps for a language with
no live result-compatibility entry: the C-recovery-swallow resolver and
the final-tree compaction pass.
The eligible set is computed from
testdata/result_compat_ownership_v1.json.
It is not a maintained list.
A future dispatcher arm cannot silently escape it.
163 of 206 registered languages are eligible today.
Go is not one of them.
dispatch.gostays live, sogrammargen_lrand the other three canonical
Go fixtures still take the full tail.
A deep-tree digest comparison (elided against unelided) is exact across
every eligible language's smoke sample.
It is also exact across every real-corpus file this campaign measured, up
to 484 KB.
This is a correctness-neutral simplification, not a measured performance
win: the result-compatibility dispatch and its error-summary walk already
run once, during materialization, for every language; the tail's own copy
of that work was already unreachable in the common case before this
change, eligible language or not.
Two eligible-language timing probes (OCaml, Zig) and the Go warm-route
benchmark all read within this shared host's noise floor, consistent with
that finding.
See the PR for the full reading anddocs/compat-tail-elision.mdfor the
corrected performance and correctness analysis. -
The condense-candidate dispatch path no longer passes a closure through
two wrapper layers per event.
Each shift, cohort, and reduction entry point now validates the scheduler
owner and calls the uncheckpointed operation directly.
Behavior is unchanged; every identity, work-count, and allocation check
still passes.
Local timing on a shared host showed no significant change across four
fixtures.
The host carried heavy background load throughout the run, so the result
is not a sealed measurement. -
Removed the 64 KiB source-length eligibility decline from the compact
admission switch.
A fresh full parse of any size now attempts the compact route first.
The scheduler's stop-control poll bounds a large or pathological input:
it compares the compact core's own real retained-memory footprint
against the same soft memory budget production honors, and falls back to
production with a matchingParseStopMemoryBudgetstop reason when the
budget is exceeded.
Every decline path now releases the compact core's retained capacity
before returning, not only its logical record count, so a production
fallback does not run alongside megabytes of memory an earlier declined
attempt on the same parser left allocated.
An operator watchingAdmissionCandidateCounters()sees this directly:
a large input that declines bumps the fallback count exactly as a small
one always did.
This bounds retained footprint, not the compact scheduler's own transient
per-token allocation during a declined attempt; closing that remaining
gap trades against how large an input the compact route can still serve,
and is an open follow-up, not resolved by this change.
Routing only changes; every canonical tree digest stays identical.
Additional notes2 sources agree
Open source →Removed
- One member of the Go result-compatibility arm is retired: the member
that widened the root span across a trailing end-of-file newline.
extendNodeToTrailingWhitespaceruns unconditionally after the compatibility pass and accepts a superset of the byte set the Go member tested, so the Go member could never reach a span the later pass did not already reach. A census recorded 35,244 gate entries and zero rewrites. The four pinned canonical Go deep-tree digests are byte-identical, and the exhaustive C-oracle fresh and incremental parity sweep stays green.
Added
-
FactProgramcompiles selected definition, call, heritage, and import work into a dense 16-bit instruction stream. One program can process compatible trees with one traversal while the individual extraction APIs remain stable. The 20-seed combined benchmark reduced Go tree inspection time by 71.19% for definitions and calls. The all-facts path reduced time by 84.14%, bytes by 1.88%, and allocations by 49.41%. Parse-plus-extraction stayed unchanged. -
Opt-in Lean 4 support now provides a grammar blob, scanner, highlights, outline tags, and focused corpus tests in
grammars/lean. The default 206-language registry remains unchanged. -
The V10 fleet harness now runs bounded Google Cloud spot workers across all registered languages. It enforces wall, memory, disk, and automatic deletion limits. Accepted epoch
20260808T202958Z-v10-full-5003ffbacompleted all 1,435 measurements for 206 languages. -
scripts/run_randomized_benchmarks.shnow runs one process for each shuffle seed and randomizes benchmark order. The standard comparison uses 20 seeds,GOMAXPROCS=1, a 750 millisecond benchmark time, and memory reporting. -
Merge-event census instrumentation now records merge decisions and refusal gates on the production and C-oracle paths. Its first constructed receipt reports 15 Go merges against 191 C merges across 104 sources. It records no source where Go merges more often than C.
-
A regression guard covers issue #660. It checks anonymous comma nodes in Python imports and subscripts on both parser routes.
-
The included-ranges route now has committed test coverage.
Parser.SetIncludedRangeshad no test for any language, and injection uses that call for every injected child. A Markdown document with two or more Go fences reaches it in production. Four new gates cover the route: a root-symbol gate, a positive control that proves the Go arm still rewrites the tree there, a C-oracle table that pins the measured root of both parsers across four range geometries, and a deletion guard. The route is not at parity with C, and the new tests do not claim it is. The root span matches C only when the first range starts at byte 0 and the last ends at end of file, which is a shape an injection child never receives. One pinned geometry publishes anERRORroot where C publishessource_file. That case is an open defect in included-range clipping, recorded so the fix moves the pin. -
The Go result-compatibility arm's three members are now registered as named census subpasses, so a census receipt names the member that rewrote the tree instead of only the arm. Census receipts are recorded on the production route only; the default candidate route leaves
ParseRuntime().NormalizationPassesnil.
Fixed
-
C enum lists with three or more enumerators no longer publish
ERRORorMISSINGnodes. A clean forest result can replace a recovered tree only after it covers the full source and contains no recovery nodes. This fixes issue #667. -
Node.HasErrorOrMissingreports both recovery node forms. Thegrammargen parse -strictcommand now rejects either form. -
JavaScript, TypeScript, and TSX scanners now bind external results through each language's positional symbol table. Regenerated blobs no longer mistype shifted external symbols.
-
The full-parse retry selector no longer releases an incumbent when a candidate aliases it. This restores selected roots across retry and compact fallback paths.
-
The accepted-error retry ladder now honors explicit stack and merge caps. It also keeps bounded pass counts and configured wall budgets across retries.
-
Kotlin published an
ERRORroot instead ofsource_filewhen a parse usedParser.SetIncludedRangeswith more than one range and the parse entered recovery. The recovered-root normalization that owns this result was removed on 2026-08-02 as dead code. The census behind that removal measured the fresh, over-64-KiB, incremental and pinned routes. It never measured the included-ranges route, and the member is live there. The member is restored. On the committed witness,testdata/included_ranges/kotlin_work_queue_test.ktwith three ranges, the root returns tosource_file, which is the kind the locked Kotlin C reference runtime publishes for the same input. Production reaches this route through injection. The member retags the root. Its downstream consequence is not always toward the reference runtime: on one measured file the retag lets a later stage flatten a cleanclass_declarationinto root-level members. The route now has committed test coverage for Kotlin, in the root package and in the C-parity lane. -
The C-recovery missing-token search (
cHandleError/cDoAllPotentialReductions,parser_recover_c.go) cloned the whole GSS stack without a work limit. A 4-byte erlang input and a 56-byte jsdoc input drove heap use past 2 GB in seconds. Two new loop ceilings now bound the search directly.cRecoverMaxReductionCandidateAttemptscaps candidate attempts within onecDoAllPotentialReductionscall.cRecoverMaxMissingTokenTrialscaps total trials across onecHandleErrorsearch.cRecoverMaxReductionCandidateAttemptsis the active mechanism. It is what stops every known witness and every corpus file measured so far.cRecoverMaxMissingTokenTrialshas not fired once on any of them. It stays in place as an unexercised backstop for input this codebase has not sampled yet, not as a mechanism this fix currently relies on. A newParser.budgetScratchpointer feeds GSS-scratch allocation into the existing 512 MB soft memory budget. The check already in the main parse loop now covers the C-recovery candidate search too.Measured through the shipped regression test on the
erlang_pfx_017_71bwitness (testdata/recovery_memory_bound_witnesses/): heap growth after this fix is 144.1 MB on the production route and 138.9 MB on the compact route, in well under half a second. Before this fix, the same input reached 695 MB and 52.0 seconds. Both post-fix numbers are the real, reproducible figures. An earlier draft of this fix reported smaller ones.parser_memory_budget_runtime.go'sruntimeMemoryHardCeilingEnabledfunction keeps its exact prior behavior. This fix adds a comment there recording why an earlier draft's source-length-independent hard ceiling was tried and dropped. It reopened issue #454's determinism symptom class on sub-64 KiB input. It also cost 3.6-9x more time on ordinary small parses, with no offsetting protection over the two loop ceilings above. -
The GSS-forest link cap could silently drop the widest hidden-symbol alternative when it tied a narrower one on score and error cost. The cap kept the earlier arrival by default in every tie. json5's flat-array grammar shape hits this tie constantly on ordinary input. Other forest-default languages hit it rarely or not at all on their current tables. The cap now keeps the wider alternative when two links tie on the same symbol and the same end byte. A narrower-tie or cross-symbol tie keeps the prior behavior unchanged.
Parser.ForestCapTieStats()is a new method. It reports how often the tie fires and how often the fix changes the outcome. SetGOT_FOREST_CAP_TIE_DUMP=1to also record a bounded per-decision receipt list. -
Corrected the root-cause comment on the javascript declared-conflict election witness test.
grammargen/lr.goalready retains thelabeled_statement/_property_nameGLR fork that the real tree-sitter-javascript grammar declares. That fix landed 2026-03-16. The shippedjavascript.binblob predates the fix by two weeks. Nothing resynced the blob afterward, so the raw parse still diverges from the C oracle today. A new unit test pins the retention rule directly against the generator. Regressions now surface without a blob rebuild. -
The Swift optional-binding vs trailing-closure fix (#542) added a shift/reduce precedence branch to
grammargen/lr.go. It ran before the declared-conflict retention check. It also matched ordinary undeclared conflicts in other grammars and picked the wrong side. JavaScript'supdate_expressionvsbinary_expressionconflict is one example. The branch now runs last, after declared-conflict retention and the ordinary precedence ladder get a chance to resolve the conflict first. The Swift case from #542 still resolves correctly. A new test now guards javascript and typescript regeneration against the C reference parser on real source files.
Changed
-
Parser stop checks now skip inactive callbacks and keep the common callback direct. Result materialization reads the wall clock every 64 checkpoints. Cancellation and sticky stop checks still run at every checkpoint.
-
GLR recovery now computes C-compatible error cost and visible counts in one tree walk. Memo indexing uses pointer-bit folds and checks the primary way first. Graph-structured stack (GSS) nodes store clean-zero merge results without a larger node layout. Extra-link mutations invalidate the result.
-
C-recovery promotes an error stack to the graph-structured stack before reduction forks. Deep recovery branches now share their immutable prefix. The Swift recovery witness reduced time by 9.96%, bytes by 59.65%, and mean peak resident memory by 22.09%. The 20-seed combined suite reduced KDL recovery time by 1.20%, bytes by 13.14%, and allocations by 1.58%. Other parser timings stayed neutral.
-
The randomized benchmark suite now accepts an exact recovery corpus file and language. The 20-seed comparison against the release boundary reduced the timing geomean by 1.77%. Elixir recovery improved by 15.21%, KDL recovery by 9.12%, full parse by 1.16%, and incremental no-edit by 6.51%.
FactProgramparse and extraction improved by 1.23%. The parser-core control stayed neutral. No timing, byte, or allocation metric had a significant regression. -
The guarded parser-core bytecode experiment now supports
REDUCE_CHAINandREDUCE_SHIFT. The corridor remains off by default. Each superinstruction also requires its own experiment gate. -
Synthetic-root replay now hashes frames and gap cursors, memoizes gap tokens and advance transitions, reuses scratch, and pools external token sources. Paged advance and close streams bound retained memo storage. Advance memoization cut the hard Elixir target latency by 22.62 percent. Close paging cut bytes per operation by 5.15 percent and allocations by 60.48 percent. Its latency remained neutral across 20 balanced pairs. All 16 combined-suite latency rows also remained neutral.
-
Exact C and V runtime profiles now avoid certified duplicate retry work. Other grammars keep the conservative retry ladder.
-
Performance counters now expose maximum resident memory, replay closure distribution, memo capacity skips, and parser stop attribution.
-
The compact fresh-path route now skips two tail steps for a language with no live result-compatibility entry: the C-recovery-swallow resolver and the final-tree compaction pass. The eligible set is computed from
testdata/result_compat_ownership_v1.json. It is not a maintained list. A future dispatcher arm cannot silently escape it. 163 of 206 registered languages are eligible today. Go is not one of them.dispatch.gostays live, sogrammargen_lrand the other three canonical Go fixtures still take the full tail. A deep-tree digest comparison (elided against unelided) is exact across every eligible language's smoke sample. It is also exact across every real-corpus file this campaign measured, up to 484 KB. This is a correctness-neutral simplification, not a measured performance win: the result-compatibility dispatch and its error-summary walk already run once, during materialization, for every language; the tail's own copy of that work was already unreachable in the common case before this change, eligible language or not. Two eligible-language timing probes (OCaml, Zig) and the Go warm-route benchmark all read within this shared host's noise floor, consistent with that finding. See the PR for the full reading anddocs/compat-tail-elision.mdfor the corrected performance and correctness analysis. -
The condense-candidate dispatch path no longer passes a closure through two wrapper layers per event. Each shift, cohort, and reduction entry point now validates the scheduler owner and calls the uncheckpointed operation directly. Behavior is unchanged; every identity, work-count, and allocation check still passes. Local timing on a shared host showed no significant change across four fixtures. The host carried heavy background load throughout the run, so the result is not a sealed measurement.
-
Removed the 64 KiB source-length eligibility decline from the compact admission switch. A fresh full parse of any size now attempts the compact route first. The scheduler's stop-control poll bounds a large or pathological input: it compares the compact core's own real retained-memory footprint against the same soft memory budget production honors, and falls back to production with a matching
ParseStopMemoryBudgetstop reason when the budget is exceeded. Every decline path now releases the compact core's retained capacity before returning, not only its logical record count, so a production fallback does not run alongside megabytes of memory an earlier declined attempt on the same parser left allocated. An operator watchingAdmissionCandidateCounters()sees this directly: a large input that declines bumps the fallback count exactly as a small one always did. This bounds retained footprint, not the compact scheduler's own transient per-token allocation during a declined attempt; closing that remaining gap trades against how large an input the compact route can still serve, and is an open follow-up, not resolved by this change. Routing only changes; every canonical tree digest stays identical.
- One member of the Go result-compatibility arm is retired: the member
- v0.48.14 Aug 2026
Release notes
Open source →Emergency patch release for issue #660. This release restores reference-compatible field names for anonymous separator nodes in Python import and subscript forms. It includes focused regression tests and parity validation. It was built from the v0.48 maintenance branch and does not include the unresolved v10 performance work.
Additional notes2 sources agree
Open source →Fixed
- Anonymous separator nodes in Python import and subscript forms stay unfielded. The patch restores field-name behavior from the reference C runtime. This fixes issue #660.
- v0.48.1-0.20260811150718-f13d192e7e7611 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260811075126-4a15cf1eb13211 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260809090851-e1ce651493749 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260804231015-9036bbb47f2a4 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260803060937-603658c6b8b53 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260803054413-150406a97d6b3 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260803051024-0b40edaaa1eb3 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260803041029-70d8c0beb65e3 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260803030559-4347068b2a583 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802224531-8070e1ad51f42 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802205926-a2e3a2e8f57d2 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802184415-375a0e7c49322 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802162755-f6f9d81010482 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802145823-0f4e975efc9b2 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802113901-f83312959b192 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802101936-6fab988d30d42 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802090059-1c8cb42160462 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802064336-fe443305a90d2 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802052128-c3fe495ef1082 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802042202-193c44b340532 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802024807-78214007ee972 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260802005429-df619d1cda562 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260801231757-44b8054fe7091 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260801203246-c840dff7e9271 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260801184806-3325e0b107cd1 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260801100523-ba71701a0e411 Aug 2026pre-release
Nothing published for this version
- v0.48.1-0.20260801075154-083187e5c3ee1 Aug 2026pre-release
Nothing published for this version
- v0.48.01 Aug 2026
Release notes
Open source →Added
-
A validated Swift corpus now guards real-code parsing.
Twelve files come from swiftlang/swift 6.3 and apple/swift-algorithms 1.2.1.
A ratcheting expectations test fails on any regression or unrecorded fix.
Five upstream grammar gaps are recorded in issues
#574 through
#578. -
The dispatcher census now reports distinct Ada, Apex, Bash, and Cooklang
materialization subpasses.
Compatibility-free probes record active and inert producer behavior. -
grammargen -js-clinow resolvesgrammar.jswith Tree-sitter 0.26 or newer.
It imports the temporary canonicalgrammar.jsonthrough the existing path.
The explicit flag warns that grammar evaluation executes JavaScript. -
grammargen -js-clinow identifies a missing JavaScript runtime when
Tree-sitter cannot start Node. The command help and README list both
prerequisites. -
The canonical compact real-corpus matrix now records 70 direct routes,
30 fallbacks, exactly 10 skips, and no divergence or error.
The bounded current receipt covers 110 rows.
Performance
-
Live-header scoping reduces compact full-parse allocation counts by
11.27 percent on the 235,626-byte Go fixture.
Allocated bytes fall by 26.64 percent.
Parse time remains statistically unchanged for that fixture.
The rewrite fixture regresses by 19.99 percent.
The query-compile fixture regresses by 15.77 percent. -
The parser now skips four redundant source reconstruction passes for
certified isolated C# recovered roots.
The receipt requires one one-byte error, no missing nodes, and matching raw
top-level spans.
The 137 KiB deletion witness still matches the pinned C parser.
Median full-parse time falls from 8.24 seconds to 5.01 seconds.
Median memory falls from 608 MB to 195 MB.
Median allocation count falls from 286,009 to 9,663.
BenchmarkIssue454CSharpRecoveredFullParseusesGOMAXPROCS=1,
-benchmem,-benchtime=1x, and-count=5. -
The graph-structured stack shape walk now skips a duplicate cache lookup
after a known head miss.
The standard full-parse benchmark improves by 5.12 percent across 20 samples.
Allocations remain at nine per parse. -
Graph-structured stack hashing now selects inline or pooled walk storage
before it collects nodes.
The 235,626-byte Go fixture allocates 24.84 KiB instead of 110.34 KiB.
Allocations fall from 511 to 169 per parse.
Parse time and maximum resident set size remain unchanged. -
Outer parser-state re-lex transactions now reuse one 4 KiB scanner-state
buffer.
The 235,626-byte Go fixture allocates 7.583 MiB instead of 48.073 MiB.
Allocations fall from 10,404 to 513 per parse.
Two stable pairs improve parse time by 9.52 to 15.46 percent.
Maximum resident set size falls from 220,236 KiB to 185,920 KiB. -
Direct parser-state re-lex probes now use their existing outer transaction.
This removes a redundant scanner-state snapshot.
The 235,626-byte Go fixture allocates 48.07 MiB instead of 86.69 MiB.
Allocations fall from 20,290 to 10,400 per parse.
Three stable benchmark pairs improve parse time by 10.90 to 15.26 percent. -
The compact scheduler now stores its common rollback frontier inline.
This removes one allocation from a fresh full parse. -
The fresh compact runner now reuses its scheduler storage across parses.
Full-parse allocations fall from 15 to 14 per operation.
Parse time and allocated bytes remain statistically unchanged. -
The parser now reuses its bound stop-check callback across compact full
parses.
This removes one allocation and 16 bytes per operation.
Full-parse time and maximum resident set size remain unchanged.
Incremental parses retain zero allocations. -
Fresh compact full parses now store the scheduler receipt inside the
scheduler allocation.
Full-parse allocations fall from 17 to 16 per operation.
Parse time, allocated bytes, and maximum resident set size remain unchanged. -
The compact full-parse receipt now stores its acceptance value in the
scheduler receipt allocation.
Full-parse allocations fall from 18 to 17 per operation.
Parse time and allocated bytes remain statistically unchanged. -
PR #498 moved the single-header compact dispatch cell onto the stack.
This removes one allocation from the common full-parse path.
The stable Go benchmark improves full-parse time by 15.57 percent.
Allocated bytes fall by 20.25 percent.
The allocation count falls by 7.89 percent. -
Certified graph-structured stack convergence now merges duplicate C# full-parse
stacks while it retains their packed alternatives.
The 137 KiB witness improves from 1.629 seconds to 98.7 milliseconds.
Allocated bytes fall from 135.6 MB to 50.4 MB.
The allocation count falls from 7,820 to 1,429 per parse.
Incremental parses retain their previous merge policy. -
The compact full-parse runner now reuses buffers during canonicalization and
tree materialization.
Warm materialization drops from 136,584 to 272 bytes per operation.
Its allocation count drops from 47 to 8.
Total warm allocation drops from 20,440 to 5,208 bytes per operation.
Total parse time remains statistically unchanged. -
The compact scheduler stores its one-element seed frontier inside the
scheduler allocation.
The warm full-parse benchmark drops from 20,352 to 20,328 bytes per operation.
Allocations drop from 66 to 65 per operation.
Parse time remains statistically unchanged.
Fixed
-
Swift now recovers an
if/elsewhose comparison condition ends with a
parenthesised member access in the then-branch (a call argument or a
parenthesised negation). The then-block no longer swallows the trailing
elseas a call's trailing closure.
This fixes #560. -
A Swift optional generic type such as
Range<Int>?now parses cleanly.
The token source defers the closer to the DFA only when a reduce action
closes an opentype_argumentsproduction.
This fixes #556. -
A Swift constrained extension with a multiline
whereclause now parses
cleanly. The scanner carries the resolved previous rune across comment
handoffs instead of re-reading a raw source byte.
This fixes #557. -
Swift nested
if letchains inside methods now parse cleanly.
The recovery pass brackets only the right-hand side of the binding.
This fixes #558. -
Three or more nested Swift generic type arguments such as
A<B<C<Int>>>
now parse cleanly. The split fires only when an unclosed<sits open, so
custom operators such as>>>stay intact.
This fixes #559. -
A Swift method that contains a
forloop over a range, followed by another
method, now parses cleanly. The recovery pass also fires when thefor
statement forms with an error inside it.
This fixes #561. -
Raw error-cost walks now retain each captured child shape reference.
A later mutable node update cannot create a recursive shape cycle.
The Rust aggressive corpus completes all 25 bounded parses without a crash. -
TypeScript and TSX now parse
in,out, andin outvariance
annotations on type parameters.
The source overlay uses the semantics from upstream pull request 361.
This fixes issue #539. -
TypeScript and TSX now separate adjacent generic call signatures at a
newline.
The grammar uses the dedicated function-signature separator.
Generic automatic-semicolon behavior remains unchanged.
This fixes issue #540. -
Compact reductions now merge only with live scheduler headers.
Removed historical versions no longer consume the shared boundary link cap.
This retires four real-corpus fallbacks without a tree divergence. -
Shared DFA token election now prefers one composable close angle over a
wider close-angle token.
Nested TypeScript union arguments now retain the generic-call lineage.
This fixes issue #541.
Apex nested generic declarations now match the pinned C tree without a
result rewrite.
This retires the Apex generic local declaration compatibility pass. -
The full-parse retry ladder now retains a widened candidate until it reads
the candidate's runtime receipt.
This enables the existing combined stack-and-merge retry for the ZodUnion
fixture.
This fixes issue #544. -
Swift optional bindings now keep the statement body separate from a trailing
closure. The generator now uses exact advanced LR item precedence.
Production, compact, and pinned C tests cover the correction.
The regenerated Swift blob retains its exact runtime profile certification.
This fixes #542. -
The DFA lexer now splits adjacent Swift generic closers by parser state.
It preserves>>when the active state accepts the shift operator.
The pinned C oracle now matches without divergence.
This fixes #543. -
Native
grammar.jsimport now ignores comments in semantic AST children.
All 206 pinned grammars show import coverage increasing from 62 to 70. -
AWK recovery now captures the original splice parent before it constructs a
replacement concatenation.
This prevents self-parent links during recovered expression materialization.
A locked 7,392-byte production fixture now verifies bounded completion and
a stable tree digest. -
Compact recursive insertion now proves external token identity from exact
scanner checkpoints.
Mismatched or missing checkpoints fail closed.
Locked Kotlin, OCaml, Perl, and Rust fixtures now reach their next parser gate. -
Forest result selection now preserves an existing same-symbol container.
Its children must exactly match adjacent visible root containers.
This removes the inert HTTP section-coalescing compatibility pass. -
The native reduction path now sets Dart switch-expression body fields.
It now sets the target field for nested Elixir calls.
This change removes two inert language-local field repairs. -
Inherited reduction fields now fill anonymous gaps between repeated direct
descendants.
They do not cross a leading separator without direct descendant evidence.
This change removes three Scala field repairs and the SQLINTOcleanup. -
Compact graph insertion now persists exact predecessor merges across a
bounded 16-level path.
Non-exact nested edges and deeper paths still fail closed.
Locked C#, Elixir, Perl, and Scala fixtures now reach their next parser gate. -
The compact admission census now separates runnable no-table-action stops
from paused frontiers.
The real-corpus matrix labels production error trees.
Clean graduation coverage no longer counts recovery fixtures as parser gaps. -
Compact graph branches now re-lex one exact token span for each parser state
when the shared symbol has no action.
Each alternative keeps the shared byte range and scanner checkpoint.
The medium Scala corpus now reaches compact acceptance.
A separate proof for joined reduction paths still gates direct publication. -
Exact grammar profiles can now flatten certified same-span unary wrappers
during reduction materialization.
The F# profile removes its declaration-name compatibility walk.
Expression and dotted identifiers retain their wrappers.
Compact and forest routes retain fail-closed behavior. -
Native C-style recovery now owns Angular, BibTeX, Chatito, and Electronic Data
Sheet materialization for every registered recovery witness.
This removes four inert result-compatibility dispatcher arms.
Compact and forest routes retain fail-closed behavior. -
Native parser results now retain the expected Hurl and INI root types.
This removes both expected-root fallback compatibility arms as one class.
Compact and forest routes retain fail-closed behavior. -
Native recovery now owns Forth and Luau recovery-action materialization.
Forth keeps C-equivalent missing terminators and empty-definition errors.
Luau keeps recoveredendtokens as identifiers.
This removes both result-compatibility dispatcher arms. -
Parser recovery now owns skipped error materialization for Robot variables
and Scheme quote-family forms.
This removes both compatibility dispatcher arms as one defect class.
Compact and forest routes retain fail-closed behavior. -
C-style recovery now marks an absorbed
ERRORtoken as named.
Recovered INI trees now match the pinned C parser at this node boundary. -
The tracked dispatcher census now includes the locked JavaScript
convergence fixture.
The receipt exposes seven active compatibility rewrites on a direct route. -
Native Crystal scanner lookahead now skips whitespace after hash and
named-tuple openers.
Exact token boundaries retire the Crystal compatibility dispatcher arm. -
PRs #497 and #500 add locked GraphQL and Svelte direct-route fixtures.
Each fixture records its source commit and SHA-256 digest.
Dedicated C-oracle tests require exact trees and zero fallback. -
PR #499 initializes the native Typst scanner indentation stack on creation.
Native scanner semantics remove the nested-list comma artifact.
This retires the remaining Typst compatibility dispatcher arm. -
Accepted-error C# full parses now retry with a certified merge width after
cap-one convergence.
Exact grammar identity gates the policy.
Explicit environment settings keep precedence.
This preserves recovered declarations while clean full parses remain on the
faster path. -
Native ReScript materialization now owns value identifier path aliases.
This change adds two small corpus fixtures.
It removes the ReScript compatibility arm. -
Native Linker Script recovery now owns named error nodes and root spans.
This change adds clean and recovered corpus fixtures.
It removes the Linker Script compatibility arm. -
The parser covers every byte in each recovered EBNF source.
This change removes the EBNF compatibility arm. -
Native visible-wrapper election now owns D storage classes.
This retires the matching result-normalization subpass.
Native reduction also owns D variable-type qualifiers.
Native call targets now match C for qualified, template, and simple callees.
This retires the D dispatcher arm. -
The Cooklang smoke fixture now uses a valid ingredient instruction.
The previous period required production recovery and was omitted from the
resulting tree.
The valid fixture routes directly while the recovered form still falls back.
The smoke scorecard reports 200 direct routes and one fallback. -
Compact admission now treats zero-width extras as progress when their token
end advances the parser boundary.
COBOL fixed-format padding now routes directly without weakening the
same-byte no-progress guard.
The smoke scorecard reports 199 direct routes and two fallbacks. -
Compact admission now supports bounded no-lookahead reductions.
One runnable head can reduce a synthetic EOF and re-elect at the same byte.
Transparent gotos mark the reduced node as an extra.
A root reduction requires authenticated EOF on the next election.
Doxygen, JSDoc, and VHDL now route directly.
The smoke scorecard reports 198 direct routes and three fallbacks. -
Compact admission now supports two certified acceptance-frontier shapes.
HTTP and Robot can drop EOF siblings with no actions.
Meson can select the sole primary accepted derivation.
Exact blob profiles and field-aware C-oracle receipts guard these choices.
The current smoke scorecard reports 195 direct routes and no divergence. -
Compact admission now permits certified converged-path reduction split drops
for the exact Bash, Erlang, Haskell, and JavaScript artifacts.
Field-aware C-oracle receipts cover each selected compact tree.
Three real-corpus files and the Haskell smoke fixture now route directly. -
Compact reduction outputs now carry their multi-pop fact directly.
This avoids two full work snapshots on every reduction.
The stable full-parse control improves by 8 percent againstmain.
Full-parse allocation falls by 12 percent with no new allocations. -
The dispatcher census now records each live D and Objective-C subpass.
Exact fingerprints retain spans, points, fields, flags, and parser states.
The census does not materialize compact final-child references. -
The parser now folds raw descendant content into certified
materializing-shape hashes.
This prevents shallow GSS merges from discarding Objective-C method types.
The parser now owns those identifiers before result compatibility.
This retires a fifth Objective-C normalization subpass. -
Generic result selection preserves both valid Objective-C
sizeofbranches.
It selects the C-equivalent expression branch for an unknown type name.
This retires the final Objective-C subpass and its dispatcher arm. -
DFA keyword promotion now owns Arduino primitive types before compatibility.
Native materialization owns Objective-C protocol type identifiers.
This retires Arduino's dispatcher arm and the matching Objective-C subpass. -
Erlang macro replacement election now stays in the parser.
It distinguishes function clauses from case and receive clauses.
Reduction already emits exact top-level form spans.
These owners retire the Erlang result-compatibility arm. -
Compact parse-state replay now visits each derivation node once.
Its depth-first worklist retains only the active derivation path.
The stable full-parse benchmark improves without new allocations or
incremental regressions. -
Clean roots now keep hidden, childless whitespace extras as span coverage.
They do not publish those extras as children.
Visible comments and error-root evidence remain unchanged.
Final-child filtering now preserves fields without materializing lazy ranges. -
The shared root-extra classifier now drops zero-width scanner tokens from
child lists. The repetition-skip fold also stops the historical Typst
comma artifact. These producer rules retire two returned-tree walks.
Typst keeps its dispatcher arm for other repairs. -
Root spans now exclude unowned leading token padding through one shared
materialization rule. This removes seven language-local repairs.
Compact admission now accepts the same first-token start as the C oracle.
Squirrel's result-compatibility dispatcher arm is retired. -
Rust dot ranges now parse without a result repair.
The exact collapsed-child policy retains each bare..token.
The merged-left-side conflict rule selects chained dot-range shifts. -
The no-live-action re-lex no longer checks the grammar's name. This recovery
step re-lexes the lookahead when no live stack has any parse action for it,
and it was gated to JavaScript. The condition it guards is grammar
independent:noLiveStackCanAcceptLookaheadalready proves that no live stack
can consume the token, so re-lexing cannot take a token another version was
going to use, whatever the grammar. Every grammar now gets the same recovery
step. This retires one per-language gate from the parser core. -
Large GLR parses allocate far less. Two hot-path buffers grew without
amortization or reuse:- The three merge-scratch helpers (
ensureMergeResultCap,
ensureMergeSlotCapandensureMergeLargeSlotCap) allocated exactly the
requested length. Every merge pass sizes them from the live-stack count, so
a parse whose stack count climbs reallocated the whole buffer on each
increment, which makes total allocation grow with the square of the peak
stack count. One element of the large-slot buffer holds two 256-entry
arrays, so this dominated wide parses. They now double. gssNodeCanReachbuilt a new fallback map each time a link graph outgrew
its 64-entry local array. On grammars that reach that size routinely, the
map became the largest single allocation in the parse. The fallback map is
now pooled and reused.gssNodeHashgrew a fresh walk buffer on the heap whenever an unhashed
chain was longer than its 32-entry inline array. Ordinary parses stop at
the first already-hashed node, so this only bites where an error path
rebuilds deep chains: a PHP edit that introduces a transient error spent
124 MB there. The walk buffer is now pooled.
Measured on a C# corpus of repeated method declarations, with the grammar
loaded before measuring: allocation per source byte falls from 119,036 to
17,207, total allocation for an 8.7 KB input falls from 989 MB to 143 MB, and
the parse runs in 200 ms instead of 335 ms. On a 137 KiB input, allocation
falls from 5462 MB to 1111 MB and the parse takes 4.0 s instead of 6.5 s.
Java, C++, Go and JavaScript allocate exactly as before. - The three merge-scratch helpers (
-
A GLR stack that needs a different tokenization of the same bytes now gets
one. tree-sitter C lexes once per parse version, so two versions in different
states can receive different symbols for the same characters. This engine
lexes one token for every stack, which is cheaper and correct while all live
stacks accept that token. Where one stack's state required a different symbol,
that stack found no parse action, paused, and the condense step dropped it
because an unpaused rival was still alive. The rival then reached a dead end
and the whole file became oneERRORnode. Scala shows the failure most
clearly, because+,-,!and~are its only prefix operators: in
if (a) c + 2the correct derivation needs+as the generic
operator_identifier, while the rival needs the dedicated+token that
exists soprefix_expressioncan spell unary plus.while (a) c + 2failed
the same way, andif (a) c * 2always worked because*is not a prefix
operator. The parser now re-lexes at the stack's own byte offset with the
stack's own lex mode before pausing it. It adopts the result only when the new
token covers the same byte span, which keeps every version at the same offset,
and only when the stack's state has a real action for the new symbol. The
re-lex reads the internal lexer only, so no external scanner state changes.
Clean parses are unaffected, because the probe runs only where a stack would
otherwise pause. Any grammar whose characters lex differently by state gains
the same protection. -
Parse results no longer depend on garbage-collection timing. The soft
per-parse memory budget stopped a parse whenruntime.MemStats.HeapAllocor
runtime.MemStats.Sysgrew past the budget. Both values cover the whole
process, and the garbage collector paces both, so the stopping point was not
a function of the input. The same bytes returned a different tree on each
run. Five parses of one 137 KiB C# file returned five different trees, of
1, 11048, 19178, 22928 and 31928 nodes. Each tree reported
HasError() == falseover only part of the input. The budget arms at 64 KiB,
so every language was exposed above that size. Only the absolute hard ceiling
(GOT_PARSE_MEMORY_HARD_CEILING_MB) now stops a parse from a runtime memory
reading, because that ceiling guards against running out of memory and is not
a shaping decision. The arena budget, the scratch budget, and the node and
stack limits continue to bound memory. Those layers measure what the parse
itself allocated, so they stop the same input at the same place every time.
A downstream user reported this behavior in issue #454.
Changed
- v0.48.0 adds fields to
FullParseAcceptedErrorRetryProfile,ParseRuntime,
andDiagnosticParserCoreGenericWork. Change unkeyed literals to keyed
literals before you upgrade.
Removed
-
The Bash command-name concatenation repair. Native reduction now
constructs the complete command name before result compatibility.
The historical producer, all result routes, and the isolated C oracle match.
The 25-case Bash corpus matches baseline83548f55exactly.
Three unrelated Bash subpasses remain live. -
The D template-call type result repair. Generic result election now
preserves a visible named unary wrapper over its direct-child alternative.
Production, forest, incremental, and isolated C-oracle receipts match.
Four unrelated D subpasses remain live. -
Two Objective-C result repairs. Exact stack-node equivalence preserves
deep alternatives for generic alias-target selection.
Native selection now owns@encodeidentifiers and function-pointer
expression shapes.
Production, incremental, and field-aware C-oracle receipts match.
Native selection also owns single and concatenated@strings.
Raw-shape equivalence now preserves compound struct type specifiers.
Two unrelated Objective-C subpasses remain live. -
The D module-bound result repair. Native reduction already excludes
leading comments and trailing trivia from eachmodule_defspan.
Production, compact, forest, incremental, and C-oracle routes match.
Incremental parsing reuses the old tree.
The D dispatcher remains live for unrelated shape repairs. -
The HCL root normalization pass. Shared root finalization now removes
hidden whitespace extras at every root position.
Native reduction already produces each exact HCL body span.
Production, compact, forest, incremental, and locked C receipts match.
The three-file census found no mismatch across 114 body nodes.
This removes the HCL result-compatibility dispatcher arm. -
The Haskell section-span result repair. Native reduction and root
finalization already produce the exactimportsanddeclarationsranges.
The real-corpus census found no remaining rewrite.
Production, compact, incremental, and locked C receipts match.
The forest route retains its existing section reduction-cap limit.
This removes the remaining Haskell dispatcher arm. -
The source-driven collapsed-token repair family for HCL, CPON, C#, and
PowerShell. Reduction now preserves each required anonymous token child.
The same-name collapse keeps CPON null nodes childless.
This removes the CPON dispatcher arm.
The other three arms remain live for unrelated repairs.
Compatibility-free, production, compact, forest, incremental, and isolated
C-oracle receipts return the same trees. -
The CUE, Git Commit, and R alias-map result repairs. Their pinned blobs
now carry the nonterminal alias metadata from each C parser table.
Materialization keeps the required named child under each collapsed wrapper.
Production, compact, forest, incremental, and locked C receipts match.
CUE also proves nonzero old-tree reuse.
Git Commit and R record their external scanner reuse limit. -
The trailing root and child span compatibility family. Materialization
now owns the exact spans for Caddy, Comment, Fortran, Just, Nginx, Nim,
Pascal, Pug, and RST. The compact scheduler admits progressing zero-width
external extras. Forest publication omits zero-width synchronization extras
as children while it retains their source-range ownership. Native producer,
production, compact, forest, incremental, reuse, and isolated C-oracle
receipts support the removal of four dispatcher arms. -
The Lua, Make, and Zig field-projection passes. Reduction now projects
inherited and direct fields through hidden productions.
The Zig grammar metadata emits initializer lists withoutfield_constant.
Compatibility-free, production, compact, forest, incremental, and locked C
receipts return the same fields.
Make and Zig preserve old-tree reuse.
Lua records its external scanner reuse limitation. -
The Haskell and Erlang root field repairs. Reduction now retains each
inherited field conflict and projects it by an exact named-symbol match.
Root acceptance preserves producer field metadata when it absorbs trivia.
Compatibility-free, production, incremental, and isolated C-oracle receipts
preserve the expected root fields. -
The Scala returned-tree span repair subfamily. A language-neutral
in-place rewrite refresh now preserves a valid producer-owned span and can
widen it. This change deletes the Scala function-end and case-clause helpers.
It also removes the second-pass root-end call and its duplicate case-clause
block. Production, compact, forest, changed incremental, fresh, and
locked C routes return the exact ranges and points. Scala incremental reuse
remains unsupported and reports zero reuse. -
The duplicate Scala returned-tree repair calls. Recovery, field, and
annotation repair now runs only in the canonical compatibility pass.
Mandatory fixtures and the authenticated corpus report zero mutations when
the deleted calls run again. -
The shared returned-tree fixpoint. The last Scala arm became inert after
checkpoints A and B. The publication paths no longer call a repeated
post-finalization normalizer. -
The HTML returned-tree range fixup. Materialization now extends recovered
custom elements through each structural_implicit_end_tagchild.
Production, compact, forest, and incremental routes return the exact
absolute ranges. The incremental route also proves nonzero old-tree reuse.
The locked C reference parser returns the same recovered ranges and points. -
The generic terminal-leaf tree mutation. Reduction and alias
materialization now own the terminal shape. Production, compact, forest,
incremental, scanner-aware corpus, and locked Go C-oracle receipts find no
retired shape. The exact retry error summary and stop polling remain as a
read-only full-tree walk. -
Three dead per-language result-normalization dispatcher arms (R2 of
docs/root-normalization-retirement.md). The three are OCaml's collapsed
named-leaf restoration, Ruby's top-level module bound shrink, and HTML's
ERROR-root nested-custom-tag reconstruction
(normalizeHTMLRecoveredNestedCustomTags). At the R2 checkpoint, HTML's
separate range function stayed live. The R1 item above now removes that
function independently. A real-corpus census measured zero rewrites for all
three dispatcher arms. Native-parse tests confirm that the reduce engine
already produces the corrected shape without them.A fourth candidate, Elixir, stayed live. Its census also measured zero
rewrites over the real corpus. A native-parse regression test found the
cause: the corpus sample lacked the triggering construct. Two consecutive
top-level comments — a common file-header shape — still lose their hidden
_newline_before_commentsibling without the normalizer. The ownership
registry keeps all four entries as historical receipts.
Changed
-
JavaScript program-end finalization now has one authoritative compatibility
owner. The redundant returned-tree second pass is retired; production,
compact final-child-ref, forest, and incremental publication continue to use
the canonical JavaScript compatibility pipeline before the tree is exposed. -
Clean hidden whitespace-only root tails are now owned by root finalization,
retiring a generic compatibility pass while preserving error-root recovery
extras and lazy compact child references.
Performance
-
Same-length single-byte replacements now mark the affected path without
recomputing unchanged spans. Other edits and compact child references keep
the general editor. The pinned incremental benchmark improves 2.10 percent
with zero allocations. -
Fresh parse finalization now computes the retry error summary while it wires
parent links. This removes one complete tree traversal.
Deferred and incremental paths retain their separate summary walk.
Under-flagged errors and stop polling keep their existing behavior.
The pinned Go full-parse benchmark improves 0.81 percent. -
Parser retry policy now snapshots override presence with each parsed value.
This removes repeated environment lookups from the incremental hot path.
The pinned edited incremental benchmark improves 1.28 percent with zero
allocations.
Additional notes2 sources agree
Open source →Added
-
A validated Swift corpus now guards real-code parsing. Twelve files come from swiftlang/swift 6.3 and apple/swift-algorithms 1.2.1. A ratcheting expectations test fails on any regression or unrecorded fix. Five upstream grammar gaps are recorded in issues #574 through #578.
-
The dispatcher census now reports distinct Ada, Apex, Bash, and Cooklang materialization subpasses. Compatibility-free probes record active and inert producer behavior.
-
grammargen -js-clinow resolvesgrammar.jswith Tree-sitter 0.26 or newer. It imports the temporary canonicalgrammar.jsonthrough the existing path. The explicit flag warns that grammar evaluation executes JavaScript. -
grammargen -js-clinow identifies a missing JavaScript runtime when Tree-sitter cannot start Node. The command help and README list both prerequisites. -
The canonical compact real-corpus matrix now records 70 direct routes, 30 fallbacks, exactly 10 skips, and no divergence or error. The bounded current receipt covers 110 rows.
Performance
-
Live-header scoping reduces compact full-parse allocation counts by 11.27 percent on the 235,626-byte Go fixture. Allocated bytes fall by 26.64 percent. Parse time remains statistically unchanged for that fixture. The rewrite fixture regresses by 19.99 percent. The query-compile fixture regresses by 15.77 percent.
-
The parser now skips four redundant source reconstruction passes for certified isolated C# recovered roots. The receipt requires one one-byte error, no missing nodes, and matching raw top-level spans. The 137 KiB deletion witness still matches the pinned C parser. Median full-parse time falls from 8.24 seconds to 5.01 seconds. Median memory falls from 608 MB to 195 MB. Median allocation count falls from 286,009 to 9,663.
BenchmarkIssue454CSharpRecoveredFullParseusesGOMAXPROCS=1,-benchmem,-benchtime=1x, and-count=5. -
The graph-structured stack shape walk now skips a duplicate cache lookup after a known head miss. The standard full-parse benchmark improves by 5.12 percent across 20 samples. Allocations remain at nine per parse.
-
Graph-structured stack hashing now selects inline or pooled walk storage before it collects nodes. The 235,626-byte Go fixture allocates 24.84 KiB instead of 110.34 KiB. Allocations fall from 511 to 169 per parse. Parse time and maximum resident set size remain unchanged.
-
Outer parser-state re-lex transactions now reuse one 4 KiB scanner-state buffer. The 235,626-byte Go fixture allocates 7.583 MiB instead of 48.073 MiB. Allocations fall from 10,404 to 513 per parse. Two stable pairs improve parse time by 9.52 to 15.46 percent. Maximum resident set size falls from 220,236 KiB to 185,920 KiB.
-
Direct parser-state re-lex probes now use their existing outer transaction. This removes a redundant scanner-state snapshot. The 235,626-byte Go fixture allocates 48.07 MiB instead of 86.69 MiB. Allocations fall from 20,290 to 10,400 per parse. Three stable benchmark pairs improve parse time by 10.90 to 15.26 percent.
-
The compact scheduler now stores its common rollback frontier inline. This removes one allocation from a fresh full parse.
-
The fresh compact runner now reuses its scheduler storage across parses. Full-parse allocations fall from 15 to 14 per operation. Parse time and allocated bytes remain statistically unchanged.
-
The parser now reuses its bound stop-check callback across compact full parses. This removes one allocation and 16 bytes per operation. Full-parse time and maximum resident set size remain unchanged. Incremental parses retain zero allocations.
-
Fresh compact full parses now store the scheduler receipt inside the scheduler allocation. Full-parse allocations fall from 17 to 16 per operation. Parse time, allocated bytes, and maximum resident set size remain unchanged.
-
The compact full-parse receipt now stores its acceptance value in the scheduler receipt allocation. Full-parse allocations fall from 18 to 17 per operation. Parse time and allocated bytes remain statistically unchanged.
-
PR #498 moved the single-header compact dispatch cell onto the stack. This removes one allocation from the common full-parse path. The stable Go benchmark improves full-parse time by 15.57 percent. Allocated bytes fall by 20.25 percent. The allocation count falls by 7.89 percent.
-
Certified graph-structured stack convergence now merges duplicate C# full-parse stacks while it retains their packed alternatives. The 137 KiB witness improves from 1.629 seconds to 98.7 milliseconds. Allocated bytes fall from 135.6 MB to 50.4 MB. The allocation count falls from 7,820 to 1,429 per parse. Incremental parses retain their previous merge policy.
-
The compact full-parse runner now reuses buffers during canonicalization and tree materialization. Warm materialization drops from 136,584 to 272 bytes per operation. Its allocation count drops from 47 to 8. Total warm allocation drops from 20,440 to 5,208 bytes per operation. Total parse time remains statistically unchanged.
-
The compact scheduler stores its one-element seed frontier inside the scheduler allocation. The warm full-parse benchmark drops from 20,352 to 20,328 bytes per operation. Allocations drop from 66 to 65 per operation. Parse time remains statistically unchanged.
Fixed
-
Swift now recovers an
if/elsewhose comparison condition ends with a parenthesised member access in the then-branch (a call argument or a parenthesised negation). The then-block no longer swallows the trailingelseas a call's trailing closure. This fixes #560. -
A Swift optional generic type such as
Range<Int>?now parses cleanly. The token source defers the closer to the DFA only when a reduce action closes an opentype_argumentsproduction. This fixes #556. -
A Swift constrained extension with a multiline
whereclause now parses cleanly. The scanner carries the resolved previous rune across comment handoffs instead of re-reading a raw source byte. This fixes #557. -
Swift nested
if letchains inside methods now parse cleanly. The recovery pass brackets only the right-hand side of the binding. This fixes #558. -
Three or more nested Swift generic type arguments such as
A<B<C<Int>>>now parse cleanly. The split fires only when an unclosed<sits open, so custom operators such as>>>stay intact. This fixes #559. -
A Swift method that contains a
forloop over a range, followed by another method, now parses cleanly. The recovery pass also fires when theforstatement forms with an error inside it. This fixes #561. -
Raw error-cost walks now retain each captured child shape reference. A later mutable node update cannot create a recursive shape cycle. The Rust aggressive corpus completes all 25 bounded parses without a crash.
-
TypeScript and TSX now parse
in,out, andin outvariance annotations on type parameters. The source overlay uses the semantics from upstream pull request 361. This fixes issue #539. -
TypeScript and TSX now separate adjacent generic call signatures at a newline. The grammar uses the dedicated function-signature separator. Generic automatic-semicolon behavior remains unchanged. This fixes issue #540.
-
Compact reductions now merge only with live scheduler headers. Removed historical versions no longer consume the shared boundary link cap. This retires four real-corpus fallbacks without a tree divergence.
-
Shared DFA token election now prefers one composable close angle over a wider close-angle token. Nested TypeScript union arguments now retain the generic-call lineage. This fixes issue #541. Apex nested generic declarations now match the pinned C tree without a result rewrite. This retires the Apex generic local declaration compatibility pass.
-
The full-parse retry ladder now retains a widened candidate until it reads the candidate's runtime receipt. This enables the existing combined stack-and-merge retry for the ZodUnion fixture. This fixes issue #544.
-
Swift optional bindings now keep the statement body separate from a trailing closure. The generator now uses exact advanced LR item precedence. Production, compact, and pinned C tests cover the correction. The regenerated Swift blob retains its exact runtime profile certification. This fixes #542.
-
The DFA lexer now splits adjacent Swift generic closers by parser state. It preserves
>>when the active state accepts the shift operator. The pinned C oracle now matches without divergence. This fixes #543. -
Native
grammar.jsimport now ignores comments in semantic AST children. All 206 pinned grammars show import coverage increasing from 62 to 70. -
AWK recovery now captures the original splice parent before it constructs a replacement concatenation. This prevents self-parent links during recovered expression materialization. A locked 7,392-byte production fixture now verifies bounded completion and a stable tree digest.
-
Compact recursive insertion now proves external token identity from exact scanner checkpoints. Mismatched or missing checkpoints fail closed. Locked Kotlin, OCaml, Perl, and Rust fixtures now reach their next parser gate.
-
Forest result selection now preserves an existing same-symbol container. Its children must exactly match adjacent visible root containers. This removes the inert HTTP section-coalescing compatibility pass.
-
The native reduction path now sets Dart switch-expression body fields. It now sets the target field for nested Elixir calls. This change removes two inert language-local field repairs.
-
Compact graph insertion now persists exact predecessor merges across a bounded 16-level path. Non-exact nested edges and deeper paths still fail closed. Locked C#, Elixir, Perl, and Scala fixtures now reach their next parser gate.
-
The compact admission census now separates runnable no-table-action stops from paused frontiers. The real-corpus matrix labels production error trees. Clean graduation coverage no longer counts recovery fixtures as parser gaps.
-
Compact graph branches now re-lex one exact token span for each parser state when the shared symbol has no action. Each alternative keeps the shared byte range and scanner checkpoint. The medium Scala corpus now reaches compact acceptance. A separate proof for joined reduction paths still gates direct publication.
-
Exact grammar profiles can now flatten certified same-span unary wrappers during reduction materialization. The F# profile removes its declaration-name compatibility walk. Expression and dotted identifiers retain their wrappers. Compact and forest routes retain fail-closed behavior.
-
Native C-style recovery now owns Angular, BibTeX, Chatito, and Electronic Data Sheet materialization for every registered recovery witness. This removes four inert result-compatibility dispatcher arms. Compact and forest routes retain fail-closed behavior.
-
Native parser results now retain the expected Hurl and INI root types. This removes both expected-root fallback compatibility arms as one class. Compact and forest routes retain fail-closed behavior.
-
Native recovery now owns Forth and Luau recovery-action materialization. Forth keeps C-equivalent missing terminators and empty-definition errors. Luau keeps recovered
endtokens as identifiers. This removes both result-compatibility dispatcher arms. -
Parser recovery now owns skipped error materialization for Robot variables and Scheme quote-family forms. This removes both compatibility dispatcher arms as one defect class. Compact and forest routes retain fail-closed behavior.
-
C-style recovery now marks an absorbed
ERRORtoken as named. Recovered INI trees now match the pinned C parser at this node boundary. -
The tracked dispatcher census now includes the locked JavaScript convergence fixture. The receipt exposes seven active compatibility rewrites on a direct route.
-
Native Crystal scanner lookahead now skips whitespace after hash and named-tuple openers. Exact token boundaries retire the Crystal compatibility dispatcher arm.
-
PRs #497 and #500 add locked GraphQL and Svelte direct-route fixtures. Each fixture records its source commit and SHA-256 digest. Dedicated C-oracle tests require exact trees and zero fallback.
-
PR #499 initializes the native Typst scanner indentation stack on creation. Native scanner semantics remove the nested-list comma artifact. This retires the remaining Typst compatibility dispatcher arm.
-
Accepted-error C# full parses now retry with a certified merge width after cap-one convergence. Exact grammar identity gates the policy. Explicit environment settings keep precedence. This preserves recovered declarations while clean full parses remain on the faster path.
-
Native ReScript materialization now owns value identifier path aliases. This change adds two small corpus fixtures. It removes the ReScript compatibility arm.
-
Native Linker Script recovery now owns named error nodes and root spans. This change adds clean and recovered corpus fixtures. It removes the Linker Script compatibility arm.
-
The parser covers every byte in each recovered EBNF source. This change removes the EBNF compatibility arm.
-
Native visible-wrapper election now owns D storage classes. This retires the matching result-normalization subpass. Native reduction also owns D variable-type qualifiers. Native call targets now match C for qualified, template, and simple callees. This retires the D dispatcher arm.
-
The Cooklang smoke fixture now uses a valid ingredient instruction. The previous period required production recovery and was omitted from the resulting tree. The valid fixture routes directly while the recovered form still falls back. The smoke scorecard reports 200 direct routes and one fallback.
-
Compact admission now treats zero-width extras as progress when their token end advances the parser boundary. COBOL fixed-format padding now routes directly without weakening the same-byte no-progress guard. The smoke scorecard reports 199 direct routes and two fallbacks.
-
Compact admission now supports bounded no-lookahead reductions. One runnable head can reduce a synthetic EOF and re-elect at the same byte. Transparent gotos mark the reduced node as an extra. A root reduction requires authenticated EOF on the next election. Doxygen, JSDoc, and VHDL now route directly. The smoke scorecard reports 198 direct routes and three fallbacks.
-
Compact admission now supports two certified acceptance-frontier shapes. HTTP and Robot can drop EOF siblings with no actions. Meson can select the sole primary accepted derivation. Exact blob profiles and field-aware C-oracle receipts guard these choices. The current smoke scorecard reports 195 direct routes and no divergence.
-
Compact admission now permits certified converged-path reduction split drops for the exact Bash, Erlang, Haskell, and JavaScript artifacts. Field-aware C-oracle receipts cover each selected compact tree. Three real-corpus files and the Haskell smoke fixture now route directly.
-
Compact reduction outputs now carry their multi-pop fact directly. This avoids two full work snapshots on every reduction. The stable full-parse control improves by 8 percent against
main. Full-parse allocation falls by 12 percent with no new allocations. -
The dispatcher census now records each live D and Objective-C subpass. Exact fingerprints retain spans, points, fields, flags, and parser states. The census does not materialize compact final-child references.
-
The parser now folds raw descendant content into certified materializing-shape hashes. This prevents shallow GSS merges from discarding Objective-C method types. The parser now owns those identifiers before result compatibility. This retires a fifth Objective-C normalization subpass.
-
Generic result selection preserves both valid Objective-C
sizeofbranches. It selects the C-equivalent expression branch for an unknown type name. This retires the final Objective-C subpass and its dispatcher arm. -
DFA keyword promotion now owns Arduino primitive types before compatibility. Native materialization owns Objective-C protocol type identifiers. This retires Arduino's dispatcher arm and the matching Objective-C subpass.
-
Erlang macro replacement election now stays in the parser. It distinguishes function clauses from case and receive clauses. Reduction already emits exact top-level form spans. These owners retire the Erlang result-compatibility arm.
-
Compact parse-state replay now visits each derivation node once. Its depth-first worklist retains only the active derivation path. The stable full-parse benchmark improves without new allocations or incremental regressions.
-
Clean roots now keep hidden, childless whitespace extras as span coverage. They do not publish those extras as children. Visible comments and error-root evidence remain unchanged. Final-child filtering now preserves fields without materializing lazy ranges.
-
The shared root-extra classifier now drops zero-width scanner tokens from child lists. The repetition-skip fold also stops the historical Typst comma artifact. These producer rules retire two returned-tree walks. Typst keeps its dispatcher arm for other repairs.
-
Root spans now exclude unowned leading token padding through one shared materialization rule. This removes seven language-local repairs. Compact admission now accepts the same first-token start as the C oracle. Squirrel's result-compatibility dispatcher arm is retired.
-
Rust dot ranges now parse without a result repair. The exact collapsed-child policy retains each bare
..token. The merged-left-side conflict rule selects chained dot-range shifts. -
The no-live-action re-lex no longer checks the grammar's name. This recovery step re-lexes the lookahead when no live stack has any parse action for it, and it was gated to JavaScript. The condition it guards is grammar independent:
noLiveStackCanAcceptLookaheadalready proves that no live stack can consume the token, so re-lexing cannot take a token another version was going to use, whatever the grammar. Every grammar now gets the same recovery step. This retires one per-language gate from the parser core. -
Large GLR parses allocate far less. Two hot-path buffers grew without amortization or reuse:
- The three merge-scratch helpers (
ensureMergeResultCap,ensureMergeSlotCapandensureMergeLargeSlotCap) allocated exactly the requested length. Every merge pass sizes them from the live-stack count, so a parse whose stack count climbs reallocated the whole buffer on each increment, which makes total allocation grow with the square of the peak stack count. One element of the large-slot buffer holds two 256-entry arrays, so this dominated wide parses. They now double. gssNodeCanReachbuilt a new fallback map each time a link graph outgrew its 64-entry local array. On grammars that reach that size routinely, the map became the largest single allocation in the parse. The fallback map is now pooled and reused.gssNodeHashgrew a fresh walk buffer on the heap whenever an unhashed chain was longer than its 32-entry inline array. Ordinary parses stop at the first already-hashed node, so this only bites where an error path rebuilds deep chains: a PHP edit that introduces a transient error spent 124 MB there. The walk buffer is now pooled.
Measured on a C# corpus of repeated method declarations, with the grammar loaded before measuring: allocation per source byte falls from 119,036 to 17,207, total allocation for an 8.7 KB input falls from 989 MB to 143 MB, and the parse runs in 200 ms instead of 335 ms. On a 137 KiB input, allocation falls from 5462 MB to 1111 MB and the parse takes 4.0 s instead of 6.5 s. Java, C++, Go and JavaScript allocate exactly as before.
- The three merge-scratch helpers (
-
A GLR stack that needs a different tokenization of the same bytes now gets one. tree-sitter C lexes once per parse version, so two versions in different states can receive different symbols for the same characters. This engine lexes one token for every stack, which is cheaper and correct while all live stacks accept that token. Where one stack's state required a different symbol, that stack found no parse action, paused, and the condense step dropped it because an unpaused rival was still alive. The rival then reached a dead end and the whole file became one
ERRORnode. Scala shows the failure most clearly, because+,-,!and~are its only prefix operators: inif (a) c + 2the correct derivation needs+as the genericoperator_identifier, while the rival needs the dedicated+token that exists soprefix_expressioncan spell unary plus.while (a) c + 2failed the same way, andif (a) c * 2always worked because*is not a prefix operator. The parser now re-lexes at the stack's own byte offset with the stack's own lex mode before pausing it. It adopts the result only when the new token covers the same byte span, which keeps every version at the same offset, and only when the stack's state has a real action for the new symbol. The re-lex reads the internal lexer only, so no external scanner state changes. Clean parses are unaffected, because the probe runs only where a stack would otherwise pause. Any grammar whose characters lex differently by state gains the same protection. -
Parse results no longer depend on garbage-collection timing. The soft per-parse memory budget stopped a parse when
runtime.MemStats.HeapAllocorruntime.MemStats.Sysgrew past the budget. Both values cover the whole process, and the garbage collector paces both, so the stopping point was not a function of the input. The same bytes returned a different tree on each run. Five parses of one 137 KiB C# file returned five different trees, of 1, 11048, 19178, 22928 and 31928 nodes. Each tree reportedHasError() == falseover only part of the input. The budget arms at 64 KiB, so every language was exposed above that size. Only the absolute hard ceiling (GOT_PARSE_MEMORY_HARD_CEILING_MB) now stops a parse from a runtime memory reading, because that ceiling guards against running out of memory and is not a shaping decision. The arena budget, the scratch budget, and the node and stack limits continue to bound memory. Those layers measure what the parse itself allocated, so they stop the same input at the same place every time. A downstream user reported this behavior in issue #454.
Changed
- v0.48.0 adds fields to
FullParseAcceptedErrorRetryProfile,ParseRuntime, andDiagnosticParserCoreGenericWork. Change unkeyed literals to keyed literals before you upgrade.
Removed
-
The Bash command-name concatenation repair. Native reduction now constructs the complete command name before result compatibility. The historical producer, all result routes, and the isolated C oracle match. The 25-case Bash corpus matches baseline
83548f55exactly. Three unrelated Bash subpasses remain live. -
The D template-call type result repair. Generic result election now preserves a visible named unary wrapper over its direct-child alternative. Production, forest, incremental, and isolated C-oracle receipts match. Four unrelated D subpasses remain live.
-
Two Objective-C result repairs. Exact stack-node equivalence preserves deep alternatives for generic alias-target selection. Native selection now owns
@encodeidentifiers and function-pointer expression shapes. Production, incremental, and field-aware C-oracle receipts match. Native selection also owns single and concatenated@strings. Raw-shape equivalence now preserves compound struct type specifiers. Two unrelated Objective-C subpasses remain live. -
The D module-bound result repair. Native reduction already excludes leading comments and trailing trivia from each
module_defspan. Production, compact, forest, incremental, and C-oracle routes match. Incremental parsing reuses the old tree. The D dispatcher remains live for unrelated shape repairs. -
The HCL root normalization pass. Shared root finalization now removes hidden whitespace extras at every root position. Native reduction already produces each exact HCL body span. Production, compact, forest, incremental, and locked C receipts match. The three-file census found no mismatch across 114 body nodes. This removes the HCL result-compatibility dispatcher arm.
-
The Haskell section-span result repair. Native reduction and root finalization already produce the exact
importsanddeclarationsranges. The real-corpus census found no remaining rewrite. Production, compact, incremental, and locked C receipts match. The forest route retains its existing section reduction-cap limit. This removes the remaining Haskell dispatcher arm. -
The source-driven collapsed-token repair family for HCL, CPON, C#, and PowerShell. Reduction now preserves each required anonymous token child. The same-name collapse keeps CPON null nodes childless. This removes the CPON dispatcher arm. The other three arms remain live for unrelated repairs. Compatibility-free, production, compact, forest, incremental, and isolated C-oracle receipts return the same trees.
-
The CUE, Git Commit, and R alias-map result repairs. Their pinned blobs now carry the nonterminal alias metadata from each C parser table. Materialization keeps the required named child under each collapsed wrapper. Production, compact, forest, incremental, and locked C receipts match. CUE also proves nonzero old-tree reuse. Git Commit and R record their external scanner reuse limit.
-
The trailing root and child span compatibility family. Materialization now owns the exact spans for Caddy, Comment, Fortran, Just, Nginx, Nim, Pascal, Pug, and RST. The compact scheduler admits progressing zero-width external extras. Forest publication omits zero-width synchronization extras as children while it retains their source-range ownership. Native producer, production, compact, forest, incremental, reuse, and isolated C-oracle receipts support the removal of four dispatcher arms.
-
The Lua, Make, and Zig field-projection passes. Reduction now projects inherited and direct fields through hidden productions. The Zig grammar metadata emits initializer lists without
field_constant. Compatibility-free, production, compact, forest, incremental, and locked C receipts return the same fields. Make and Zig preserve old-tree reuse. Lua records its external scanner reuse limitation. -
The Haskell and Erlang root field repairs. Reduction now retains each inherited field conflict and projects it by an exact named-symbol match. Root acceptance preserves producer field metadata when it absorbs trivia. Compatibility-free, production, incremental, and isolated C-oracle receipts preserve the expected root fields.
-
The Scala returned-tree span repair subfamily. A language-neutral in-place rewrite refresh now preserves a valid producer-owned span and can widen it. This change deletes the Scala function-end and case-clause helpers. It also removes the second-pass root-end call and its duplicate case-clause block. Production, compact, forest, changed incremental, fresh, and locked C routes return the exact ranges and points. Scala incremental reuse remains unsupported and reports zero reuse.
-
The duplicate Scala returned-tree repair calls. Recovery, field, and annotation repair now runs only in the canonical compatibility pass. Mandatory fixtures and the authenticated corpus report zero mutations when the deleted calls run again.
-
The shared returned-tree fixpoint. The last Scala arm became inert after checkpoints A and B. The publication paths no longer call a repeated post-finalization normalizer.
-
The HTML returned-tree range fixup. Materialization now extends recovered custom elements through each structural
_implicit_end_tagchild. Production, compact, forest, and incremental routes return the exact absolute ranges. The incremental route also proves nonzero old-tree reuse. The locked C reference parser returns the same recovered ranges and points. -
The generic terminal-leaf tree mutation. Reduction and alias materialization now own the terminal shape. Production, compact, forest, incremental, scanner-aware corpus, and locked Go C-oracle receipts find no retired shape. The exact retry error summary and stop polling remain as a read-only full-tree walk.
-
Three dead per-language result-normalization dispatcher arms (R2 of
docs/root-normalization-retirement.md). The three are OCaml's collapsed named-leaf restoration, Ruby's top-level module bound shrink, and HTML's ERROR-root nested-custom-tag reconstruction (normalizeHTMLRecoveredNestedCustomTags). At the R2 checkpoint, HTML's separate range function stayed live. The R1 item above now removes that function independently. A real-corpus census measured zero rewrites for all three dispatcher arms. Native-parse tests confirm that the reduce engine already produces the corrected shape without them.A fourth candidate, Elixir, stayed live. Its census also measured zero rewrites over the real corpus. A native-parse regression test found the cause: the corpus sample lacked the triggering construct. Two consecutive top-level comments — a common file-header shape — still lose their hidden
_newline_before_commentsibling without the normalizer. The ownership registry keeps all four entries as historical receipts.
Changed
-
JavaScript program-end finalization now has one authoritative compatibility owner. The redundant returned-tree second pass is retired; production, compact final-child-ref, forest, and incremental publication continue to use the canonical JavaScript compatibility pipeline before the tree is exposed.
-
Clean hidden whitespace-only root tails are now owned by root finalization, retiring a generic compatibility pass while preserving error-root recovery extras and lazy compact child references.
Performance
-
Same-length single-byte replacements now mark the affected path without recomputing unchanged spans. Other edits and compact child references keep the general editor. The pinned incremental benchmark improves 2.10 percent with zero allocations.
-
Fresh parse finalization now computes the retry error summary while it wires parent links. This removes one complete tree traversal. Deferred and incremental paths retain their separate summary walk. Under-flagged errors and stop polling keep their existing behavior. The pinned Go full-parse benchmark improves 0.81 percent.
-
Parser retry policy now snapshots override presence with each parsed value. This removes repeated environment lookups from the incremental hot path. The pinned edited incremental benchmark improves 1.28 percent with zero allocations.
-
- v0.47.2-0.20260801063327-a2cfa76d3f691 Aug 2026pre-release
Nothing published for this version
- v0.47.2-0.20260731195604-1a0eba41931c31 Jul 2026pre-release
Nothing published for this version
- v0.47.2-0.20260731094744-7c47a369bee331 Jul 2026pre-release
Nothing published for this version
- v0.47.2-0.20260730202825-96671b92f2c630 Jul 2026pre-release
Nothing published for this version
- v0.47.2-0.20260730035411-8dedd4f8b78b30 Jul 2026pre-release
Nothing published for this version
- v0.47.2-0.20260729214138-5bfc3f21e6bd29 Jul 2026pre-release
Nothing published for this version
- v0.47.2-0.20260729212038-45c2b3f547d229 Jul 2026pre-release
Nothing published for this version
- v0.47.2-0.20260729194426-c7073363b23f29 Jul 2026pre-release
Nothing published for this version
- v0.47.2-0.20260729190157-1a7214c5276d29 Jul 2026pre-release
Nothing published for this version