NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #1949 most downloaded on PyPI
Pre-compiled tree-sitter grammars for 372 programming languages
Last release today
04 Oct 2026
Ships fairly regularly
a new release about every 1 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
2 years old
81 releases · first in 2024
fix(intel): Swift and TypeScript declarations, configuration DataNode paths by @robertsfeir in #186
Full Changelog: v1.16.0...v1.21.0
baml, from BoundaryML/baml-treesitter, Apache-2.0), bringing the pack to 372
languages (#200). Upstream ships an ABI 15 parser.c; it is regenerated at ABI 14 like the rest of
the pack. Known limitation: the grammar rejects the documented backtick prompt form until
BoundaryML/baml#5012 is fixed upstream.chown (#202). Group- or other-writable directories are
still rejected.DocstringFormat, StructureKind and SymbolKind gain a from_other() factory, and the
unit enums are hashable. WASM classes gain copyForTransfer().locals/tags queries. cuda stays held: its upstream
parser.c is past the regeneration size threshold and would lose its injections and tags queries.DocstringFormat, StructureKind and SymbolKind are sealed types, not
enums. Java uses sealed interfaces with record variants and C# uses abstract records with sealed
record variants, so StructureKind.FUNCTION / StructureKind.Function becomes
new StructureKind.Function(). Other now carries its string payload, and Java's values(),
getValue() and case-insensitive fromValue() are gone, so match with pattern matching or
instanceof/is. The JSON wire format is unchanged. Java ErrorException now extends
TreeSitterLanguagePackRsException.cache_dir, clean_cache, configure, download, download_all,
download_group, init, manifest_groups, manifest_languages, prefetch and process raise the
typed tree_sitter_language_pack.Error subclasses (DownloadError, LanguageNotFoundError, ...)
instead of RuntimeError, matching get_language. except RuntimeError: no longer catches them.
The .pyi stubs also drop the never-present type attribute and TypedDict variants on the kind
types and tighten constructor parameter types.data and bodySpan setters no longer accept null/undefined. Use the
new clearData() and clearBodySpan() instead.type_tag property, an other property and static factories, not as string constants. The Ruby RBS
enum classes become enum_* type aliases. Kotlin Android unit enums now return the wire string
from toString() ("Line", not "LINE").rustler_precompiled ~> 0.10. Ruby builds against magnus 0.9
and the MCP server against rmcp 3.5. libc is held at 0.2.189 because 0.2.190 breaks iOS builds
(rust-lang/libc#5601).download([...]) fetches languages the manifest knows but the cache does not hold. It previously
returned the requested count without downloading anything, leaving downloaded_languages() empty on
a fresh cache (#201).get_language() releases the GIL while it downloads a missing grammar, so a cache miss no
longer freezes every other thread and the asyncio loop for up to the HTTP timeout (#203).One column per month.
fix(intel): Swift and TypeScript declarations, configuration DataNode paths by @robertsfeir in #186
Full Changelog: v1.16.0...v1.20.0
UPPER_SNAKE_CASE. StructureKind.Function becomes
StructureKind.FUNCTION, and likewise across CommentKind, DataNodeKind, DiagnosticSeverity,
DocstringFormat, ExportKind and SymbolKind. The serialized wire values are unchanged, so no
data migration is needed and persisted JSON stays readable; only Java source naming the old
identifiers has to be updated.process() returns typed value types instead of opaque Rust handles.
ProcessResult, StructureItem, DataNode and fifteen other types are now Codable structs with
stored properties rather than typealiases to swift-bridge handles. Field access moves from method
to property syntax (result.structure, not result.structure()), collections are real Swift arrays,
and StructureKind / SymbolKind / DocstringFormat are Swift enums with associated values rather
than strings. Values now cross the bridge as JSON and are decoded in Swift, so process() is
throws on malformed input rather than returning a handle that fails on first access.Codable conformance could not read serde's wire format. Both are resolved in 0.87.1.process(), leaving the structure path unexercised — all seventeen passed against
an Alef 0.87.0 build whose process() threw on any source containing structure. The new test asserts
the decoded StructureKind cases and item names, which is what exercises the element decoder; nim
cannot cover this (its grammar uses none of the node kind names structure_kind_at() matches) so it
goes through mojo, already in the package's statically compiled language set."Function") and payload variants as a single-keyed object
({"Other": "macro"}); the generated conformance previously fell through to Swift's synthesized
Codable, which expects {"function": {}} and matched neither form. (Alef 0.87.1)FormatMetadata as the flattened shape serde
actually emits, with the sibling-field form asserted directly so a regression to the nested form
fails loudly instead of silently degrading to the variant name. (Alef 0.87.1)errors.Is while still reading the detail the native layer produced. (Alef 0.87.1)fix(intel): Swift and TypeScript declarations, configuration DataNode paths by @robertsfeir in #186
Full Changelog: v1.16.0...v1.19.1
DataNode, ProcessResult and StructureItem become Codable structs with typed properties
instead of typealiases to opaque Rust handles -- but does not update the Swift e2e suite it
generates alongside them, which still calls result.language() and result.structure() on
what are now properties. The two halves of its own output do not compile together, and e2e/
is generated with a CI freshness gate, so the mismatch cannot be patched downstream. Reported
upstream; the upgrade lands once the e2e emitter follows the binding emitter.release/swift/<version> branches. publish.yaml moves the
release tag onto the same checksummed commit it pushes the branch to, so each branch named a
commit already reachable through v<version>; a consumer pinned to one can switch to
exact: "<version>" with no other change. release/swift/1.19.0 is retained.agda external scanner. VEC_POP is a bare
len-- and VEC_BACK reads data[len - 1], so the dedent loop could drain the stack, wrap
len to UINT32_MAX and dereference the buffer plus 16 GiB. 1023 or more repeated '
characters -- or NUL bytes -- crash the parser with SIGBUS. The same scanner also seeded its
column-0 sentinel only when deserialize was called with a NULL buffer, while tree-sitter
resets a scanner with a non-NULL inline buffer of length 0, leaving the following scan to read
data[-1]. A survey of all 186 vendored scanners found both defects only in agda.load_from_dir called
Language::from_raw on a downloaded parser with only a null check, so an incompatible grammar
was accepted and failed later at ts_parser_set_language -- or, just outside the compatible
range, parsed wrongly. The ABI version is now compared against the linked runtime's
MIN_COMPATIBLE_LANGUAGE_VERSION..=LANGUAGE_VERSION and rejected with both versions named. The
bounds are read from the runtime, so a runtime upgrade moves them rather than leaving a stale
literal behind.scripts/sync_zig_zon_hashes.py now maintains alef.toml's
[crates.e2e.registry.packages.zig.platform_hashes] alongside test_apps/zig/build.zig.zon.
Nothing had ever populated that table, so it carried five STALE_HASH_REGENERATE placeholders;
Alef responds to a placeholder by omitting the .hash line entirely, which is the only reason
the manifest had to be declared user_owned in the first place. The five real digests are in
place and the gate now fails on drift in either file.SIGABRT where abort_on_error defaults to 1 and prints ERROR: UndefinedBehaviorSanitizer,
neither of which the previous detector matched, so the agda crash above was exactly the shape
it let through; any parse killed by a signal now counts. Its sanitizer environment is scoped to
the ts-pack invocation as well -- as a step-level env: it also reached the shell, python3,
find, grep and tee, each running uninstrumented with the runtime LD_PRELOADed into it
and exitcode=42 in force, so a diagnosis raised in a helper would have been reported as a
scanner finding.Upgraded the tree-sitter runtime to 0.27, moving tree-sitter-language to 0.1.8 in the same step. The two pins are not independent: 0.27 requires tree-
tree-sitter-language to 0.1.8 in the sametree-sitter-language ^0.1.8, and 0.1.8wasm/unsupported/ so they no longer compile against theTREE_SITTER_LANGUAGE_VERSION 15 with a minimum compatible version of 13 -- so every bundledtree-sitter directly must move to 0.27 in the same step. Thetree-sitter crate declares links = "tree-sitter", and Cargo permits only one package perlinks value in a dependency graph, so pairing this release with a 0.26 pin fails to resolveNDEBUG on wasm32. 0.27's WASM libc is a documented subsetassert, so a scanner keeping its assertions emitted an unresolved __assert_fail.env module import rather than a link error, and the package then failed to loadCannot find module 'env'. Native builds keep their assertions.memchr, strcmp, iswalnum and friends) now thatts_language_is_parseable was the first symbol SwiftPM's copy could not supply --wasm32 builds now find their headers under either wasi-sysroot include layout. wasi-libcinclude/wasm32-wasi to include/wasm32-wasip1, and probing only the former meant a'stdlib.h' file not found. Toolchains whose clang supplies its own sysroot wereNode::child_count now returns tree-sitter's u32 widened to the usize this crate has alwaysRefreshed 22 grammar pins to their upstream latest: al , blade , cfml , dotenv , gitcommit , javadoc , kcl , kdl , lean , magik , mlir , nix , ocaml ,
al, blade, cfml, dotenv,gitcommit, javadoc, kcl, kdl, lean, magik, mlir, nix, ocaml,ocaml_interface, perl, proto, sas, sql, swift, sysml and t32. abl is noouter_join node its highlights.scmcuda is now theparser.c grew past the regenerationocaml_interface highlights overlay from upstream's reworked query.queries/highlights.scm for all three of its(shebang), which only the implementation grammarcontains assertions now compare a JSON-serialized value instead ofConvert.ToString, which reported a type name rather than content for tagged unions suchStructureKind. The PHP e2e runner exports the verified extension path so processes it-n load the same build.rmcp to 3.3.tree-sitter-language 0.1.7 pin. tree-sitter 0.27tree-sitter-language 0.1.8 and removed its Package.swift, so SwiftTreeSitterLanguage istree-sitter floor in alef.toml and realignedtest_apps/node's lockfile with its regenerated manifest, which disagreed and brokepnpm install --frozen-lockfile. Realigned the generated Go e2e module pin, which theRetain tree-sitter 0.26 and the current grammar revisions. The attempted 0.27 upgrade is deferred because SwiftTreeSitter 0.25 requires the 0.25 C run
tree-sitter-language to 0.1.7 because 0.1.8 rejects the retained runtime. (#189)dirs 7 and zstd 0.14.extension_ambiguity() API and its serde-gated JSON companion to expose theprocess() returns a ProcessResult object again in Python, so attribute access works. In 1.16.1 the generator emitted options.ProcessResult as class Pr
process() returns a ProcessResult object again in Python, so attribute access works.options.ProcessResult as class ProcessResult(TypedDict, total=False), which is a type annotation and not a runtime class -- process() therefore handeddict, and every documented access (result.language, result.chunks,result.metrics.total_lines) raised AttributeError. Only Python was affected; the TypedDictProcessResult is a @dataclass(frozen=True, slots=True) again, as itresult.language == "python")process tests failed because the emitter wrote String(e.kind).includes("Function") againstkind, which is an object -- String({ type: "Function" }) is "[object Object]", so everyexpected false to be true. Both emitters now read the value the way the binding presents it -- the dataclasse.kind?.["type"] === "Function" in TypeScript -- and both suites are 523@xberg-io/tree-sitter-language-pack skipped both 1.16.0 and 1.16.1 -- npm's latest is still
1.15.8 -- for two different reasons. 1.16.0 never got past Validate version consistency, the
unrun task version:sync recorded in the 1.16.1 notes below. 1.16.1 cleared that gate and then
failed E2E gate — Node on the 18 TypeScript failures above; that job blocks the npm publish, so
the package never reached the registry. The wasm package is gated separately and did ship 1.16.1.
1.16.2 is the first release in which both gates pass.
The test-app pin check could no longer match the two alef-generated installers, so the release gate was permanently blocked rather than merely stale.
test_apps/php'sVERSION="${1:-...}" default into its own PINNED_VERSION= assignment and--fix therefore could not repair them either. The patterns are nowtask version:sync was never run for 1.16.0, leaving the dart, elixir, kotlin_android andThe plain-Java artifact reaches Maven Central again. It was the only registry still stuck at 1.14.3: every 1.15.x publish skipped Publish Maven packag
The plain-Java artifact reaches Maven Central again. It was the only registry still stuck at
1.14.3: every 1.15.x publish skipped Publish Maven package, because the E2E gate — Java it
sits behind failed first on 8 assertions of the shape expected: <null> but was: <[]>.
The cause was a disagreement inside each generated Java record. A component backed by a Rust
Vec carrying #[serde(default, skip_serializing_if = "Vec::is_empty")] was emitted as
@Nullable, and the canonical constructor stored whatever it was handed — null included —
while the Jackson builder defaulted the same component to List.of(). A record built through
the builder, or round-tripped through JSON, therefore never compared equal to the same record
built through the constructor. Regenerating against alef 0.67.2 drops @Nullable on those
components and adds a compact constructor normalizing null to an empty collection, so both
construction paths agree. Affected records: DataNode, DocstringInfo, ImportInfo,
ProcessResult, StructureItem.
The hand-written Java unit tests asserted the old, wrong contract. They required such a
component to arrive as null; the Rust fields behind them are plain Vec, never
Option<Vec>, so empty is the truthful representation. They now assert
assertEquals(List.of(), ...), which fails on null as well as on a non-empty list — the
assertions were tightened, not relaxed to accept either shape. mvn test in packages/java
reports 150 tests, 0 failures, 0 errors.
Prerelease mode for the registry-mode test apps. task test-apps:prerelease:run (and
:verify, :status, :clean) stages a throwaway copy of each test_apps/ app under
.prerelease/ and redirects its dependency resolution at the in-repo package source, so the
suite is runnable between a version bump and the publish — the window in which the pinned
version exists on no registry and alef test-apps run cannot resolve anything. Covers rust,
go, python, ruby, dart, elixir and swift; status names the remaining targets and why each
needs a built artifact rather than a source path.
scripts/check_test_app_pins.py — a gate that fails when a test app is pinned to a release
other than the one being built. It re-derives all 18 pins across every test_apps/ app plus
the alef.toml registry pins from Cargo.toml, deliberately independent of alef, so a change
in what alef sync-versions is willing to write surfaces as a failure instead of as silent
drift. A pattern that matches nothing is an error, not a pass — that was the Dart failure mode.
--fix repins everything; --release X checks against the version being published and also
catches a Cargo.toml that disagrees with the tag.
Wired in three places: task version:sync repins after alef sync-versions, the Check version sync CI step gates every push, and the publish workflow's validate-versions job gates the
release itself against the tag. Also exposed as task test-apps:check-pins / :fix-pins.
scripts/sync_zig_zon_hashes.py — regenerates and verifies the test_apps/zig/build.zig.zon
package hashes from the tarballs their URLs name. A Zig package hash is a content digest, so
it cannot be derived from a version string the way every other test-app pin can; the only
reproducible source is zig fetch <url>, which prints the exact hash Zig will demand. --fix
rewrites them, the default verifies. Assets that 404 are reported as not-yet-published rather
than as failures, which is the normal state between a version bump and its publish, so the
check is safe to run at any point in the release cycle. Exposed as
task test-apps:check-zig-hashes / :fix-zig-hashes and wired into the ci-zig job — the only
job with a Zig toolchain. Because the digest only exists once the release has published its Zig
assets, --fix runs after a publish, never during release prep.
The publish workflow now refreshes the Zig package hashes itself. A new
refresh-zig-package-hashes job runs after publish-zig — the earliest point at which the
tarballs the digests are taken over exist — recomputes every hash and commits
test_apps/zig/build.zig.zon back to main, modelled on the Swift
update-swift-package-manifest job. Catching the drift only made the manual --fix step
forgettable again; this removes the step. The job cannot break a release that would otherwise succeed: nothing depends
on it, it is continue-on-error, and it commits one file or none. It skips when main has
already moved to a version this release did not publish, and reports every outcome — refreshed,
already current, skipped, failed — in the run summary, since continue-on-error would
otherwise let a failure pass unnoticed. --require-published is new on the script for this
caller: after a successful publish a 404 is a missing upload, not the pre-publish window, and
is a failure rather than a skip.
test_apps/zig could not build at all — its five package hashes had been stale since 1.14.3.
alef sync-versions repoints each .url at the release being cut, but a Zig package hash is a
content digest over the fetched tarball, so nothing recomputed it and all five sat at the 1.14.3
values while the URLs advanced to 1.15.7. zig build failed outright:
error: hash mismatch: manifest declares tree_sitter_language_pack-1.14.3-jCz0Y85sQQ...
but the fetched package has tree_sitter_language_pack-1.15.7-jCz0Y86TSQ...
This is not the cosmetic staleness it looks like — the declared hashes are byte-identical to the
real 1.14.3 tarball hashes, so every release from 1.15.0 to 1.15.7 shipped an app that resolves
nothing. Nothing reported it because no workflow builds test_apps/zig: ci-zig builds
packages/zig from the working tree and ci-e2e runs e2e/zig, so the one app that resolves
these URLs was never exercised. All five are regenerated via zig fetch, and ci-zig now
verifies them.
Five registry-mode test apps were validating an already-published release, not the one being
built. test_apps/{elixir,swift_e2e,php} were pinned at 1.15.1 and
test_apps/{dart,kotlin_android} at 1.15.2, against a 1.15.7 tree. Each gate passed — it
proved the old release still installs, which is not what a registry-mode gate is for.
All five are now repinned to the release being built.
The cause is not the matched-nothing regex fixed for Dart below. Every one of these five sync
rules does match its file. alef sync-versions reaches these paths through its catch-all
branch, which refuses to rewrite a file that carries no alef provenance marker on the grounds
that it reads as hand-written; it emits a WARN naming each skipped path and exits 0. So the
rules looked applied and never were. alef.toml's own
[crates.e2e.registry.packages.*] pins are skipped for the same reason.
Compounding it, the Check version sync CI step diffs only packages/ and crates/, so
test_apps/ drift was outside the gate's scope even when sync did rewrite it.
The test_apps/dart/pubspec.yaml version-sync rule required a caret the generated pubspec does
not contain, so it matched nothing. Fixing the rule was necessary but not sufficient: the file
is one of the five skipped by the catch-all branch above, so it stayed at 1.15.2 afterwards.
Regenerated against alef 0.66.0 (from 0.64.0, via 0.65.0). This clears the Swift E2E gate failure that blocked the v1.15.6 release: is_empty on a non-
is_empty on a non-optional array reached through anBool? into XCTAssertTrue, which does not typecheck, so the gate?? true). alef verify --exit-code returns 0 against the regenerated tree.level: front-matter key instead of emittinglevel: null, and every backend now orders IR items before emission — both are deliberateA failed publish can no longer announce itself as a successful release.
announce-discord gated on !contains(needs.*.result, 'failure'), which is satisfied when
every dependency is skipped -- the exact shape of the v1.15.5 run, where all 13 publish jobs
skipped and the release reached zero registries. It now additionally requires
release-finalize (the job that asserts every enabled target actually published) to have
succeeded.
publish-wasm now genuinely waits on publish-node. publish-node was listed in its
needs: but never restated in its if:; because the job opens with always(), the needs:
entry alone gated nothing and WASM could publish to npm after the node publish had failed. The
gate is deliberately negative-form (!= 'failure') so a legitimately skipped publish-node
-- npm already at this version, or node not among the release targets -- does not cascade into
skipping WASM.
68 of the 85 files alef verify reported frozen are now recorded as alef-owned. Every one
of them is a create-once seed, so plain alef adopt refuses them and the report's own remedy is
unreachable without --clobber-create-once-seeds. Only the files that already match generated
output byte-for-byte were adopted, so no content changed: 66 are recorded in
.alef-ownership.toml (formats that cannot carry a marker), and packages/go/go.mod plus
packages/zig/build.zig.zon gained the marker header. The 17 left frozen all differ from
generated output and need a content decision -- notably packages/java/checkstyle.xml, whose
generated form reverts the ${config_loc} + optional="false" SuppressionFilter fix, and
checkstyle-suppressions.xml, whose generated form drops the NativeLib LineLength
suppression. alef verify stays red on those 17.
task docs:snippets:validate:lang runs again. It invoked alef snippets validate, a
subcommand that does not exist (alef snippets has list, check, parse, audit, gaps),
so the target could only ever fail with unrecognized subcommand. It now calls
alef snippets check --strict --cache off --lang {{.LANGUAGES}}, matching
docs:snippets:validate with the per-language filter added. The --snippets path is dropped
because check reads [workspace.docs.snippets], and LEVEL is dropped because check has no
level flag -- each validator picks the deepest level its toolchain supports.
alef verify no longer reports 8,415 stale files. alef all ran at ada0fac and the
1.15.6 version bump landed after it at e9816af; alef.toml is a hashed generation input, so
every generated file was stamped against pre-bump inputs and nothing re-stamped them. The tree
is regenerated after the bump, which rewrites the alef:hash: line in ~8,380 files. The
remaining diffs are real drift the earlier regen never covered: all of test_apps/* (alef all
does not run the test-apps stage -- alef test-apps generate is a separate command), plus the
packages/*/README.md version strings. Regeneration order is now bump-then-generate; running
alef all before alef sync-versions re-opens this.
Validate (Lint & Format) no longer fails on every release commit. The job passed
python-extra-projects: packages/python to the shared validate workflow, which runs
uv pip install -e packages/python -- a maturin build of crates/ts-pack-core-py, and so a run
of crates/ts-pack-core/build.rs. That job never downloads the language-parsers artifact, so
parsers/ is empty and build.rs falls through to the parser-sources-<version>.tar.zst release
asset for the version in Cargo.toml -- which does not exist yet on a release commit, because
the version is always bumped before its release is published. Run 32562369389 (v1.15.6) died on
that 404 at step 9 of 32, so Format check and Lint never ran at all. The input is dropped:
its stated purpose (letting pyrefly resolve tree_sitter) was already satisfied by the root dev
group in pyproject.toml, and ._native resolves from the checked-in _native.pyi stub, so
pyrefly check packages/python reports 0 errors without the wheel being built.
Rust snippet validation no longer downloads parser sources. The rust snippet session now
sets TSLP_OFFLINE=1. Its cargo check builds ts-pack-core through a path dependency and so
runs the same build.rs against the same empty parsers/ tree -- the identical 404 on a release
commit, one step later in the same job. TSLP_LANGUAGES is unset there, so no grammar is
compiled into that check either way; skipping the sources changes nothing it validates.
Quick Start's kotlin-android process and quickstart tabs now import the generated corpus
(generated/kotlin-android/process/config_all_python.md and
generated/kotlin-android/parsing/parsing_python_function.md) instead of two hand-written
stand-ins. Those stand-ins were kept back in 064e3c3 because the generated twins failed with
unresolved reference 'io'; that classpath defect was fixed today (651ddba), so the
stand-ins were deleted. Not re-verified with a session-based alef snippets check run -- watch
the next CI run for kotlin_android snippet failures.
alef.toml now documents next to [workspace.docs.snippets] dirs why docs-site/src/snippets/ cli/*.md stay outside the validated snippet tree: the CLI is not an alef binding target, so nodirs -- nothing recorded that the gap was deliberate.Windows parser binaries build. The UTF8PROC_STATIC fix in 1.14.2 covered the two static-link paths but not the third one, which builds each grammar's
UTF8PROC_STATIC fix in 1.14.2 covered the two
static-link paths but not the third one, which builds each grammar's shared library from a raw
compiler invocation and compiles utf8proc.c straight into it. MSVC therefore still rejected all
183 scanner grammars with error C2491, and the 1.14.2 release produced no windows-x86_64 or
windows-aarch64 parser binaries ([#174]).windows-x86_64 and windows-aarch64 on every push, mirroring
the publish job. Windows previously ran on no CI runner, so the dynamic-link build was first
attempted at release time; the build log is also written to a file and uploaded, because the
failing step exceeded GitHub's per-step log archive cap and left no recoverable diagnostic.poly's whole-project lint
phase — golangci-lint, rubocop, steep, dart-analyze and credo — runs in CI rather than only in the
git hooks.Windows artifacts build again. utf8proc.c is compiled into a static archive, but UTF8PROC_STATIC was never defined, so on MSVC utf8proc.h declared eve
utf8proc.c is compiled into a static archive, but
UTF8PROC_STATIC was never defined, so on MSVC utf8proc.h declared every utf8proc_* symbol
__declspec(dllimport) and the same translation unit then defined it — error C2491. Every
Windows job in the 1.14.1 release (parser binaries, C FFI, CLI, C# native) failed, which is why
no windows-x86_64 entry ever reached the pre-built parser manifest (#174).std::iswspace(c) expanded to a std::ts_pack_iswspace that does not exist, and heavy libstdc++
headers re-declaring those classifiers after the macros were in scope failed to compile on musl.
The redirected names are now visible in namespace std, and norg — whose scanner pulls in
<cwctype>, <locale> and <regex> — is excluded from the shim entirely, so it falls back to
libc consistently. This unblocks the all-grammar Docker image and the Docker publish.env import. Most tree-sitter scanners include
<wchar.h> themselves, and wasi-libc's copy unconditionally redefines iswdigit and friends as
trampoline macros, silently clobbering the shim's redirects and leaving a reference to a symbol
that does not exist on wasm32. It surfaced as Cannot find module 'env' from the wasm-bindgen
glue. The shim now burns that include guard up front on __wasi__.composer.json still pointed at the
emptied packages/php/src, so every PHP test failed with a missing-class error. It now resolves
to crates/ts-pack-core-php.docs-site/pnpm-workspace.yaml, and scoped
poly lint to skip whole-project linters whose toolchains CI does not install.Ruby binding no longer exports generated types into the global Object namespace (issue #173, the Parser collision with the parser gem); generated type
Object namespace (issue #173,
the Parser collision with the parser gem); generated types stay namespaced under
TreeSitterLanguagePack.<wctype.h>, making parse trees (and downstream formatting)
identical across macOS and glibc. Pinned tree-sitter-cli to 0.26.11 in CI.Add 65 tree-sitter grammars, raising coverage from 306 to 371. New languages include Slint, MoonBit, Aiken, Koka, Koto, Unison, Leo, Motoko, Eiffel, P
just
is re-sourced to the canonical casey/tree-sitter-just.kitty.conf), rsHTML (.rs.html), and dotenv (.env) files by name.tracing spans and events always-on across parsing, grammar loading, and
download operations, making observability a first-class product surface. tracing is now a
non-optional dependency (near-zero cost without a subscriber). Levels follow the shared contract:
INFO for coarse state changes (download/init/prefetch/clean, network bundle fetch), DEBUG for
per-call flow (process, get_language, grammar shared-library loads). Span names and field keys
are part of the public API.ts-pack CLI installs a tracing subscriber for every command (previously only the MCP
server did), so RUST_LOG surfaces library diagnostics on stderr for all subcommands; machine-
readable result output on stdout is unchanged. tracing/tracing-subscriber are no longer gated
behind the mcp feature.println!/eprintln!/print!/eprint!/dbg! are denied in production code across the
workspace (clippy print_stdout/print_stderr/dbg_macro); tracing is the sole diagnostic
surface. The ts-pack CLI's command results and prompts remain its stdout/stderr output contract
(opted in crate-wide), while the MCP command's diagnostics continue through tracing.Patch release fixing version strings that 1.13.6 shipped stale in generated bindings.
Patch release fixing version strings that 1.13.6 shipped stale in generated bindings.
nativeModuleVersion (drives the native-asset download URL), Python __version__/docstring, and the C# e2e app package reference now report 1.13.7.packages/go setup moduleVersion realigned so gofmt passes.Fix the GraphQL parser rejecting fields that carry more than one directive (#172) by vendoring a patched grammar in-tree under grammars/graphql/ and t
Fix the GraphQL parser rejecting fields that carry more than one directive (#172) by vendoring a patched grammar in-tree under grammars/graphql/ and teaching the vendoring pipeline to build from a local grammar source. Wire the embedded MCP server for rmcp 3.0 (new ReadResourceResponse/GetPromptResponse MRTR wrappers) and bump base64 to 0.23 in the JNI crate. Move the alef code generator to 0.48.5 and regenerate all language bindings.
Regenerate all language bindings on alef 0.48.4, fixing Java (Maven) publishing by lowering the maven-enforcer version floor and C# (NuGet) publishing
Regenerate all language bindings on alef 0.48.4, fixing Java (Maven) publishing by lowering the maven-enforcer version floor and C# (NuGet) publishing by rendering a runtime.json template at pack time. Update grammar revisions (gnuplot, javadoc, scala, tmux) to latest upstream.
runtime.json.template so runtime.json can be
rendered before the NuGet pack).runtime.json from its template before packing the NuGet package so C# runtime
metadata is published correctly.Regenerate all language bindings on alef 0.48.2.
ruby: platform-specific native gems now package one native extension per supported Ruby ABI and load the extension matching the current runtime. This
Parser#parse self-conversion TypeErrors from loading
an extension compiled against a different Ruby ABI.
Fixes #171.Cargo.lock,
pnpm-lock.yaml, uv.lock, Ruby Gemfile.lock).ruby: the Magnus binding now exposes the Parser instance methods (set_language/parse/parse_bytes/reset). alef previously skipped &mut self methods on
Parser instance methods
(set_language/parse/parse_bytes/reset). alef previously skipped &mut self methods on
opaque types, so Parser was generated with no callable methods; parse_bytes also now accepts a
Ruby String. This fixes the root cause behind
#168 at the generator level
(alef 0.38.4), superseding the hand-edit stopgap in
#169.CodingKeys honoring serde renames, Optional<Vec<struct>>
accessors no longer double-encoding, and the Elixir e2e streaming entry-point suffix).ruby: the published 1.13.0 gem still shipped required_ruby_version = [">= 3.2.0", "< 4.0"] despite #162, because alef regeneration reverted the hand-e
required_ruby_version = [">= 3.2.0", "< 4.0"]
despite #162, because alef
regeneration reverted the hand-edit. Fixed at the generator level (alef 0.38.0 defaults to
">= 3.2.0" with no upper bound and makes the constraint configurable), so the gem now installs
on Ruby 4.x.nil and pre-encoded JSON strings as-is
instead of double-encoding, matching the keyword-arg path (alef 0.38.0).ruby: expose the parser API methods (Parser#parse, tree/node accessors) on the Ruby binding so consumers can drive parsing directly instead of only fe
Parser#parse, tree/node accessors) on the Ruby binding
so consumers can drive parsing directly instead of only fetching languages.
Fixes #166.< 4.0 RubyGems constraint; the native
extension builds and the generated Ruby e2e suite passes on Ruby 4.0.6.
Fixes #162.freezed to the stable ^3.2.5 constraint instead of a ^4.0.0-dev prerelease.
The prerelease was rejected by flutter_rust_bridge_codegen's dependency validator, which broke
the Dart binding build.linux-x64-musl and linux-arm64-musl platform sub-packages.
CI now cross-compiles both musl targets (via zig / cargo-zigbuild), and the main package's
optionalDependencies reference the real version instead of the 0.0.1 placeholder, so the
native binaries resolve on Alpine/musl systems.tree_sitter property API (root_node/type).rmcp to 2.2, go-tree-sitter to v0.25.0, @napi-rs/cli to ^3.7.3,
@types/node to ^26.0.0, and vitest to ^4.1.10.@xberg-io/docs-theme,
and rewrite the landing page and README to a value-first voice.build: build.rs no longer forces spurious rebuilds when the crate is consumed from a registry (crates.io) rather than a git checkout. It skips cargo:r
build.rs no longer forces spurious rebuilds when the crate is consumed from a registry (crates.io) rather than a git checkout. It skips cargo:rerun-if-changed for the query-overlays directory when it does not exist, skips rerun triggers for parser directories under OUT_DIR, and falls back to CARGO_MANIFEST_DIR when the overlay-directory search walks past the filesystem root. Fixes #159.crossbeam-epoch to 0.9.20 (RUSTSEC-2026-0204).See CHANGELOG.md for details.
build.rs no longer forces spurious rebuilds when the crate is consumed from a registry
(crates.io) rather than a git checkout. It skips cargo:rerun-if-changed for the query-overlays
directory when it does not exist, skips rerun triggers for parser directories that live under
OUT_DIR, and falls back to CARGO_MANIFEST_DIR when the overlay-directory search walks past the
filesystem root. Previously these emitted rerun triggers on nonexistent or generated paths, so cargo
rebuilt the crate on every invocation. Fixes
#159.wasm: skip the tmux grammar on wasm32. Its generated parser.c is ~30MB — under TSLP_WASM_MAX_PARSER_BYTES (40MB) so the size gate doesn't catch it, bu
tmux grammar on wasm32. Its generated parser.c is ~30MB — under TSLP_WASM_MAX_PARSER_BYTES (40MB) so the size gate doesn't catch it, but it still overruns the clang wasm backend during codegen. Added to DEFAULT_WASM_SKIP_GRAMMARS so the wasm build is clean; it degrades gracefully (absent from STATIC_LANGUAGES).getrandom js feature for wasm32-unknown-unknown. Build-dependencies (ureq) transitively depend on getrandom, which requires the js feature on that target; without it wasm32 builds failed with "the wasm*-unknown-unknown targets are not supported by default".build.rs now reports failed_languages.txt write errors separately and always surfaces the underlying grammar compilation error even when the write fails.Full Changelog: https://github.com/xberg-io/tree-sitter-language-pack/compare/v1.12.1...v1.12.2
fix(intel): include Ruby namespaces in structure by @pboling in https://github.com/xberg-io/tree-sitter-language-pack/pull/129
ecosystem page cross-linking the sibling docs sites by @v-tan in https://github.com/xberg-io/tree-sitter-language-pack/pull/135Full Changelog: https://github.com/xberg-io/tree-sitter-language-pack/compare/v0.17.10...v1.12.1
prefetch: prefetch(&[&str]) downloads (if needed) and loads every requested grammar in one pass, so a parallel workload only parses. Probes real on-di
prefetch(&[&str]) downloads (if needed) and loads every requested grammar in one pass, so a parallel workload only parses. Probes real on-disk loadability instead of has_language, fixing a short-circuit where known-but-not-downloaded grammars were skipped.get_query(language, QueryKind) -> Result<Option<Arc<Query>>> compiles a bundled .scm query once and caches the Arc<Query> process-wide. Rust-only; bindings keep the raw query-string accessors.get_indents_query / get_folds_query expose the bundled indents.scm / folds.scm; QueryKind gains Indents and Folds.get_language now takes a lock-free fast path for static and already-loaded dynamic grammars; the global load mutex guards only the not-yet-loaded dynamic path.prefetch(&[&str]) downloads (if needed) and loads every requested grammar in one
pass, so a subsequent parallel workload only parses. It probes real on-disk loadability rather
than has_language, fixing a short-circuit where known-but-not-downloaded grammars were skipped.get_query(language, QueryKind) -> Result<Option<Arc<Query>>> compiles a
bundled .scm query once and caches the Arc<Query> process-wide (negative results cached too).
Rust-only — bindings continue to expose the raw query-string accessors.get_indents_query / get_folds_query now expose the bundled
indents.scm / folds.scm (harvested from the grammar bundle alongside the existing kinds);
QueryKind gains Indents and Folds.get_language now takes a lock-free fast path for statically-compiled and
already-loaded dynamic grammars; the global load mutex guards only the not-yet-loaded dynamic
library path. Removes per-call mutex contention on the hot parse path.python: get_language() and other download failures now surface the public exception type in tracebacks — tree_sitter_language_pack.DownloadError inste
get_language() and other download failures now surface the public exception type in tracebacks — tree_sitter_language_pack.DownloadError instead of _native.DownloadError (#147).ByteArraySerializer.java so the generated ObjectMapper compiles for every package.Regenerated against alef 0.30.1.
get_language() (and other download failures) now surface the public exception type
in tracebacks — tree_sitter_language_pack.DownloadError instead of _native.DownloadError
(#147). Regenerated against alef 0.30.1.ByteArraySerializer.java so the generated ObjectMapper compiles for every
package (alef 0.30.1).tree-sitter-language-pack 1.11.0 — stable release. Tree-sitter grammars + code-intelligence primitives. Promotes 1.11.0-rc.2.
tree-sitter-language-pack 1.11.0 — stable release. Tree-sitter grammars + code-intelligence primitives. Promotes 1.11.0-rc.2.
Stable release promoting 1.11.0-rc.2 (fully published). Version synced across all manifests.
tree-sitter generate with default concurrency of 3, causing each of the 306 grammars
to generate in parallel (~1 GB RSS per instance), exhausting the 7 GB RAM limit on GitHub-hosted
runners and triggering SIGTERM (exit 143) at ~13 minutes. Reduced defaults to 8 clone concurrency
(from 16) and 2 generate concurrency (from 3) to stay within resource budgets. Fixes CI, CLI,
Docker, E2E, Swift, Rust, and Validate workflow failures.Release candidate 1.11.0-rc.2. Regenerated with alef 0.29.3; corrected Homebrew tap path and CLI install docs.
Release candidate 1.11.0-rc.2. Regenerated with alef 0.29.3; corrected Homebrew tap path and CLI install docs.
fix(intel): include Ruby namespaces in structure by @pboling in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/129
ecosystem page cross-linking the sibling docs sites by @v-tan in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/135Full Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v0.17.10...v1.10.9
Tree.walk() (and other handle-returning calls) no longer crash the JVM.
Opaque handle types (Tree, Node, TreeCursor) crossed the JNI boundary as String/JSON
while the Rust shim returned a raw jlong, so the JVM dereferenced a primitive as an object
reference and faulted with EXCEPTION_ACCESS_VIOLATION. Regenerated with alef 0.27.1, the
kotlin-android bridge now returns primitive Long handles (required and optional, via a 0L
sentinel) and constructs the wrapper directly. Fixes #146.get_language("unknown") raised
_native.DownloadError, a different class object than the DownloadError exported from the
package, so except DownloadError: never caught it. Regenerated with alef 0.27.1, the native
variants derive from the native base Error and the package re-exports the native classes
(with matching type stubs), so except DownloadError:/except Error: work. Fixes #147.fix(intel): include Ruby namespaces in structure by @pboling in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/129
ecosystem page cross-linking the sibling docs sites by @v-tan in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/135Full Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v0.17.10...v1.10.8
wasm32 builds no longer OOM on pathologically large grammars. Compiling the bundled grammars
to wasm32 previously included every parser.c, but a few are huge generated sources (e.g.
abl at ~130 MB) that need 18-25 GB＋ of clang RAM at any optimization level — a single one
OOMs standard ≤16 GB CI runners (serialization via CARGO_BUILD_JOBS=1 cannot help when one file
alone exceeds the budget). build.rs now skips any grammar whose parser.c exceeds a size limit
on wasm32 (default 40 MB, configurable via TSLP_WASM_MAX_PARSER_BYTES; 0 disables the gate),
emitting a cargo:warning per skipped grammar plus a summary. Skipped grammars are absent from
STATIC_LANGUAGES (no dangling FFI symbol) and degrade gracefully at runtime. The 40 MB default
keeps every common language (including the ~40 MB sql grammar) and excludes only the handful of
unbuildable outliers (abl, systemverilog, razor, fsharp, verilog, gnuplot, latex).
(crates/ts-pack-core/build.rs)
Swift publish now creates the release/swift/<version> branch carrying the substituted
XCFramework checksum. The alef-generated Swift e2e/test-app pins
.package(url: …, branch: "release/swift/<version>") (the non-destructive layout shared with the
other polyglot repos), but the publish workflow only force-moved the v<version> tag and never
created that branch — so SwiftPM could not resolve the package and the Swift test-app failed with
an empty TreeSitterLanguagePack target. The checksum commit is now also pushed to
refs/heads/release/swift/<version>. (.github/workflows/publish.yaml)
fix(intel): include Ruby namespaces in structure by @pboling in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/129
ecosystem page cross-linking the sibling docs sites by @v-tan in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/135Full Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v0.17.10...v1.10.7
Full Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v1.10.5...v1.10.6
Full Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v1.10.5...v1.10.6
Regenerated all bindings with alef 0.26.0. Picks up alef's sync-versions byte-stability fix: sync-versions no longer rewrites externally-formatted sca
sync-versions no longer rewrites externally-formatted scaffold/manifest files and preserves external SwiftPM dependency pins instead of clobbering them with the workspace version. This clears the CI version-sync freshness gate.sync-versions no longer rewrites externally-formatted scaffold/manifest files (this repo formats
via external tools with [workspace.format] enabled = false), and it preserves external SwiftPM
dependency pins instead of clobbering them with the workspace version. This clears the CI
version-sync freshness gate.Generated binding doc comments no longer emit Rust intra-doc link syntax. alef copied core rustdoc comments verbatim into the per-language binding cra
cargo doc in the binding crates with rustdoc::broken-intra-doc-links. The references are now de-linked to plain code spans during emission, preserving genuine URL/anchor Markdown links. Picked up from the alef 0.25.60 regen.fix(intel): include Ruby namespaces in structure by @pboling in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/129
ecosystem page cross-linking the sibling docs sites by @v-tan in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/135Full Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v0.17.10...v1.10.1
EXCEPTION_ACCESS_VIOLATION) when traversing a parsed tree via opaque
handles (#146). Parser.parse, Tree.walk, Tree.rootNode, Node.parent/child, and
TreeCursor.node freed the returned native handle in a finally block immediately after wrapping
it, so the returned Tree/Node/TreeCursor referenced already-freed memory and the next native
call dereferenced it and crashed. The wrapper now owns the handle and frees it once on close().
Fixed in the alef Java backend and picked up by the 0.25.55 regen; value/DTO returns
(byteRange/startPosition/process) still correctly free the FFI temporary after reading it.packages/kotlin-android, add --kotlinlang-style to ktfmt, switch alef.toml kotlin format/check from gradle-ktlintFormat to ktfmt so alef and prek agree, and exclude the vendored Gradle wrapper from shellcheck. detekt remains for static analysis. (.pre-commit-config.yaml, alef.toml)Language passthrough across the C-ABI binding family (#143). get_language
now returns each ecosystem's native tree-sitter Language instead of an opaque alef handle, so
the result drops straight into the host runtime's parser: Go (*tree_sitter.Language via
go-tree-sitter), Zig (?*const tree_sitter.Language via zig-tree-sitter), Java
(jtreesitter.Language), C# (TreeSitter.Language), Kotlin Android (ktreesitter.Language),
and Swift (SwiftTreeSitter.Language) — joining the existing Python and Node passthrough. Each
binding gained a dependency on its host tree-sitter runtime, injected into the generated manifest.
Configured via [crates.*.capsule_types.Language] in alef.toml; regenerated against alef 0.25.55.tree-sitter dependency so the capsule shim can name tree_sitter::ffi::TSLanguage (the pointee
it casts value.into_raw() to), and the zig build.zig.zon carries the resolved zig-tree-sitter
content hash.fix(intel): include Ruby namespaces in structure by @pboling in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/129
ecosystem page cross-linking the sibling docs sites by @v-tan in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/135Full Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v0.17.10...v1.9.1
getLanguage(name:) function in the TreeSitterLanguagePack
module. An alef 0.25.38 codegen regression added opaque types to the Swift forwarder
exclusion set, dropping get_language (the only free function returning the opaque
Language type) from the generated public API in v1.9.0. Regenerated against alef 0.25.43.fix(intel): include Ruby namespaces in structure by @pboling in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/129
ecosystem page cross-linking the sibling docs sites by @v-tan in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/135Full Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v0.17.10...v1.9.0
alef pin 0.25.28 → 0.25.38 and regenerated all bindings. Picks up alef 0.25.29–0.25.38: enum associated (static factory) methods surfaced across backends, the swift opaque no-op shim so $_free is synthesised for handle types with no visible methods (e.g. Language), swift streaming-owner already_declared re-declaration, java marshal_optional_bytes template registration, and java @Nullable type-use placement on qualified types.task upgrade across every language workspace; lock files regenerated and committed.packages/kotlin-android/.gradle/ cache and the .basemind/ index (untracking the accidentally-committed Gradle cache files), and exclude the deterministic .ai-rulez/.generated-manifest.json from the oxfmt pre-commit hook so it no longer fights the ai-rulez-generate hook.UnsupportedOperationException stubs for Self-returning DTO/enum methods. There is no JNI/FFM symbol for DTO methods yet, so the throwing stubs compiled but misled callers and broke any path that reached them. The Java backend now skips these methods until marshaling lands.true default for boxed @Nullable Boolean #[serde(default)] record fields. A non-optional #[serde(default)] bool = true field is boxed to @Nullable Boolean, so JSON that omitted it deserialised to null and the accessor returned null instead of true.E2E fixture coverage for: language alias resolution (shell→bash) via has_language / get_language / get_parser (3 fixtures); download edge cases — empt
shell→bash) via has_language / get_language / get_parser (3 fixtures); download edge cases — empty list, multiple-language, and unknown-language error path (3 fixtures); error-handling for 120KB sources and get_language("") (2 fixtures); and TypeScript function parsing (1 fixture). Brings fixture count from 403 to 412, covering 100% of the public download, get_*, and has_language surface across all 10 language bindings.Node: getLanguage(name) now returns a real tree-sitter Language that new Parser().setLanguage(lang) accepts at runtime. The previous capsule shim used napi::bindgen_prelude::External::new (rejected by node-tree-sitter's UnwrapLanguage), wrote the External to __parser, and did not type-tag the value. Adopts alef v0.15.49 where the napi capsule codegen emits raw napi_create_external + napi_type_tag_object and reads property_name/type_tag from [crates.node.capsule_types].
Python: PackConfig and ProcessConfig type hints now resolve to the .options dataclasses, fixing mypy --strict errors at every init(...) / process(...) call site (adopts alef #72).
Python: restore SupportedLanguage as Literal[...] of all 306 grammars at tree_sitter_language_pack.SupportedLanguage. The symbol was dropped during the alef 0.15.x codegen migration and re-importing it raised ImportError in 1.8.0 (#121).
Python: get_parser("python").parse(b"...") returns a real tree_sitter.Tree again instead of raising AttributeError. get_parser / get_language now return native tree_sitter.Parser / tree_sitter.Language instances via PyO3 capsule pass-through (alef v0.15.39 wires capsule_types through gen_bindings) (#121).
tree-sitter@0.25.0 (the tree-sitter npm package) ships a binding.cc written against pre-C++20 stdlib (no std::ranges, concept, requires) and fails to compile against Node 24/26's V8 headers. Node 22 is the latest supported runtime until upstream node-tree-sitter updates its cflags_cc or ships prebuilds.packages/java/pom.xml maven-pmd-plugin minimum-tokens bumped from 100 → 250: alef's java codegen emits ~200-token try/catch cleanup blocks on DownloadManager / LanguageRegistry. Refactoring the codegen to share a helper is tracked separately.macOS x86_64 native binaries across all polyglot bindings (Python wheels, npm napi, Ruby gem, Maven JAR, NuGet, C FFI, Go FFI, libts-pack bottle) — re
ts-pack (CLI) and libts-pack (FFI library) via brew install --build-bottle + brew bottle --json, replacing the prior synthetic tarball approach. Eight bottles per release across arm64_sequoia, sequoia, arm64_linux, x86_64_linux. brew install now pours instead of source-buildinglibts-pack Homebrew formula bundling tree-sitter language pack as a C library (headers + dylib/so + static archive)kotlin_package_class_intel.json), Java package declarations (java_package_intel.json), and a process call exercising the typed extractions map (process_with_extractions.json)--target nodejs build to npm so consumers no longer hit the bundler-only import * from "env" failure on require()StructureKind::Module for Kotlin package_header and Java package_declaration so callers can build fully-qualified names for JVM languages (#112)name field → type_identifier → identifier → scoped_identifier) so Kotlin classes and Java/Kotlin packages no longer surface with null names (#111)natives/{rid}/ entries inside the published JAR — actions/download-artifact produces nested artifact paths, and the previous staging loop preserved them, so every platform hit UnsatisfiedLinkError on load. Flatten via find and add presence/jar tf guard steps so the regression cannot ship silently again (#114)extractions as a typed Map<String, ExtractionPattern> / Map<String, PatternResult> across Java, Python, Go, TypeScript, Ruby, PHP, C#, Elixir, FFI, and WASM (was Optional<String> on Java, blocking pattern extractions through the high-level API). Driven by the alef 0.12.4 codegen fix for AHashMap-typed fields (#115){ lines emitted by alef 0.14.33 codegen so generated .cs files compilenative.rb no longer recurses into itself via define_singleton_method — magnus codegen now skips re-export when binding name matches the native module methodindex.js now contains real platform-dispatch logic so require() resolves the correct .darwin-arm64.node/.linux-x64-gnu.node/etc. instead of failing on the un-suffixed bundle name'env' module imports that broke require() from Node consumerslinux-x86_64 natives because of stage-loop path mishandling; flatten artifact downloads and add a jar tf guardmetadata.config size limit — exclude the parser sources tarball from the packagecrates/ts-pack-php/README.md links in root README.md — path moved to packages/php/README.md after alef migration (#106).task/php.yml build, build:dev, and clean tasks pointing to removed crates/ts-pack-php/ — corrected to crates/ts-pack-core-php/ (#106)packages/php/composer.json and packages/php/README.md package name to canonical Packagist vendor slug (kreuzberg/ not kreuzberg-dev/) (#106)mlocati/php-extension-installer prerequisite in install docs and correct minimum PHP version to 8.4+ (#106)binding.go with current alef generatorGo: fix FFI build defaults — add TSLP_LINK_MODE and TSLP_LANGUAGES env vars to Go task
TSLP_LINK_MODE and TSLP_LANGUAGES env vars to Go task (#102)LDFLAGS paths — point to workspace target/release/ instead of crate-local path (#102)ffi.go (already in ts_pack.h) (#102)Init, Download) (#102)cache_dir() to registry on creation (#102)additional_dependencies for all textlint plugins (#102)fix(go): move package root from packages/go/v1/ to packages/go/ by @kh3rld in https://github.com/kreuzberg-dev/tree-sitter-language-pack/pull/97
Full Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v1.6.0...v1.6.2
-fno-strict-aliasing to prevent undefined behavior (#100)Go: move package root from packages/go/v1/ to packages/go/ so the Go module proxy can resolve go.mod at the correct path
packages/go/v1/ to packages/go/ so the Go module proxy can resolve go.mod at the correct path (#97)SRCDIR-relative include/lib paths (one fewer ../ after directory restructure)features = ["all"] from e2e Rust test Cargo.toml — use download feature for runtime parser fetchinglang-* features to unblock crates.io publish (300 feature limit)rustls-webpki to patch RUSTSEC-2026-0098 and RUSTSEC-2026-0099 (#99)language_definitions.json in cratedownload/TSLP_LANGUAGES documentation in READMEspackages/go/v1/ to packages/go/ so the Go module proxy can resolve go.mod at the correct path — go get github.com/xberg-io/tree-sitter-language-pack/packages/go now works (#97)SRCDIR-relative include/lib paths (one fewer ../ after directory restructure)features = ["all"] from e2e Rust test Cargo.toml — use download feature for runtime parser fetchinglang-* features to unblock crates.io publish (300 feature limit)rustls-webpki to patch RUSTSEC-2026-0098 and RUSTSEC-2026-0099 (#99)language_definitions.json in cratedownload/TSLP_LANGUAGES documentation in READMEsThread-local parser cache — parse_string() now reuses parsers across calls for the same language, avoiding the cost of creating a new parser every tim
parse_string() now reuses parsers across calls for the same language, avoiding the cost of creating a new parser every timerun_query() caches compiled tree-sitter queries in both a thread-local cache (fast, no locking) and a global shared cache (cross-thread reuse)CompiledExtraction::extract() and intel::parse_source() now use the thread-local parser cacheCompiledExtraction — avoids rebuilding capture name vectors on every extraction callextract_from_tree() callcompiled_query() now propagates Error::LockPoisoned instead of silently ignoring poisoned RwLockQueryCursor byte-range no longer leaks between patterns when reusing the cursor in extract_from_tree()std::collections::HashMap with ahash::AHashMap in parser cache for consistencyget_language() call removed from parse_string() hot path — only called on cache misslanguage_definitions.json (frozen at 248 languages) causing missing extension mappings for 58 newer languages.gd (gap/gdscript), .conf (hocon/nginx), .sv/.svh (verilog/systemverilog)build.rs now prefers sources/language_definitions.json over the crate-local copy to prevent future stalenesstype_spec declarations (structs, interfaces, type aliases) are now extracted as symbols with correct SymbolKindweb, systems, scripting, data, jvm, functional)language-pack.toml) setupFull Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v1.5.0...v1.6.0
parse_string() — avoids re-creating parsers on repeated calls for the same languagerun_query() — avoids recompiling tree-sitter queriesparse_with_language() internal API for callers that already have a Language objectCompiledExtraction — avoids rebuilding on every extraction calltype_spec declarations extracted as symbols with correct SymbolKind (struct, interface, type)compiled_query() now propagates Error::LockPoisoned instead of silently ignoring poisoned RwLockQueryCursor byte-range no longer leaks between patterns when reusing the cursor in extract_from_tree()std::collections::HashMap with ahash::AHashMap in parser cache for consistencyget_language() call removed from parse_string() hot path — only called on cache missCompiledExtraction::extract() and intel::parse_source() now use the thread-local parser cacheQueryCursor reused across patterns within a single extract_from_tree() callString allocation removed from node_types.contains() check in chunkinglang-* Cargo features and group features (all, web, systems, scripting, data, jvm, functional, wasm) — language selection is now via TSLP_LANGUAGES env var at build time; the download feature (default) fetches parsers at runtime57 new permissively-licensed grammars — 305 languages total
less grammar: regenerated parser from ABI 11 to ABI 14 (was incompatible with tree-sitter 0.26)corn smoke fixture: replaced invalid snippet with valid corn syntaxFull Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v1.4.2...v1.5.0
Publish `ts-pack-cli` to crates.io — cargo install ts-pack-cli now works
ts-pack-cli to crates.io — cargo install ts-pack-cli now works (#87)install.sh reference from documentationcargo install ts-pack-cli
or via Homebrew:
brew install kreuzberg-dev/tap/ts-pack
Include language_definitions.json in the published crate so build.rs can find extension mappings, ambiguity data, and C symbol overrides when installe
language_definitions.json in the published crate so build.rs can find extension mappings, ambiguity data, and C symbol overrides when installed from crates.ioFull Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v1.4.0...v1.4.1
detect_language_from_extension and detect_language_from_path exposed across all language bindings (Python, Node.js, Ruby, Go, Java, Elixir, PHP, C#, W
detect_language_from_extension and detect_language_from_path exposed across all language bindings (Python, Node.js, Ruby, Go, Java, Elixir, PHP, C#, WASM, C FFI)detectLanguage, detectLanguageFromContent, extensionAmbiguity, getHighlightsQuery, getInjectionsQuery, getLocalsQuery, extract, validateExtraction, toSexp, errorCountdetectLanguage, detectLanguageFromExtension, detectLanguageFromPathdetect_language exposed in Python public API (#85)ts-pack-php (hyphens)DetectLanguageFromContent missing error checkFull Changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/compare/v1.3.3...v1.4.0
detect_language in Python public API (#85)ts-pack-php (hyphens)Download path for c_symbol languages: C_SYMBOL_OVERRIDES now includes ALL languages from language_definitions.json, not just compiled ones — fixes dow
C_SYMBOL_OVERRIDES now includes ALL languages from language_definitions.json, not just compiled ones — fixes download and loading of csharp, vb, embeddedtemplate, nushell from published packages (PyPI, npm, RubyGems, etc.)downloaded_languages() returns canonical names (csharp) instead of c_symbol names (c_sharp)JSON.parse on native Hash return from process()ProcessResult fields directly (no metadata wrapper)rustler_precompiled 0.8.4 → 0.9.0Full changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/blob/main/CHANGELOG.md
C_SYMBOL_OVERRIDES table now includes ALL languages from language_definitions.json, not just compiled ones — fixes download and loading of csharp, vb, embeddedtemplate, nushell from PyPI/npm/RubyGems packagesdownloaded_languages() returns canonical names (csharp) instead of c_symbol names (c_sharp)JSON.parse on native Hash return from process()ProcessResult fields directly (no metadata wrapper)rustler_precompiled updated to 0.9.0 (Elixir)Dynamic parser loading for languages with c_symbol overrides (csharp, vb, embeddedtemplate, nushell) — build named libraries with raw name but runtime
c_symbol overrides (csharp, vb, embeddedtemplate, nushell) — build named libraries with raw name but runtime expected c_symbol name (#80)process(), extract(), validate_extraction() return native Ruby Hash instead of raw JSON stringProcessResult fields are flat (matching Rust JSON) instead of incorrectly nested under Metadataextract/2 and validate_extraction/1 NIF declarationsaMOPel/tree-sitter-nim (ABI v14)LowerHex compatibility after dependency updatec_symbol override languages (csharp, vb, embeddedtemplate, nushell)c_symbol naming mismatchespie install kreuzberg/tree-sitter-language-pack)Full changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/blob/main/CHANGELOG.md
c_symbol overrides (csharp, vb, embeddedtemplate, nushell) — build was naming libraries with the raw name but runtime loader expected the c_symbol name (#80)tspack import in non-process test filesextract/2 and validate_extraction/1 NIF declarations\n is interpreted correctlyparanim/tree-sitter-nim (ABI v11) to aMOPel/tree-sitter-nim (MIT, ABI v14)c_symbol override languages (csharp, vb, embeddedtemplate, nushell)ci-all-grammars.yaml to catch c_symbol naming mismatchesRuby binding: process(), extract(), validate_extraction() now return native Ruby Hash instead of raw JSON string
process(), extract(), validate_extraction() now return native Ruby Hash instead of raw JSON string*ProcessResult struct fields instead of invalid json.Unmarshal on non-string returnload_from loaderFull changelog: https://github.com/kreuzberg-dev/tree-sitter-language-pack/blob/main/CHANGELOG.md
Extraction query API: run user-defined tree-sitter queries and get structured results
extract_patterns() / extract() across Python, Node.js, Rust, Ruby, Elixir, PHP, WASM, C FFIvalidate_extraction() for config validation without executionCompiledExtraction for pre-compiled query reuse (Rust)ProcessConfig.extractions for combining custom queries with standard analysisprocess_imports_contains_source assertion uses contains instead of equalitydetectLanguageFromPath and detectLanguageFromExtension exportsprocess() result assertionscrate field resolution with load_from overrideNushell grammar c_symbol override — linker error undefined symbol: tree_sitter_nushell
c_symbol override — linker error undefined symbol: tree_sitter_nushell.as_deref() on String type (compile error on CI)c_symbol_for behind dynamic-loading/download features (dead code warning)crate: field with Cargo [lib] name (underscores, not hyphens)--cfg flag patch to publish workflow for Rustler 0.37.3 compatibilitywithout_gil(): add catch_unwind to ensure GIL is reacquired on panic//!, /*!, and doc_comment node typestext.contains()Arc<Vec<PathBuf>> for extra lib dirs (avoids Vec clone per language lookup)AHashSet<&str> in available_languages() (avoids 248+ String allocations)NodeInfo.kind uses Cow::Borrowed (zero-copy from tree-sitter's &'static str)with_tree()/try_with_tree() helpers replace 9 duplicate lock patternswithout_gil() helper replaces 5 duplicate GIL release patternsextension_ambiguity_json() helper replaces duplicated JSON serialization in 4 bindingsMetadataCollector struct reduces function from 11 to 7 parameters49 new permissively-licensed grammars — 248 languages total
scripts/check_grammar_updates.py) with weekly CI workflowdocs/supported-languages.md) integrated into docs CIdetectLanguageFromExtension, detectLanguageFromPath, getHighlightsQuery, extensionAmbiguityprocess test category with process() API coverage across all 11 language bindingsintel/ and metadata/ directories into unified process/ categoryindex.js exports for detection and query functionstree_sitter_nushell link error).as_deref() on String type errorcatch_unwind//!, /*!, and doc_comment node typesintel_*/meta_* to process_*Arc<Vec<PathBuf>> for extra lib dirs (avoids Vec clone per lookup)available_languages() uses AHashSet<&str> internally (avoids 248+ String allocations)NodeInfo.kind uses Cow::Borrowed (zero-copy from tree-sitter's &'static str)without_gil() helper, with_tree()/try_with_tree() lock helpersextension_ambiguity_json() core helper replaces duplicated JSON serialization in 4 bindingsMetadataCollector struct reduces chunking function from 11 to 7 parametersNew language: al (AL / Business Central) — 198 languages total
al (AL / Business Central) — 198 languages totaltask lint:licenses) verifies all grammars use permissive licensesnim grammar (MPL-2.0 copyleft) with paranim/tree-sitter-nim (MIT)prolog grammar (AGPL-3.0 copyleft) with Rukiza/tree-sitter-prolog (ISC)All 198 grammars verified permissive (MIT, Apache-2.0, BSD, ISC, or similar). No copyleft licenses.
See CHANGELOG.md for full details.
al (AL / Business Central) — 198 languages totalscripts/lint_grammar_licenses.py, task lint:licenses) verifies all grammars use permissive licensesnim grammar (alaviss, MPL-2.0 copyleft) with paranim/tree-sitter-nim (MIT)prolog grammar (codeberg foxy, AGPL-3.0 copyleft) with Rukiza/tree-sitter-prolog (ISC)Dynamic loader: resolve c_symbol overrides for csharp, embeddedtemplate, and vb so get_language() works for dynamically loaded grammars (fixes #80)
c_symbol overrides for csharp, embeddedtemplate, and vb so get_language() works for dynamically loaded grammars (fixes #80)--cfg flag for newer cargo; fix NIF filename mismatchci-all-grammars.yaml) that tests all 197 grammars end-to-endrust:e2e:all-grammars task for running the full grammar suite locallySee CHANGELOG.md for full details.
c_symbol overrides for csharp, embeddedtemplate, and vb so get_language() works for dynamically loaded grammars (fixes #80)ci-all-grammars.yaml) that tests all 197 grammars end-to-end, preventing regressions like #80rust:e2e:all-grammars task for running the full grammar suite locallyElixir NIF: fix Rustler crate name mismatch (ts_pack_elixir → ts-pack-elixir) causing compilation failure
ts_pack_elixir → ts-pack-elixir) causing compilation failureinclude_str! with relative paths that break in the cargo package tarballSee CHANGELOG.md for details.
WASM build: ahash uses compile-time-rng instead of runtime-rng (avoids getrandom on wasm32)
c_symbol override for grammars with non-standard C symbol names (csharp → tree_sitter_c_sharp, vb → tree_sitter_vb_dotnet, embeddedtemplate → tree_sitter_embedded_template)#[cfg(feature)] flags.pyi and py.typed included in both wheel and sdist (PEP 561)ExtensionAmbiguityResult model classgenerate: true for csharp, vb, embeddedtemplatestd::HashMap/HashSet to ahash::AHashMap/AHashSet for faster hashing in registry007ec6See CHANGELOG.md for full details.
c_symbol override for grammars with non-standard C symbol names (csharp, vb, embeddedtemplate)dynamic-loading feature disabled (WASM builds).pyi and py.typed now included in both wheel and sdistExtensionAmbiguityResult model classgenerate: true for csharp, vb, embeddedtemplate grammarsstd::HashMap/HashSet to ahash::AHashMap/AHashSet for faster hashing in registryasciidoc, awk, caddy, cedar, cedarschema, devicetree, dot, idris, jinja2, jq, lean, postscript, prolog, rescript, ssh_config, textproto, tlaplus, vb,
asciidoc, awk, caddy, cedar, cedarschema, devicetree, dot, idris, jinja2, jq, lean, postscript, prolog, rescript, ssh_config, textproto, tlaplus, vb, wasm-interface-types, zsh
language_definitions.json is now the single source of truth for 239 file extensions across 197 languages. Extension lookup is generated at build time with strict validation.
detect_language_from_content(content) — shebang-based language detectionextension_ambiguity(ext) — query whether a file extension is ambiguousget_highlights_query(lang) — bundled tree-sitter highlight queriesget_injections_query(lang) — bundled injection queriesget_locals_query(lang) — bundled locals queriesProcessConfig.language: String → Cow<'static, str> (zero alloc for literals)memchr for fast byte scanning in text splitter and chunkingHashSet dedup in available_languages() (O(1) vs O(n))LanguageRegistry uses Arc<RwLock<Vec>> interior mutability — no more global RwLock wrapper.pyi and py.typed included in both wheel and sdist (PEP 561)See CHANGELOG.md for full details.
sources/language_definitions.json is the single source of truth for 239 file extensions across 197 languagesbuild.rs generates extension lookup with strict validation (panics on duplicates, non-ASCII, uppercase, dots)detect_language_from_content(content): shebang-based language detection (#!/usr/bin/env python3 → "python")extension_ambiguity(ext): query whether a file extension is ambiguous (e.g. .m → objc with matlab alternative)get_highlights_query(lang), get_injections_query(lang), get_locals_query(lang) — embed .scm queries at build timeambiguous field in language_definitions.json for declaring known extension ambiguitiesLanguageRegistry uses Arc<RwLock<Vec<PathBuf>>> for interior mutability — no more global RwLock wrapper, eliminates lock poisoning riskProcessConfig.language: String → Cow<'static, str> (zero allocation for string literals)NodeInfo.kind, QueryMatch.captures: String → Cow<'static, str>available_languages() uses HashSet for O(1) dedup instead of O(n) Vec containsmemchr dependency for fast byte scanning in text splitter and chunkingclone_vendors.py now copies queries/ directories alongside src/.pyi (TypedDicts), TypeScript .d.ts (interfaces), Ruby .rbs (record types), C# Models.cs (string enums replace object)RwLock poisoning — test helpers now use local LanguageRegistry::new().os (bsl) and .cls (apex/LaTeX conflict) extensionsStable release — complete rewrite from Python to Rust with polyglot bindings.
Stable release — complete rewrite from Python to Rust with polyglot bindings.
See CHANGELOG.md for full details.
ts-packrelease events and includes full smoke tests before pushYour coding agent can read these notes before it upgrades. Set up the MCP server →