NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
pub.dev
pub.dev, #1399 most downloaded on pub.dev
Cross-platform, on-device LLM inference for Dart and Flutter. Run GGUF (llama.cpp) and LiteRT-LM models locally on Android, iOS, macOS, Windows, Linux and web.
Latest stable
0.19.5
Last release 11 days ago
11 Nov 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
9 months old
91 releases · first in 2026
Drag across the ruler to compare, or tap a release to open it.
Keep the default WebGPU bridge assets at v0.1.83 (unchanged from 0.19.4).
v0.1.83 (unchanged from 0.19.4).v0.19.1-3, whose LiteRtLm has the GPU accelerator built in: GPU generation is unchanged, the unused Metal accelerator no longer ships, the macOS hook bundle is 8.7 MB smaller and iOS apps 8.1 MB smaller, and the iOS LiteRT-LM frameworks no longer use the User Defaults required-reason API (#1798).responseFormat now also runs on the macOS arm64 GPU, iOS arm64 CPU and GPU, and the Android arm64, Linux x64/arm64 and Windows x64 CPU, and with Gemma 4 (bundle template), which can think before its constrained reply. It accepts the JSON Schema subset that both llama.cpp's GBNF converter and LiteRT-LM's LLGuidance enforce (nested objects, arrays, $ref, anyOf, anchored patterns, uuid/date formats, bounds). Other schemas, including oneOf, throw LlamaUnsupportedException naming the schema path, and other hosts, Android, Linux and Windows GPUs and NPU throw it before generation. Qwen3 with thinking is refused, and a reply cut off inside the JSON reports finish reason length (#174).getPerformanceContext() on native LiteRT-LM returns the counters of a generation cancelled mid-stream again, as before the 0.19 runtime, instead of null (#1903).engine.getContextSize() on LiteRT-LM Web withTextHistory() loads returns the context size the runtime's executor enforces when its load probe finds one below the requested size, such as Qwen3 0.6B's 4096 on WASM CPU, instead of the requested ModelParams.contextSize (#1861).<tool_call><function= call is dropped), its required arguments come before its optional ones, and a string value keeps its whitespace and any </parameter> not followed by a line break. Nemotron 3 Nano now parses as Qwen3-Coder XML, so a <tool_call> also ends its thought, and the tool-call grammar of these templates requires that shape and starts at <tool_call> (#1898).<seed:tool_call> scope is never closed keeps a call complete through </function> and drops one cut off before it, an argument declared as a string keeps its value as generated instead of being read as JSON (\n42\n stays a string), and with ToolChoice.required the grammar lets a thought and text come before the calls (#1898).</mm:think> and any later <mm:think> or </mm:think> is content, so a reply such as </mm:think></mm:think><mm:think> streams the same whole or split by character, with ToolChoice.none too (#1905).<tool_call>name calls with <arg_key>/<arg_value> arguments no longer parse as Hermes JSON, a <tool_call> ends a forced-open thought, a Hermes JSON call stays content, and a trailing <|role_end|> is consumed. A string argument is the raw text up to </arg_value>; any other is JSON (#1857).Runtime baseline: native llama.cpp v0.9.0 and Web use upstream v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8; native LiteRT-LM uses v0.19.1-3 and stable-diffusion uses v0.3.0-3. Web LiteRT-LM uses @litert-lm/core@0.19.0. Manifest SHA-256: ae6f488408672f9ba28d8f9e664a2b85f4e12048f290e259e18dd0b750c038e5.
Your coding agent can read these notes before it upgrades. Set up the MCP server →
Keep the default WebGPU bridge assets at v0.1.83 (unchanged from 0.19.3).
v0.1.83 (unchanged from 0.19.3).LlamaBackend.withTextHistory() on @litert-lm/core@0.19.0: create returns the model's tool calls and takes tool results in the next request, and supportsToolCalling reports whether the loaded runtime reads tool declarations; ToolChoice.required and parallelToolCalls still throw LlamaUnsupportedException. Replies cut off at maxTokens or the context size now end with finish reason length, and automatic tool loops (sendWithTools, completeWithTools) run when a load probe finds that the runtime reports per-turn token counts; a reply that ends exactly at maxTokens also reports length (#1790, #919).generate() wraps the prompt in one user turn of the bundle's chat template, because @litert-lm/core@0.19.0 has no raw streaming route (#1863).ModelParams.chatTemplate, as the legacy runtime does; an argument error from the worker isolate keeps LlamaArgumentException.name; and the raw-prompt refusal says whether the prompt is empty or holds only the model's start token (#1868, #1861).</think> before any <think>, a later <think> or a second </think>, is now content in generic, Llama 3, GLM 4.5, Kimi K2, LFM2, Laguna, Granite, MiniCPM5, Mistral Nemo, Apertus, Hunyuan V3 and DeepSeek R1, V3.2 and V4 replies, and after the start of a Hermes or Qwen3-Coder XML reply. The stream used to depend on how the reply was chunked and could repeat text as both content and reasoning. MiniMax M3 does the same: a <mm:think> after leading whitespace or later in the reply, and a </mm:think> after text, stay content (#1865, #1887).<tool_call> on that is not a valid call, such as an invalid call or text after the calls, is dropped instead of kept as content; only the first call counts without parallelToolCalls; Qwen3-Coder replies have no reasoning; and StepFun3.5-Flash with thinking off ends its thought at </think> or <tool_call>. With ToolChoice.required, these replies may think and write text before their calls instead of having to start with <tool_call>, so one that spends maxTokens first ends with length and no call, and a Qwen3-Coder call that leaves out the first <tool_call> is now a tool call (#1531).<seed:tool_call> followed by a line break, as llama.cpp does, in both the parser and the tool-call grammar: a scope such as <seed:tool_call><function= is now content, and a call after it is still found. An unknown tool or a missing required argument still parses as a call; Seed-OSS requests refuse ToolDefinition.fromJsonSchema tools, so validate the tool name and arguments before running a call (#1542).Runtime baseline: native llama.cpp v0.9.0 and Web use upstream v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8; native LiteRT-LM uses v0.19.1-2 and stable-diffusion uses v0.3.0-3. Web LiteRT-LM uses @litert-lm/core@0.19.0. Manifest SHA-256: ae6f488408672f9ba28d8f9e664a2b85f4e12048f290e259e18dd0b750c038e5.
Ship the 0.19.2 changes below, which never reached pub.dev: pub.dev rejected the 0.19.2 CHANGELOG.md as larger than 256 KiB, so notes for releases bef
CHANGELOG.md as larger than 256 KiB, so notes for releases before 0.10.0 moved to the changelog archive.v0.1.83 (unchanged from 0.19.1).dart run llamadart:doctor_native --runtime litert_lm on macOS reporting ASSET_NOT_DECLARED for a working runtime: it now checks the runtime the build hook keeps in its cache and reports target_source: build_hook_cache (#1855).generate(), like other backends, instead of wrapping it in the bundle's chat turn, so create() requests that fall back to a rendered prompt are no longer templated twice. A prompt that already starts with the model's start token, such as Gemma's <bos>, no longer gets a second one; an empty prompt throws LlamaArgumentException; and pre-1.0 runtime overrides without the conversation template setter refuse raw generate() with LlamaUnsupportedException (#1853).getMetadata()['tokenizer.chat_template'] and engine.chatTemplate(...), whose tokenCount matches the runtime's prefill for Gemma 4 and Qwen3.5. Format detection and parsing keep the file-name template, now reported as llamadart.chat_template.parsing, so code that detects the format from tokenizer.chat_template should read that key first. Native JSON Schema chat now refuses bundles whose text chats the runtime renders with its own template, matching generation (#1836).</thi when generation stops mid-tag, in an unended thought's reasoning, as llama.cpp v0.9.0 does; it used to be dropped (#1531).</think> that comes before any <think> in a Hermes, Qwen3-Coder XML or DeepSeek V3 reply as content, with every thinking tag after it, when the prompt opened no thought (such as Qwen3 with thinking on), as llama.cpp does. The result used to depend on how the stream was chunked and could repeat the text before the tag as both content and reasoning (#1858).Runtime baseline: native llama.cpp v0.9.0 and Web use upstream v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8; native LiteRT-LM uses v0.19.1-2 and stable-diffusion uses v0.3.0-3. Web LiteRT-LM uses @litert-lm/core@0.19.0. Manifest SHA-256: ae6f488408672f9ba28d8f9e664a2b85f4e12048f290e259e18dd0b750c038e5.
Keep the default WebGPU bridge assets at v0.1.83 (unchanged from 0.19.0).
v0.1.83 (unchanged from 0.19.0).v0.19.1-2, which reports LiteRT linked in process; the C API is unchanged. Linux and Windows native embeddings need v0.19.1-2 or later, the bundled runtime: an explicit older LLAMADART_LITERT_LM_LIB_DIR runtime, including v0.19.1-1, is refused with LlamaUnsupportedException, where custom v0.19.0 and v0.19.1 runtimes used to crash (#1367).libLiteRt (about 7 to 9 MB) in Linux and Windows LiteRT-LM apps, whose runtime links LiteRT in; an older LLAMADART_LITERT_LM_LIB_DIR runtime still loads the libLiteRt it ships with (#1815).ComputeDevice.gpu throws LlamaUnsupportedException at load, and a GPU hint under ComputeDevice.auto runs on the CPU with a warning (#572).dispose failing with "worker stopped before native cleanup was confirmed" when cleanup outlasts 5 s, such as releasing a freshly written CPU weight cache on Linux ext4; freeTimeout now bounds only the wait for native teardown to start (#1814).clear, remove and prune on Windows failing with "Access is denied" when another process or isolate was deleting the same entry at that moment (#1821).0.19.1 with a Flutter floor of >=3.38.6, the first Flutter whose Dart (3.10.7) satisfies their sdk: ^3.10.7; Flutter 3.38.0 to 3.38.5 could not resolve them (#1834). The LiteRT-LM companion pins v0.19.1-2 and requires core 0.19.1.Runtime baseline: native llama.cpp v0.9.0 and Web use upstream v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8; native LiteRT-LM uses v0.19.1-2 and stable-diffusion uses v0.3.0-3. Web LiteRT-LM uses @litert-lm/core@0.19.0. Manifest SHA-256: ae6f488408672f9ba28d8f9e664a2b85f4e12048f290e259e18dd0b750c038e5.
Add native llama.cpp speech output for LFM3-Audio, Soprano, Chatterbox Turbo, Chatterbox Multilingual and Pocket TTS (Lfm3AudioTtsAdapter, SopranoTtsA
Lfm3AudioTtsAdapter, SopranoTtsAdapter, ChatterboxTtsAdapter, PocketTtsAdapter), with predefined voice selection (TextToSpeechRequest.voice, TextToSpeechCapabilities.voices) and Chatterbox and Pocket speaker-reference cloning. LFM3-Audio and Soprano need llamadart-native v0.8.0 and Chatterbox and Pocket v0.9.0; older runtimes refuse them with LlamaUnsupportedException (#1615, #1783).v0.9.0: ARM64 CPU F16 dot products of large values no longer overflow to inf or NaN (llamadart-native#174), and Qwen3-TTS sampling applies the GGUF's suppress tokens.v0.1.83 on the same upstream llama.cpp as native, so Web Qwen3-TTS also applies the GGUF's suppress tokens (llama-web-bridge#211).v0.19.1-1 (upstream LiteRT-LM v0.19.1); the C API is unchanged.@litert-lm/core@0.19.0; WebGPU loads keep the runtime's default backend, so standard .litertlm bundles load there. Web embeddings, tool loops, media and custom templates stay unsupported. In Safari 27, GPU text, history and the chat app pass; on CPU, the runtime's compat build can leave Qwen3 0.6B thinking through its whole token budget (#1025, #1802).microBatchSize above 8, media (with the image encoder on the CPU), speculation and embeddings for native Qwen3.5 on Android Vulkan with the bundled runtime; runtimes older than ggml 0.28.0 (llamadart-native v0.8.0) keep the text-only restriction (#948).0.19.0 with matching native binaries.Runtime baseline: native llama.cpp v0.9.0 and Web use upstream v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8; native LiteRT-LM uses v0.19.1-1 and stable-diffusion uses v0.3.0-3. Web LiteRT-LM uses @litert-lm/core@0.19.0. Manifest SHA-256: ae6f488408672f9ba28d8f9e664a2b85f4e12048f290e259e18dd0b750c038e5.
Fix synthesized speech playback in the macOS chat example.
LlamadartEngine<C> lifecycle management and eager text/chat and Decision tasks with events, completion and request-scoped cancellation.v0.19.0 and C API 1.0 with legacy compatibility, tagged Android pointer support, validated settings and typed unsupported-path errors.v0.8.0-3 for speculative rollback and model-creation ownership fixes, retaining the genuine nextn query while retiring the unused embedding query; isolate native bundle caches from older apps, verify Apple framework identity and report missing Windows speculative helpers.v0.3.0-3 for original SD checkpoint containers while retaining the SDXS tiny-decoder policy and iOS full-VAE Metal restrictions.require-corp isolation and hidden-tab termination during large model loads (#1025).v0.1.82 with bounded preallocation for large wasm32 model writes.0.18.0 with matching native binaries.Runtime baseline: native llama.cpp v0.8.0-3 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM uses v0.19.0 and stable-diffusion uses v0.3.0-3. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: fda19e047aba5a511020854437cc89e2f53e2f67f5f6cf5fec336faef3a6c362.
v0.8.0-3 can overflow to inf on large-magnitude inputs (llamadart-native#174). This is not a regression, and no public-model failure has been observed. A fix will ship in a later native release (llamadart-native#177).Preserve quoted LFM2/LFM2.5 tool-end markers inside tool arguments during parsing and streaming.
v0.1.79 (unchanged from 0.17.4).Runtime baseline unchanged from 0.17.4: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.
Restore documentation destinations linked from published package READMEs and protect retained release notes.
0.17.2 with unchanged native runtimes.v0.1.79 (unchanged from 0.17.3).Runtime baseline unchanged from 0.17.3: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.
Reject projector requests on fixed LiteRT-LM custom backends before resolving files, preserving the loaded model (#1556).
v0.1.79 (unchanged from 0.17.2).Runtime baseline unchanged from 0.17.2: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.
Fix Windows native runtime overrides failing model loads with no backends are loaded (#1649).
no backends are loaded (#1649).message.thinking, including turns with tool calls (#1623).tensorSplit sums in direct llama.cpp loads instead of silently selecting a free-memory split (#1554).v0.1.79 (unchanged from 0.17.1).Runtime baseline unchanged from 0.17.1: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.
Aligned the default WebGPU bridge assets to v0.1.79: Web now runs llama.cpp v0.8.0 like native, and a media embedding batch that takes longer than two
v0.1.79: Web now runs llama.cpp v0.8.0 like native, and a media embedding batch that takes longer than two minutes on the CPU stays on the worker instead of moving to the main thread (llama-web-bridge#165). Upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6 matches native v0.8.0, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.leehack/llamadart-native@v0.8.0-1 (llama.cpp v0.8.0), restoring Metal model loading on Apple devices. Apple apps get it with llamadart_llama_cpp_flutter 0.17.1; a locked companion 0.17.0 keeps v0.7.0-3 and builds with an upgrade warning. llama.cpp v0.8.0 refuses a hand-written grammar nested or chained more than 256 levels deep with LlamaInferenceException; generated schema grammars stay below that.numberOfThreads or numberOfThreadsBatch at 0, native llama.cpp now uses at most as many threads as Platform.numberOfProcessors reports, instead of always 4, which made inference tens of times slower on hosts with fewer than 4 cores; multimodal projectors and decision encoders follow the same cap. Hosts with 4 or more cores use 4, which lowers opt-in GpuBackend.blas on hosts above 4 cores from all cores to 4; set an explicit count to raise it (#1612).getResolvedGpuLayers() now reports 0, a warning names the requested backend, and getAvailableBackends() no longer lists a module that failed to load. getResolvedGpuLayers() also reports 0 for a GpuBackend.auto load on a host with no GPU device (#1598).ModelSplitMode.row on native llama.cpp devices without split buffers, which include every bundled GPU backend whose module loads, BLAS and an explicitly selected or fallback CPU, now throws LlamaUnsupportedException naming the device before the model loads, instead of a generic load failure; use ModelSplitMode.layer. A bundled module that fails to load or finds no device loads row on the CPU instead (#1576), and a speculative draft model now loads when its row target runs on the CPU of a host with no GPU backend, instead of failing with "Failed to load draft model" (#1593).GpuBackend.auto now runs on the CPU with 0 GPU layers, because Metal returns garbled output on Intel Macs with AMD GPUs. An explicit GpuBackend.metal or ComputeDevice.gpu still uses Metal and logs a warning; arm64 macOS is unchanged (#1597).ToolChoice.required with parallelToolCalls: true now returns every call instead of only the first; the flag also widens the grammar under ToolChoice.auto when the first system message carries the Force JSON schema. marker. A longer list needs more tokens: a reply maxTokens cuts off before the list closes finishes with length and returns no calls, including those already complete, so raise maxTokens and retry. Their tool-call parsing also no longer throws on malformed model output: a number too large to represent, a value nested too deeply or a lone quote (#1606).parallelToolCalls: true now lets the model put whitespace between calls, as llama.cpp does, so it can write the newline its template uses instead of stopping after one call. Templates without parallel calls, such as the stock 1.5B and 8B distills, still allow one call (#1581).</think> split across chunks no longer streams as content when the parse drops it, as in a DeepSeek R1 reply that starts with </think>, and TranslateGemma, Muse Glimmer without tools, and PEG-parsed templates without tools such as Ministral 3 Reasoning, Devstral and Nemotron 3 Nano stream thinking tags their parse keeps as text (#1582).Breaking (backend SPI): LoRA and multimodal projector members move off LlamaBackend into the optional BackendLoraSupport and BackendMultimodalSupport
LlamaBackend into the optional BackendLoraSupport and BackendMultimodalSupport interfaces. On a backend or routed runtime without one, LlamaEngine LoRA calls and projector loads throw LlamaUnsupportedException, before any download, and removeLoraSource of an adapter never applied does nothing. The built-in backends keep the same support and exception type; LlamaEngine.loadMultimodalProjector refuses a LiteRT-LM model with the same LlamaUnsupportedException message as loadMultimodalProjectorSource and setModel, including on a custom backend that reports the LiteRT-LM runtime and implements BackendMultimodalSupport, and through a router it no longer says the projector is unsupported by the active runtime (#362, #1541).leehack/llamadart-native@v0.7.0-3, a wrapper rebuild of llama.cpp v0.7.0, read through llamadart-native hook contract 2: the hook downloads a backend sidecar archive, such as Windows x64 CUDA, only when that backend is selected, so Windows x64 apps that do not select cuda download 15 MiB instead of 703 MiB, and an overridden release that needs an unsupported contract fails the build with NativeHookContractException (#812). Linux arm64 bundles ship FP16 CPU variant modules, and every backend mode loads the best variant the CPU supports, falling back to the baseline, so mixed auto and explicit loads share one CPU backend. Apple apps get the runtime with llamadart_llama_cpp_flutter 0.17.0.v0.3.0-1 (stable-diffusion.cpp master-1017), which loads FP8 image weights as F16 on Metal: FP8 checkpoints now generate on Metal instead of being refused. The bounded image recipes are requalified on it, and Flutter iOS/macOS apps need llamadart_stable_diffusion_flutter 0.17.0 (#1446).v0.1.76: removes the retained projector copy behind the repeated Web media embedding batch abort on the wasm32 core, which stays unqualified (#1189) and can still exhaust its heap on larger contexts (llama-web-bridge#200); a speculative draft that does not fit throws LlamaUnsupportedException instead of leaking until the runtime aborts, and a model whose context does not fit fails its load instead of aborting (#198); and uncached model loads report progress as they download (#195). Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 90822656cef6eaf43a47eafc7fe32f2ef9c2ed56a74c6b3cae29115ba61f818a.ModelParams can load here before any download with await engine.checkLoad(model, params: params): the LlamaLoadCheckReport is unsupported exactly when setModel would throw before downloading (LiteRT-LM on macOS x86_64, Windows arm64 or the x86_64 iOS simulator, an explicit device the platform lacks, or an option the runtime ignores), and names the runtime and device the load would use. Custom backends join through BackendLoadCheck. setModel now also rejects a LiteRT-LM model with LlamaUnsupportedException before any download, keeping the loaded model, for ComputeDevice.npu or liteRtLmEmbeddingEngine on the Web and for liteRtLmEmbeddingEngine on native platforms without qualified embeddings (Linux and Windows); these used to fail at load after the previous model was unloaded (#861).ModelParams.gpuDevices (listGpuDevices indices of one backend) and set each GPU's share with tensorSplit on native llama.cpp, validated on one Metal GPU and on a Vulkan Tesla T4, with Mesa lavapipe as the second device for two-device lists and splits (two-GPU hardware has no evidence yet, and CUDA is not hardware-qualified); decision heads and multimodal projectors of the model run on a listed GPU (#1517), and indices the host lacks, a split of another length, the Web and LiteRT-LM throw LlamaUnsupportedException (#916).ModelParams.cpuMoeLayers and cache recently used experts on the GPU with the experimental moeCacheBytes budget on native llama.cpp with Metal or CUDA, validated on Metal only (CUDA and multi-GPU hosts have no hardware evidence yet); a budget on Vulkan, other backends or tensor split, and either field on the Web or LiteRT-LM, throw LlamaUnsupportedException (#1123).LlamaEngine.stateIdentity() reports the native runtime build, session format, context size and a digest of the context settings that shape the saved state (KV cache types, flash attention, sequences, SWA and RoPE), so an app can check a llama.cpp state file another process saved before loading it. It needs native llama.cpp with llamadart-native v0.7.0-3; Web, LiteRT-LM, custom backends and older runtimes throw LlamaUnsupportedException (#1211).LlamaToolResultContent.content or a handler's LlamaToolOutput on native llama.cpp with a vision projector and Hermes-style templates (smoke-tested on macOS CPU and Metal with Qwen3.5-0.8B weights, a qwen3vl_merger projector and the Qwen3-VL template), rendered as llama-server renders them; check LlamaEngineCapabilities.toolResultMedia. Other formats, runtimes, audio, video and URL-only images throw LlamaUnsupportedException (#1008).ChatTemplateEngine.parseRendered(rendered, output), which reads how its render or engine.chatTemplate result left the thought: after the stock Seed-OSS template, Plan</seed:think>Sunny. is content, as in llama-server, where ChatTemplateEngine.parse returns reasoning Plan (#1565).ComputeDevice.auto retries a GPU engine that fails to start on the CPU with a warning, and reports LiteRT-LM cpu; explicit gpu and npu still fail (#1034).HiresFixImageGenerationRequest and ImageHiresFix (latent upscale, bounded to 1024x1024 and 4 images) on macOS CPU or Metal with SD 1.x or SD 2.x full-VAE checkpoints; check ImageGenerationCapabilities.supportsHiresFix, and other recipes throw LlamaUnsupportedException (#1228).ReferenceImageEditingRequest and ImageReferenceEditingRecipe.fluxKontext (one RGB reference at the output size, at most 1024x1024) on macOS Metal with the Kontext transformer, ae, CLIP-L and T5-XXL; check ImageGenerationCapabilities.supportsReferenceImageEditing, and other models, devices and reference counts throw LlamaUnsupportedException (#1222).ImageModelRole.controlNet and steer generation with ControlNetImageGenerationRequest on the bounded macOS CPU recipe; other ControlNet recipes are refused at load (#1224).ImageGenerationEngine.load(placement: ImageModelPlacement(...)) on Metal and Vulkan, validated with SD-Turbo on Metal and measured on one Vulkan Tesla T4 (multi-GPU hosts have no evidence yet); a placement on a CPU engine, Android, the CPU builds or the Web, and a GPU budget with the fit off or on Vulkan without a discrete GPU, throw LlamaUnsupportedException (#806).enableThinking as enable_thinking on every request, including responseFormat and generic tool-call routes, as in llama.cpp, so templates such as GLM-4.6, GLM-4.7-Flash, GLM-OCR, SmolLM3, Apertus and DeepSeek-R1 distills render thinking as llama-server does for the same flag. enableThinking still defaults to true, so Apertus now deliberates and DeepSeek-R1 distills open a thought unless you pass false (#1465).enable_thinking or thinking chat template kwarg now overrides enableThinking, and a template's own thinking setting passed in chatTemplateKwargs, such as reasoning_effort for Hy3 and Cohere2 MoE or thinking_mode for MiniMax M3, is no longer replaced by the value llamadart derives from enableThinking, so both render as in llama-server; DeepSeek V3.1 forces its thought open from enableThinking alone (#750).responseFormat or tool-call grammar after a thought the prompt leaves open now lets the model finish the thought first, as llama-server does, instead of returning the JSON as reasoning; a thought the template opens stays open with enableThinking: false, as in llama-server (#1465). A required tool call after such a thought also lets Cohere2 MoE, Nemotron 3 Nano, MiniCPM5 and Hy3 models finish it first; MiniCPM5 previously wrote the call inside the thought and returned no tool call.\n and a repeated <think> stay in the reasoning as generated, thinking tags after the first </think> stay in the content instead of being parsed as more reasoning, a partial </think> that ends an unfinished thought is dropped, and a <tool_call> also ends a Qwen3.5 thought (#750).<think> blocks the model writes in the content, as llama-server does, instead of dropping them as hidden reasoning (#750).</think> as content: content waits for </think>, <think> or the end of the reply, matching the final message at any chunking (#1532).<|END_THINKING|> as content: content waits for a thinking tag, <|START_TEXT|> or <|START_ACTION|>, where llama-server ends the thought. Command R7B and Seed-OSS templates that open the thought with thinking off hold their reply until a thinking tag, and Ministral and Solar Open replies, whose parse keeps such text as content, no longer wait (#1553, #1557). With thinking off, a Cohere2 MoE thought that ends at <|START_TEXT|> or <|START_ACTION|> is reasoning instead of content, as in llama-server; thinking tags inside a Cohere2 MoE text or action stay text; and a response schema no longer leaves the thought open (#1566).<|START_TEXT|> or <|START_ACTION|> the model writes without <|END_THINKING|>, as in llama-server, so its text and tool calls no longer land in the reasoning; with enableThinking: false and a reasoning: true chat template kwarg, its prompt ends with the template's own empty thought, without a trailing newline (#1523).<seed:think>, kept as generated up to the first </seed:think>, as in llama-server; other thinking tags stay in the content (#1561). To a prompt that opens the thought, a repeated <seed:think>, and any text before it, stay in the reasoning up to the first </seed:think>, as in llama.cpp, instead of the tag being dropped, and a thinking tag inside a tool-call argument after that thought no longer drops the call.<tool_call> blocks of Qwen3-Coder XML replies, as from Qwen3-Coder, Qwen3.5 and StepFun3.5 (#1530), and <seed:tool_call> blocks of Seed-OSS replies (#1540). Nemotron 3 Nano's parallel-call grammar admits them too, where it used to admit only replies that parsed to no calls.ToolChoice.required now returns the tool call on DeepSeek-R1 distills, LFM2 and Functionary v3.2 instead of running to the length limit or returning the call as text: their tool-call grammars match llama.cpp's call framing, without a literal \n or a free-text LFM2 id, and DeepSeek-R1 takes one call unless parallelToolCalls is set (#1577). On Firefunction v2 it no longer makes the model write a free-text call id: its tool-call grammar, like llama.cpp's, takes only the call's name and arguments (#1565).ToolChoice.auto instead of forcing a tool call on templates such as Qwen3-VL and Qwen3.5, and use the same tool-call grammar, triggers, parser and stops as text requests (#1463).stateLoadFile no longer lets the next prompt-prefix reuse skip evaluating the prompt on the cache llama.cpp cleared, which returned wrong scores and text without an error (#1491).sharedCache(), userSelected(...) or appGroup(...) directories, now download each entry once instead of returning a file another process is still writing, and never cache a file whose length differs from the one the server reported (#1444). A cache hit on a read-only model cache root returns the cached entry instead of reporting that no cached model is available.DefaultModelDownloadManager.remove, prune and clear no longer delete a cache entry another isolate or process is downloading: remove and clear remove the rest and throw ModelCacheEntryInUseException, and prune skips it (#1467). Overlapping remove, prune and clear calls on one cache root take turns instead of reporting each other's entries in use (#1490), and no longer fail or over-remove when they overlap in one process; a stray .REMOVAL directory in a case-insensitive cache root no longer keeps an entry reported in use (#1522); clear removes a stray .removal directory instead of reporting it in use forever (#1507); and on Windows a model file that can never be deleted, such as a read-only one, throws its FileSystemException instead of ModelCacheEntryInUseException (#1490).assets.json.absent record, which notes that the release publishes no assets.json, is re-checked daily, or at once when it is dated in the future, and a record whose re-check fails or returns a page that is not JSON is kept and renewed instead of failing the build or retrying on every offline build; an assets.json that is truncated, empty, non-UTF-8 or not a manifest fails the build instead of being treated as unpublished; and a CUDA build that cannot reach its assets.json uses the cached CUDA sidecar instead of leaving CUDA out (#1490, #1506, #1522).extracted/ directory still holds what the cached archive beside it extracted to, and otherwise extracts that archive again, so a library replaced or added in the cache, such as a test stub, is no longer bundled (#1558). A cache from an earlier release is extracted again once, without network access. A cached native archive that matches its digest but lacks runtime libraries fails the build instead of being downloaded again on every build (#1490), and an incomplete cached core archive that matches its cached SHA256SUMS no longer tells an offline build to pre-seed SHA256SUMS (#1506).ios-arm64-sim and ios-x64-sim, instead of ios-arm64 and ios-x64 (#1565).libCLiteRTLM_mac.dylib as the library a sandboxed Flutter macOS app without the LiteRT-LM companion fails to load, instead of libLiteRt.dylib (#1565).Reject unsupported tool-role image/audio content before chat rendering or generation instead of silently losing or relocating it.
llamadart:doctor and add opt-in isolated desktop native diagnostics with privacy-safe loader and ABI observations.LlamaArgumentException.v0.1.73 (unchanged from 0.16.4).Web runtime baseline (unchanged): upstream llama.cpp v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab, native v0.7.0-2; Web LiteRT-LM @litert-lm/core@0.18.0. Bridge manifest SHA-256: f89c41d151b7df0cbe4f5e97f95563e31e0968c13e65a630e4e0dba28adb1d44.
Opt into ordered system/user/assistant text history on LiteRT-LM Web with LlamaBackend.withTextHistory(); the chat app adopts it, and unavailable cont
LlamaBackend.withTextHistory(); the chat app adopts it, and unavailable controls fail explicitly (#1152).v0.1.73 (unchanged from 0.16.3).Web runtime baseline (unchanged): upstream llama.cpp v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab, native v0.7.0-2; Web LiteRT-LM @litert-lm/core@0.18.0. Bridge manifest SHA-256: f89c41d151b7df0cbe4f5e97f95563e31e0968c13e65a630e4e0dba28adb1d44.
Recommend the corrected Qwen3-0.6B LiteRT-LM model bundle for native CPU to preserve split-token emoji; replace the older model artifact rather than r
collectLlamaRuntimeSetupReport() and dart run llamadart:doctor [--json]. Reports separate pinned publication policy from unchecked native loading, GPU/backend selection and model behavior. Exit 0 means report generation, not runtime validation (#124).TemplateTimeChatSession and explicit clocks for strftime_now (#859).LlamaReranker on native rank-pooled GGUF models, preserving classifier scores, labels and document indices (#323).SeamlessImageGenerationRequest; the initial SDXS embedded-tiny-decoder recipe is qualified at 512×512 on CPU and Metal (#1234).TiledImageGenerationRequest and ImageVaeTiling, qualified on SD-Turbo full VAE/TAESD with macOS CPU and Metal. Decoder buffers are about one quarter of untiled size, with longer decoding; other SD 2.x checkpoints remain unmeasured, and unsupported families, backends and circular-axis combinations are refused (#1231).ImageConditioningCacheParams; zero disables reuse, while ordinary settings preserve the runtime default.splitMode: ModelSplitMode.none and negative mainGpu; unavailable or inactive GPUs still throw (#915).v0.1.73 (unchanged from 0.16.2).Web runtime baseline (unchanged): upstream llama.cpp v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab, native v0.7.0-2; Web LiteRT-LM @litert-lm/core@0.18.0. Bridge manifest SHA-256: f89c41d151b7df0cbe4f5e97f95563e31e0968c13e65a630e4e0dba28adb1d44.
Update the default LiteRT-LM native runtime to leehack/litert-lm-native@v0.18.1-4. Windows x64 apps no longer need the Visual C++ Redistributable for
leehack/litert-lm-native@v0.18.1-4. Windows x64 apps no longer need the Visual C++ Redistributable for LiteRT-LM, which now ships its own Visual C++ runtime DLLs (litert-lm-native#87); llama.cpp already did, so only the opt-in stable_diffusion runtime still needs it. On Linux and Windows, LiteRT-LM CPU inference uses the configured thread count instead of one thread: Qwen3 0.6B CPU decode is 3.3× faster on an 8-vCPU Linux arm64 host and 2.2× faster on a 4-vCPU Windows x64 host (litert-lm-native#89, #1375). Flutter Apple apps get it with llamadart_litert_lm_flutter 0.16.2.dart build cli, flutter build linux or flutter build windows load LiteRT-LM from their own bundle, so a copied or installed app no longer fails with Failed to load dynamic library 'package:llamadart/litert_lm_LiteRtLm' once the project's hook cache is out of reach (#1397).LlamaUnsupportedException on Linux, Windows and Android x86_64 instead of failing at engine creation or crashing the process; they stay available on macOS arm64, iOS arm64 and Android arm64 (verified on a Galaxy S24 without SME and on arm64 emulators) (#1367).length instead of stop when a reply reaches maxTokens or fills the context, and automatic sendWithTools/completeWithTools loops now run on it, stopping truncated without running tools on a cut-off reply. This needs a runtime whose stream_proxy_finish_reason_abi_version export returns 1 (litert-lm-native v0.18.0-3 or later, including the default pin); on older runtimes loops still throw LlamaUnsupportedException. LiteRtLmRuntimeClient adds supportsFinishReasons and lastFinishReason. Web LiteRT-LM is unchanged (#919).v0.1.73: Web embed() and embedBatch() throw LlamaUnsupportedException for rank-pooled rerankers instead of returning an invalid vector built from classifier scores, and LlamaEmbeddingInputTooLongException for an input longer than the one micro-batch it must fit, instead of embedding only part of it or aborting the runtime, as native does (llama-web-bridge#194). Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: f89c41d151b7df0cbe4f5e97f95563e31e0968c13e65a630e4e0dba28adb1d44.dart run llamadart:web_setup: it downloads the Web bridge files this version pins into web/webgpu_bridge/, checks each against the pinned release manifest and adds the loader to web/index.html; reruns keep current files, and --check verifies offline. It refuses a page that already loads the bridge, so remove a hand-written loader first. With its loader, a page whose files are missing fails the first model load at once instead of after 12 seconds (#855).usage.promptTokens counts the tokens of an image or audio prompt, as llama-server does, instead of its positions, which M-RoPE vision models such as Qwen3.5 report far fewer of (#696).LlamaVideoContent rejection text: on desktop it points to a projector without a vision encoder instead of saying video support is built out, and it names Android and iOS as the platforms whose artifacts lack video (#320).Update the default LiteRT-LM native runtime to leehack/litert-lm-native@v0.18.1-2: on SME2 CPUs (verified on Apple M4 and iPhone 16 Pro), CPU chat and
leehack/litert-lm-native@v0.18.1-2: on SME2 CPUs (verified on Apple M4 and iPhone 16 Pro), CPU chat and audio embeddings are deterministic again (#1340) and Gemma 4 CPU decode is faster again (litert-lm-native#84; 1.56× on M4, 1.2× on iPhone 16 Pro), while its CPU prefill is 15–30% slower than v0.18.1. Output and speed on non-SME2 CPUs (Linux arm64, Windows x64) are unchanged. Image and audio embeddings shift slightly from core 0.16.0, so re-embed stored vectors of image or audio content; text-only vectors are unchanged. Flutter Apple apps get it with llamadart_litert_lm_flutter 0.16.1.v0.1.72: Web repeat and presence penalties now count the prompt, as native does, so output for the same seed changes at the default penalty of 1.1, and tool chats (ToolChoice.auto) leave the prompt out, also as native does. In worker mode, Stop right after a generation starts now stops it, and a bare cancel() streams the text generated before the worker yields. Text containing U+0000 is no longer truncated in tokenize, embed, score and prompts; speech text containing it is rejected. Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 85075d7d278bcacab706b79cda65c4a1bf8098a90f1e9fdbd504c198b8c8b03a.draftDflash and draftDspark generation throws LlamaArgumentException instead of aborting the process when a LiliCorr draft block (dflash.block_size, 16 when absent) does not fit one draft micro-batch: SpeculativeDecodingConfig.draftMicroBatchSize, or the inherited ModelParams.microBatchSize when it is unset (#1330, #1341).…the liteRtLm* arguments of ModelParams are deprecated in favor of LiteRtLmModelParams. Loads reject ModelParams fields the model's runtime would ignor…
LlamaCppModelParams and LiteRtLmModelParams, ModelParams that take only the fields their runtime applies; the liteRtLm* arguments of ModelParams are deprecated in favor of LiteRtLmModelParams. Loads reject ModelParams fields the model's runtime would ignore, such as liteRtLm* fields on a GGUF model or preferMemory64 off the Web, and either type used for the other runtime, with LlamaUnsupportedException naming each field; LlamaBackend() also rejects these and LiteRT-LM platform gaps before any download when the model format is known (#858, #861).leehack/litert-lm-native@v0.18.1, built from upstream LiteRT-LM v0.18.1 with the ASR and stream finish-reason fixes of v0.18.0-3. Flutter Apple apps get it with llamadart_litert_lm_flutter 0.16.0. Native GPU embeddings still throw LlamaUnsupportedException because EmbeddingGemma 2 still crashes in the prebuilt WebGPU accelerator.leehack/stable-diffusion-native@v0.3.0, built from stable-diffusion.cpp master-982-0518b34: Windows opens model files with non-ASCII paths, and the legacy pickle reader no longer rejects checkpoints above 2 GiB (stable-diffusion-native#23). Images keep the flash attention and direct convolution settings ImageModelParams selects instead of the runtime's new automatic per-module choice. Flutter Apple apps need llamadart_stable_diffusion_flutter 0.16.0; this core rejects earlier SD companions, whose sd_ctx_params_t is smaller.v0.1.70: a character split across tokens streams once complete instead of as a wrong one, and detokenize returns U+FFFD for invalid or incomplete bytes. Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 7ac4e6e88205396f7221cd4e9e863bdf284df3498ced58cae2ffa1840996a8bf.GenerationParams.penalty (other than the default), presencePenalty and, in chat on bundles with a thinking channel such as Gemma 4, thinkingBudget; penalties count only the tokens of the current send, capabilities reports each control, and a value it cannot apply throws LlamaUnsupportedException instead of UnsupportedError (#1155).SpeculativeDecodingConfig.draftMicroBatchSize (llama.cpp --spec-draft-n-ubatch) to give native llama.cpp speculative draft contexts a smaller micro-batch and compute buffer than the target; unset keeps the target's, and WebGPU and LiteRT-LM reject it (#1124).ToolChoice.required tool call and end the thought, so the call is no longer parsed as reasoning and lost (#742).ToolDefinition.fromJsonSchema may list in required a property that only additionalProperties describes, as JSON Schema allows; the tool-call grammar requires that key instead of rejecting the schema (#1090).ChatSession subclasses whose addMessage stores a copy of a tool result, adds a note before it or keeps the history length fixed: the rollback could leave an orphaned tool result or note behind (#1268).WEBGPU_BRIDGE_PATCH_SAFARI_COMPAT and WEBGPU_BRIDGE_MIN_SAFARI_VERSION options of scripts/fetch_webgpu_bridge_assets.sh and the chat app's Safari patching of the bridge: the default assets already contain both fixes, and overrides to older asset tags no longer get them.Report a disposed LlamaEngine consistently: deprecated loaders disposed as they finish throw LlamaStateException, loadDecisionHeadBackend and attached…
leehack/llamadart-native@v0.7.0-2: a fixed TextToSpeechRequest.seed gives the same Qwen3-TTS audio on every synthesis of a loaded engine, including after a cancelled one (#632), and Windows arm64 bundles no longer ship libomp140.aarch64.dll, a Visual Studio debug DLL that apps may not redistribute. Apple apps get it with llamadart_llama_cpp_flutter 0.15.2; companions 0.15.0 and 0.15.1 keep building with an upgrade warning.leehack/litert-lm-native@v0.18.0-3: native speech-to-text loads dedicated ASR .litertlm containers such as Whisper tiny, which failed engine creation with tf_lite_prefill_decode not found (litert-lm-native#77). Flutter Apple apps get it with llamadart_litert_lm_flutter 0.15.2; companions 0.15.0 and 0.15.1 keep v0.18.0-2.v0.1.69: loading a missing or
truncated Web state file throws LlamaStateException instead of aborting the
runtime, and chat image and audio URLs relative to the page load in worker
mode. Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches
native v0.7.0-2; the asset manifest records the byte-equivalent
v0.7.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0.
Manifest SHA-256:
994e798389d440a256f103f3789a1f56da2ab00dd6bb6e52f334ed4e58867c46.v0.18.0 and later).SpeechToTextEngine.capabilities with Qwen3AsrAdapter reports a projector that is not Qwen3-ASR's, such as a Qwen3-TTS mmproj, as unsupported instead of failing later with an empty transcript; when the projector type cannot be read, an empty transcript's error details name a possible model mismatch (#641).transcribeOnce, instead of keeping it and its input audio until the next task.setLoraSource and removeLoraSource match a path adapter spelled with ./, ../ or repeated separators instead of silently leaving it applied or applying it twice (#907).ModelParams.loras that LiteRT-LM cannot take before any adapter downloads, also for extensionless native .litertlm files, on the Web and in embedding mode.flutter test on macOS for such an app no longer loads the bundled llama.cpp or stable_diffusion library. Such an app with llamadart_litert_lm_flutter must exclude the x86_64 iOS simulator, as direct dependents already must; the build error names the EXCLUDED_ARCHS setting.preferredBackend has no device, such as GpuBackend.vulkan on Apple platforms: getBackendName() no longer reports CPU when layers are offloaded to another GPU (#1281), and the ComputeDevice.gpu error names the registered backend, such as GpuBackend.metal, instead of suggesting ComputeDevice.auto (#1017).package:llamadart/backend.dart exports BackendRuntimeIdentity, BackendModelFormatRouting, BackendChatScope, BackendDirectMediaInput and BackendDeferredEngineCreation, so custom backends can report their runtime and load an explicit ModelSource.format (#917).LlamaEngine consistently: deprecated loaders disposed as they finish throw LlamaStateException, loadDecisionHeadBackend and attached speech engines throw it instead of LlamaUnsupportedException, and dispose() no longer waits for a projector download that ignores cancellation.ChatSession subclasses and implementations whose addMessage moves or repeats messages, or whose history returns copies or hides messages; the turn could stay in the history while rolledBack was true (#1287).stop instead of length when a llama.cpp speculative-decoding reply ends exactly at maxTokens, so completeWithTools no longer rolls it back as truncated.ToolChoice.required tool call; the call was previously parsed as reasoning and lost.LlamaInferenceException instead of overflowing the stack, and Command R7B and Cohere2 MoE replies parse in linear time instead of seconds.allOf components such as {"description": "..."}, matching llama.cpp; reject \u-escaped spellings of declared property names as extra keys in open-object grammars; and reject integer bounds at -2^63 on native platforms with LlamaUnsupportedException, as on Web, instead of an opaque error or a wrong constraint.llamadart_native_path avoids it.Keep Llama 3 (JSON and builtin <|python_tag|>), FireFunction v2, Functionary v3.1/v3.2, FunctionGemma and Gemma tool calls out of streamed answer text
<|python_tag|>), FireFunction v2, Functionary v3.1/v3.2, FunctionGemma and Gemma tool calls out of streamed answer text, and LFM2 and Kimi K2 tool calls out of streamed answer text and reasoning.all header when a malformed call falls back to raw text.final and analysis frame text as it arrives instead of at the end of the reply.v0.1.66 (unchanged from 0.15.4).
Runtime pins are unchanged: native v0.7.0-1 uses upstream
v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab; Web LiteRT-LM stays
@litert-lm/core@0.18.0.
Manifest SHA-256:
6a82e16cd850c79ebb8d5f5613dccc9adc42030bf5788bc3cffcda21f19f2f1d.Keep GPT-OSS reasoning and tool frames out of answer text, including native header continuations.
v0.1.66 (unchanged from 0.15.3).
Runtime pins are unchanged: native v0.7.0-1 uses upstream
v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab; Web LiteRT-LM stays
@litert-lm/core@0.18.0.
Manifest SHA-256:
6a82e16cd850c79ebb8d5f5613dccc9adc42030bf5788bc3cffcda21f19f2f1d.Refresh media-embedding examples, migration links and upgrade guidance.
v0.1.66 (unchanged from 0.15.2).
Runtime pins are unchanged: native v0.7.0-1 uses upstream
v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab; Web LiteRT-LM stays
@litert-lm/core@0.18.0.
Manifest SHA-256:
6a82e16cd850c79ebb8d5f5613dccc9adc42030bf5788bc3cffcda21f19f2f1d.Aligned the default WebGPU bridge assets to v0.1.66 to preserve worker timeouts and model-load tuning during CPU recovery. Upstream v0.7.0@620d2225cc4
v0.1.66 to preserve worker
timeouts and model-load tuning during CPU recovery.
Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native
v0.7.0-1; Web LiteRT-LM stays @litert-lm/core@0.18.0.
Manifest SHA-256:
6a82e16cd850c79ebb8d5f5613dccc9adc42030bf5788bc3cffcda21f19f2f1d.v0.2.0-4 and reject
images after recorded Metal failures. Keep qualified older Apple companions
working across patch upgrades with an upgrade warning.Report native embedding overflow with typed limits and input counts.
v0.1.65 (unchanged from 0.15.0) for image/audio
embeddings (API floor v0.1.64+), qualified on Chrome CPU wasm memory64
against native CPU/Metal v0.7.0-1 with upstream
v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab. Web LiteRT-LM remains
@litert-lm/core@0.18.0. Immutable Web asset manifest:
185cfc060fb90b7c8d240b23e7eeb85ef402eb3ef52e8acb1a5590cd5da56328.Embed ordered text, images and audio with typed inputs and modality probes on native llama.cpp, native CPU LiteRT-LM (opt in with liteRtLmEmbeddingEng
liteRtLmEmbeddingEngine) and compatible Web llama.cpp assets.dart build cli bundles, which also affected 0.14.v0.1.65 for image/audio
embeddings (API floor v0.1.64+), qualified on Chrome CPU wasm memory64
against native CPU/Metal v0.7.0-1 with upstream
v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab. Web LiteRT-LM remains
@litert-lm/core@0.18.0. Immutable Web asset manifest:
185cfc060fb90b7c8d240b23e7eeb85ef402eb3ef52e8acb1a5590cd5da56328.LlamaStateException.ABI break: Update the native llama.cpp runtime to v0.7.0 with matching bindings and Apple companions; v0.6.0 and older native overrides are incompatib
v0.1.63, embedding llama.cpp
v0.7.0 and qualified against native v0.7.0, preserving Web/native
v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab parity. Web LiteRT-LM
remains @litert-lm/core@0.18.0. Immutable Web asset manifest:
65d146539eaba034d2c42bfc88a8ff376b937254ea089772401e07cade79f0b9.responseFormat schemas with unsupported or shadowed
assertions now throw LlamaUnsupportedException with a path before generation
instead of being silently unconstrained; supported formats, bounds, allOf
and tuples are now enforced.void Function(double progress)?;
existing Function(double progress)? overrides still compile; prefer void.liteRtLmQwen3ChatTemplate for stable thinking control on renamed
native LiteRT-LM Qwen3 bundles; reject unsupported Web overrides explicitly.Report recursive chat-template failures as LlamaInferenceException instead of StackOverflowError on VM and JavaScript (#1112).
LlamaInferenceException
instead of StackOverflowError on VM and JavaScript
(#1112).finishReason: tool_calls
(#887).enableThinking: false for renamed native LiteRT-LM Qwen 3 bundles
with a canonical Qwen 3 chatTemplate override or identifying metadata
(#901).v0.1.59, unchanged from 0.13.1:
they embed llama.cpp v0.6.0, are qualified against native v0.6.0, and
keep Web/native llama.cpp v0.6.0@d81235049384534c167caea52b85a694f6103d14
parity with native pin v0.6.0-3; Web LiteRT-LM is
@litert-lm/core@0.18.0. Immutable Web asset manifest:
f823b10417b212ff1f66aa1c46bf78af7d9c393497dfcf0fa3a4f77adeb260eb.With llamadart_native_tag or llamadart_native_repository set, the build hook keeps the release's SHA256SUMS beside the cached archive and verifies a c
llamadart_native_tag or llamadart_native_repository set, the build
hook keeps the release's SHA256SUMS beside the cached archive and verifies
a cached archive against it without network access, so offline builds work
again. When it cannot fetch SHA256SUMS, the build fails with
NativeArchiveDigestException naming where to pre-seed it
(#1040).cancelGeneration() during a Web generation now ends the stream normally,
as on native, instead of throwing LlamaInferenceException: on the WebGPU
worker path, which rejects with an AbortError, and on LiteRT-LM Web, whose
pending stream read rejects. Observers report the run as cancelled
(#875).getPerformanceContext() decode and prompt-eval times on a native
runtime without the llamadart exit-teardown exports (v0.5.0 or older),
where an asynchronous GPU backend such as Metal counted its work as sampling
time and decode speed read about 10x too high
(#671).ModelParams.gpuLayers now means all layers on every runtime,
as in llama.cpp, exactly like ModelParams.maxGpuLayers. Native llama.cpp
now also offloads the KV cache and reports the GPU backend instead of
CPU, LiteRT-LM under ComputeDevice.auto uses its GPU backend instead
of the CPU, and the Web applies its GPU safety caps and CPU fallback
(#920).responseFormat JSON schemas the grammar converter cannot represent,
such as an untyped property, a non-local $ref or one nested too deeply for
the stack, as LlamaUnsupportedException, and cyclic or other non-JSON
schemas as LlamaArgumentException, instead of StateError or a stack
overflow. A combined tool or schema grammar that llama.cpp would refuse to
load, such as about 2000 Hermes schema tools, is rejected with
LlamaUnsupportedException before generation instead of a sampler
initialization failure
(#584,
#1093).Conversation roles must alternate user/assistant/...: when a
template that rejects the tool role fails such a request, it is rendered
again with each run of results as one user turn and the earlier calls as
JSON in the model turn
(#885).StateError (generation is already in progress). An overlap with a
running generation that was not cancelled now throws LlamaStateException
(#656).v0.1.59, unchanged from 0.13.0:
they embed llama.cpp v0.6.0, are qualified against native v0.6.0, and
keep Web/native llama.cpp v0.6.0@d81235049384534c167caea52b85a694f6103d14
parity; Web LiteRT-LM is @litert-lm/core@0.18.0. Immutable Web asset
manifest:
f823b10417b212ff1f66aa1c46bf78af7d9c393497dfcf0fa3a4f77adeb260eb.Pair this release with Apple companions llamadart_llama_cpp_flutter, llamadart_litert_lm_flutter and llamadart_stable_diffusion_flutter 0.13.0, which
Pair this release with Apple companions llamadart_llama_cpp_flutter,
llamadart_litert_lm_flutter and llamadart_stable_diffusion_flutter
0.13.0, which link native runtimes v0.6.0-3, v0.18.0-1 and v0.2.0-3,
respectively.
Behavior change: Apple companion versions now share the core
MAJOR.MINOR, starting at 0.13.0, with independent patch versions. Each
companion depends on llamadart within its core minor
('>=0.13.0 <0.14.0'), so pub refuses a mismatched companion and core
instead of the iOS or macOS build failing, and the Apple build error or
warning for a mismatched or older llama.cpp companion names that minor,
such as ^0.13.0. Apps that take llamadart from a path or git checkout
add it under dependency_overrides
(#857).
Behavior change: LiteRT-LM caps each cache directory's GPU program cache
files at 1 GiB by default
(ModelParams.defaultLiteRtLmMaxProgramCacheBytes), pruning least recently
modified files first before each engine create, so an existing oversized
cache is pruned on the first create after upgrading. An explicit
liteRtLmMaxProgramCacheBytes now bounds the directory total instead of
each file; pass ModelParams.unlimitedLiteRtLmProgramCacheBytes (-1) to
keep every file. Engines sharing a cache directory no longer delete the
program caches of a model another one is running, and
liteRtLmCacheDir: ':nocache' no longer creates a directory, which failed
the load on Windows (#552).
Add ToolDefinition.fromJsonSchema for owned, validated tool input schemas,
with explicit errors for unsupported constraints, formats and runtimes
(#1007).
Behavior change: ToolParam tool grammars are stricter: GLM-4.5, Laguna
and Hunyuan V3 integer parameters reject fractions, ToolParam.nullType
accepts only null, and Gemma 2/3 multi-tool grammars require the
{"tool_call": ...} wrapper. Open JSON Schema objects, including
responseFormat schemas, no longer accept a declared property through the
extra-property slot unless the declared names exceed the grammar size budget.
Behavior change: a ToolChoice.required reply cut short by a cancel or
maxTokens ends with stop or length and no tool calls instead of
throwing StateError; other output a PEG parser cannot parse, including in
ChatTemplateEngine.parse, throws LlamaInferenceException
(#890).
Behavior change: on native llama.cpp, a runtime missing an mtmd function,
with no complete fallback mtmd library, fails image and audio requests, and
supportsVision, with LlamaUnsupportedException naming the function and
pinned runtime
(#640).
Behavior change: on Web, SpeechToTextEngine.load,
TextToSpeechEngine.load and LlamaEngine.setModel with a projector report
progress from 0 to 1, as DecisionEngine.load does
(#903).
On macOS x86_64, LiteRT-LM GPU requests fail up front with
LlamaUnsupportedException, and LiteRT-LM is no longer reported or selected
as GPU there (#711).
With llamadart_litert_lm_flutter 0.0.15, the macOS build only warns to
upgrade the companion instead of claiming the app needs the LiteRT-LM build
phase (#1064).
WebGpuLlamaBackend(bridgeScriptUrl: ...) loads the bridge in dart2js
builds (#735).
The image model memory check treats a GPU reporting 0 bytes free, as Vulkan does over budget, as having no free GPU memory: loads warn and, on Linux, are refused above system memory (#980).
LlamaEngine.getPerformanceContext() returns null after a generation
cancelled before it reached the backend, instead of the previous request's
counters (#677).
Clearer errors: a chat template that fails to render names its source
(#820), a
SpeechToTextEngine attached to a disposed LlamaEngine reports the
disposal (#924), and an
ImageGenerationEngine.load SHA-256 mismatch names the main file by
position, not its path or URL
(#907).
v0.1.59, unchanged from 0.12.2:
they embed llama.cpp v0.6.0, are qualified against native v0.6.0, and
keep Web/native llama.cpp v0.6.0@d81235049384534c167caea52b85a694f6103d14
parity; Web LiteRT-LM is @litert-lm/core@0.18.0. Immutable Web asset
manifest:
f823b10417b212ff1f66aa1c46bf78af7d9c393497dfcf0fa3a4f77adeb260eb.Pair this release with Apple companions llamadart_llama_cpp_flutter 0.0.23, llamadart_litert_lm_flutter 0.0.16 and llamadart_stable_diffusion_flutter
Pair this release with Apple companions llamadart_llama_cpp_flutter
0.0.23, llamadart_litert_lm_flutter 0.0.16 and
llamadart_stable_diffusion_flutter 0.0.3, which link native runtimes
v0.6.0-3, v0.18.0-1 and v0.2.0-3, respectively.
The default llama.cpp runtime is now leehack/llamadart-native@v0.6.0-3,
whose Apple frameworks declare their minimum OS version (iOS 16.4, macOS
13.3) (#1037).
Flutter iOS and macOS apps that keep llamadart_llama_cpp_flutter 0.0.22
(native v0.6.0-2, the same binaries) still build, with a warning to depend
on ^0.0.23 for the frameworks' minimum OS version metadata. Any other
mismatched companion still fails the build, and the error names the
companion version to depend on
(#1063).
The default LiteRT-LM runtime is now leehack/litert-lm-native@v0.18.0-1,
whose macOS runtimes are arm64 only. Universal Flutter macOS apps with
llamadart_litert_lm_flutter now launch on Intel Macs and under Rosetta
instead of aborting; LiteRT-LM throws LlamaUnsupportedException on x86_64
macOS (litert-lm-native#72).
The build hook verifies every llama.cpp archive it downloads or reuses from
its cache against a pinned SHA-256, or the release's SHA256SUMS when
llamadart_native_tag or llamadart_native_repository overrides the
release. A mismatched cached archive is downloaded again; a fresh download
that does not match fails the build with NativeArchiveDigestException
(#1015).
A .litertlm bundle in a directory the process cannot write, such as an
iOS app bundle, no longer fails GPU engine creation on iOS and Linux when
liteRtLmCacheDir is unset: the GPU engine reuses caches already there or
runs without them (#976).
Image generation on Metal: ImageModelParams.vaeDirectConvolution: true
throws LlamaUnsupportedException at load on iOS instead of returning
corrupt images (#1044),
and after a failed command buffer generation says the model must be
reloaded and fails later generations on that engine with the same message
(#1045).
A model cache directory that cannot be created, such as an unwritable
ModelLoadOptions.cacheDirectory, throws LlamaModelException naming the
directory instead of a raw FileSystemException
(#894). A download that
fails TLS certificate verification says so in its LlamaModelException,
and TLS certificate and handshake failures are no longer retried
(#978).
A ModelResolver that maps a remote source to a local mirror file now
loads it when the download options set cacheDirectory, credentials or
other download-only options, instead of throwing
LlamaUnsupportedException
(#897).
ModelParams.loras with more than one adapter or a scale other than 1.0
throws LlamaUnsupportedException before any adapter downloads when the
model loads in LiteRT-LM, by backend, format or .litertlm extension
(#891).
The chat example no longer sends its stored Min-P to text to speech when the loaded runtime does not support Min-P (#747).
v0.1.59, unchanged from 0.12.1:
they embed llama.cpp v0.6.0, are qualified against native v0.6.0, and
keep Web/native llama.cpp v0.6.0@d81235049384534c167caea52b85a694f6103d14
parity; Web LiteRT-LM is @litert-lm/core@0.18.0. Immutable Web asset
manifest:
f823b10417b212ff1f66aa1c46bf78af7d9c393497dfcf0fa3a4f77adeb260eb.Pair this release with Apple companions llamadart_llama_cpp_flutter 0.0.22, llamadart_litert_lm_flutter 0.0.15 and llamadart_stable_diffusion_flutter
Pair this release with Apple companions llamadart_llama_cpp_flutter
0.0.22, llamadart_litert_lm_flutter 0.0.15 and
llamadart_stable_diffusion_flutter 0.0.3, which link native runtimes
v0.6.0-2, v0.18.0 and v0.2.0-3, respectively.
Flutter macOS apps with llamadart_litert_lm_flutter now load LiteRT-LM on
CPU and GPU from their own bundle, so sandboxed, copied and distributed apps
work without a build phase; pair it with companion 0.0.15, which embeds
the macOS runtime
(#1029).
v0.1.59, unchanged from 0.12.0:
they embed llama.cpp v0.6.0, are qualified against native v0.6.0, and
keep Web/native llama.cpp v0.6.0@d81235049384534c167caea52b85a694f6103d14
parity; Web LiteRT-LM is @litert-lm/core@0.18.0. Immutable Web asset
manifest:
f823b10417b212ff1f66aa1c46bf78af7d9c393497dfcf0fa3a4f77adeb260eb.Pair this release with Apple companions llamadart_llama_cpp_flutter 0.0.22, llamadart_litert_lm_flutter 0.0.14 and llamadart_stable_diffusion_flutter
Pair this release with Apple companions llamadart_llama_cpp_flutter
0.0.22, llamadart_litert_lm_flutter 0.0.14 and
llamadart_stable_diffusion_flutter 0.0.3, which link native runtimes
v0.6.0-2, v0.18.0 and v0.2.0-3, respectively.
Behavior change: when the app, or any member of its pub workspace,
depends on llamadart together with flutter_edge_ai_llama or
flutter_edge_ai_litertlm, the native build hook bundles llama.cpp only
unless llamadart_native_runtimes names litert_lm, and fails the build
when litert_lm is named next to flutter_edge_ai_litertlm, which ships the
same libraries (Alongside Flutter Edge AI).
On an iOS simulator, GpuBackend.auto now runs llama.cpp on the CPU, because Metal returns incorrect output there (#950). An explicit GpuBackend.metal or ComputeDevice.gpu still uses Metal and logs a warning; iOS devices and macOS are unchanged.
Android validation builds can retain bounded, identity-checked reports in app-private storage.
Clarify that native KV-state restoration does not guarantee identical next-token scores or continuation.
Portable validation launchers can use captured native logs for required GPU qualification.
Adopt native runtime v0.6.0-2 and image runtime v0.2.0-3 for cooperative application shutdown.
Shut down the Linux chat app’s Flutter engines before process exit, including active model work.
Wait for llama.cpp and image-generation workers to exit before disposal completes, including concurrent disposal calls.
Exclude Android Adreno 750 driver 2150604839 from llama.cpp Vulkan loads: automatic selection falls back to CPU with a warning, and an explicit GPU request throws instead of crashing or producing incorrect text.
Update the default LiteRT-LM runtime to
leehack/litert-lm-native@v0.18.0 (upstream LiteRT-LM v0.18.0). On the
macOS GPU backend the program cache of Qwen3 0.6B and Gemma 4 E2B no longer
grows on every engine create; Qwen3.5 0.8B's still does
(#552), and on the GPU
backend that bundle can still return the prompt instead of an answer on
later requests to one engine, as in 0.11
(#672). The Windows
bundle adds webgpu_dawn.dll, which the GPU accelerator now loads.
Behavior change: with LiteRT-LM v0.18.0, a ModelParams.chatTemplate
on a Qwen 3 .litertlm text bundle receives each message's content as a
list of {"type": "text", "text": ...} parts instead of a string. A
template that concatenates content as a string fails the generation, and
one that interpolates it renders the list into the prompt; read both shapes
(Chat templates).
The built-in templates are unaffected.
Qwen3 0.6B on the LiteRT-LM GPU of an Adreno 750 (Galaxy S24): with
v0.18.0 GPU engine creation fails there, where v0.17.0-8 generated wrong
text. The first use throws LlamaUnsupportedException under
ComputeDevice.gpu and LlamaModelException under auto. Load that model
with ComputeDevice.cpu, which works on the device; Gemma 4 E2B runs on
that GPU (#553).
Behavior change: a LiteRT-LM engine the runtime cannot create under
ComputeDevice.auto or ComputeDevice.cpu now throws LlamaModelException
on first use. LlamaEngine.create, generate and complete threw
LlamaInferenceException there, so a catch of that type no longer sees
this failure; ChatSession, tokenize, detokenize, getTokenCount and
chatTemplate with a token count threw a StateError. With v0.18.0 that
includes a liteRtLmCacheDir the runtime cannot write, on the CPU backend
too.
Fix a LiteRT-LM bundle named without .litertlm failing engine creation
when the process cannot write the bundle's directory and liteRtLmCacheDir
is unset, as on Linux: it now runs without runtime caches
(#973).
LiteRT-LM release sync accepts runtimes built on upstream v0.17.1 and
v0.18.0, and names the upstream version when its prebuilt-override table
is behind the runtime owner's
(#586).
Move the documented Web LiteRT-LM default from @litert-lm/core@0.15.0 to
@litert-lm/core@0.18.0. An app still chooses the module through
window.__llamadartLiteRtLmModuleUrl; the chat app's web/index.html and
the docs now name 0.18.0.
Adopt leehack/stable-diffusion-native@v0.2.0-3 for the opt-in
stable_diffusion image runtime.
Check the memory of Vulkan GPUs before loading an image model
(stable-diffusion-native#9).
On Linux ImageGenerationEngine.load refuses, with LlamaModelException,
only a model that does not fit the discrete GPUs plus system memory. A
model larger than the memory of the GPU that computes still loads, with
one warning, and runs slower; on Windows it is warned about and never
refused. Not validated on a physical Vulkan GPU yet;
ImageModelParams(checkMemory: false) skips the check and the warning.
Quote the errors stable-diffusion.cpp logged in the LlamaModelException
of an image model it cannot load, with files named by role
(stable-diffusion-native#3).
Send the image runtime's messages to the LlamaLogging.configure handler
at the configured levels, after each load and generation. The runtime no
longer prints ggml's device messages to stderr: at the default level
none it logs nothing.
Update the default llama.cpp runtime to
leehack/llamadart-native@v0.6.0-2 (llama.cpp v0.6.0), and the
llamadart_llama_cpp_flutter Apple SwiftPM pin with it.
Behavior change: a llama.cpp state file saved by llamadart 0.11.x or
earlier no longer loads: llama.cpp v0.6.0 changed the session format.
On native, stateLoadFile throws LlamaStateException naming the file's
format version and the one the runtime reads; evaluate the prompt again and
save a new file (Save and restore prompt state).
Fix a C++ exception inside llama.cpp ending the process on native. The
failed call now throws the typed LlamaException of its operation with
llama.cpp's message, such as LlamaInferenceException from detokenize
for a token outside the vocabulary, or from a lazy-grammar trigger pattern
that is not a regular expression. A context the failed call left undefined
then throws LlamaStateException until its model is loaded again
(llamadart-native#102).
Android llama.cpp Vulkan: the 8-token text-prompt decode cap now applies
only on a GPU whose subgroup size llama.cpp v0.6.0 mis-tiles (16, or below
8) or cannot be read. Other GPUs decode a text prompt in full micro-batches
(#948).
llama.cpp Vulkan needs Vulkan 1.2 from the loader and the GPU driver. When
the Vulkan GPU a load would run on reports less, ComputeDevice.gpu now
throws LlamaUnsupportedException naming the GPU and both versions, and
ComputeDevice.auto loads the model on the CPU with a warning, instead of
starting a driver llama.cpp cannot use; a load with other usable GPUs
leaves that one out
(#782).
v0.1.59: they embed llama.cpp
v0.6.0, are qualified against native v0.6.0, and keep Web/native
llama.cpp v0.6.0@d81235049384534c167caea52b85a694f6103d14 parity; Web
LiteRT-LM is @litert-lm/core@0.18.0. Immutable Web asset manifest:
f823b10417b212ff1f66aa1c46bf78af7d9c393497dfcf0fa3a4f77adeb260eb.llama-server does. LlamaEngine.create, ChatSession and
ChatTemplateEngine.parse trimmed it in every chat format except Gemma 4,
so the text before a caller stop sequence lost a trailing space or newline:
alpha before the stop cedar17 came back as alpha
(#951). A reply the model
ends with a newline now keeps it, as does a JSON-mode reply where the
grammar allows whitespace after the value (createStructuredJson and
LlamaStructuredOutput.parse still decode it); trim replies you compare
exactly. ChatSession and its tool loop store assistant turns in history
without that whitespace, so the next prompt is unchanged.LlamaInferenceException naming the
ModelParams.microBatchSize it needs. With default batch sizes that is an
image above 512 tokens. A context whose batch equals its micro-batch still
decodes such an image in several batches, and now logs a warning that this
can reduce accuracy
(#958).all in llamadart_native_runtimes now includes the
opt-in stable_diffusion image runtime, so llamadart_native_runtimes: all
ships GGUF, .litertlm and image generation together. An app that already
sets all, or a list containing it, downloads 14.7 to 15.6 MB more per
target, or 25.8 to 27.8 MB for the Vulkan build Linux and Windows use by
default, and grows by about 37 MB as measured on macOS arm64 and Android
arm64. android-x64 and Windows arm64 publish no image runtime and skip it
with a warning. A Flutter iOS or macOS app skips it with a build warning
unless llamadart_stable_diffusion_flutter is a dependency; a pub
workspace member, whose user-defines live in the workspace root pubspec, is
not recognised as one and bundles it. Leaving the key unset, or setting
both, still bundles only llama_cpp and litert_lm
(#856).llama-server
renders it (#964). For
Qwen3.5 and Qwen2.5-Omni it was attached before the whole prompt, and for
GLM-OCR it was wrapped in the image start and end tokens twice, so prompts
for media turns of these models change. A chat template no longer writes
its own text for a media part: Qwen's Picture N: under add_vision_id,
the Kimi-K2 wrapper, Reka's run of image tokens and Cohere's
<|IMG_PATCH|> are gone, as on llama-server. A placeholder string that
a chat message quotes (<img>, <image>, [IMG], <__media__>) is text
instead of a part. Where the runtime would still read such a string as a
part, it reaches the model with a zero-width space (U+200B) after its
first character, which the model may repeat in its answer: a quoted
<__media__> in create on native; every placeholder the bridge
rewrites in create on Web; and every placeholder generate reads in the
prompt chatTemplate returns for messages with media and in create on
a custom backend without BackendChatPromptGeneration, which
package:llamadart/backend.dart now exports. A prompt written for
generate keeps its placeholders, and chatTemplate output for messages
without media is unchanged.LlamaUnsupportedException naming the encoder, where
create and generate threw LlamaInferenceException. A chat template
that raises or has invalid syntax throws LlamaInferenceException with the
template's message instead of a raw Exception
(#963).Qwen3.5-0.8B_int8.litertlm builds from before September
2026. With LiteRT-LM v0.18.0 a template that interpolates the content put
a serialized [{"text": ..., "type": "text"}] list into every prompt, so
repeated requests degraded sooner than with v0.17.0-8; one that
concatenates it failed every request; one that only handles a string
rendered empty messages. Native platforms now give such a template the text
again. Not covered: Web, and templates in the runtime's single-turn
is_appending_to_prefill form, for which a warning is logged
(#991, Chat templates).Give a chat template that reads content part lists but rejects image and audio parts the media marker in its text instead of failing in the template,
chatTemplate prompt as the media
marker on any template when the messages hold no image or audio part
(#959). Beside an image
or audio part, a template that is given typed content parts still receives
the video part (#963).ModelParams.microBatchSize is set
(#948). Only text-prompt
decoding is covered: prompts with image or audio input, embeddings, decision
models, text-to-speech and speculative-decoding verification batches are
not, so speculative decoding can still produce wrong output on an affected
GPU.onExitRequested is still
recommended (#826).v0.1.54, unchanged from 0.11.0:
they embed llama.cpp v0.5.0, are qualified against native v0.5.0, and keep
Web/native llama.cpp v0.5.0@7fe450e19305b828c199d602c23a8337aaa1f03b parity
and Web @litert-lm/core@0.15.0. Immutable Web asset manifest:
8a9f83c15035eeb034a6563e6f753382d7d7f9be81503ef76902138da7841176.Disposal now cancels model and LoRA downloads started by deprecated loadModelSource, without waiting for resolution that ignores cancellation (#895).
Fix a Flutter macOS app aborting in ggml-metal when it quits while a
llama.cpp model loads or after a hot restart during a load, and a Dart
program aborting when it ends or kills an isolate during a load: the
llama.cpp runtime now frees the models, contexts, projectors and decision
heads still allocated at exit
(#813,
llamadart-native#96).
A native host's C exit() while a llama.cpp call is running is not covered.
Update the default llama.cpp runtime to leehack/llamadart-native@v0.5.0-2,
a rebuild of llama.cpp v0.5.0 whose Apple XCFramework carries a privacy
manifest (File Timestamp, reason C617.1). A runtime older than v0.5.0-1
keeps the previous exit behavior and logs a warning at the first model load.
Fix a process aborting when a Dart program dies of an error, or a Flutter macOS app quits or hot restarts, while an image model loads or generates, and a macOS Metal process aborting in ggml-metal when a native host exits with an image model loaded: the stable_diffusion runtime now records progress instead of calling back into Dart, and frees the image models still allocated at exit (stable-diffusion-native#10). Image progress events now arrive up to about 50 ms after the runtime reports them. A quit during an image generation waits for the generation to finish; dispose the engine first to quit at once.
Update the stable_diffusion runtime to
leehack/stable-diffusion-native@v0.2.0-1, a rebuild of the same
stable-diffusion.cpp commit whose Apple XCFramework carries a privacy
manifest (File Timestamp, reasons C617.1 and 3B52.1). Image generation
needs this runtime or a later one.
Documented that LiteRT-LM on iOS has run on a device only on iOS 18.3.2; iOS 16.4 remains the declared, untested deployment floor (#831).
Documented Apple privacy manifests: one reaches an app inside a companion
package's XCFramework, never through the hook path, and an app whose
llama.cpp or stable_diffusion runtime has none declares File Timestamp
reason C617.1 (plus 3B52.1 for stable_diffusion) itself.
Flutter iOS and macOS apps pair core 0.11.0 with
llamadart_llama_cpp_flutter 0.0.21, llamadart_litert_lm_flutter
0.0.13 and llamadart_stable_diffusion_flutter 0.0.2, which link the
runtimes this release pins; an older llama.cpp or stable_diffusion companion
fails the Apple build.
Adopt LiteRT-LM v0.17.0-7 with provider-free iOS artifacts; Gemma FST
constrained decoding remains unavailable on iOS.
Update the default LiteRT-LM runtime to leehack/litert-lm-native@v0.17.0-8.
Its iOS SwiftPM frameworks carry Apple privacy manifests (File Timestamp
C617.1 and 3B52.1, System Boot Time 35F9.1, User Defaults CA92.1);
an app that runs LiteRT-LM on the hook path declares them itself.
Fix Android LiteRT-LM GPU engines keeping their graphics memory after deletion, which got an app killed after one or two model reloads on a Galaxy S24 (litert-lm-native#59).
Documented that LiteRT-LM's default GPU selection on Android generates wrong
text for Qwen3 0.6B on Adreno 750; load it with ComputeDevice.cpu
(#553).
Documented that llama.cpp Vulkan on Android is experimental and
device-dependent, with known failures on Pixel 9 Pro, Galaxy A53 and Galaxy
S24; auto stays on the CPU
(#948).
LiteRT-LM release sync can remove an obsolete iOS provider target from modern Swift packages while preserving required macOS runtime libraries.
Reentrant engine disposal now shares one teardown and immediately reports disposed state, including calls from logging or backend cancellation hooks.
Native requests, generation, and speech synthesis fail promptly when their worker exits unexpectedly, allowing disposal to finish.
Native recurrent and hybrid models reject nonzero speculative rollback capacity before context creation, avoiding runtime graph-budget aborts.
Native embeddings reject inputs that exceed a required one-pass micro-batch, avoiding non-causal attention aborts and incorrect MEAN/CLS pooled vectors.
Disposal now cancels model and LoRA downloads started by deprecated
loadModelSource, without waiting for resolution that ignores cancellation
(#895).
Prevented execution of incomplete parallel tool-call replies when generation reaches a reported limit; the tool loop rolls back the whole turn.
Automatic tool loops now reject pinned LiteRT-LM native and Web runtimes before generation because they cannot report token-limit truncation reliably; manually managed completion remains available (#919).
Web GGUF completions preserve runtime limits as length, so tool loops
report truncated and roll back cut-off turns; typed Web speech recognition
raises a runtime truncation error with the partial transcript.
Redacted known URL secrets repeated in model filenames from download cancellation messages.
Model unload, replacement, and disposal now report interrupted tool loops as cancelled, preserving partial answers and rolling back unfinished turns.
Projector loads now reject model changes during loading instead of reporting success after the model has been unloaded.
Fixed: URL-valued ModelSource.path diagnostics redact credentials and
signed queries while preserving the loading path and cache identity
(#846).
Fixed LiteRT-LM speech recognition startup failures hanging indefinitely; failed workers now release their resources and allow another attempt.
Kept cancelled replies out of reset chats and rolled back turns cancelled before any output; structured JSON cancellation now reports a state error. A model change during draft-model resolution also rolls back the turn; resets during context preparation preserve the replacement conversation.
Redacted URL credentials and signed query strings from redirected download failures, download snapshots and invalid model-source errors, including slashless URLs; LiteRT-LM Web model names reject decoded URL delimiters.
LlamaEngine.load(LlamaModel(source, projector:), params:, download:, onProgress:, store:) creates an engine and loads a model with its
projector in one atomic call, and setModel loads or replaces the model of
an engine: the loaded model keeps serving until every new file has
downloaded, except on the Web, where the runtime unloads it before it
fetches the new one. A class that implements LlamaEngine must add
setModel, and an override of loadMultimodalProjectorSource its new
download parameter
(#846).
Deprecated: LlamaEngine.loadModel, loadModelSource,
loadModelFromUrl and loadMultimodalProjector; use LlamaEngine.load or
setModel. loadMultimodalProjectorSource(options:) is now download:.
They still work until 1.0
(#846).
Fixed: a load checks that it can proceed before it downloads:
LlamaEngine.load and setModel reject a projector for a LiteRT-LM model
and ComputeDevice.npu for a GGUF first, and the deprecated
loadModelSource throws for an already loaded engine first
(#846).
Fixed: LlamaEngine.dispose() stops the downloads of a running
setModel at once, and unloadModel() and dispose() stop a
loadMultimodalProjectorSource download instead of waiting for it; the
load throws LlamaStateException
(#895,
#896).
Behavior change: on the Web, a ModelSource.path loads as a URL
relative to the document, or as a blob: URL, for models, projectors, LoRA
adapters and draft models; it used to throw LlamaUnsupportedException
(#846).
Fixed: the llama.cpp WebGPU bridge resolves a relative model,
projector, LoRA or draft model URL against the document, as LiteRT-LM Web
does; its worker resolved one against webgpu_bridge/, so the fetch
failed (#846).
A local ModelSource.path whose file name holds %2F or %5C, or whose
directory is named %2e or %2e%2e, loads through LlamaEngine.load,
setModel, loadMultimodalProjectorSource, setLoraSource,
ModelParams.loras, draft models and the image, decision and speech
engines' load (#846).
Breaking (Preview): ImageGenerationEngine.generate returns
Future<ImageGenerationTask>; await it before reading events or calling
cancel (#850).
Breaking: ImageGenerationTask, SpeechToTextTask and
TextToSpeechTask no longer report a failure as an error on events; read
it from done, or from the new result, which returns the result or
throws the failure, or LlamaStateException when the task is cancelled
(#850).
Fixed: SpeechToTextTask.cancel() stops only that recognition; it no
longer cancels chat and other requests on the same LlamaEngine
(#850).
ModelParams.device (ComputeDevice) selects the device for every
runtime: auto keeps each runtime's default, and an explicit cpu, gpu
or npu runs there or throws LlamaUnsupportedException instead of
falling back to another device
(#849).
Deprecated: ModelParams.liteRtLmBackend, LiteRtLmBackendPreference
and LiteRtLmBackend(preferredBackend:); use ModelParams.device. They
still work until 1.0
(#849).
Behavior change: DecisionModelParams(device: ComputeDevice.gpu) runs
the encoder on a GPU, Vulkan on Android, or throws
LlamaUnsupportedException from the encoder load; it ran on the CPU on
Android and threw before loading where GPU modules load with the model
(#849).
Behavior change: LlamaEngine loads run ModelParams.validate() before
any download or native call, so an invalid combination throws
LlamaArgumentException instead of LlamaModelException
(#849).
Breaking: ModelParams.validate() throws LlamaArgumentException,
ModelDownloadController throws LlamaArgumentException or
LlamaStateException, and Web backend calls before a model load throw
LlamaStateException, instead of ArgumentError or StateError; a failed
load's details is now the cause's message instead of a {type, message}
map, and the not-ready error names LlamaEngine.load and setModel
(#843).
Deprecated, behavior change: sourceLangCode and targetLangCode on
LlamaEngine.create, createStructuredJson, chatTemplate and
BackendNativeChatGeneration.generateChat; pass
chatTemplateKwargs: {'source_lang_code': 'en', 'target_lang_code': 'ko'},
which TranslateGemma templates now read, as llama.cpp does. A custom
BackendNativeChatGeneration now gets the codes from LlamaEngine only in
chatTemplateKwargs
(#853).
Deprecated: LlamaLogging.configure(level:, nativeLevel:, handler:)
replaces LlamaEngine.configureLogging and the engine's setLogLevel,
setDartLogLevel and setNativeLogLevel; levels are now library-wide, so
the last call wins and reaches every running engine's worker, including the
default native backend's (#845).
Deprecated: speech engines follow the shared engine pattern:
SpeechToTextEngine.load(SpeechToTextModel(...)) and
TextToSpeechEngine.load(TextToSpeechModel(...)) download every
ModelSource and own what they load, attach(engine, adapter:) borrows a
loaded LlamaEngine, dispose() cancels the running task, and
transcribeOnce and synthesizeOnce return the final result. Adapters
(Qwen3AsrAdapter, LiteRtLmAsrAdapter, Qwen3TtsAdapter, or your own
SpeechToTextPromptAdapter) replace SpeechToTextModelProfile,
TextToSpeechModelProfile, the modelProfile constructors and
SpeechToTextEngine.liteRtLm, which keep working until 1.0
(#848).
Breaking: SpeechToTextEngine gains dispose(), isDisposed,
adapter and transcribeOnce, and TextToSpeechEngine gains dispose(),
isDisposed, adapter and synthesizeOnce, so a class that implements
either must add them
(#848).
Breaking: DecisionEngine.load(DecisionModel(encoder:, head:, config:), params:, download:, onProgress:) loads a decision model from
ModelSources into an engine it owns, atomically, and dispose() frees it
all; DecisionEngine.attach(engine, head:, config:) adds a head to a loaded
LlamaEngine; decision engines report an instance capabilities. The
String-path load(engine, headPath:, configPath:) is removed; use
attach (#847).
Breaking (Preview): image generation follows the shared engine
pattern: ImageGenerationEngine.load(ImageGenerationModel(source, components: [...]), params:, download:, onProgress:) downloads every
ModelSource into the model cache, with combined progress, cancellation
and cache reuse, and detects each file's role from its header. Generation
settings move to ImageGenerationRequest. download's bearer token and
headers never reach more than one origin: such remote files throw
LlamaArgumentException. The model presets, ImageGenerationModelFamily,
ImageGenerationModelFiles, ImageGenerationDefaults,
ImageGenerationOptions (now ImageModelParams, and the engine's options
getter params), ImageGenerationDevice (now ComputeDevice) and String
paths are removed; MIGRATION.md maps each former preset to its files and
request settings
(#883).
Breaking: LlamaEngine.dispose() is idempotent and terminal, like
every other engine's: each call returns the same future, and afterwards
loads, requests and DecisionEngine.attach throw LlamaStateException
while capabilities reports the engine as disposed. getBackendName,
getAvailableBackends, isGpuSupported, getVramInfo, listGpuDevices
and getResolvedGpuLayers used to answer after dispose() and now throw
LlamaStateException too. A load running when it is called throws
LlamaStateException, and its model is unloaded. LlamaEngine gains
isDisposed, so a class that implements it must add it
(#851).
Breaking (Preview): ImageGenerationEngine.capabilities is async, and
ImageGenerationEngine.runtimeCapabilities() is removed; use
checkRuntime(). ImageGenerationCapabilities and DecisionCapabilities
implement EngineCapabilities
(#851).
Behavior change: LlamaEngine.supportsVision and supportsAudio report
what capabilities reports: true for a LiteRT-LM bundle that takes media
directly, and false instead of throwing when the runtime cannot probe the
projector (#851).
Behavior change: responseFormat maps with an unknown type or
key, such as json_shema or a misspelled schma, now throw
LlamaUnsupportedException before generation instead of generating
unconstrained output; a null-valued key counts as absent
(#836,
#864).
ChatSession.create takes responseFormat, and the new
ChatSession.createStructuredJson decodes the reply. A turn that fails or
is cancelled before its first chunk, such as a strict format on LiteRT-LM,
removes its user message from the history, and one cancelled or failing
mid-stream keeps the partial reply, so alternating-role templates keep
working
(#836,
#864).
LlamaCompletionChunk.model, observer model names and load logs report
llama_model, and web LiteRT-LM omits general.name, when a URL's last
path segment repeats its userinfo credential; web LiteRT-LM
litert_lm.model_url shows a relative URL as given and a blob: or data:
URL as its scheme (#822).
loadModelSource throws LlamaUnsupportedException instead of
ArgumentError for a local path whose file name holds %2F or %5C or
whose directory is named %2e or %2e%2e; loadModel still loads it
(#822).
Behavior change: on Android and iOS, the default model cache is now
llamadart/models in the app's cache directory instead of the temporary
directory, which Android empties on every app update and iOS purges; add
DefaultModelDownloadManager.globalCacheDirectory to move every default
download (#838).
Native LlamaBackend() picks llama.cpp or LiteRT-LM from the model file's
header, not its extension, so extensionless downloads load in the right
runtime and mislabelled files throw LlamaModelFormatException; name a Web
URL's format with ModelSource.url(..., format: ModelFormat.liteRtLm)
(#837).
Add LlamaEngine.runtime and LlamaEngine.capabilities, one snapshot of
what the loaded model's runtime supports: image and audio input,
embeddings, multi-turn chat, tools, structured output, grammars, every
sampling control including penalty, stream batching and speculative
strategies. Native LiteRT-LM reports the image, audio and speculative
decoding support the bundle declares, best-effort (litert-lm-native#60),
and a request the runtime then fails for lack of one throws
LlamaUnsupportedException naming it instead of an opaque error.
backendGenerationCapabilities is deprecated; on native LiteRT-LM it now
reports streamBatching and, for bundles without a declared drafter, no
speculative strategy (#841).
Read completions without choices.first.delta: chunk.text,
chunk.thinking, chunk.toolCalls and a typed chunk.finishReason
(LlamaFinishReason); stream.text(), stream.textDeltas() and
stream.collect() (a LlamaCompletion with assembled tool calls and an
assistant message); and the one-shot engine.complete(messages) and
session.send('...')
(#840).
session.sendWithTools(text, tools: ...) and completeWithTools(parts, ...) run the model's tool calls with each ToolDefinition.handler,
concurrently for parallel calls, until it answers, and return a
LlamaToolLoopResult whose stopReason also reports maxRounds,
unhandled calls, context overflow, a reply cut off at maxTokens
(truncated, rolled back) and cancellation
(#842).
Breaking: ChatSession gains createStructuredJson, and LlamaEngine
gains runtime, capabilities, setLoraSource and removeLoraSource, so
a class that implements either must add them.
Breaking: ToolDefinition.handler is nullable, so tools the app runs
itself can leave it out; code that calls tool.handler(params) must check
it for null first (#842).
GenerationGrammarTrigger.typed(type: GrammarTriggerType.word, ...)
replaces the raw-int constructor, now deprecated; an unknown raw trigger
type throws LlamaUnsupportedException on llama.cpp instead of being
ignored (#844).
LoRA adapters (setLoraSource, removeLoraSource,
LoraAdapterConfig.source), speculative draft models
(SpeculativeDecodingConfig.draftModel, withDraftModel,
withDraftModelDownload) take a ModelSource, so they download and cache
like models, and LiteRtLmAsrRuntimeConfig.source takes local
ModelSource files
(#852).
Deprecated: the String path forms of LoRA adapters, speculative draft
models and LiteRT-LM ASR files
(#852).
Breaking: package:llamadart/llamadart.dart is the app API. The raw
ffigen bindings move to package:llamadart/llama_cpp_bindings.dart (native
only, outside semantic versioning), and the custom-backend SPI moves to the
new package:llamadart/backend.dart: every Backend* type except
BackendPerfContextData and BackendTextToSpeechModel, LiteRtLmBackend,
LiteRtLmRuntimeClient, LiteRtLmRuntimeMetrics, LiteRtLmRuntimeResult,
LiteRtLmAsrRuntimeSession, LiteRtLmAsrPushResult,
LiteRtLmAsrProcessResult and LiteRtLmAsrProcessState. The app API now
exports TemplateToolCallSerialization
(#355).
Breaking: the LlamaEngine text-to-speech and decision hooks,
modelHandle and contextHandle move to the LlamaEngineBackendHooks
extension in package:llamadart/backend.dart, so neither a subclass nor an
implements LlamaEngine fake can override them; fake a backend that
implements BackendTextToSpeech or BackendDecision instead
(#355).
Breaking: the deprecated LiteRtLmBenchmarkClient,
LiteRtLmBenchmarkMetrics, LiteRtLmBenchmarkResult,
LiteRtLmRuntimeClient.conversationTokenCount and
LiteRtLmRuntimeClient.replaceConversationWithClone are removed
(#355).
v0.1.54, unchanged from 0.10.0:
they embed llama.cpp v0.5.0, are qualified against native v0.5.0, and keep
Web/native llama.cpp v0.5.0@7fe450e19305b828c199d602c23a8337aaa1f03b parity
and Web @litert-lm/core@0.15.0. Immutable Web asset manifest:
8a9f83c15035eeb034a6563e6f753382d7d7f9be81503ef76902138da7841176.Add the llamadart_stable_diffusion_flutter 0.0.1 companion package: Flutter iOS and macOS apps that add it link the image generation runtime through S
llamadart_stable_diffusion_flutter 0.0.1 companion package:
Flutter iOS and macOS apps that add it link the image generation runtime
through Swift Package Manager, since App Store Connect rejects the iOS
framework the build hook bundles. Without it the hook keeps bundling the
runtime, and an iOS build reports an Xcode build warning (shown by Xcode and
xcodebuild, not by plain flutter build or flutter run output).leehack/stable-diffusion-native@v0.2.0.ModelParams.chatTemplate, which engine.create and
engine.chatTemplate ignored in favor of the GGUF template
(#710).LlamaBackend.applyChatTemplate now throws
LlamaUnsupportedException for a template override it cannot render
(#710).qwen.gguf, in LlamaCompletionChunk.model instead of
the full local path or redacted URL; compare it with the file name, not
the path. Like LlamaOperation.model, it now reads local paths as paths
and leaves out data: and blob: URLs
(#718).litert_lm.model_url
metadata that web LiteRT-LM reports.%, such as
C:\models\qwen 100%.gguf, or whose URL file name decodes to one, instead
of throwing ArgumentError; loadModelSource and ModelCacheEntry keep
% in local and cache paths literal instead of percent-decoding them into
another file (#819).main returns or throws
with a model, decision head or image model still loaded; llamadart now
frees them as the program ends, and an undisposed ImageGenerationEngine
no longer keeps the program running
(#613).AppLifecycleListener.onExitRequested,
since quitting a macOS app with a Metal model loaded aborts the process.dart run skills@ get.ModelParams.loras at model load on native
llama.cpp and WebGPU, where they were silently ignored; an adapter that
cannot be applied, or WebGPU bridge assets before v0.1.54, fail the load
instead (#709).LlamaUnsupportedException instead of
LlamaModelException when native or web LiteRT-LM rejects a ModelParams
field, including more than one LoRA adapter or a non-default adapter
scale.stable_diffusion native runtime
(stable-diffusion.cpp) to llamadart_native_runtimes for image
generation; it is never bundled by default or by all, and
llamadart_stable_diffusion_backends picks its CPU or Vulkan build on Linux
and Windows (#777).ImageGenerationEngine, on
the opt-in stable_diffusion runtime: SDXS and SD-Turbo presets,
phase-labelled progress, cancellation, PNG output, a memory check before
loading, and an output size that defaults to the model's
(ImageGenerationDefaults.width and height); not available on the web
(#778).ImageGenerationEngine.warmUp, which compiles the GPU pipelines of
the first image ahead of time, and document first-image latency per
platform (#790).ImageGenerationEngine.checkRuntime, which probes the image runtime
without blocking the calling isolate; load now probes the same way, and
the chat example uses it, so the first probe's Metal library compile (about
16 s with an empty shader cache) no longer freezes the UI
(#798).LlamaUnsupportedException for image or audio parts
sent to a GGUF model with no projector loaded: native llama.cpp answered
from the text alone, and WebGPU threw an untyped error surfaced as
LlamaInferenceException. Native llama.cpp and LiteRT-LM also throw it
for LlamaImageContent.url.LlamaEngine.supportsEmbeddings. Native llama.cpp embeddings of an
encoder-decoder model now throw LlamaUnsupportedException, and input
longer than the context throws LlamaInferenceException, instead of a
plain Exception.llm
text-encoder file for Z-Image and Qwen-Image, sampler, scheduler and flow
shift on the request and model defaults, and flash attention and direct
VAE convolutions, which now turn on automatically where measured faster or
smaller, such as a 1024x1024 Vulkan decode in about 1 s instead of up to
56 s in stable-diffusion.cpp's native CLI
(#802).MemAvailable and
half of physical memory less the app's own memory, so SD-Turbo loads on
8 GB phones whose MemAvailable leaves out memory the system frees on
demand; 6 GB phones still refuse it in most cases
(#792).warmUp leaves the size unset, and the basic
example's image CLI downloads them
(#802).v0.1.54, unchanged from 0.9.0:
they embed llama.cpp v0.5.0, are qualified against native v0.5.0, and keep
Web/native llama.cpp v0.5.0@7fe450e19305b828c199d602c23a8337aaa1f03b parity
and Web @litert-lm/core@0.15.0. Immutable Web asset manifest:
8a9f83c15035eeb034a6563e6f753382d7d7f9be81503ef76902138da7841176.Document generic JSON tool calling as an intentional fallback, including its prompt and model-reliability limits; runtime behavior is unchanged (#755)
contextSize using smaller processing batches, while preserving
full-context defaults for unknown models, including embedding models
(#752).LlamaModelException with its real cause, not as a COOP/COEP
worker-thread error; only a real worker-thread failure still names COOP/COEP
(#753).preferredBackend GPU module is not
bundled and the model loads on CPU instead, as happens for cuda with the
default Windows bundle; the native runtime docs now say when CUDA is bundled
(#756).llamadart_server example exiting at startup on Windows; it stops
on Ctrl+C there, and on SIGINT or SIGTERM elsewhere
(#757).presencePenalty, minP and thinkingBudget, and runtime LoRA
adapters (setLora, removeLora, clearLoras), on WebGPU with bridge
assets whose capability probes report them; other assets still reject them
(#722).backendGenerationCapabilities.speculativeDecodingStrategies; other assets
still reject it (#722).GenerationParams.minP on WebGPU when the bridge lacks
Min-P, with LlamaUnsupportedException instead of ignoring it, and ignore a
stop sequence equal to a preservedTokens entry there, as native llama.cpp
does (#661).LlamaEngine.backendGenerationCapabilities, which reports whether the
loaded runtime applies presencePenalty, minP and thinkingBudget; the
example chat app uses it to send Min-P and enable its slider only where
supported (#661).\n and \r in Qwen3-Coder XML reasoning, as llama.cpp does
with the Qwen3.5 template; before, they became line breaks unless a tool
call ended the thought
(#743).ChatSession history
no longer start with the blank lines after </think>. Only whitespace at
either end, and text that may be a tag or tool-call opening, waits for more
output, so reasoning still streams token by token. Without tools, Hermes,
DeepSeek R1, Qwen3-Coder XML and the other formats the template engine
guide lists also drop a start tag repeated at the start of a forced-open
thought, as the parse does. The guide lists the exceptions
(#754).ChatSession history. Content and reasoning are
trimmed as for Hermes, and text after a call arrives at the end of the
stream. See the template engine guide for the formats and exceptions
(#732).<tool_call> envelope into
streamed content or ChatSession history; only a possible envelope opening
and trailing whitespace wait for more output. With tools, streamed content
is now trimmed as the parse trims it, and text the parse keeps after a tool
call, including a malformed envelope, arrives at the end of the stream.
Streamed reasoning, and so ChatSession thinking, is trimmed per thought as
the parse trims it. The exception is a forced-open thought that never
closes and starts with whitespace: the parse keeps it untrimmed, but it
streams without its leading and trailing whitespace, so
" \n Hello there. \n\n" streams as "Hello there.". Before, it streamed
as the parse gives it, except for some thoughts containing a backslash,
depending on chunking. After a forced-open
thought, text after a tool call arrives at the end
(#701).LlamaModelException when a WebGPU model load fails with a bridge
error that has no specific mapping, and LlamaInferenceException or
LlamaStateException for such Web embedding, next-token scoring and state
errors, with URL credentials and signed query values redacted from the
details and the load-failure console log
(#704).LlamaEngine model and projector load errors and logs and the model field
of completion chunks for every URL form, including scheme-relative
//user:pass@host/... URLs and relative paths with a query, and out of
native model download errors. A projector load error that is not a
LlamaException now throws LlamaModelException. The details of a
model or projector load failure is now a {type, message} map instead of
the original error, and a native download that fails with a network error
carries the error text as a String in details
(#704).DecisionEngine.load throw LlamaStateException when another model is
loaded while it runs, even under the same backend handle
(#626).phys_footprint on macOS and iOS, RssAnon plus RssShmem
plus VmSwap on Linux and Android, read after malloc_trim(0) where the C
library provides it (glibc, not Android), and PrivateUsage plus
SharedCommitUsage on Windows (PrivateUsage alone on builds without it).
Evicting file-backed pages, such as the
mmapped weights, compressing memory under pressure, or glibc keeping freed
memory across reloads no longer fails peak_memory_bound without memory
growth, and each report names its counter
(#633,
#762).leak_slope_bound when the least-squares footprint
slope over cleanup cycles 1-8 exceeds 7 MiB per cycle; it failed only when
every cycle grew by more than 7 MiB, and passed leaks of 16 MiB per reload
(#762).<|tool_list_start|>, as llama.cpp does, so LFM2.5-1.2B-Instruct and
LFM2.5-1.2B-Thinking tool prompts drop the stray "Respond in JSON format"
instruction (#716).v0.1.54, on the final create chunk and to observers
(#696).null in chat templates, as llama.cpp does: QwQ-32B
renders them instead of throwing, and LFM2 and Devstral prompts drop a stray
null or <function text>
(#715).dinja 1.2.0, so more chat prompts match llama.cpp: tojson
output such as tool declarations uses llama.cpp's spacing, number format
and non-ASCII text; Qwen3-Coder, GLM-4.6, GLM-4.7-Flash, MiniMax-M2,
Nemotron-3-Nano, Command R7B, Cohere2 MoE and Ling 3.0 prompts lose stray
indentation; Functionary v3.1 adds no tool instructions without tools;
Granite 3.3 spells out the month in its date; Hunyuan Hy3 keeps the system
prompt first instead of merging it into the user turn; and Bielik 11B v3
tool-call turns without text render instead of throwing.getPerformanceContext() evalTokens and sampleCount without speculative
decoding, as the speculative path already did
(#706).create chunk as
LlamaCompletionChunk.usage on native llama.cpp
(#696).LlamaEngine(observers: ...), which reports chat and text completions,
embeddings and model loads, with their usage, to tracing and metrics code
(#696).leehack/llamadart-native@v0.5.0 (llama.cpp v0.5.0) with Apple companion
0.0.20, regenerated matching Dart FFI bindings, refreshed the llamadart_llama_cpp_flutter Apple SwiftPM
checksum, and aligned current README/website native override docs.Add LlamaEngine.scoreNextToken(...) for next-token log-probabilities on
native llama.cpp and WebGPU bridge assets v0.1.52+, matching llama-server
n_probs; check
supportsNextTokenScoring first
(#694).
Add example/laya_command_bar, a Flutter text field that reshapes into a
reminder, message, calculation or other command as you type, read by a
Laya decision model, by EmbeddingGemma and labelled examples, or by small
LLMs' next-token scores, including the decision model decider-2b.
Throw LlamaModelException when native llama.cpp cannot find or load a
multimodal projector, and LlamaUnsupportedException when the runtime lacks
the mtmd functions; LlamaEngine.supportsAudio also throws the latter.
Speech-to-text capabilities now say when no projector is loaded, using the
new LlamaEngine.hasMultimodalProjector
(#325).
Honour LlamaEngine.cancelGeneration() issued right after listening to a
create, generate or ChatSession.create stream, before it reaches the
backend, instead of running the whole generation
(#602).
Cancel an active text-to-speech synthesis on LlamaEngine.unloadModel() and
dispose() instead of waiting for it to finish
(#628).
Cancel an active Qwen3-ASR transcription on LlamaEngine.unloadModel() and
dispose() instead of completing it with the transcript cut at the unload
(#670).
Send LiteRT-LM tool calls and tool results in the runtime's own message format, so Gemma 4 reads tool output and Qwen3 tool histories no longer fail (#681).
Start a native llama.cpp generation requested right after a cancel once the
cancelled run stops, instead of failing with generation is already in progress. An overlap with a running generation that was not cancelled now
throws LlamaStateException
(#655).
Render Qwen3 prompts as llama.cpp does: an earlier assistant tool-call
turn without reasoning no longer gets an empty <think> block
(#691).
Require dinja 1.1.0. Its Jinja string comparison makes three more chat
templates render as llama.cpp does: MiniMax-M1 adds no empty
system block for an empty or whitespace-only system message; NVIDIA
Nemotron Nano v2 drops the blank line before a tool call, the blank lines
before its tool instructions when tools come with an empty or
whitespace-only system message, and an empty final assistant turn without
a generation prompt; and Functionary v3.2 tool declarations drop stray
// Format=<|NONE|> lines and spell out nested object parameters
(#351).
Cancel a generation's backend run as soon as its stream subscription is
cancelled, instead of at its next token, which during prompt evaluation
meant after the whole prompt. A native llama.cpp generation requested right
after such a cancel now waits for it instead of throwing
LlamaStateException, and native llama.cpp sees a cancel between text
prompt micro-batches (ModelParams.microBatchSize, 512 tokens by default)
or, with speculative decoding, between batches (ModelParams.batchSize)
(#663,
#660).
Render every result of a tool message holding several
LlamaToolResultContent parts, as one tool message per result like
llama.cpp, instead of only the first; LlamaChatMessage.toJson lists them
all (#683).
Render Gemma 4 tool calls and tool results as llama.cpp does, so Gemma 4 GGUF models can read tool output (#669).
Stop a Qwen3-TTS audio decode at its next chunk boundary when native text-to-speech is cancelled, instead of finishing the native step in progress first. This needs llamadart-native v0.4.1-1 or later; older runtimes keep the previous behaviour (llamadart-native#86, #322).
Add an experimental DecisionEngine for Laya-style decision models (a
ModernBERT encoder GGUF plus a safetensors head) on native llama.cpp, with
typed ChoiceKey, ScoreKey and NoulKey questions
(#604).
Add example/basic_app/bin/llamadart_decision_example.dart, a console demo
that triages a support ticket with DecisionEngine
(#604).
Add example/laya_tetris, a Flutter app in which a Laya decision model
plays real-time Tetris through DecisionEngine
(#604).
Run example/laya_tetris on Web through the WebGPU bridge, with a live demo
at https://leehack-flutter-laya-tetris.static.hf.space.
Add a notebook in example/laya_tetris/training/ that fine-tunes a Laya
decision head for the Tetris example and exports it for DecisionEngine
(#604).
Run DecisionEngine on WebGPU through the bridge decision API
(apiVersion 1), which bridge assets v0.1.47+ include
(#604).
Stop native image and audio requests from seeding the repeat penalty with leftover memory, which made output depend on the previous request (#603).
After a failed native prompt decode, the next reusePromptPrefix request no
longer runs on the wrong KV cache or keeps failing
(#601).
Reject embed() and embedBatch() on rank-pooled reranker GGUFs with
LlamaUnsupportedException on native, instead of returning memory read
past llama.cpp's classifier-score buffer
(#583).
Throw LlamaInferenceException from native embed() and embedBatch()
when input to an encoder-only model or a model without a KV cache (such as
BERT-family and ModernBERT GGUFs) does not fit one microBatchSize pass,
instead of aborting the process or embedding only the last chunk
(#607).
v0.1.54 for the decision API,
next-token scoring, presence penalty, Min-P, thinking budgets, runtime LoRA
adapters, speculative decoding and the Web runtime fixes below; the bridge
also adds its supportsCompletionUsage flag, which llamadart does not use yet
(#729). The assets embed llama.cpp v0.5.0, are
qualified against native v0.5.0, and keep Web/native llama.cpp
v0.5.0@7fe450e19305b828c199d602c23a8337aaa1f03b parity and Web
@litert-lm/core@0.15.0. Immutable Web asset manifest:
8a9f83c15035eeb034a6563e6f753382d7d7f9be81503ef76902138da7841176.LlamaInferenceException whose details contain (invalid grammar), and the
loaded model stays usable, instead of aborting the WebGPU bridge runtime
(llama-web-bridge#125).TypeError. Later calls fail with
No model loaded. Call loadModelFromUrl first.; call unloadModel(), then
loadModel() and any projector again to recover
(llama-web-bridge#123,
llama-web-bridge#127).Grammar rejected every candidate token only
when the grammar cannot continue
(llama-web-bridge#118).jfk.wav. Every GGUF
STT pack run executes the four checks and each one gates functional_pass;
the LiteRT-ASR and TTS packs pass no edge fixtures
(#325).stt, tts and litert-asr speech validation pack run from 8
to 15 lifecycle checks: an immediate cancel, three
cancel/dispose/load/generate cycles, and bounds that fail the run when a
cancellation takes over 500 ms to end its task or the peak resident set
exceeds 1.10x the one sampled after the first generation
(#594).ToolChoice.auto on a prompt that
needs no tool, and run the tool cases on the GGUF chat profiles
(#602,
#655,
#654).decision-gguf-{cpu,metal,vulkan,cuda,webgpu} validation profiles that
check DecisionEngine token ids, raw logits and answers against the Laya
0.3.5 reference, plus batching, reload and typed rejections, on desktop,
mobile, Web WebGPU and GCE CUDA
(#604).chat-gguf-webgpu validation profile, hash Web validation
models while they stream so GGUFs over 2 GiB pass preparation, list every
bundled profile in the validation app, and verify iOS GGUF GPU placement
from the XCTest console log.tts unloads and disposes the engine
during a synthesis, cancels one during its audio decode and bounds the
resident set those checks add, and stt must fail with
LlamaSpeechTranscriptTruncatedException at maxOutputTokens and at the
context size. stt runs now execute 28 checks and tts runs 25
(#628,
#636,
#322).topK: 1 for zero-temperature LiteRT-LM Web generation, matching
the native clamp
(#548).Model … loaded from …; native engine creation is deferred until the first generation or tokenizer call instead of loaded successfully when
the native LiteRT-LM backend finishes loadModel, since it creates the
engine lazily
(#569).dxcompiler.dll and dxil.dll in the Windows x64 LiteRT-LM runtime
cache and desktop validation bundle checks, matching the hook's v0.17.0-6
inventory. The runtime does not preload them: Dawn's D3D12 backend loads the
pair at GPU engine creation
(#570).libgomp.so.1; libgomp1 on Ubuntu/Debian, libgomp on Fedora and Arch)
and that Linux LiteRT-LM GPU needs a hardware Vulkan ICD: with only Mesa
llvmpipe the runtime segfaults after model load instead of failing cleanly
(llamadart-native#82,
#572).ggml/wrapper symbol is missing from
both the primary FFI asset and every fallback library. Candidates are named
by asset URI or file name only and loader errors are classified, never
quoted, so no directory or loader search path reaches the diagnostic
(#416).LlamaEngine.configureLogging handler. A worker takes the Dart logger level
when it starts and LlamaEngine.setDartLogLevel/setLogLevel update a
running worker; the default none sends nothing and debug records are
capped at 1000 per worker. The LiteRT-LM program-cache pruning warnings are
now ordinary warn records gated by that level instead of the native log
level. Adds LlamaLogger.level and the BackendDartLogLevel capability
(#567).Bearer <token> and the value in token=, key=,
secret=, password=, api_key= and apikey=<value> outside HTTP URLs in
native startup diagnostics with <redacted-secret>; URL and
control-character handling is unchanged
(#551).hook/build.dart into
lib/src/hook/native_release_pins.dart, the only file the pin sync now
rewrites.ModelParams.liteRtLmCacheDir to choose the native LiteRT-LM runtime
cache directory and opt-in ModelParams.liteRtLmMaxProgramCacheBytes, which
deletes *_mldrift_program_cache.bin files above the cap before each engine
create and logs a warning per deleted file. Defaults are unchanged: the same
per-platform directory and no pruning
(#552).topK: 1 for zero-temperature
LiteRtLmRuntimeClient.createConversation calls, which returned incoherent
text on the LiteRT WebGPU sampler with the default top-k.startupDiagnostics=[...] suffix overflows: teardown entries, now prefixed
teardown: , are dropped first, duplicates are recorded once, entries are
capped at 2048 characters, and each omitted run renders as ...
(#415).Failed to preload Windows backend module startup diagnostic per
missing candidate
(#550).promptTemplate on the non-native
LiteRtLmRuntimeClient.createConversation placeholder, so callers passing
it compile for Web as they do on native
(#549).TemplateCaps.detect results in a per-isolate LRU keyed by exact
template source and bounded at 16 entries, so repeated chat-template renders
skip both Jinja parses and all four capability probes. Detections in which
any analysis step failed are not cached and keep logging on every call
(#448).supportsTools and supportsToolCalls for chat templates that
reject two tool calls in one assistant message (Llama 3.2) or a user turn
directly after a tool call (Ministral 3). The tools capability probe now
renders a single tool call, and a separate parallel probe clears only
supportsParallelToolCalls when it throws
(#557).[TOOL_CALLS] block unless
parallel tool calls are enabled; it previously always allowed repeats while
the parser kept only the first call
(#559).pattern keyword when generating GBNF, for anchored
patterns built from literals, positive character classes, (...) groups
nested at most 32 deep, grouped alternation and */+/?/{m,n}
repetition; any other pattern, including deeper nesting, falls back to the
rule the schema would have produced without it, so minLength and
maxLength still apply there. An applied pattern replaces
minLength/maxLength as it does in llama.cpp, so a schema carrying both
pattern and maxLength is no longer length-bounded. A schema carrying
pattern but no explicit type now yields a string rule instead of
throwing Unrecognized schema. Mistral Nemo and Magistral tool-call ids are
now grammar-constrained to exactly nine alphanumerics
(#582).SpeechToTextEngine now fails a native Qwen3-ASR transcript that reaches
the context size or maxOutputTokens with
LlamaSpeechTranscriptTruncatedException instead of completing with
truncated text, and create() reports finishReason: 'length' when native
llama.cpp stops at either limit. The chat app caps Qwen3-ASR recordings at
the validated 30 seconds
(#636).ToolChoice.auto on WebGPU from forcing a tool call: it now skips
the lazy tool-call grammar and parses tool calls best-effort. WebGPU
rejects GenerationParams.grammarLazy and a non-root grammarRoot with
LlamaUnsupportedException; backends report this through the new
BackendLazyGrammarSupport
(#654).libcudart.so.12, libcublas.so.12)
that the Linux cuda backend needs on the default loader path; llamadart
does not ship them, and validation bundles refuse LD_LIBRARY_PATH
(#587).packages/llamadart_validation before
building the report, instead of reporting FAILED with a null error on a
fresh checkout. A failed report step is now the run's error, with its exit
code and a redacted stderr tail
(#688).GpuBackend.metal or GpuBackend.hip
on llama.cpp: they looked up ggml registries named Metal and HIP, but
ggml names them MTL and ROCm, so loading fell back to automatic device
selection. A HIP load on a ROCm build now reports its backend as HIP
instead of CPU
(#611).error 138 as the documented
cross-origin isolation (COOP/COEP) UnsupportedError, as
thread constructor failed already was, instead of rethrowing the raw
bridge error
(#598).LlamaModelException when WebGPU cannot fetch or load a multimodal
projector, instead of the raw JavaScript error. Its details drop these
parts of the projector URL the app passed, as written, JSON-escaped,
percent-encoded or percent-decoded: the userinfo and password, as whole
tokens of any length; the ?query and #fragment, where they directly
follow a non-space character; the query and each &-separated part that
contain =, as whole tokens; and bare query values and the fragment of 10
or more characters, as whole tokens. A whole token has no ASCII letter or
digit directly before or after it. Shorter bare values printed on their
own, such as the 1 of ?v=1, stay. Other URLs in the details lose
userinfo, query and fragment on a best-effort basis
(#642).content when Qwen2.5 wraps a Hermes
tool call in double braces (<tool_call>{{"name": ...}}</tool_call>, with
any number of extra closing braces) without a grammar. Calls are extracted as
before; a double-brace call with other malformed envelope text keeps that
text. This deliberately differs from upstream llama.cpp, which rejects that
output and extracts no call
(#662).Align native leehack/llamadart-native@v0.4.1 on upstream b29c606e28a01b1bc8c1351026a0fa6e616bf6c4, with matching Dart bindings and Apple companion 0.0
leehack/llamadart-native@v0.4.1 on upstream
b29c606e28a01b1bc8c1351026a0fa6e616bf6c4, with matching Dart bindings
and Apple companion 0.0.19. This resolves the 0.8.23 grammar limitation:
native {2000} repetitions are accepted again
(llamadart-native#76).v0.1.44 for matching
Web/native v0.4.1@b29c606e28a01b1bc8c1351026a0fa6e616bf6c4 parity.
Immutable Web asset manifest:
8d61f453753ac7a7d839ac12318b70986a814748d86029993118c19454293aa9.v0.17.0-6 with Apple companion 0.0.11,
Qwen3 tokenizer compatibility, corrected Linux loading, and explicit Linux
and Windows GPU selection while retaining CPU defaults. The Windows x64
runtime bundles dxil.dll and dxcompiler.dll, which D3D12 GPU engine
creation requires
(litert-lm-native#47).
Web LiteRT-LM stays at @litert-lm/core@0.15.0.Fail Apple builds before native symbol lookup when the resolved llama.cpp companion does not match the core native runtime, with an actionable upgrade
Fail Apple builds before native symbol lookup when the resolved llama.cpp companion does not match the core native runtime, with an actionable upgrade diagnostic instead of allowing ABI-incompatible frameworks.
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@v0.4.0, regenerated matching Dart FFI bindings, refreshed
the llamadart_llama_cpp_flutter Apple SwiftPM checksum, and
aligned current README/website native override docs. Updated multimodal calls
for the matching ABI. Saved native sessions from older runtimes must be
regenerated. Apple companion 0.0.18 supplies the matching native runtime.
Aligned the default WebGPU bridge assets to v0.1.43 for Web/native
llama.cpp v0.4.0@5266f24da75dc449bd56cbed7addb9c8e4a6a73e parity.
Web @litert-lm/core@0.15.0 and native LiteRT pins are unchanged. Immutable
manifest: 111eefc3588842cebfe665b363378edca34924764610263e1eda5280dfcfaa27.
Known upstream limitation: llama.cpp v0.4.0 can reject large grammar
repetitions, such as root ::= "a"{2000}. The post-v0.4.0 upstream
correction is tracked in llamadart-native#76 and is not part of this release.
Updated llamadart_llama_cpp_flutter to 0.0.17 with the Apple SwiftPM v0.3.0 runtime pin.
Updated llamadart_llama_cpp_flutter to 0.0.17 with the
Apple SwiftPM v0.3.0 runtime pin.
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@v0.3.0, regenerated matching Dart FFI bindings, refreshed
the llamadart_llama_cpp_flutter Apple SwiftPM checksum, and
aligned current README/website native override docs.
Aligned the default WebGPU bridge assets to v0.1.41 for corrected
TypeScript declarations and TTS recovery guidance, retaining Web/native
llama.cpp v0.3.0@c1d0e7a004015f23bc0233470b747b596f29b264 parity and Web
@litert-lm/core@0.15.0. Immutable manifest:
fe97604daabaad6aefa223a8637d5fd9dcac09dd4a61b2ef19cd6aabb39392b9.
Consolidated native release tag grammar across Dart, Python, Bash, workflows,
and documentation via a machine-readable fixture contract (#404).
Patched the website's vulnerable nanoid and uuid dependency paths. Until Docusaurus replaces its unpatched image parser, automatic local Markdown imag…
Aligned the default WebGPU bridge assets to v0.1.39 (immutable manifest
b355d01040604f6ae2c5c5fe5bb42b858101a96f03f67e4b27b32fe41ce3b2bf),
restoring Web/native llama.cpp upstream v0.2.0@bb4caa7540188872173c44d161602d9271386413
parity with native anchor llamadart-native@v0.2.0-1 while preserving
approved Web @litert-lm/core@0.15.0 packaging.
Fixed native Qwen3-ASR transcription by applying the model chat template to audio turns, while preserving the validated raw-prompt Web bridge contract. Empty ASR output now fails explicitly instead of reporting an empty result.
Fixed Qwen 2.5/3 LiteRT-LM ToolChoice.required requests silently running
without their required-call grammar and finishing with no call. They now fail
with an actionable LlamaUnsupportedException before generation when the
backend cannot enforce the declared tool schema; Gemma 4 compatibility and
auto/none tool routing are unchanged.
Fixed Gemma 4 thinking-budget output so split channel controls and tool-call envelopes stay out of visible assistant content while preserving ordinary whitespace. The chat example now also validates custom tool declarations, executes declared host handlers exactly once, appends tool results, and performs a bounded continuation for both llama.cpp and LiteRT-LM backends.
Patched the website's vulnerable nanoid and uuid dependency paths. Until
Docusaurus replaces its unpatched image parser, automatic local Markdown
images are rejected; website contributors should use static pathname URLs.
Fixed llamadart_native_runtimes values none, off, and the string
false selecting every runtime family instead of none;
the build hook fails with its No native runtimes selected error again, as
it did before 0.8.0. A YAML boolean false clears the selection too. Unset,
empty, and all-unrecognised config still select every family.
The published package no longer ships the doc/ directory; that contributor
and maintainer documentation is maintained on GitHub, and the packaged files
that link to it now use absolute URLs.
Fixed unanchored docs/ and website/ publish-exclusions that matched those
directory names at any depth and dropped tool/docs/ plus the
llamadart_server example's OpenAPI spec and Swagger UI sources from the
package, leaving the published example unable to analyze. Both patterns are
now root-anchored.
Narrowed the dinja dependency constraint to >=1.0.0 <1.1.0 so chat
template capability detection cannot silently resolve against an unverified
Jinja parser minor. A 1.0.x patch can still reorganise the private sources
the analyzer imports; a new coupling test turns that into a named failure.
Fixed Command R7B, Hermes, and Hunyuan V3 tool grammars so distinct tool or parameter names cannot collide after conversion to internal GBNF rule names.
Fixed DeepSeek V3.2 DSML tool calls using their upstream
<|DSML|function_calls> envelope while preserving DeepSeek V4's distinct
<|DSML|tool_calls> grammar and parser behavior.
Fixed partial GLM 4.5, Poolside Laguna, and Muse Glimmer tool envelopes leaking into streamed assistant content, while preserving completed calls, malformed final output, and ordinary text surrounding Muse recipient channels.
Fixed schema-constrained tool calls for Kimi K3, MiniMax M1/M3, DeepSeek
V3.2/V4, and Muse Glimmer, including exact escaped names, required fields,
declared value types, matching MiniMax M3 element tags, zero-argument calls,
and strings containing delimiter characters. Required-tool mode now accepts
each format's reasoning/content prefix while still requiring a call.
MiniMax M3, DeepSeek DSML, Muse Glimmer, Poolside Laguna, and GLM 4.5 now
reconstruct argument values from the declared tool schema instead of
guessing from text. Added ToolParam.nullType for null-only JSON Schema
properties.
llama.cpp backend initialization failures now complete the worker startup
handshake with a typed LlamaBackendInitializationException and collected
native-loader diagnostics. A failed or incompatible worker is torn down
instead of being reported ready and leaving later requests waiting forever.
Added template-aware parsing for Kimi K3, MiniMax M1/M3, DeepSeek V3.2/V4, Muse Glimmer, and Poolside Laguna, preventing their native tool calls from silently falling back to plain content.
Fixed XML-style tool-call parsing to honor raw-versus-JSON argument values and final-value delimiters. Apriel 1.5 and Xiaomi MiMo now parse nested JSON values without splitting on inner commas, while malformed payloads remain ordinary assistant content.
Made native video-input capability truthful without claiming end-to-end
support. LlamaVideoContent requests now fail with an actionable
LlamaUnsupportedException, LlamaEngine.supportsVideo reports public
consumability as false, and the llama.cpp worker uses the behavioral
mtmd_helper_support_video result instead of exported helper symbols. The
published v0.2.0-1 archive has not been qualified for end-to-end video;
full path/byte input remains blocked on cross-platform FFmpeg/ffprobe
packaging and Dart frame lifecycle wiring.
Native release synchronization and build-hook overrides now accept stable
vMAJOR.MINOR.PATCH artifacts and ordered vMAJOR.MINOR.PATCH-N wrapper
rebuilds plus nightly bNNNN-N rebuilds, while preserving historical
bNNNN and bNNNN-llamadart.N artifacts. Sync rejects invalid tags,
leading-zero nightly versions, rollback, wrapper/nightly latest results,
incompatible manifest contracts, missing bundles, and release/manifest
checksum or version skew without changing the default pin.
Fixed Web/native backend API parity. WebAutoBackend now forwards grammar
constraint support from its active runtime, so strict structured output fails
early with an actionable error on unsupported Web backends, and the Web-safe
LiteRtLmRuntimeClient stub now exposes the native client's thinking-tag
configuration method.
A failed llama.cpp model load now reports the startup diagnostics collected during native library discovery, so a missing or unloadable runtime library explains itself instead of surfacing as a bare load failure. Platforms that record no diagnostics keep their previous message unchanged.
llama.cpp worker errors now keep their type. Every backend method routes an
ErrorResponse through the file's own error mapper instead of rebuilding a
bare Exception, tokenize and detokenize no longer discard the worker's
error entirely, and a core UnsupportedError raised for an unavailable native
capability is classified rather than flattened. State-file failures now throw
LlamaStateException.
Fixed multimodal media placeholders being normalized inconsistently. <img>,
<|img|>, <start_of_image> and indexed markers such as <|image_1|> are now
rewritten to the mtmd marker on every path, rather than depending on which
layer rendered the prompt. MiniMax-M2 and MiniCPM-5 also now detect a
forced-open thinking block using the same rule as every other handler.
aLoRA adapters are now rejected with LlamaUnsupportedException instead of
being applied like ordinary LoRA adapters. An aLoRA adapter must activate only
after its invocation tokens appear in the prompt, so applying it from the
start of generation silently changed output. Missing metadata-inspection
symbols in custom native runtimes also fail closed with the same typed error,
and rejected adapters are released when the cleanup ABI is available. LoRA
errors from the worker keep their typed exception instead of arriving as a
bare Exception, and a failed adapter load now throws
LlamaModelException.
Deprecated LiteRtLmRuntimeClient.conversationTokenCount() and
replaceConversationWithClone(). Both are unused and are scheduled for
removal in the next major release; open an issue if you depend on either.
NativeLlamaBackend.modelLoadFromUrl now throws LlamaUnsupportedException
instead of UnimplementedError, bringing it into the LlamaException
hierarchy. It is the same exception type LlamaEngine.loadModelFromUrl
already throws for this condition; each keeps its own message.
Updated the default llama.cpp native runtime to the immutable
leehack/llamadart-native@v0.2.0-1 release (llama.cpp v0.2.0), adding LFM2
DSpark support plus current upstream correctness and backend performance
fixes. Matching Dart FFI bindings, including the new multimodal
projector-device field, and the Apple SwiftPM artifact checksum were
refreshed. Linux libmtmd.so.0 now loads without the old
libmtmd.so.SOVERSION compatibility alias.
Removed the abandoned Dart-side MTP/n-gram speculative-decoding scaffolding
from the llama.cpp backend. Speculative decoding is unchanged: it continues to
run through the native wrapper, and SpeculativeDecodingStrategy keeps every
existing option.
Bumped llamadart_llama_cpp_flutter to 0.0.16 so the v0.2.0-1 Apple
SwiftPM pin actually publishes; 0.0.14 was already on pub.dev, so release
automation skipped it and Apple builds would have kept the b10514 runtime.
Corrected the WebGPU bridge docs, which claimed the pinned v0.1.37 bridge
assets match the default native llama.cpp runtime. They embed b10514 and now
trail the native v0.2.0-1 pin.
Chat-template capability detection now logs a debug message naming the
probe (string-content, typed-content, system-role, tools) when its
render throws, so a template that fails to render is distinguishable from
one that genuinely lacks the capability.
Added code_assets 2.x compatibility while retaining support for 1.x native asset toolchains.
Added code_assets 2.x compatibility while retaining support for 1.x native
asset toolchains.
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@b10514, adding BailingMoE3,
GraniteSWA/GraniteMoeSWA, speculators-format DSpark checkpoints, and current
upstream multimodal/backend improvements. Matching Dart FFI bindings and the
llamadart_llama_cpp_flutter Apple SwiftPM artifact were refreshed.
Updated the default WebGPU bridge assets to v0.1.37, embedding llama.cpp
b10514 to restore native/Web parity. The bridge provisions an explicit 1 MiB
Wasm stack for wasm32 and memory64, preventing Qwen3-ASR memory64 context
construction from overflowing the default stack.
Improved Web microphone transcription by warming up browser capture before showing the recording-ready state, trimming the warmup silence, and rejecting too-short, silent, or unsupported PCM WAV captures before inference.
Added logical batch-size (n_batch) and micro-batch-size (n_ubatch)
controls for llama.cpp/WebGPU models in the Flutter chat example.
Improved Android Auto backend selection by probing the packaged Vulkan device before memory planning, avoiding unnecessary CPU fallback on capable models.
Updated the native LiteRT-LM runtime to v0.16.0-native.2. The Apple companion
packages the iOS Gemma constraint provider and Metal accelerator/sampler
plugins required by the published runtime.
Added an experimental SpeechToTextEngine.liteRtLm path with capability
discovery, worker-isolated CPU inference, bounded mono 16 kHz float PCM,
partial/final transcript events, finalization, and cancellation.
Added experimental live dictation to the native Flutter chat example for chat models using selectable Moonshine Tiny and Parakeet TDT 0.6B sidecars. Live dictation is CPU-only, English-only, capped at five minutes, and unavailable on Linux and Web.
Improved Flutter chat example model downloads with bounded retries for transient network failures, safe resume after truncated responses, and a distinct integrity-verification state after transfer reaches 100%.
Redesigned the Flutter chat example onboarding and Lab surfaces, preserved a completed model card's viewport position when downloads reorder the catalog, and stopped streaming responses from pulling users away from chat history.
Added an experimental typed TextToSpeechEngine for native llama.cpp and
WebGPU with Qwen3-TTS models, including capability discovery, speaker
references, cancellable progress, complete 24 kHz PCM output, and WAV
encoding. The Flutter chat example adds synthesis, playback, replay, and WAV
save controls. Apple apps discover the TTS ABI in the embedded llama
framework; current LiteRT-LM artifacts fail explicitly as unsupported.
Added an experimental typed SpeechToTextEngine with an explicit Qwen3-ASR
adapter profile for whole-file llama.cpp transcription. The Flutter chat
example includes a checksum-pinned Qwen3-ASR 0.6B preset plus file and
microphone transcription on native and WebGPU. Web accepts WAV bytes only;
native LiteRT-LM live dictation remains a separate implementation.
Added Ask with voice to the native Flutter chat example for Gemma 4 E2B through LiteRT-LM direct media and audio-capable GGUF + projector paths. It sends a microphone recording through multimodal chat and remains separate from typed speech-to-text.
Added experimental, opt-in native llama.cpp DSpark speculative decoding
through SpeculativeDecodingConfig.draftDspark(...) with a compatible
external draft model.
Updated the default llama.cpp native runtime pin to leehack/llamadart-native@b10333, regenerated matching Dart FFI bindings, refreshed the llamadart_l
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@b10333, regenerated matching Dart FFI bindings,
refreshed the llamadart_llama_cpp_flutter Apple SwiftPM checksum, and aligned
current README/website native override docs.
Updated WebGPU bridge assets to v0.1.27 (llama.cpp b10333), keeping the
native and Web GGUF runtimes on the same upstream revision.
Fixed corrupt Qwen3.5 output on Android Vulkan by preserving the KQV offload required for correct hybrid model inference while retaining the remaining conservative Android context settings.
Updated the default llama.cpp native runtime pin to leehack/llamadart-native@b10276, picking up Qwen3-TTS model-loading primitives, explicit bundled-M
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@b10276, picking up Qwen3-TTS model-loading
primitives, explicit bundled-MTP loading, automatic model-specific token
suppression, and recent model, multimodal, speculative-decoding, backend,
and runtime fixes. Regenerated matching Dart FFI bindings, migrated model
loading to llama.cpp's load-mode ABI and the penalty sampler to the new
vocabulary-sized ABI, and refreshed the llamadart_llama_cpp_flutter Apple
SwiftPM checksum. Speech generation is not yet exposed through the public
Dart API.
Updated the default LiteRT-LM runtimes to
leehack/litert-lm-native@v0.15.0-native.3 and
@litert-lm/core@0.15.0. The native artifact includes a corrected v0.15
streaming callback bridge and an Android Dawn rollback for Mali-G715 GPU
device loss; incompatible callback runtimes now fail safely before
generation, and concrete macOS app, framework, and cache libraries take
precedence over process-linked assets.
Disabled automatic WebGPU fetch-backed model loading by default. Streamed loading remains the safe default; controlled range-capable deployments can opt in explicitly.
Updated WebGPU bridge assets to v0.1.26 (llama.cpp b10276), refreshing
both WebAssembly runtimes while preserving the existing bridge API.
Updated the default llama.cpp native runtime to leehack/llamadart-native@b10075 with matching bindings and Apple artifacts.
Updated the default llama.cpp native runtime to
leehack/llamadart-native@b10075 with matching bindings and Apple artifacts.
Added Tencent Hunyuan V3 chat-template, reasoning, and tool-call support.
Fixed Gemma 4 LiteRT-LM text generation in the Web chat app after model loading completed successfully.
Restored GGUF loading in deployed Web chat apps by packaging the pinned WebGPU runtime assets with Flutter Web builds.
Updated the default llama.cpp native runtime to leehack/llamadart-native@b9982, including regenerated bindings and safer multimodal UTF-8 prompt handl
Updated the default llama.cpp native runtime to
leehack/llamadart-native@b9982, including regenerated bindings and safer
multimodal UTF-8 prompt handling.
Improved llama.cpp batching defaults and ChatSession context management for
more predictable generation under constrained contexts.
Added llama.cpp presencePenalty sampling and thinking-budget controls,
including the server's thinking_budget_tokens extension. Unsupported
WebGPU and LiteRT-LM paths now fail explicitly.
Reworked the runnable TUI coding agent with a focused Pi-style workflow, Unsloth Qwen3.6 defaults, shared model-source loading, and clearer reasoning and final-answer presentation.
Improved the OpenAI-compatible server with standard client-managed tool-call
transcripts, named tool_choice, configurable thinking behavior, and shared
model-source loading.
Added clipboard media attachments to the runnable chat app. Desktop and web users can paste screenshots or copied image/audio files with Cmd/Ctrl+V, m
Added clipboard media attachments to the runnable chat app. Desktop and web
users can paste screenshots or copied image/audio files with Cmd/Ctrl+V,
mobile users can choose Paste attachment, and ordinary text paste remains
unchanged.
Restored the runnable macOS chat app build phase that embeds and signs
LiteRT-LM runtime libraries inside the sandboxed app bundle, and enabled
LiteRT-LM Metal selection on iOS with the consolidated upstream runtime.
Updated the default LiteRT-LM runtime to
leehack/litert-lm-native@v0.14.0-native.2, which fixes Android GPU plugin
symbol resolution and uses the checksum-pinned official Apple XCFrameworks.
Fixed native LiteRT-LM generation incorrectly treating the requested maximum response length as a forced benchmark decode count. Short responses no longer wait for every allowed token before streaming, and LiteRT-LM chat flushes its first token immediately.
Added an app-owned FIFO model-download queue to the runnable chat app. Only one model transfers at a time, queued cards show their position and can leave the queue independently, and a responsive shell progress pill keeps the active download visible after the settings panel closes.
Replaced the runnable chat app's broad built-in model catalog with a focused Unsloth-first set. Added cross-platform Gemma 4 E4B plus native-desktop Gemma 4 12B/26B-A4B/31B and Qwen3.6 35B-A3B presets. The model library now promotes downloaded models, supports name/capability search and Mobile & Web/Desktop filters, explains incompatible choices, and uses quieter cards with compact compatibility, capability, and recommended-setting summaries. Custom entries retain independent remove-from-library and downloaded-file actions.
Enabled Gemma 4 audio attachments in the runnable chat app for the current
native GGUF projector and LiteRT-LM bundle, while keeping LiteRT-LM Web
correctly text-only. Model capability settings now distinguish direct media
input from external mmproj input and persist that distinction across app
launches.
Redesigned the runnable chat app with a quieter responsive shell, compact runtime details, full-screen mobile settings, streamlined message/composer surfaces, accessible controls, protected conversation deletion, and copy/regenerate actions for assistant responses. Model downloads now remain active when settings closes or switches between drawer and pinned layouts.
Hardened the runnable chat app for narrow windows, 200% text scaling, and macOS assistive technology; removed stale assistant placeholders after empty or failed generations; kept context accounting stable when regenerating a response; and redacted signed model URL parameters from labels and generation errors.
Clarified the chat app's backend and GPU-layer controls, exposed the active loaded backend beside its preference, and restored GPU offload when switching from a zero-layer CPU configuration back to Auto or a GPU backend. Native Max now maps to full llama.cpp offload, while Auto uses model size, available device memory, safe system headroom, and requested context to choose full or partial offload and only reduces context when the model does not fit. Auto intent now persists separately from resolved layer/context values so device headroom is recalculated on every model load and after app restarts.
Updated the runnable chat app's web runtimes to pinned WebGPU bridge assets v0.1.18 (llama.cpp b9915) and @litert-lm/core@0.14.0, keeping hosted and l
Updated the runnable chat app's web runtimes to pinned WebGPU bridge assets
v0.1.18 (llama.cpp b9915) and @litert-lm/core@0.14.0, keeping hosted
and local inference on reproducible runtime versions.
Improved the runnable chat app's model-cache UX by reusing cached GGUF and
LiteRT-LM web bundles for benign catalog URLs such as ?download=true,
preserving browser model caches during app cache cleanup, adding a text-only
projector skip path, and polishing download/load progress states.
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@b9935, regenerated matching Dart FFI bindings, refreshed
the llamadart_llama_cpp_flutter Apple SwiftPM checksum, and
aligned current README/website native override docs.
Fixed load lifecycle guards so repeated model loads preserve the active model state, URL load unsupported-runtime diagnostics stay typed, and unload c
Fixed load lifecycle guards so repeated model loads preserve the active model state, URL load unsupported-runtime diagnostics stay typed, and unload cancels active generation before freeing llama.cpp handles.
Tightened LiteRT-LM runtime validation and local smoke coverage by requiring complete macOS arm64 runtime caches, requiring an explicit model path for the LiteRT-LM chat feature smoke scenario, normalizing the chat app's LiteRT-LM auto context size, and adding the missing iOS-compatible SwiftPM Gemma provider target. Flutter macOS LiteRT-LM companion-package builds now fall back to hook-managed native assets, and the companion package no longer links the incomplete macOS LiteRT-LM SwiftPM artifact set.
Reworked the README into a shorter entry point, fixed stale docs/examples found during the documentation review, and aligned release, Android smoke, WebGPU mem64, native sync, and capability-support wording with the current workflows and runtime behavior. WebGPU runtime LoRA calls now throw an unsupported-operation error instead of reporting no-op success.
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@b9873-llamadart.2, keeping the b9873 llama.cpp
ABI/bindings while picking up wrapper fixes for native release
provenance and backend-selected speculative sampler acceptance. Refreshed
the llamadart_llama_cpp_flutter Apple SwiftPM checksum and aligned current
README/website native override docs.
Aligned llama.cpp n-gram speculative rejected-tail rollback with upstream
server behavior by trimming rejected target tokens before falling back to
checkpoint replay, avoiding early token-count/EOG drift in accepted-draft
ngram-map-k runs. The speculative benchmark can now emit full generated
text with --include-output for parity investigations.
Fixed llama.cpp n-gram speculative configuration mapping so
draftTokenMax no longer implicitly overrides upstream ngramSizeM, and
documented upstream comparison commands plus measured n-gram benchmark
results.
Added a durable flag-based llama.cpp speculative benchmark runner and wired
it into the local E2E scenario list, testing matrix, README, and website
performance guide. The runner can now generate a llama.cpp-compatible
static n-gram cache file for ngram-cache E2E validation, and renders
benchmark prompts with configured or loaded GGUF chat templates before the
generic fallback instead of silently falling back to a hard-coded Gemma
prompt.
Documented compatible DFlash GGUF metadata, a known-good public target/draft
model pair, and troubleshooting guidance for incompatible dflash-draft or
missing dflash.target_layers artifacts.
Hardened generic llama.cpp external draft-model speculative decoding so
draft-context processing does not request unused logits, avoiding a
draft-simple native abort while preserving target verification logits.
Extended llama.cpp speculative performance diagnostics with draft-attempt, target-verification-token, and replay-token counters. Enabled speculative runs now preserve explicit zero counters instead of collapsing them to null, and local benchmark JSON includes the new fields.
Hardened LiteRT-LM generation validation so llama.cpp-only speculative decoding knobs fail loudly instead of silently degrading to LiteRT-LM's boolean speculative toggle.
Added llama.cpp upstream speculative decoding parity through
SpeculativeDecodingConfig constructors for draft-simple, EAGLE3, MTP,
DFlash, ngram-simple, ngram-map-k, ngram-map-k4v, ngram-mod, ngram-cache,
and mixed n-gram plus one draft-model strategy, including generic native
wrapper bindings, docs, and local benchmark matrix coverage.
Added LlamaStructuredOutput and LlamaEngine.createStructuredJson(...)
helpers for strict JSON-object / JSON-schema generation with final-output
validation and typed decoding.
Added LlamaEngine.loadMultimodalProjectorSource(...) so GGUF
multimodal projector files can use the same ModelSource resolver,
native download/cache manager, authentication, checksum, and progress
options as loadModelSource(...), while preserving the existing
loadMultimodalProjector(...) path/string API.
Improved the runnable chat app's Manage Models cache UX so model and mmproj asset cache states are shown separately, missing multimodal projectors can be re-cached without re-fetching already cached model assets, and runtime media capability mismatches surface as user-readable warnings. Custom signed or tokenized Hugging Face URLs now require confirmation before they are saved.
Updated the default LiteRT-LM native runtime pin to leehack/litert-lm-native@v0.14.0-native.1, refreshed matching native-assets checksums, aligned the
Updated the default LiteRT-LM native runtime pin to
leehack/litert-lm-native@v0.14.0-native.1, refreshed matching native-assets
checksums, aligned the llamadart_litert_lm_flutter Apple SwiftPM checksums,
and documented the newly exposed native LiteRT-LM load/generation controls:
per-request max output tokens, native sampler params, thread count, one
default-scale initial text LoRA adapter, activation data type, prefill chunk
size, parallel file-section loading, and Android LiteRT dispatch library
directory.
Hardened LiteRT-LM runtime packaging and local runtime preparation for the 0.14 line, including Linux/Windows runtime dependency resolution, Android Dawn companion libraries, Apple SwiftPM checksums, and macOS fallback app runtime copying.
Hardened release automation by adding CODEOWNERS coverage for publication-sensitive files and making pub.dev/GitHub Release propagation waits configurable with longer defaults.
Added post-merge release automation so a merged release-prep PR can publish missing companion package versions, push the core release tag, wait for pub.dev, and confirm the GitHub Release without a separate manual tag step.
Added LlamaEngine.getModelFileType() for llama.cpp/GGUF models, exposing
native model file type / quantization metadata from llama_model_ftype and
llama_ftype_name when available.
Refreshed llama.cpp b9860 template parity for DeepSeek V4 and MiniCPM5,
including MiniCPM5 XML tool-call detection, rendering, and parsing.
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@b9860, regenerated matching Dart FFI bindings, refreshed
the llamadart_llama_cpp_flutter Apple SwiftPM checksum, and
aligned current README/website native override docs.
docs: use hf model sources in beginner examples by @leehack in https://github.com/leehack/llamadart/pull/251
Full Changelog: https://github.com/leehack/llamadart/compare/v0.8.10...v0.8.11
leehack/llamadart-native@b9829, refreshed the
llamadart_llama_cpp_flutter Apple SwiftPM checksum, and aligned current
README/website native override docs for the release.fix: use platform-appropriate model cache defaults by @leehack in https://github.com/leehack/llamadart/pull/248
Full Changelog: https://github.com/leehack/llamadart/compare/v0.8.9...v0.8.10
Potentially breaking behavior change: native model cache defaults changed
without breaking Dart source compatibility. DefaultModelDownloadManager() now
prefers the platform shared cache on desktop/server instead of the process temp
directory, and mobile DefaultModelDownloadManager.auto() without an explicit
app-private directory now uses a best-effort temporary/cache fallback instead
of throwing. Apps or tests that asserted the old temp path or mobile exception
should pass an explicit cache directory or follow MIGRATION.md.
Added optional androidAppPrivateCacheDirectory and
iosAppPrivateCacheDirectory arguments to
DefaultModelDownloadManager.auto(...) so apps can provide platform-specific
mobile cache roots without constructor-level Platform.isAndroid /
Platform.isIOS branching.
Updated the default native DefaultModelDownloadManager() constructor to use
the per-user shared model cache on desktop/server platforms and the mobile
app-private cache fallback, so plain LlamaEngine(...) remote source loads use
a platform-appropriate default while preserving a temporary fallback for hosts
that cannot expose a desktop cache environment.
chore: restore pub score compatibility by @leehack in https://github.com/leehack/llamadart/pull/247
Full Changelog: https://github.com/leehack/llamadart/compare/v0.8.8...v0.8.9
Broadened the hooks dependency constraint to support both the existing
build-hooks package family and the latest stable release, restoring the
pub.dev dependency freshness score without breaking downstream packages that
still resolve hooks 1.x.
Made web-safe backend stubs the default conditional import/export targets,
preserving native dart:io selection while avoiding false WASM compatibility
deductions in pub.dev analysis.
Add cross-platform model cache strategies by @leehack in https://github.com/leehack/llamadart/pull/242
Full Changelog: https://github.com/leehack/llamadart/compare/v0.8.7...v0.8.8
Added a CI release-doc version consistency check so current README/website
install snippets and companion package READMEs stay aligned with package
pubspec.yaml versions, and documented that companion/core package publishing
happens only after release-prep merge with explicit maintainer approval for
each tag.
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@b9803, regenerated matching Dart FFI bindings, refreshed
the llamadart_llama_cpp_flutter Apple SwiftPM checksum, and
aligned current README/website native override docs.
Added DefaultModelDownloadManager.auto(...) plus explicit model cache root
constructors for shared desktop caches, app-private mobile caches,
user-selected model libraries, and App Group containers. Implicit shared cache
resolution now fails loudly on mobile and web where the OS cannot provide a
hidden cross-developer model folder.
Fixed multimodal chat-template rendering so templates that force-open reasoning, for example Qwen3.5 VLM prompts ending with , preserve enable_thinkin
<think>, preserve enable_thinking and stream generated reasoning through delta.thinking instead of delta.content.Validation highlights:
dart pub publish --dry-runPublish to pub.dev workflowDocs Version Cut workflow./tool/docs/validate_links.shPatch release for the llama.cpp native runtime sync to leehack/llamadart-native@b9776.
Patch release for the llama.cpp native runtime sync to leehack/llamadart-native@b9776.
leehack/llamadart-native@b9776.llamadart_llama_cpp_flutter Apple SwiftPM checksum.Validation highlights:
dart pub publish --dry-runmainmain./tool/docs/validate_links.shPublish to pub.dev workflowDocs Version Cut workflowleehack/llamadart-native@b9776, regenerated matching Dart FFI bindings,
refreshed the llamadart_llama_cpp_flutter Apple SwiftPM checksum, and
aligned current README/website native override docs.Patch release for the mtmd split-library fallback ABI used by multimodal llama.cpp loads.
Patch release for the mtmd split-library fallback ABI used by multimodal llama.cpp loads.
mtmd.dll and other split mtmd native bundles use the same bitmap helper signature as the generated native binding path.Validation highlights:
dart pub publish --dry-run./tool/docs/validate_links.shUpdated the default llama.cpp native runtime pin to leehack/llamadart-native@b9744, regenerated matching Dart FFI bindings, refreshed the llamadart_ll
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@b9744, regenerated matching Dart FFI bindings,
refreshed the llamadart_llama_cpp_flutter Apple SwiftPM checksum, and
aligned current README/website native override docs.
Expanded llama.cpp chat-template parity coverage for the latest upstream
template fixtures, including Cohere2 MoE, LFM2.5 tool-call, and Granite 4.1
templates. LFM2.5 templates that use plain List of tools: [...] prompts
with <|tool_call_start|> / <|tool_call_end|> now route through the LFM2
handler like upstream llama.cpp, and ToolChoice.required now uses
grammar-constrained LFM2 tool-call generation.
Fixed streaming tool-call parsing so partial North/Cohere bare action arrays are not emitted as content before the complete tool call is parsed.
Expanded the local GGUF chat feature smoke to cover thinking, tool-call, and optional multimodal turns through the unified local E2E runner.
Patch release for Windows CUDA backend discovery in native-assets builds.
Patch release for Windows CUDA backend discovery in native-assets builds.
PATH.cudart64_12.dll,
cublas64_12.dll, and cublasLt64_12.dll resolve correctly.Validation highlights:
dart pub publish --dry-runmainPublish to pub.devDocs Version CutDocs PagesUpdated the default llama.cpp native runtime pin to leehack/llamadart-native@b9694, regenerated matching Dart FFI bindings, refreshed the llamadart_ll
Updated the default llama.cpp native runtime pin to
leehack/llamadart-native@b9694, regenerated matching Dart FFI bindings,
refreshed the llamadart_llama_cpp_flutter Apple SwiftPM checksum, and
updated the default WebGPU bridge asset pin to
leehack/llama-web-bridge-assets@v0.1.17 (llama.cpp b9699). The WebGPU
backend now caps unset large-model browser batches so Gemma 4 mem64 loads do
not fall back to context-sized compute buffers.
Added BackendGpuEnumeration.listGpuDevices({probeBackends}) (exposed via
LlamaEngine.listGpuDevices) to enumerate GPU-class devices for offload
selection: backend, per-backend mainGpu index, name, description, device
id, type, and free/total memory per device. With an empty probeBackends
only already-registered backends are inspected, so an unsupported GPU runtime
cannot crash the process during enumeration; pass specific backends to opt
into loading just those modules first. Web/WebGPU return an empty list.
Added Cohere2 MoE / North Code chat-template detection and parsing so
<|START_TEXT|> responses and <|START_ACTION|> tool-call arrays are
handled separately from older Command-R templates.
Fixed docs references that still pointed at llamadart_litert_lm_flutter 0.0.1 and the pre-native.1 LiteRT-LM release after the 0.8.0 native pin sync m
llamadart_litert_lm_flutter 0.0.1 and
the pre-native.1 LiteRT-LM release after the 0.8.0 native pin sync moved
LiteRT-LM Apple/runtime artifacts to v0.13.1-native.1..litertlm image/audio chat parts through LiteRT-LM
Conversation message JSON so bundles with native media processors can accept
LlamaImageContent / LlamaAudioContent path and encoded-byte inputs without
a separate mmproj projector.One column per month.