NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
pub.dev, #1399 most downloaded on pub.dev
Cross-platform, on-device LLM inference for Dart and Flutter. Run GGUF (llama.cpp) and LiteRT-LM models locally on Android, iOS, macOS, Windows, Linux and web.
Latest stable
0.19.5
Last release 11 days ago
11 Nov 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
9 months old
91 releases · first in 2026
Drag across the ruler to compare, or tap a release to open it.
Added responseFormat routing to LlamaEngine.create(...) for grammar-capable backends, deprecated the legacy chatTemplate(...) jsonSchema shortcut, and…
llamadart_llama_cpp_flutter for GGUF/llama.cpp and
llamadart_litert_lm_flutter for .litertlm/LiteRT-LM. These companion
packages live under packages/ in this repository and publish as separate
pub.dev packages.llamadart so pure Dart/native-assets
consumers can keep using the core package without taking a Flutter SDK
constraint.0.0.1; native pin sync bumps only the
affected companion package patch version. Companion package publishing uses
package-specific tags after the first manual pub.dev publish, and skips
companion versions that already exist on pub.dev.llamadart_native_runtimes to include all available
runtime families. For Flutter iOS/macOS app builds, installed companion
packages decide Apple SPM runtimes; for every other build,
llamadart_native_runtimes remains the selector.leehack/llamadart-native@b9587, regenerated matching Dart FFI bindings,
and refreshed the llamadart_llama_cpp_flutter Apple SwiftPM checksum.SpeculativeDecodingConfig.mtp(draftModelPath: ...) for llama.cpp
external draft-model MTP sessions, with draft model caching and cleanup
tied to the target model lifetime.responseFormat routing to LlamaEngine.create(...) for
grammar-capable backends, deprecated the legacy chatTemplate(...)
jsonSchema shortcut, and made strict response-format requests fail early
on LiteRT-LM instead of silently degrading to unconstrained generation..litertlm text chat through LiteRT-LM Conversation
APIs so structured history, system messages, tool declarations, and
template extra context reach the runtime without a Dart-rendered prompt.
Unsupported cases still fall back to the existing Dart chat-template path..litertlm ModelParams for
liteRtLmActivationDataType, liteRtLmPrefillChunkSize,
liteRtLmParallelFileSectionLoading, and liteRtLmDispatchLibDir,
forwarding the pinned LiteRT-LM v0.13.1 engine-settings C APIs while
keeping defaults unchanged.Your coding agent can read these notes before it upgrades. Set up the MCP server →
Added explicit pub.dev platform metadata for Android, iOS, Linux, macOS, web, and Windows so the package listing reflects actual cross-platform suppor
Metadata patch release.
Compatibility note: no Dart API breaking changes. Flutter Apple apps must target iOS 16.4/macOS 14.0 or newer, and non-Android native apps that ship .…
leehack/llamadart-native and leehack/litert-lm-native
XCFramework artifacts through darwin/llamadart/Package.swift.MinimumOSVersion mismatches in App
Store uploads. Standalone Dart macOS keeps the native-assets dylib fallback.llama_cpp and litert_lm by
default; iOS, macOS, Linux, and Windows now default to llama_cpp only..litertlm models should opt in with llamadart_native_runtimes.leehack/llamadart-native and leehack/litert-lm-native
XCFramework artifacts through darwin/llamadart/Package.swift.MinimumOSVersion mismatches in App
Store uploads. Standalone Dart macOS keeps the native-assets dylib fallback.llama_cpp and litert_lm by
default; iOS, macOS, Linux, and Windows now default to llama_cpp only.SpeculativeDecodingConfig as a backend-neutral generation option for
selecting speculative decoding strategies such as MTP while keeping the
existing GenerationParams.speculativeDecoding flag as a compatibility
switch.SpeculativeDecodingConfig.mtp(...), defaulting to a conservative
one-token draft depth unless callers tune draftTokenMax.leehack/llamadart-native@b9547, including the MTP wrapper exports and
llama-common runtime packaging.ModelParams.speculativeRollbackTokenMax so llama.cpp contexts can
reserve recurrent-state rollback snapshots required by Qwen3.5 MTP-style
models.draft-mtp backend-sampling path can abort with vk::DeviceLostError;
CPU and other supported backends remain available, and a dart-define debug
override is available for reproductions..litertlm models should opt in with llamadart_native_runtimes.Compatibility note: no public API breaking changes for existing GGUF / llama.cpp callers. LiteRT-LM support is additive, with deprecated benchmark wra…
.litertlm routing through LlamaBackend() on native and
web targets, with native bundle downloads from leehack/litert-lm-native
and web loading through @litert-lm/core.ModelParams.liteRtLmBackend so callers can select LiteRT-LM CPU,
GPU, or Android NPU execution where supported. auto chooses GPU on
Android/macOS and CPU elsewhere on native targets..litertlm loading through
loadModelSource(...), preserving the selected LiteRT-LM backend after the
cache manager resolves the local file.ChatSession token counting support.hooks.user_defines.llamadart.llamadart_native_tag,
llamadart_native_repository, and llamadart_native_path so apps can test
a different compatible native runtime source without patching llamadart..dart_tool/lib is unavailable.GenerationParams.speculativeDecoding for native LiteRT-LM. The
default remains disabled; llama.cpp, WebGPU, and LiteRT-LM web reject the
option until their speculative paths are implemented..litertlm thinking and tool calling by replacing the stub
template with the canonical Gemma 4 chat template, parsing the runtime
thought channel as reasoning, and suppressing reasoning deltas when callers
set enableThinking: false..litertlm chat-template registry seeded with
Gemma 4/3/3n and Qwen 2.5/3. Pass ModelParams.chatTemplate to override
detection for other models.supportsGrammarConstraints from the active NativeAutoBackend delegate.<|tool_call> markers as assistant content before the final
tool_calls chunk.ModelParams.preferMemory64 and ModelParams.modelBytesHint so
large WebGPU GGUF models such as Gemma 4 E2B can choose the 64-bit bridge
core before hitting the wasm32 address-space limit..litertlm chat-app turns by swallowing unsupported token-count
refreshes, avoiding unsupported minP/penalty parameters for LiteRT-LM
web generation, and replacing the stuck "Loading model 0%" label with an
indeterminate load message..litertlm load time by skipping WebGPU CacheStorage prefetch
for LiteRT-LM models, which are fetched directly by @litert-lm/core.window.__llamadartBridgeReadyPromise, requiring the bridge
prefetch API, and surfacing actionable errors for old bridge assets.?download=true URLs to be prefetched into the
browser cache while still skipping credentialed or signed URLs..litertlm loading by resolving embedded LiteRtLm and
StreamProxy frameworks from the app bundle, matching the macOS runtime
path behavior.ChatSession now forwards empty-choices completion chunks instead of
throwing, strips multiple <think> blocks, and trims history only on
user-message turn boundaries.LlamaEngine.generate wraps unexpected backend errors in
LlamaInferenceException so callers catching LlamaException see the
documented error type.$refs nested inside other
$ref targets and fails loudly on unresolvable or external $refs.minItems/maxItems, model downloads
use connection and idle-read timeouts, and partial-download resume is
restricted to files with stored validators.tool/gguf_chat_features_smoke.dart and the
chat-app-web-gemma4-webgpu-smoke E2E scenario for real-model parser and
WebGPU mem64 validation.doc/litert_lm_templates.md for backend
selection, platform/runtime support, package-size controls, benchmark
results, model templates, and current LiteRT-LM capability limits..litertlm loads instead of being silently ignored.Compatibility note: no public API breaking changes in 0.6.17; existing 0.6.16 callers remain compatible. The release only refreshes the pinned native…
leehack/llamadart-native@b9371, picking up llama.cpp b9371.MTLLibraryErrorDomain Code=3.0.6.17;
existing 0.6.16 callers remain compatible. The release only refreshes
the pinned native runtime and generated low-level bindings.Compatibility note: no public API breaking changes in 0.6.16; existing 0.6.15 callers remain compatible. The changes improve native VRAM diagnostics,…
getVramInfo() so it reports free/total VRAM from
llama.cpp GPU-class backend devices when available, using props-based
memory reporting first and the legacy memory probe as a fallback.CacheStorage failures fall back to direct network loading, and
credentialed/signed model URLs skip persistent browser cache storage.0.6.16;
existing 0.6.15 callers remain compatible. The changes improve native VRAM
diagnostics, WebGPU browser recovery, and chat app download lifecycle
behavior.chore(deps): bump path-to-regexp from 0.1.12 to 0.1.13 in /website in the npm_and_yarn group across 1 directory by @dependabot[bot] in https://github.
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.14...v0.6.15
tool/testing/run_local_e2e.dart as a discovery and orchestration
entry point for heavyweight local-only Dart E2E, Flutter device, and
Web/Playwright smoke scenarios.test-chat server/mtmd build requirements.--list and
--dry-run first.0.6.15;
existing 0.6.14 callers remain compatible. The chat-template changes fix
multimodal serialization behavior for affected templates, and the local E2E
runner is additive.docs: document WebGPU readiness checks by @leehack in https://github.com/leehack/llamadart/pull/148
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.13...v0.6.14
leehack/llama-web-bridge-assets@v0.1.16 (llama.cpp b9165),
picking up the published JS bridge build, TypeScript declaration asset,
and refreshed bridge docs.ModelDownloadController, a dependency-free helper that turns
ModelDownloadManager cache/download work into app-facing lifecycle states
for resolving, cache checks, downloads, verification, ready, failed,
cancelled, and retry flows.ModelDownloadManager adapter
so its model-management UI demonstrates the controller while preserving the
example's multi-asset and web-cache service behavior.0.6.14;
the WebGPU bridge asset update and ModelDownloadController are additive, and
existing 0.6.13 callers remain compatible.fix: cascade WebGPU batch defaults by @leehack in https://github.com/leehack/llamadart/pull/121
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.12...v0.6.13
ModelSource for local paths, HTTP(S) URLs, and Hugging Face
hf://owner/repo/path/to/model.gguf references, including deterministic
cache keys and redacted metadata/log identities for signed URLs.ModelLoadOptions, ModelCachePolicy, resolver targets, and
download/cache metadata/progress value models for package-managed model
download and cache management.DefaultModelDownloadManager support for streaming
HTTP downloads, .part files, atomic promotion, persisted metadata,
authenticated bearer/custom headers, cancellation, retry, Range resume,
cache hit/refresh/cache-only/no-cache policies, SHA-256 verification,
cache listing, removal, clearing, and age/size pruning.hf:// references now accept
?revision=... for branch/ref names containing slashes, and docs clarify
current single-file behavior, private/gated bearer-token usage, separate
mmproj asset handling, sharded-GGUF limitations, and redaction guarantees..part
files or metadata, while distinct cache entries can still download in
parallel and waiting-caller cancellation does not cancel the active download.ModelSource.path(...) option semantics: local paths now reject
remote/download-only options (non-default cache policies, cache directories,
authenticated headers, resume, and retry overrides) while continuing to
support cancellation and optional local SHA-256 verification.LlamaEngine.loadModelSource(...) to route local sources through the
existing native local loader, remote sources through the native download
cache before local loading, and simple remote sources through URL-capable web
backends when available.LlamaEngine.supportsStatePersistence,
LlamaEngine.stateSaveFile(...), and
LlamaEngine.stateLoadFile(...) so callers can persist and restore
llama.cpp KV-cache state for fast raw-prompt resume/fork workflows.BackendStatePersistence, BackendStatePersistenceSupport, and
StateLoadResult for custom backend implementers and diagnostics.ChatSession
message history must be persisted separately.v0.1.15+,
including Dart JS interop, backend forwarding, and browser integration test
coverage.0.6.13;
existing loadModel(...) callers are unchanged. Code that probes state
persistence support should prefer LlamaEngine.supportsStatePersistence over
structural backend type checks so web/router backends can report
bridge-version-dependent support accurately.fix(native): test b9016 runtime and expose mainGpu by @leehack in https://github.com/leehack/llamadart/pull/113
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.11...v0.6.12
leehack/llamadart-native@b9016,
picking up the CUDA 12.8 Blackwell-capable native bundles.leehack/llama-web-bridge-assets@v0.1.14 (llama.cpp b9016) so
native and web runtimes track the same upstream revision.ModelParams.useMmap (default true) and
ModelParams.useMlock (default false), wired to
llama_model_params.use_mmap / use_mlock. Lets callers turn off mmap
for platforms where memory-mapped weights hurt throughput, or pin
weights in RAM to avoid first-token paging spikes.ModelParams.flashAttention with the FlashAttention.{auto, enabled, disabled} enum, wired to
llama_context_params.flash_attn_type. Explicit settings win over the
existing automatic Android/Vulkan heuristics; auto preserves prior
behavior.ModelParams.cacheTypeK and ModelParams.cacheTypeV with the
KvCacheType.{f16, q8_0, q4_0} enum, wired to
llama_context_params.type_k / type_v. Enables KV-cache
quantization (Q8_0 ≈ halves KV memory; Q4_0 ≈ quarters it). When the
user requests a non-F16 KV type with flashAttention: auto, the
service auto-promotes flash attention to enabled — llama.cpp requires
it for KV quantization.ModelParams.kvUnified (nullable) for explicit override of
llama_context_params.kv_unified. null keeps the existing
auto-enable-when-multi-sequence behavior.ModelParams.ropeFrequencyBase and
ModelParams.ropeFrequencyScale (both nullable) for
context-extension overrides on llama_context_params.rope_freq_base /
rope_freq_scale. null keeps the model's trained values.ModelParams load tuning knobs through the
WebGPU bridge path, including maxParallelSequences, flash attention,
KV-cache type, KV-unified, RoPE, split-mode, and main-GPU options.batchSize /
microBatchSize cascade to n_batch = n_ctx and n_ubatch = n_batch,
avoiding first-embedding aborts for BERT-class/non-causal encoder models
while preserving explicit caller values and Qwen3.5 web tuning.ModelParams.mainGpu and wired it to llama.cpp
llama_model_params.main_gpu.ModelParams.splitMode and wired it to llama.cpp
llama_model_params.split_mode, enabling explicit single-GPU selection
with ModelSplitMode.none.0.6.12.chore(native): sync native release b8778 by @github-actions[bot] in https://github.com/leehack/llamadart/pull/106
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.10...v0.6.11
leehack/llamadart-native@b8955.<|channel>thought ... <channel|> blocks into thinking deltas instead of leaking Gemma 4 thought markers into content output.0.6.11.fix: harden multimodal context overflow handling by @leehack in https://github.com/leehack/llamadart/pull/101
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.9...v0.6.10
leehack/llamadart-native@b8638.384px max edge across Android, iOS, macOS, and Web to reduce multimodal context pressure.supportsVision / supportsAudio instead of model-family assumptions.0.6.10.chore(native): sync native release b8533 by @github-actions[bot] in https://github.com/leehack/llamadart/pull/97
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.8...v0.6.9
16.4 or newer across the README, docs site, and example docs.example/chat_app iOS Podfile and Runner project settings to use deployment target 16.4.ggml_backend_score during asset-based backend fallback so unsupported Android CPU variant libraries are skipped before initialization.auto backend resolution to prefer CPU by default while keeping Vulkan available for explicit opt-in.hooks.user_defines requires flutter clean && flutter pub get before rebuilding.0.6.9.chore(native): sync native release b8480 by @github-actions[bot] in https://github.com/leehack/llamadart/pull/92
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.7...v0.6.8
leehack/llamadart-native@b8480.0.6.8.fix: sync b8373 and harden Linux bundle loading by @leehack in https://github.com/leehack/llamadart/pull/89
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.6...v0.6.7
leehack/llamadart-native@b8373.libllamadart mappings so colocated native dependencies resolve more reliably at runtime.<tool_call> and the JSON payload.0.6.7.Fix/runtime cleanup streaming by @leehack in https://github.com/leehack/llamadart/pull/83
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.5...v0.6.6
leehack/llamadart-native@b8216.leehack/llama-web-bridge-assets@v0.1.10 (llama.cpp b8216).Q4_K_M GGUFs across the example catalog and tooling.p_eval, eval, sample, reuse) backed by llama.cpp context timings with manual timing fallback when built-in counters report zero.0.8B / 2B / 4B models by re-enabling KQV/op-offload/flash-attention where stable.0.8B and 2B, and reduced Android 0.8B context to 2048 for lower first-token latency.0.8B projector work onto CPU on Android.0.6.6.feat: add embedding APIs, batching, and docs by @leehack in https://github.com/leehack/llamadart/pull/75
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.4...v0.6.5
LlamaEngine.embed(...) and LlamaEngine.embedBatch(...) for direct vector generation.BackendEmbeddings for custom backend implementers.BackendBatchEmbeddings and worker-side batch embedding request/response path to reduce isolate round-trip overhead in embedBatch(...).ModelParams.maxParallelSequences (n_seq_max) so contexts can reserve multiple sequence slots for true multi-sequence embedding batches.example/basic_app/bin/llamadart_embedding_example.dart.example/basic_app/bin/llamadart_sqlite_vector_example.dart for local embedding retrieval with SQLite vector search.tool/testing/native_embedding_benchmark.dart to compare sequential embedding calls vs embedBatch(...) throughput (with optional --json-out).tool/testing/native_embedding_sweep.dart to run max-seq sweeps and dump CSV speedup reports for plotting.LlamaEngine.embed(...) / embedBatch(...).leehack/llama-web-bridge-assets@v0.1.8.v0.1.8 bridge bundle through local fetch-script checksum verification.example/chat_app recommended Qwen presets to the Qwen3.5 lineup (0.8B, 2B, 4B, 9B) and removed older Qwen2.5/Qwen3 defaults from the in-app library.mmproj) wiring for Qwen3.5 model cards and tuned safer multimodal defaults (contextSize: 8192, maxTokens: 1024).0.6.5.chore(pubspec): remove unused Flutter-root dependencies by @leehack in https://github.com/leehack/llamadart/pull/67
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.3...v0.6.4
Multimodal projector offload alignment:
preferredBackend: cpu or gpuLayers: 0) now also disable mmproj GPU offload.Package metadata cleanup:
pubspec.yaml (environment.flutter, flutter, path_provider, json_rpc_2, integration_test) to keep the core package pure Dart.Backend selection safety and status accuracy:
preferredBackend: cpu no longer initializes optional GPU backends during startup/model load probing.offload_kqv, op_offload, flash-attention auto path) when effective GPU layers resolve to zero, preventing GPU allocation attempts during context creation in CPU mode.ModelParams.batchSize (n_batch) and ModelParams.microBatchSize (n_ubatch) so context batch sizing can be tuned independently from contextSize while preserving legacy defaults.getAvailableBackends) vs active runtime backend (getBackendName).BackendAvailability capability and LlamaEngine.getAvailableBackends() to support safe settings UIs without forcing GPU initialization.BackendRuntimeDiagnostics capability and LlamaEngine.getResolvedGpuLayers() to expose resolved native load-time layer count for runtime diagnostics.example/chat_app to populate backend selector options from safe availability discovery while keeping active-backend status bound to effective runtime backend.Web model cache + large-model UX improvements (chat app):
ArrayBuffer pressure.Web model-load resilience:
WebGpuLlamaBackend to retry web model loads with reduced context sizes (and CPU fallback as last attempt) when bridge errors indicate browser memory pressure.llama_webgpu_core_mem64) with automatic fallback to wasm32 when unsupported.leehack/llama-web-bridge-assets@v0.1.5.custom_headers in generated Space README frontmatter.Android arm64 CPU variant policy and loader hardening:
b8138 to b8157 to consume Android arm64 CPU-variant runtime bundles.cpu_profile (full default, compact) and advanced cpu_variants override.scripts/android_runtime_smoke.sh) and smoke-plan docs for device verification.android-arm64 now defaults to cpu_profile: full, which may increase package size compared with baseline-only CPU packaging.Improve chat app download reliability and streaming performance by @leehack in https://github.com/leehack/llamadart/pull/62
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.2...v0.6.3
b8099 to b8138.example/tui_coding_agent, a nocterm-based terminal coding agent with tool-calling loop, workspace-scoped file/command tools, and runtime model switching.unsloth/GLM-4.7-Flash-GGUF:UD-Q4_K_XL) with support for custom local paths/URLs/Hugging Face shorthand.--native-tool-calling for experimentation).docs: add Docusaurus site, deploy workflows, and content refresh by @leehack in https://github.com/leehack/llamadart/pull/56
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.1...v0.6.2
create(...).streamBatchTokenThreshold,
streamBatchByteThreshold).reusePromptPrefix, enabled by default) with conservative full-replay
fallback to preserve deterministic parity.ChatSession context trimming using bounded turn-offset
search to avoid repeated linear recount loops on long histories.tool/testing/native_inference_benchmark.dart for TTFT,
throughput, and latency measurement with tunable generation settings.tool/testing/native_prompt_reuse_parity.dart and curated prompt
sets for deterministic prompt-reuse parity validation.Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.0...v0.6.1
Full Changelog: https://github.com/leehack/llamadart/compare/v0.6.0...v0.6.1
Publishing compatibility fix:
hook/src/ into
lib/src/hook/ because pub.dev currently only allows hook/build.dart
under hook files.llama.cpp parity expansion (Dart-native template/parser pipeline):
peg_parser_builder, peg_chat_parser) and integrated parser-carrying render/parse flow for PEG-native/constructed formats.ChatTemplateMatcher, ChatTemplateRoutingContext,
ChatTemplateEngine.registerHandler(...),
ChatTemplateEngine.unregisterHandler(...),
ChatTemplateEngine.clearCustomHandlers(...),
ChatTemplateEngine.registerTemplateOverride(...),
ChatTemplateEngine.unregisterTemplateOverride(...),
ChatTemplateEngine.clearTemplateOverrides(...), and
per-call customHandlerId / parse handlerId routing.chatTemplateKwargs and templateNow.Parity test coverage and tooling:
run_llama_cpp_chat_tests.sh, run_template_parity_suites.sh).peg_parser_builder, template_internal_metadata) to satisfy structure guards.Test cleanup and maintainability:
Native integration cleanup (llamadart-native migration):
tool/testing/prepare_llama_cpp_source.sh to fetch/refresh ggml-org/llama.cpp into .dart_tool/llama_cpp (or LLAMA_CPP_SOURCE_DIR) pinned to a resolved ref (LLAMA_CPP_REF, default latest release tag).tool/testing/run_llama_cpp_chat_tests.sh to use prepared .dart_tool source instead of third_party/llama_cpp, so local upstream chat-suite runs no longer depend on vendored source.LLAMA_CPP_TEMPLATES_DIR or .dart_tool/llama_cpp/models/templates instead of third_party/llama_cpp.KleidiAI/ZenDNN are CPU-path optimizations, not selectable runtime backend modules.Linux runtime/link validation and backend loader hardening:
linux-arm64 vs linux-x64) when preparing runtime dependencies.cpu/vulkan/blas) and optional cuda/hip module dependency resolution.scripts/check_native_link_deps.sh helper plus dedicated validation images:
docker/validation/Dockerfile.cuda-linkcheck and
docker/validation/Dockerfile.hip-linkcheck.Chat example backend UX cleanup:
Auto backend option from settings; only concrete runtime-detected backends are shown.Auto preference to the best detected backend at runtime.fix(android): honor preferred backend during native model loading by @leehack in https://github.com/leehack/llamadart/pull/47
Full Changelog: https://github.com/leehack/llamadart/compare/v0.5.3...v0.5.4
llama.cpp parity hardening:
ChatTemplateEngine now preserves handler-provided tokens even when grammar is attached via params, avoiding token-loss regressions in tool/thinking formats.Chat example loop/lifecycle hardening:
fmt:*, think:*, content:json, fallback:tool-result) and strengthened detach/exit disposal paths.Parity/integration test robustness:
tool_calling_integration_test now accepts both structured tool_calls deltas and XML-style <tool_call> payloads.Documentation updates:
{"response":"..."}) and documented content:json diagnostics.penalty=1.0, top_p=0.95, min_p=0.05) and added a CLI README batch parity-matrix usage example.Chat app backend/status fixes:
gpuLayers while still allowing load-time CPU enforcement.Context size auto mode:
Context Size: Auto by preserving 0 in persisted settings and passing auto behavior through to session context-limit resolution.Tool-call parsing fixes (Hermes):
{{...}} layer second, and only fall back to full _normalizeDoubleBraces when all braces are consistently doubled._normalizeDoubleBraces that bails out on mixed single/double brace payloads to prevent corruption of valid nested JSON.Tool-call parsing fixes (Magistral):
_extractJsonObject to handle \n, \r, and \t between [ARGS] and the JSON body.Example app (basic_app):
toList() buffering with await for streaming for real-time token yield.tools parameter to every follow-up create() call and bounded tool-execution loop with _maxToolRounds = 10.Test coverage:
[ARGS] with newline/nested arguments.Example rename (server):
example/api_server to example/llamadart_server.llamadart_server.example/llamadart_server.GLM 4.5 template parity:
<tool_call> payloads with <arg_key>/<arg_value> pairs.<|user|> stop handling for tool-call flows.Template/native runtime fixes:
fix: prevent .pubignore from excluding lib model sources by @leehack in https://github.com/leehack/llamadart/pull/45
Full Changelog: https://github.com/leehack/llamadart/compare/v0.5.2...v0.5.3
minP to GenerationParams with a default value of 0.0 and copyWith support.min_p sampler initialization in LlamaCppService when minP > 0.GenerationParams.minP default and copyWith behavior.feat(template): harden llama.cpp tool-call parity and grammar stability by @leehack in https://github.com/leehack/llamadart/pull/44
Full Changelog: https://github.com/leehack/llamadart/compare/v0.5.1...v0.5.2
ToolCallGrammarUtils helpers for wrapped object/array tool-call grammar generation and root-rule wrapping.llama_grammar_init_impl parse failures during tool-calling generations.chore: prepare 0.5.1 docs and publishing cleanup by @leehack in https://github.com/leehack/llamadart/pull/42
Full Changelog: https://github.com/leehack/llamadart/compare/v0.5.0...v0.5.1
.pubignore to exclude local build outputs, large model/test artifacts, and checked-out third_party sources from package uploads.feat: increase code coverage to >75% (core logic at 82%) by @leehack in https://github.com/leehack/llamadart/pull/34
Full Changelog: https://github.com/leehack/llamadart/compare/v0.4.0...v0.5.0
[BREAKING] Public API Changes:
ToolRegistry, LlamaTokenizer, and ChatTemplateProcessor are no longer part of the public package API.ChatSession now centers on create(...) streaming LlamaCompletionChunk; legacy chat(...) / chatText(...) style usage must migrate.LlamaChatMessage constructor names were standardized (.fromText, .withContent) in place of older named constructors.maxTokens in GenerationParams increased from 512 to 4096.LlamaChatMessage.toJson() no longer includes name on tool role messages.ModelParams.logLevel was removed; logging control now lives on LlamaEngine via setDartLogLevel(...) and setNativeLogLevel(...).LlamaBackend interface changed for custom backend implementers (notably getVramInfo and updated applyChatTemplate).loadModel(...) now requires unloading first.MIGRATION.md.Template/Parser Parity Expansion:
<|python_tag|> parsing for Llama 3 flows.Template Extensibility APIs:
ChatTemplateEngine.customTemplate and customHandlerId routing support and threaded handler identity into parse paths.Logging Controls:
LlamaEngine: setDartLogLevel and setNativeLogLevel, while keeping setLogLevel as a convenience method.none log level suppression so llama.cpp/ggml logs are fully muted when requested.Chat App Improvements:
Test Suite Overhaul:
feat: Chat Session Manager (#12) by @leehack in https://github.com/leehack/llamadart/pull/14
Full Changelog: https://github.com/leehack/llamadart/compare/v0.3.0...v0.4.0
LlamaBackend for strict Web isolation using "Native-First" conditional exports, ensuring native performance and full web safety.LlamaBackend() factory across all examples and scripts.getLoadedContextInfo() and robust GGUF metadata fallback in LlamaEngine.llama.cpp native backend.dart_test.yaml and @TestOn tags to enable seamless execution of all tests across VM and Chrome with a single dart test command.dup2 to /dev/null) for LlamaLogLevel.none on native platforms.dart analyze across the core library and all example applications.ModelService architecture..meta files to track download progress across app restarts.ModelCard with a visual Pause/Resume toggle.mtmd module from llama.cpp for native platforms.
loadMultimodalProjector to LlamaEngine.LlamaChatMessage.withContent and LlamaContentPart (Text, Image, Audio).mtmd module.Question: / Answer: chat template fallback for Moondream models.chat() and chatWithTools() logic from LlamaEngine to ChatSession.LlamaEngine is now a dedicated low-level orchestrator for model loading, tokenization, and raw inference.dispose() in ChatService.ChatSession class to automatically manage conversation history and system prompts.ChatSession now implements an automated sliding window to truncate history when the model's context limit is approached.llama.cpp updates, regression testing, and release artifact generation.LlamaChatMessage.role now returns a LlamaChatRole enum instead of a String. All manual role string comparisons should be updated to use the enum.LlamaChatMessage constructor (string-based) is now deprecated; use .fromText() or .withContent() instead.LlamaChatMessage.roleString is deprecated and will be removed in v1.0.llama.cpp to tag b7898.feat: implement LoRA support and training pipeline by @leehack in https://github.com/leehack/llamadart/pull/3
Full Changelog: https://github.com/leehack/llamadart/compare/v0.2.0...v0.3.0
LlamaService: The legacy LlamaService facade has been removed. Use LlamaEngine with LlamaBackend() instead for all platforms.wllama v2 features, including native chat templating and threading info.LlamaLogLevel.none suppresses all output; other levels enable default stderr logging.NativeFinalizer dependency to avoid race conditions. Explicitly call dispose() to release native resources.ModelParams to include initial LoRA configurations and introduced supportsUrlLoading for better platform abstraction.basic_app example to support testing LoRA adapters via the --lora flag.Feat/build infra refinement by @leehack in https://github.com/leehack/llamadart/pull/2
Full Changelog: https://github.com/leehack/llamadart/commits/v0.2.0
WASM Support: Full support for running the Flutter app and LLM inference in WASM on the web.
wllama integration with better error handling and progress reporting.Supported platforms: iOS, macOS, Android, Linux, Windows, Web.
llama.cpp backend.One column per month.