flutter_gemma

What's Changed

• feat(builtin-ai): web arm via Chrome Prompt API (LanguageModel) — 0.2.0 by @DenisovAV in #446

Full Changelog: v1.6.4...v1.6.5

• Tool-declaration injection is a declarative InferenceChat flag, not a hardcoded gemma4 check (no behavior change).

What's Changed

• fix(core): stop claiming background_downloader's updates stream from the host by @DenisovAV in #450
• chore: one CHANGELOG line per issue, and write that into the rules by @DenisovAV in #451
• fix(android,ios): drop KGP from the Android plugins (#440); core iOS floor 16 → 15 (#441) by @DenisovAV in #452

Full Changelog: v1.6.2...v1.6.4

• Android plugin no longer applies KGP — Flutter's Gradle plugin does (#440).
• iOS floor is 15.0; only flutter_gemma_mediapipe still needs 16.0 (#441).

• Download updates are scoped to our own task group; FileDownloader().updates stays the host's (#445).

What's Changed

• docs(website): vision/audio per-encoder backend (1.5.9 / litertlm 1.4.2) by @DenisovAV in #430
• chore(release): guard-release-docs hook + skill HARD RULES / Definition-of-Done by @DenisovAV in #431
• feat(genkit): 0.15 caps — JSON output, native toolChoice, context-window middleware (genkit_flutter_gemma 0.5.0) by @DenisovAV in #434
• docs(release-skill): publish always from an up-to-date main by @DenisovAV in #435
• fix: make the checks able to fail — CI on all 11 packages, and the guards that outlived their reason by @DenisovAV in #438
• ci(win): assert the shipped OpenVino runtime matches the dispatch's SDK by @DenisovAV in #437
• fix(litertlm): sizeInTokens asks the tokenizer instead of guessing by @DenisovAV in #439
• fix(native): the library did not arrive, and nothing said so — dead lookup paths + a hook that reported success by @DenisovAV in #436
• feat(onnx): ONNX Runtime engine (ORT-GenAI inference + plain-ORT embeddings) + runtime-agnostic embeddings 2.0.0 by @DenisovAV in #443
• fix(rag): filters that returned the wrong rows without saying so by @DenisovAV in #442
• fix(litertlm): Android generation broke when embeddings loaded the library first by @DenisovAV in #448
• feat(onnx): web arm — generation (Transformers.js) + embeddings, iOS emb fix, device tests by @DenisovAV in #449

Full Changelog: v1.5.9...v1.6.2

• Web: ModelFileType.onnx install is fileless — Transformers.js owns the repo, install just sets it active.

• getActiveModel compares every runtime param before reusing the cached model.
• Racing getActiveModel calls no longer share a model built for other params.
• Add FilterField.validateSchema — rejects duplicate field names, checked in release.
• Storage-specific name rules moved to the stores that have them, out of core.
• Document the canonical FieldEquals / FieldRange semantics both stores must honour.

• Add ModelFileType.onnx + ONNX engine chat-format routing — enables the new flutter_gemma_onnx engine.

What's Changed

• fix(litertlm): detect stream-callback ABI at runtime (LiteRT-LM v0.15.0) by @DenisovAV in #410
• chore(website): docs → flutter_gemma 1.5.7 by @DenisovAV in #423
• LiteRT-LM v0.16.0: fix the Android OpenCL leak, Windows GPU, and both NPU stacks by @DenisovAV in #425
• docs(changelog): credit @o-mid for the concurrent #391 fix (#421) by @DenisovAV in #424
• refactor(core): adopt @o-mid's _toRepoType helper + thread type through manager internal installs (#421) by @DenisovAV in #426
• feat(agent): image input in an agent turn + MCP 2025-11-25 (0.2.3) — rebuild of #407 by @DenisovAV in #427
• docs(agent): document imageBytes vision API in package README (0.2.4) by @DenisovAV in #428
• fix(litertlm): default vision + audio encoders to CPU, overridable per-encoder (1.5.9 / litertlm 1.4.2) by @DenisovAV in #429

Full Changelog: v1.5.7...v1.5.9

• Add getActiveModel(preferredVisionBackend:, preferredAudioBackend:) — per-encoder backend override (default CPU).

• docs: Windows discrete GPU is fixed in litertlm 1.4.0 — the README said it was still broken.

What's Changed

• chore(website): docs → flutter_gemma 1.5.6; document VoiceSession streamAudio by @DenisovAV in #420
• fix(core): tag installed STT/TTS/embedding models correctly so getInstalledModels lists them (#391) by @DenisovAV in #422

🙏 Credit: the #391 fix was also submitted independently by @o-mid in #421, opened before #422 landed — thank you for the fix and the report.

Full Changelog: v1.5.6...v1.5.7

• fix: getInstalledModels(stt/tts/embedding) returns installed models — were mistagged inference (#391; concurrent fix by @o-mid in #421).

What's Changed

• fix(agent): require flutter_gemma ^1.5.5 (onMaxToolTurns) by @DenisovAV in #418
• feat(speech): VoiceSession streamAudio (0.4.3) + fix(core): SDK tool-call JSON suppression (1.5.6) by @DenisovAV in #419

Full Changelog: v1.5.5...v1.5.6

• fix: Gemma 4 streaming no longer leaks raw tool-call JSON into the text channel.

What's Changed

• consolidate: AgentLoop delegates its tool loop to core (core 1.5.5 + agent 0.2.2) by @DenisovAV in #417

Full Changelog: v1.5.4...v1.5.5

• Add optional onMaxToolTurns to generateChatResponseWithTools (lets a driving consumer surface its own maxToolTurns terminal).

What's Changed

• feat(agent): stream final-answer tokens in AgentLoop (VoiceSession parity) by @DenisovAV in #415
• 1.5.4: balance committed tool-calls on a mid-stream generation error by @DenisovAV in #416

Full Changelog: v1.5.3...v1.5.4

• fix: balance committed tool-calls when the generation stream errors mid-turn (generateChatResponseWithTools).

• Add InferenceChat.generateChatResponseWithTools — the reusable function-calling driver loop.
• Add TtsModelType.inflect — Inflect-Nano-v2 TTS bundle, with cross-repo G2P file routing.
• fix: record the Gemma 4 SDK tool-call in history on the streaming path (was orphaned).

• Add TtsModelType.qwen3.

• fix: namespace companion install files (tokenizers, TTS bundle aux) per model, fixing STT/embedding tokenizer collisions.
• fix: migrate legacy (pre-namespacing) tokenizer installs on restore, so existing STT/embedding models keep working after upgrade.

• Add genai_primitives support via package:flutter_gemma/genai.dart — use ChatMessage with InferenceChat (#181).
• New sendMessage/generateContent (+ streams) covering text, vision, audio, thinking, and tool calls (#181).
• Add facade identity getters activeModelSpec/activeEmbedderSpec/activeSttSpec/activeTtsSpec + getModelPath.
• Add facade storage getStorageInfo/getOrphanedFiles/cleanupStorage/performCleanup — surface real errors, not success-shaped defaults.
• Add FlutterGemma.rag namespace (add/search/removeDocument/stats/clear); plumb removeDocument (#390 audit).
• Add uninstallEmbedder/uninstallStt/uninstallTts (delete all files); route uninstallModel off the deprecated path.
• Export ModelSource + FileSystemService from the barrel (required public parameter types).
• Document FlutterGemmaPlugin as the low-level SPI tier; FlutterGemma facade is the canonical entry.

• docs: document the on-device voice loop (VoiceSession, STT → LLM → TTS) shipped in flutter_gemma_speech 0.3.0.

• feat: on-device text-to-speech — selectable TTS model (Matcha today) via installTts()/getActiveTts()/synthesize(), raw PCM output.
• feat: core TTS contracts (SpeechSynthesizer, TtsModelType, TtsBackendProvider/TtsRegistry, ModelManagementType.tts) + RuntimeConfig.artifactPaths.
• fix: model-load failure no longer emits a spurious unhandled-async (orphaned completer in createStt/createModel/createEmbedding, #394).

• feat: on-device speech-to-text — new opt-in flutter_gemma_speech runs a selectable ASR model (moonshine today) on all native platforms via installStt()/getActiveStt()/transcribe().
• feat: STT contracts (SpeechRecognizer, SttModelSpec/SttModelType, SttBackendProvider/SttRegistry) + LiteRT engine consolidation into flutter_gemma_litertlm.
• fix: Windows fresh downloads landed in a $PWD-relative dir, not %LOCALAPPDATA% — model installed but not found at load.

• Docs: add community models to the README model list (SmolLM3-3B, Phi-4-mini-reasoning, Qwen2-VL, SmolVLM2, LLaVA-OneVision).
• Docs: bump web @litert-lm/core setup snippet to 0.14.0.

• Fix multi-GB background_downloader temp-file leak on Android (#383).
• Deterministic download task IDs (sha256 of path triple) — stable across restarts and signed-URL rotation (#383).
• getOrphanedFiles()/cleanupStorage() now surface and delete orphaned downloader fragments (#383).
• Cancel tasks before reset in cleanup so paused temp files are deleted (#383).
• Resume watchdog cancels the dead native task instead of leaking it (#383).

• Add ModelFileType.builtIn for OS system models (Gemini Nano / Apple FM)

• Cancel native .litertlm decoding before session teardown to avoid an ANR (#364, #373).
• Re-run macOS companion staging on incremental builds so LiteRtLm stays patched (#368).
• Fix FunctionGemma parser dropping array, number, boolean and object arguments (#366).
• Fix FunctionGemma streaming truncating a tool call at the first } (#366).
• Fix FunctionGemma tools prompt to match the model's chat_template (#367).
• Keep tool properties in declared order; sorting them would break existing fine-tunes (#367).
• Reject union-typed properties with a clear error instead of a raw _TypeError (#367).
• Emit parameters:{type:OBJECT} for no-argument FunctionGemma tools (#367).
• Keep an empty required:[] inside FunctionGemma items, as the template does (#367).
• Stop <end_function_call> leaking into FunctionGemma chat text and history (#366).
• Infer a FunctionGemma property type from properties/items, else require it (#367).
• Render numbers, booleans and None the way the template's Python does (#367).
• Parse a bare None argument as null instead of the string "None" (#366).
• Refuse a FunctionGemma call whose body cannot be parsed in full (#366).
• Warn instead of silently ignoring ToolChoice.required on FunctionGemma (#367).
• Send FunctionGemma tool results as the template does, not as result:{json} (#367).
• Stop opening a second model turn after a FunctionGemma tool response (#367).

• Fix AGP 9 build on android.builtInKotlin=false — apply KGP unless built-in Kotlin is on, so the plugin's Kotlin compiles (#360).

• Fix SmartDownloader resume-loop hang — cap resume attempts + arm a watchdog timer (#355).
• Fix foreground downloads on Android — configure a running notification so the service actually activates (#356).
• Scope foreground notification to explicit flag + auto-request POST_NOTIFICATIONS at runtime (#356).
• Document Android 14+ manifest requirement: host app needs FOREGROUND_SERVICE_DATA_SYNC + service merge (#356).
• Guard double-complete, add permission timeout, check resume()'s bool, arm reattach watchdog (#355 #356).

• Add skillExecutors: to FlutterGemma.initialize + the SkillExecutorProvider/SkillExecutorRegistry seam for the new opt-in flutter_gemma_agent package.
• Declare libOpenCL.so (+ -car/-pixel ICDs) in the plugin manifest — completes the #324 Mali GPU fix (#349; 1.1.1 only opened the SP-HAL door).
• Fix web "No active inference model set" right after install — await the model manager's rehydrate before reading the active model.
• Fix web Gemma 4 tool-calling — parse the raw <|tool_call> tokens the browser runtime leaves unconverted, and guard a crash when the SDK returns content as a String.

• Migrate Android module to Built-in Kotlin — apply KGP only on AGP < 9, ready for AGP 9+ (#323).
• Add optional downloadUpdatesStream/fileSystemService injection to FlutterGemma.initialize (#342, thanks @hughesyadaddy).

• Declare libvndksupport.so for the Android GPU backend — fixes Mali GPU hard-freeze on Android 12+ (#324).

• Add declared-column Filter support: FilterSchema/FilterField/FilterFieldType + configure(FilterSchema) on VectorStoreRepository.
• Add optional filterSchema: to FlutterGemma.initialize (threaded to the vector store at registration).
• Deprecate enableHnsw (no-op; vector search now runs inside the store engine).

• Add clearActiveInferenceIdentity/clearActiveEmbeddingIdentity (non-breaking defaults on ModelFileManager).
• Export DownloadError/DownloadException from flutter_gemma.dart.
• Example: downloaded-models list screen with delete/copy-path/load and gated-model 401/403 dialog.

• Add maxOutputTokens on createSession/openSession/createChat/openChat to cap generation length (#318).
• Clarify maxTokens dartdoc: it's the context window (input + output), not reply length.

• Fix empty assistant turn polluting chat history after stopGeneration() — cancelled/empty responses are no longer recorded (#325).
• Point homepage to fluttergemma.dev. No code change.

• Stable 1.0 of the modular package split (see 1.0.0-rc.1 below for full notes).
• dart2wasm-clean public graph — dart:io is off the web/wasm import graph.
• Dropped pigeon from core; value types are plain Dart in core/domain/platform_types.dart.
• SDK floor raised to Dart 3.12 / Flutter 3.44; large_file_handler ^0.5.0.

• Modular package split: core flutter_gemma + opt-in flutter_gemma_litertlm / flutter_gemma_mediapipe / flutter_gemma_embeddings / flutter_gemma_rag_qdrant / flutter_gemma_rag_sqlite.
• New FlutterGemma.initialize(inferenceEngines:, embeddingBackends:, vectorStore:) — register the opt-in packages you added; core registers none by default.
• No API changes to model / session / chat / embedding / RAG calls — see MIGRATION.md.
• Removed the legacy sqlite + local_hnsw vector store (native RAG → qdrant-edge).
• Fix Windows cancel/close hang: gate the cycle-fix dylib staging to Apple only (Windows registers straight from the cache).

• Fix embedding freezing the UI thread (#299): forward pass runs on a background isolate.
• Fix Windows build on non-UTF-8 locales (#212): add /utf-8 to the MSVC plugin target.
• Fix macOS "Cycle inside Flutter Assemble" build error (#300, thanks @fotiDim): stage native dylibs out of the cache dir.

• Android Qualcomm NPU (PreferredBackend.npu): auto-extracts QNN dispatch libs from APK at runtime (#293).
• Fix Android GPU sampler CPU fallback (#270): GPU OpenCL/WebGPU samplers now resolve correctly, restoring ~3× decode speed.
• qdrant-edge 0.7.1: drops vendored fork, wal_options now native in upstream.

• Concurrent sessions (#226): openSession()/openChat() run independent dialogues on one loaded model.
• Web .litertlm inference via @litert-lm/core early preview (WebGPU/WASM, text-only).
• Fix getActiveModel() after app restart (#227): mobile + web auto-restore from prefs.
• Report initialized backend (#288, thanks @merlinnot): InferenceModel.activeBackend getter + NPU→GPU→CPU fallback on the FFI path with BackendInitException carrying per-attempt details.
• Bump large_file_handler ^0.3.1 → ^0.4.0.

• LiteRT-LM v0.12.0 native bump (commit ffed38a): NPU dispatch now available on Linux/macOS as well as Windows.
• Fix iOS App Store upload (ITMS-90208) (#286): iOS LiteRT-LM and qdrant-edge artifacts repacked.
• Fix iOS Native Assets strip step on Xcode 26 (#289, thanks @merlinnot): emit libLiteRtLm.dylib with -Wl,-x so xcrun strip -x -S succeeds during release builds.
• Fix Windows install path when %LOCALAPPDATA% env var is relative — falls back to USERPROFILE\AppData\Local then getApplicationSupportDirectory().

Native LiteRT-LM prebuilts for flutter_gemma, built from LiteRT-LM 924e79c9 (v0.16.0) with LiteRT 0ff28117.

Consumed automatically by flutter_gemma_litertlm/hook/build.dart (Native Assets) at pub get time; SHA256-verified against the map baked into that hook. Previous release: native-v0.14.0 — there was no native-v0.15.0.

What changed

• Android OpenCL per-turn memory leak is fixed (LiteRT-LM #2699). This is the leak behind #348 and #402: RSS ratcheted ~65–300 MB per inference on the GPU path until the app was OOM-killed.
• Stream-callback ABI moved from a 4-argument callback to a 2-argument opaque chunk, upstream, with no compatibility path. libStreamProxy resolves the shape at runtime, so both old and new hosts work.
• Windows discrete GPU works again. The crash reported as LiteRT-LM #2957 was not an upstream regression: our Windows build passed --define=litert_link_capi_so=true, a name upstream had deleted. Bazel accepts unknown defines silently, so the LiteRt runtime was being linked statically, which conflicts with the separately shipped WebGPU accelerator once Dawn became its own library. Corrected to litert_runtime_link_mode=dynamic + resolve_symbols_in_exec=false. That issue has been retracted.
• Both NPU dispatch stacks are now built from the pin instead of carried forward:
  • Intel — LiteRtDispatch.dll plus a version-matched OpenVino runtime (2026.3.0.dev20260622). The carried-forward pair shipped OpenVino 2026.2.0 against a runtime pinned to 2026.3.0, which is what broke backend=npu.
  • Qualcomm — libLiteRtDispatch_Qualcomm.so rebuilt from the derived LiteRT ref, and the ten QNN runtime libraries refreshed from the same QAIRT 2.44.0.260225. The stale pair failed with Qnn System library version 1.8.0 is mismatched. The minimum supported version is 1.11.0.
• macOS deployment target is pinned. libStreamProxy.dylib had been inheriting the build host's OS since native-v0.14.0 and shipped with minos 26.0; it is now built with -mmacosx-version-min=11.0.

Verification

• Android — Snapdragon 8 Elite (SM8750), Android 16: 23/23 integration tests pass; CPU, GPU (OpenCL) and NPU all reach Engine initialized successfully, NPU engine_create in 498 ms.
• Windows — Intel Arc 140V: GPU 36.1 chunks/s; NPU functional on Lunar Lake.
• Linux — 24/24.
• Apple — all dylibs arm64; install_name_tool rewrite clean on every one (Native Assets re-runs it on each pub get); iOS minos 13.0, macOS 11.0; gpu_registry @executable_path patch present, basename dlopen absent.

Contents

SHA256 sums for every archive are in checksums_litertlm.txt.

• Native vector store: qdrant-edge by default. Replaces sqlite + local_hnsw on every native platform. Web unchanged (wa-sqlite). Old impl @Deprecated, removal in 1.0.
• Filter DSL for searchSimilar(... filter: Filter(must: [...], should: [...], mustNot: [...])). Honored on native; silently ignored on Web.
• Fix desktop install/validate path mismatch — all paths now route through FileSystemService; legacy Documents/ reads kept as fallback.

• example: add TranslateGemma 4B translation demo via task-first home navigation (#177).

• Unified embedding on LiteRT C API + Dart FFI on all native platforms (#264).

• Fix Android GPU sampler dlopen failure (#270, thanks @prithidevghosh): patchelf --add-needed libLiteRtLm.so on libLiteRtTopK{OpenCl,WebGpu}Sampler.so.
• Desktop storage (#179, co-author @ProjectEdge-Jim): use Application Support instead of Documents on Windows/macOS/Linux to avoid cloud-synced paths breaking FFI mmap.
• Multi-image, FFI session metrics, prefix replay (#262, thanks @frdteknikelektro): Message.withImages([...]) for multi-image input, chat/Conversation session metrics via FFI, persistent prefix messages replayed on session rebuild after history truncation. Backward-compatible (Message.withImage(...) still works).
• Skip sampler params on NPU backend: temperature/topK/topP/seed are silently ignored when PreferredBackend.npu is selected — LiteRT-LM NPU executor only supports internal greedy sampling.
• Windows Intel NPU end-to-end: native-v0.11.0-b Windows tarball bundles Intel dispatch (LiteRtDispatch.dll), OpenVino runtime, TBB (~30 MB); FFI client passes dispatch_lib_dir + use_hw_masking_for_npu=false at engine_create so PreferredBackend.npu works on Intel LunarLake/PantherLake silicon without manual DLL placement.
• Fix web build broken by 0.15.0: LiteRtLmFfiClient stub on web was missing the enableSpeculativeDecoding parameter — dart2js failed compilation when web target was actually built.
• CI: standalone build-litertlm-native-windows.yml workflow for Windows-only rebuilds.

• LiteRT-LM 0.11.0: MTP-capable Gemma 4 + speculative decoding on macOS / iOS / Android / Windows.
• enableSpeculativeDecoding flag on getActiveModel() (null = model default; true/false to override).
• Restore Android NPU support for .litertlm (regression from 0.14.0): PreferredBackend.npu on Android .litertlm models routes through LiteRT-LM's Backend::NPU again — same as 0.13.x's Kotlin path before the FFI migration silently dropped it. Requires a Qualcomm QNN / Google Tensor / MediaTek dispatch lib on the device; without one, engine_create fails with a dispatch error. MediaPipe .task models still don't support NPU (MediaPipe SDK limitation, unchanged).
• PreferredBackend.npu on desktop (#261): macOS / Linux / Windows backend arm wired. Same dispatch-lib requirement as Android.
• Linux known limitation: post-MTP HF Gemma 4 revisions blocked upstream — libLiteRtWebGpuAccelerator.so segfaults during graph compile (filed google-ai-edge/LiteRT-LM#2225) and multi-signature vision encoder rejected by native (engine_create error). Workaround: pin pre-MTP HF revision 7fa1d78473894f7e736a21d920c3aa80f950c0db for Linux users. Other platforms unaffected.
• hook native cache: marker-file based invalidation (.flutter_gemma_native_version) — bumps clean stale companion libs automatically.

• Fix desktop embedding on pub.dev installs (#250 follow-up): tensorflowlite_c.{dll,so,dylib} now bundled via Native Assets — regression from 0.14.0 setup-script removal.

• Fix macOS dylib loading on pub.dev installs (#255).
• Fix fromAsset install on desktop (#250 mode 2).
• Fix Android x86_64 emulator crash on embedding init (#250 mode 3).
• Reduce native lib size: iOS -63%, macOS -43%, Android -28%, Linux -16-18%.

• Fix App Store ITMS-90208 rejection on iOS (#245): downgraded patched libGemmaModelConstraintProvider.dylib minos 26.2 → 14.0 to match the other companion dylibs.
• Fix Android Google Play 16KB page-size requirement (#253): rebuilt libLiteRtLm.so with -Wl,-z,max-page-size=16384.
• Fix Web RAG build pipeline (#251): cross-platform npm run build, bundles cache_api.js + LiteRT WASM into dist/, ships package.json + vite.config.js in pub tarball.

• Fix macOS install_name_tool failure (#247): dart run / build_runner / flutter test on a pure-Dart library aborted with larger updated load commands do not fit because upstream Apple companion dylibs lack -headerpad_max_install_names. Skip them from Native Assets on macOS and bundle via Podfile post_install instead.
• Fix Gemma 4 escape-token leakage in chat history (#248): <|"|> tokens from tool_calls.arguments were written to history raw, making the model echo them on later turns. Strip recursively before persist via new SdkResponseParser.cleanRawForHistory.
• Fix modelFromAsset install on desktop (#250): the large_file_handler channel call threw MissingPluginException on macOS / Windows / Linux because the package only ships Android + iOS plugins. Now catches it and falls back to the in-memory loadAsset → writeFile path. Embedding (localagents-rag) and .litertlm FFI paths additionally fail fast with a typed message on non-arm64 Android (x86_64 emulator / armeabi-v7a) instead of a generic JVM crash — see new "Platform & Architecture Support" section in README.
• macOS Podfile setup (BREAKING for macOS app projects): app projects must update macos/Podfile post_install to the new snippet (see README → macOS Setup). iOS / Linux / Windows / Android unaffected.
• Perf debug logs: [*/perf] lines break out cost of dylib load, engine_create, prefill, decode — visible via debugPrint in both debug and release builds.

• Web build fix (#244): 0.14.0 broke web compilation by statically importing core/ffi/litert_lm_client.dart (which imports dart:ffi) through the flutter_gemma_interface.dart → mobile/flutter_gemma_mobile.dart chain. JS/Wasm targets cannot compile dart:ffi. Added conditional imports in mobile/flutter_gemma_mobile.dart that swap litert_lm_client.dart and ffi_inference_model.dart for *_stub.dart shims on dart.library.js_interop. The web plugin (FlutterGemmaWeb) registers itself as FlutterGemmaPlugin.instance before any FFI code path runs, so the stubs' constructors (which throw UnsupportedError) are never actually invoked on web.
• App Store fix (#245, ITMS-90432): 0.14.0 dropped lib*.dylib symlinks alongside the bundled .framework/ accelerators in Runner.app/Frameworks/ so LiteRT-LM's gpu_registry could dlopen them by basename — App Store Connect rejected those builds with "Unexpected file found in Frameworks". 0.14.1 patches the upstream LiteRT-LM source (runtime/components/sampler_factory.cc and litert/runtime/accelerators/gpu_registry.cc) to load Apple platform accelerators via @executable_path/../Frameworks/<X>.framework/<X> (macOS) / @executable_path/Frameworks/<X>.framework/<X> (iOS) instead of libX.dylib. Native Assets bundles the framework bundles correctly out of the box; no host-side Podfile symlinks needed. Patch applied during local Bazel rebuild — see native/litert_lm/patch_c_api.sh section 10.
• Gemma 4 native function calling: New ModelType.gemma4 routes tool definitions to LiteRT-LM SDK via litert_lm_conversation_config_set_tools (OpenAI Chat Completions JSON). SDK applies chat_template.jinja through minja, renders native <|tool>declaration:...<tool|> tokens, and parses the model's <|tool_call>...<tool_call|> response back into structured tool_calls JSON. flutter_gemma reads the result via SdkResponseParser.extractToolCalls (handles parallel calls and the multimodal content[] path) and returns FunctionCallResponse to the app — no Dart-side prompt engineering needed.
• Escape token cleanup: Strips leftover <|"|> Gemma 4 escape tokens from string arguments (recursively, including nested maps/lists).
• example/lib/models/model.dart: Gemma 4 E2B / E4B entries switched to modelType: ModelType.gemma4.

• Desktop FFI rewrite: macOS, Linux, Windows now run LiteRT-LM directly via dart:ffi against the C API. Removed Kotlin/JVM gRPC server, Azul Zulu JRE 24 download, and litertlm-server.jar bundling. Engine creation ~2 s (was ~10–15 s incl. JVM cold-start)
• iOS GPU acceleration: Metal delegate for .litertlm models on iPhone (Gemma 3 1B, Gemma 3n E2B, Gemma 4 E2B). Multimodal vision + audio work on device
• Linux + Windows GPU: WebGPU/Vulkan on Linux, WebGPU/DX12 on Windows; DXC runtime (dxil.dll + dxcompiler.dll v1.9.2602) bundled in the Windows native archive — no manual install required
• Android: drop Kotlin LiteRtLm dependency — .litertlm models on Android now go through the same Dart FFI path as desktop/iOS (was com.google.ai.edge.litertlm:litertlm-android AAR before). MediaPipe stays for .task/.bin models
• iOS Simulator: CPU-only — Metal sim has 256 MB single-allocation cap, LLM weight tensors exceed it
• Unified FFI client: All five platforms (Android, iOS, macOS, Linux, Windows) share LiteRtLmFfiClient (lib/core/ffi/)
• Native log capture: stream_proxy_redirect_stderr exposes glog/abseil output on iOS/Android via temp file; helps diagnose engine init failures
• Native prebuilts: Fetched at build time via hook/build.dart from GitHub release native-v0.10.2; SHA256-verified, bundled via Native Assets
• macOS / iOS setup: requires a small post_install block in your Podfile to symlink lib*.dylib next to the bundled .frameworks — gpu_registry calls dlopen by basename. See the macOS Setup section in README for the exact snippet (iOS works via the same pattern in example/ios/Podfile)

• ModelType.qwen3: New model type for Qwen3 models with thinking support
• Disable Qwen3 thinking at model level: /no_think appended automatically when isThinking: false — faster TTFT
• Configurable maxFunctionBufferLength (#229): createChat(maxFunctionBufferLength: 2048) for long function call args
• Fix Windows path parsing (#233): FileSource now handles backslash paths correctly
• removeDocument (#232): VectorStoreRepository.removeDocument(id:) to delete documents from vector store

• Fix Qwen3 thinking mode (#224): Qwen3 <think> tags now stripped automatically

• Fix iOS compile error (#222): XNNPack delegate type mismatch in EmbeddingModel.swift
• Fix iOS arm64 simulator (#216): Removed TensorFlowLiteSelectTfOps — simulator builds work on Apple Silicon

• Fix macOS SIGSEGV (#219): Per-conversation mutex in gRPC server prevents conversation.close() racing with sendMessageAsync on a native thread → use-after-free in C++ fixed
• Fix macOS desktop Metal accelerator: setup_desktop.sh now downloads libLiteRtMetalAccelerator.dylib from GitHub Release so GPU inference uses the Metal delegate instead of falling back to static C API
• Fix iOS pod install hanging (#220): Replaced TensorFlowLiteSwift (source pod — cloned entire TensorFlow repo) with direct TensorFlowLiteC C API in EmbeddingModel.swift
• Fix Windows >2 GB model error (#212): Clear error message when model file exceeds 2 GB on Windows (known upstream 32-bit stat() overflow in litertlm_jni.dll, google-ai-edge/LiteRT-LM#1494)
• Fix iOS arm64 simulator build (#216): Excluded arm64 from simulator archs to fix build on Apple Silicon Macs

• FileSource absolute paths: Accept both Unix (/path) and Windows (C:\path) absolute paths in FileSource validation
• Package metadata: Updated pubspec description to reflect current feature set (desktop, vision, audio, function calling, embeddings, on-device RAG)
• Desktop CI: Added Linux smoke tests for gRPC server integration

• LiteRT-LM 0.10.0: Updated Android and JVM SDK from 0.9.0 to 0.10.0
• Gemma 4 Thinking Mode: isThinking: true now works with Gemma 4 E2B/E4B models (Android, iOS, Desktop; not Web)
• Fix cancel download: Cancel download now works correctly (#196)
• Fix large_file_handler platform support: Conditional imports for pub.dev platform analysis compatibility

• Gemma 4 E2B/E4B: Added support for next-gen multimodal models (text + image + audio)
• systemInstruction: New parameter in createChat() and createSession() for setting system-level context
• ModelFileType.litertlm: New file type to properly handle .litertlm models across platforms
• iOS LiteRT-LM support: .litertlm models now work on iOS
• stopGeneration on iOS: Supported for .litertlm models
• MediaPipe GenAI 0.10.33: Updated iOS (from 0.10.24) and Android (from 0.10.29)
  • iOS: GPU backend selection via preferredBackend (Metal delegate now activated)
  • iOS: Audio modality support (addAudio + enableAudioModality)
  • Android: Unified inference engine (CPU-only path removed), improved error handling
  • Web: Updated to 0.10.27
• Example WASM compatibility: Replaced direct dart:io imports with conditional imports for WASM compilation support
• Benchmark integration test: example/integration_test/benchmark_comparison_test.dart for comparing model performance on device

• ToolChoice enum: auto / required / none parameter in createChat() to control tool calling behavior
• Parallel Tool Calls: ParallelFunctionCallResponse for multiple function calls in one response
• Strategy Pattern Parser: Per-model FunctionCallFormat implementations (Gemma, Qwen, DeepSeek, Llama, Phi, FunctionGemma)
• <tool_call> Format: Qwen/Mistral-style function call parsing
• ModelType.phi: Dedicated model type for Phi-4 with <|tool_calls|> format support
• NPU Fix: Pass nativeLibraryDir to LiteRT-LM Backend.NPU()
• Embeddings: Models return L2-normalized vectors (dot product = cosine similarity)
• Windows/Linux Embeddings Fix: TFLite C library now correctly copied to build output (#200)

• Dual-Prefix Embeddings (TaskType): Improved RAG retrieval quality with query/document prefixes
  • TaskType.retrievalQuery (default) — for search queries
  • TaskType.retrievalDocument — for document indexing
  • Follows Google RAG SDK convention (EmbedData.TaskType)
  • All platforms: Android, iOS, Web, Desktop
  • addDocument() automatically uses document prefix
• Desktop Embeddings: Run .tflite embedding models (EmbeddingGemma, Gecko) on macOS, Windows, Linux
  • LiteRT C API via dart:ffi — no gRPC, no JVM overhead
  • Pure Dart tokenizer via dart_sentencepiece_tokenizer (BPE + Unigram, auto-detect format)
  • LiteRT C library built from google-ai-edge/LiteRT v2.1.3
  • XNNPACK delegate with default options (QS8/QU8 quantization support)
  • Desktop scores match Android/Python exactly (cosine similarity 0.708)
  • CI workflow for building LiteRT C library on all 4 platform/arch combinations
• Unified VectorStore: Single Dart implementation using sqlite3 dart:ffi replacing platform-specific code
• Test Migration: Removed patrol dependency, migrated all integration tests to standard integration_test

• LiteRT-LM 0.9.0-beta: Updated from 0.9.0-alpha02 on Android and Desktop (JVM)
  • Breaking API change: Backend enum to Backend factory constructors
• Cancel Generation: Implemented on Android, Desktop, and Web
  • Android LiteRT-LM: Conversation.cancelProcess()
  • Desktop: gRPC CancelGeneration RPC
  • Web: LlmInference.cancelProcessing() (MediaPipe 0.10.26)
• MediaPipe Web 0.10.26: Pinned CDN version (was @latest)
• E2E Integration Tests: Full inference test suite
  • Parameterized tests for both MediaPipe and LiteRT-LM engines
  • Multimodal tests: vision (Android, iOS, Web, Desktop) + audio (Android, Desktop)
  • Cancel, lifecycle, dual-engine tests