marionette_flutter

See changelog on pub.dev

What's Changed

• chore: bump mcp_dart to v2.1.0 by @jakubfijalkowski with @Copilot in #71
• feat: surface Semantics(label) and Semantics(value) in get_interactive_elements by @sthefanoss in #72
• refactor: split VM service context and Marionette binding into themed files by @MatejLNCD in #73
• feat: add support for secondary tap by @lukaszgarstecki in #77
• feat: promote schema-bearing custom extensions to first-class MCP tools by @MatejLNCD in #78
• docs: restructure README into slim landing page + docs/ guides by @mchudy in #80
• feat: add swipe command to the CLI by @mchudy in #82
• feat: add hot restart support by @mchudy in #81
• fix: exit stdio server on stdin EOF; drop custom Copilot transport by @mchudy in #86
• fix: sanitize custom-extension tool names for strict MCP clients by @mchudy in #87
• Add Dockerfile for Docker MCP Registry submission by @mchudy in #83
• feat: add keyboard input support (press_key) by @mchudy in #88
• chore: prepare 0.6.0 release by @mchudy in #91
• fix: declare meta as a direct dependency of marionette_flutter by @mchudy in #92

New Contributors

• @sthefanoss made their first contribution in #72
• @MatejLNCD made their first contribution in #73
• @lukaszgarstecki made their first contribution in #77

Full Changelog: v0.5.0...v0.6.0

• Promote schema-bearing custom extensions to first-class MCP tools via the new ExtensionInputSchema/ExtensionParam DSL
• Add press_key tool for dispatching hardware key events with modifiers
• Add hot_restart tool for fully restarting the app from main()
• Add secondary_tap tool for right mouse button clicks on desktop
• Add swipe CLI command for element- and coordinate-based swipe/drag gestures
• Surface Semantics(label:) and Semantics(value:) in get_interactive_elements
• Exit the stdio MCP server on stdin EOF to prevent orphaned processes
• Bump mcp_dart to ^2.1.0 and drop the bespoke Copilot-compat stdio transport
• Add a Dockerfile for building marionette_mcp as a container image

• Promote schema-bearing custom extensions to first-class MCP tools. Registering an extension with an inputSchema (declared via the new typed ExtensionInputSchema / ExtensionParam DSL) now surfaces it as an individually-named, discoverable tool with a validated input schema — the agent gets argument hints and enum autocomplete, invalid arguments are rejected before reaching your app, and declared defaults are applied before the callback runs. Promoted tool names are sanitized for strict clients (e.g. VS Code Copilot's [a-z0-9_-] -only rule) while the extension is still invoked by its real name. Schema-less extensions stay supported through the generic call_custom_extension tool.

• Add the press_key tool (plus the press-key CLI command) for dispatching real hardware key events to the focused element — named keys ( enter , tab , escape , arrows, etc.) and single characters, with optional control/shift/alt/meta modifiers. Unlike enter_text , which rewrites a field's value, this drives form submission, focus traversal, dismiss, and app shortcuts.

• Add the hot_restart tool (plus the hot-restart CLI command), mirroring hot reload but fully restarting the app from main() and resetting all state. Requires the app to be launched via flutter run ; the connector re-resolves the new root isolate and re-registers its extensions after the restart.

• Add a swipe CLI command, wiring the existing swipe/drag gesture into the CLI in both element-based ( --key / --text / --type + --direction ) and coordinate-based ( --start-x / --start-y / --end-x / --end-y ) modes.

• Add the secondary_tap tool (plus the secondary-tap CLI command) for right mouse button clicks on desktop. It dispatches a mouse pointer with the secondary button pressed, triggering Flutter's onSecondaryTap (e.g. context menus). Touch tap is unchanged.

• Surface Semantics(label:) and Semantics(value:) in get_interactive_elements . Widgets that render via inline-span trees ( Text.rich , custom-painted text, etc.) where toPlainText() loses structure can now be made fully readable by agents through an explicit accessibility annotation, without altering the rendered widget.

• Bump the mcp_dart dependency to ^2.1.0 .

• Exit the stdio MCP server when its stdin reaches EOF (e.g. the MCP host crashes or disconnects without sending a signal), preventing orphaned marionette_mcp processes from accumulating. Removed the bespoke Copilot-compat stdio transport in favour of mcp_dart 's built-in StdioServerTransport — mcp_dart 2.1.0 parses GitHub Copilot's object-typed tasks capabilities natively, so the workaround is no longer needed.

• Add a Dockerfile so marionette_mcp can be built as a container image and submitted to the Docker MCP Registry; the image installs the published package from pub.dev and runs the server over stdio.

See changelog on pub.dev

What's Changed

• feat: extract text callback exposes element instead of widget by @thangmoxielabs in #61
• Fix input validation, error handling, and resource cleanup by @alperduzgun in #55
• feat:add swipe/drag gesture tool by @BottlePumpkin in #54
• fix: format vm_service_context.dart by @Copilot in #65
• Rework publishing flow and add min-version compat job by @jakubfijalkowski in #67
• feat: add pinch zoom gesture simulation by @alperduzgun in #56
• feat: add double tap gesture support by @alperduzgun in #53
• feat: add video recording with TCP/WS transport and Android auto-fallback by @Peetee06 in #62
• feat: add press_back_button to simulate system back navigation by @alperduzgun in #59
• chore: bump versions to 0.5.0 by @mateusz-pietras in #68

New Contributors

• @thangmoxielabs made their first contribution in #61
• @BottlePumpkin made their first contribution in #54
• @Peetee06 made their first contribution in #62

Full Changelog: v0.4.0...v0.5.0

• Breaking: Pass Element instead of Widget to the extractText configuration callback, giving access to the full element context. Migration: change (widget) to (element) and use element.widget to access the widget.
• Add gesture support across all layers: swipe/drag, long_press, double_tap, pinch_zoom, and press_back_button.
• Add video recording with TCP/WS transport and Android auto-fallback.
• Improve screencast stability with broken-pipe race handling, concurrent stop safety, and orphaned session cleanup.
• Improve runtime robustness around input validation, resource cleanup, and non-monotonic timestamp handling.
• Strengthen release CI with explicit minimum-version compatibility checks and Flutter version sourced from .fvmrc.

• Breaking: Pass Element instead of Widget to the extractText configuration callback, giving access to the full element context. Migration: change (widget) to (element) and use element.widget to access the widget.

• Add gesture support across all layers: swipe/drag , long_press , double_tap , pinch_zoom , and press_back_button .

• Add video recording with TCP/WS transport and Android auto-fallback.

• Improve screencast stability with broken-pipe race handling, concurrent stop safety, and orphaned session cleanup.

• Improve runtime robustness around input validation, resource cleanup, and non-monotonic timestamp handling.

• Strengthen release CI with explicit minimum-version compatibility checks and Flutter version sourced from .fvmrc .

See changelog on pub.dev

What's Changed

• Fix custom text extraction by @jakubfijalkowski in #26
• task: update codeowners by @mateusz-pietras in #31
• Decouple the packages from logging by @jakubfijalkowski in #28
• Allow calling arbitrary flutter services through the MCP by @jakubfijalkowski in #30
• docs: document MarionetteBinding test incompatibility by @Copilot in #38
• Fix pointer device collision with macOS mouse cursor by @RyanCarrier in #32
• Fix onChanged callback not being called on enter_text by @mchudy in #41
• Add focused selector to enter_text by @mchudy in #42
• Introduce CLI by @mchudy in #39
• Make ScrollSimulator scroll backwards to find the item if needed by @mchudy in #43
• Fix taps/scrolls when there are modal barriers by @jakubfijalkowski in #48
• ci: add test execution steps to CI workflow by @alperduzgun in #49
• Improve Pub metadata by @mateuszwojtczak in #60
• Prepare release 0.4.0 by @Copilot in #57

New Contributors

• @mateusz-pietras made their first contribution in #31
• @RyanCarrier made their first contribution in #32
• @mchudy made their first contribution in #41
• @alperduzgun made their first contribution in #49

• Add marionette_cli package with multi-instance CLI support
• Add call_custom_extension tool for arbitrary VM service extensions
• Add custom extension registry with registerMarionetteExtension
• Add focused selector to enter_text
• Add version compatibility check on connect
• Decouple logging from marionette_flutter into marionette_logger and marionette_logging
• Improve ScrollSimulator with reverse scrolling direction and configurable max scroll attempts
• Fix tap matching widgets behind modal route barriers
• Fix onChanged not being called on enter_text
• Fix pointer device collision with macOS mouse cursor
• Fix extracted text not being exposed in interactive elements
• Ensure correct status code range in MarionetteExtensionError

• Add marionette_cli package with multi-instance CLI support

• Add call_custom_extension tool for arbitrary VM service extensions

• Add custom extension registry with registerMarionetteExtension

• Add focused selector to enter_text

• Add version compatibility check on connect

• Decouple logging from marionette_flutter into marionette_logger and marionette_logging

• Improve ScrollSimulator with reverse scrolling direction and configurable max scroll attempts

• Fix tap matching widgets behind modal route barriers

• Fix onChanged not being called on enter_text

• Fix pointer device collision with macOS mouse cursor

• Fix extracted text not being exposed in interactive elements

• Ensure correct status code range in MarionetteExtensionError

See changelog on pub.dev

• Add screenshot resizing configuration
• Fix SIGTERM handling on Windows
• Add GitHub Copilot support
• Require Dart 3.10 for the MCP server
• Fix hot_reload tool response handling

• Add screenshot resizing configuration

• Fix SIGTERM handling on Windows

• Add GitHub Copilot support

• Require Dart 3.10 for the MCP server

• Fix hot_reload tool response handling

See changelog on pub.dev

• Add CI workflow with analyze, format, and version checks
• Add version generation script with validation and robustness
• Replace example symlinks with actual directories in both packages
• Improve project structure and dependencies

See changelog on pub.dev

• Add example symlinks for improved pub.dev package scoring
• Fix documentation lints for angle brackets in code references

See changelog on pub.dev

• Add compatibility for Copilot initialize call
• Make configuration parameter optional in MarionetteBinding.ensureInitialized()

• Add compatibility for Copilot initialize call

• Make configuration parameter optional in MarionetteBinding.ensureInitialized()

Release v0.2.1 with new README

• Update README

Update CHANGELOG and bump version to 0.2.0

• Support tapping by widget type
• Support tapping by coordinates
• Return more info in get_interactive_elements
• Add hot reload tool, and return logs only since last hot reload
• get_interactive_elements returns only hittable elements now
• The tools have better naming to conform to the MCP implementations
• scroll_to does not assume that the widget is in the tree
• Text is a stopwidget now

• Support tapping by widget type

• Support tapping by coordinates

• Return more info in get_interactive_elements

• Add hot reload tool, and return logs only since last hot reload

• get_interactive_elements returns only hittable elements now

• The tools have better naming to conform to the MCP implementations

• scroll_to does not assume that the widget is in the tree

• Text is a stopwidget now

Require Flutter when building marionette_flutter

• Initial version of Marionette MCP
• Support for getting the interactive widget tree
• Support for entering text
• Support for tapping and scrolling
• Support for getting logs from logging package

• Initial version of Marionette MCP

• Support for getting the interactive widget tree

• Support for entering text

• Support for tapping and scrolling

• Support for getting logs from logging package

58

likes

160

points

153k

downloads

Documentation

API reference

Publisher

leancode.co

Weekly Downloads

Metadata

Flutter extensions for AI agent interaction via MCP — lets Claude, Copilot, and Cursor tap, scroll, type, and screenshot your app.

Repository (GitHub)

View/report issues

Topics

#mcp #ai #claude #agent #copilot

License

Apache-2.0 ( license )

Dependencies

flutter , meta , web

More

Packages that depend on marionette_flutter

← Metadata

58

likes

160

points

153k

downloads

Documentation

API reference

Publisher

leancode.co

Weekly Downloads

Metadata

Flutter extensions for AI agent interaction via MCP — lets Claude, Copilot, and Cursor tap, scroll, type, and screenshot your app.

Repository (GitHub)

View/report issues

Topics

#mcp #ai #claude #agent #copilot

License

Apache-2.0 ( license )

Dependencies

flutter , meta , web

More

Packages that depend on marionette_flutter

Back