agent_device

Co-Authored-By: Claude Fable 5 [email protected]

Ports the actionable slice of the upstream 0.18.0 → 0.18.2 gap (102 commits triaged; the bulk — command-descriptor/PlatformPlugin refactors, daemon internals, web/CDP, cloud providers — is out of scope for this mobile-only, daemon-less port). See PORTED_COMMITS.md for the per-commit registry.

Performance (iOS)

• open --relaunch now works and is fast: the flag was previously accepted but ignored (and not exposed on the CLI). Simulators relaunch via a single simctl launch --terminate-running-process call, the XCUITest runner stays hot across the relaunch, and a 5s booted-memo removes repeat simctl list devices -j spawns — 0.54s wall on an iPhone 16e simulator, with exactly one device listing per invocation. Real devices keep the conservative runner teardown. (#1010, #1024)
• The cached runner build no longer rebuilds on package version bumps (content fingerprint only), and the runner app dropped its asset catalogs + compiles unit-test code out of runtime builds. Warm-cache runner relaunch: ~4.4s. (#900)

Reliability (iOS)

• Keyboard dismissal in the Swift runner no longer swipes the keyboard down or falls back to coordinate taps (both could trigger unintended UI actions); only safe native dismiss controls are used, with a clear UNSUPPORTED_OPERATION otherwise, and the dismiss-control search now spans all buttons, not just toolbars. (#957)
• Retained runner lifetime is bounded: a runner idle past 5 minutes (env AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS, 0 disables) is stopped and relaunched at the next reconnect instead of being adopted. (#1025)
• The physical-device install timeout is a named constant keeping the 180s end-to-end budget (upstream splits 120s exec under a 180s daemon budget; daemon-less, the exec timeout is the whole budget). (#964)

Observability

• Opt-in exec diagnostics: AGENT_DEVICE_EXEC_TRACE=1 (or --debug) records every external command spawn (duration, exit path, first 6 args) to the per-invocation diagnostics log. The CLI now opens a root diagnostics scope, which also un-deadens all pre-existing diagnostic events (screenshot fallbacks, fill verification) in CLI runs. (#1019)

Triaged as not applicable

Runner keepalive log tuning (#1017 — the keepalive timer was never ported), prepare-deadline threading (#967 — no daemon request envelope), and the daemon/web/cloud refactor campaign (~80 commits).

feat(ios): external/CI-signed runner flags + fix stale pinch test; v0…

Closes the upstream 0.16.10 → 0.18.0 gap (179 commits triaged; the bulk — Maestro, daemon internals, web/CDP, MCP, the 0.18.0 TypeScript refactors — is out of scope for this mobile-only, daemon-less port).

New Capabilities (ported from agent-device)

• External / CI-signed iOS runner: --ios-xctestrun-file, --ios-xctest-derived-data-path, and --ios-xctest-env-dir use a prebuilt .xctestrun artifact and skip the build entirely. The artifact is trusted as-is, so a pre-signed device runner is reused without re-signing, and the env-injected copy can land in a writable scratch dir for read-only artifacts. (#806)
• scroll --duration-ms: honor a caller-provided scroll/swipe duration (0–10000 ms) on both platforms instead of a fixed default. (#866)

Bug Fixes

• Selector values now decode JSON-style escapes (\n \t \r \uXXXX, including surrogate pairs), not just escaped quotes. (#711)
• Block taps on covered/occluded snapshot targets via a geometric occlusion pass, wired into the iOS and Android snapshot pipelines. (#708)
• iOS runner command failures caused by a crashed target app are reclassified to IOS_TARGET_APP_CRASH with an actionable hint, read from the runner log tail. (#793, #797)
• Fail fast with an actionable hint when Developer Mode for Apple development tools is disabled (the runner would otherwise hang). (#792)
• Recover Android snapshots from system-only / empty UI-automator helper output. (#861)
• iOS adjustSlider corrected after the runner re-sync (the runner field is sliderSteps, distinct from the sequence command's steps array).

Internal

• iOS XCUITest runner Swift re-synced to upstream 4648051b: sequence/scroll fused runner commands, structured snapshot capture plans, a private-AX snapshot fallback, shared flat-snapshot filtering, gesture stabilization, and synthesized-tap orientation rotation.
• The DevToolsSecurity developer-mode preflight is cached per process (one check per session instead of per cold runner launch); the covered-node occlusion search is memoized.

• fix(find): type=<T> now matches a node's type (alias for role).
  Previously type was not a recognized selector key, so find type=Button
  fell through to a literal-substring search and silently returned nothing.
• Ship the corrected Android snapshot helper manifest sha256 (committed in
  the prior CI fix) to pub.dev so published-package users also get the
  prebuilt-APK fast path instead of the auto-build/uiautomator fallback.

Co-Authored-By: Claude Opus 4.8 (1M context) [email protected]

Bug Fixes

• Android snapshot now uses the bundled prebuilt helper APK directly. Its manifest carried a placeholder "sha256": "local-build" that failed validation, so the helper was silently discarded — falling back to an on-device auto-build (which fails in many environments) and then to a flaky uiautomator dump. Snapshots are now faster and far more reliable on Android, with no build toolchain required.
• find type=<T> now matches a node's type (e.g. type=Button). Previously type was not a recognized selector key, so the expression fell through to a literal-substring search and silently returned no matches; it now aliases role.

Fix iOS command flakiness and harden device/runner concurrency.

• Session store: replace the non-blocking FileLock.exclusive (which made
  concurrent commands sharing a session fail with errno 35) with a FileMutex
  that queues intra- and cross-process.
• CLI: auto-detect the backend across platforms when --platform is omitted,
  so --serial / --device "" resolve regardless of platform;
  remember the resolved platform in the session to keep the fast path.
  DEVICE_NOT_FOUND now lists devices from every platform with labels.
• iOS runner: tolerant isAlive (no false-negative relaunch churn), a
  per-device cross-process launch lock with double-check, and per-device
  command serialization — all on a shared FileMutex.
• Regenerate version.dart from pubspec as part of make compile so
  ad --version can't drift (was stuck at 0.0.4).

Co-Authored-By: Claude Opus 4.8 (1M context) [email protected]

Reliability / Bug Fixes

• Concurrent commands sharing a session no longer fail with FileSystemException: lock failed … errno 35. The session-store advisory lock used the non-blocking FileLock.exclusive, so two overlapping ad invocations collided instead of waiting. Introduced a FileMutex (intra- and cross-process) that queues with bounded retry; concurrent commands now serialize cleanly.
• iOS commands without --platform no longer fail with a misleading DEVICE_NOT_FOUND that listed only Android devices. The CLI now auto-detects the backend across platforms, so --serial <udid> and --device "<name>" resolve regardless of platform. The resolved platform is remembered in the session so repeat commands skip re-enumeration and keep the fast path.
• DEVICE_NOT_FOUND now lists devices from every platform with platform labels and a clearer hint, instead of only the default backend's devices.
• iOS runner no longer "churns": isAlive tolerates a busy single-threaded runner (longer probe timeout + retries) instead of false-negatively declaring it dead and relaunching. Runner (re)launch is serialized per device with a cross-process lock and a double-check, so concurrent invocations don't each spawn their own xcodebuild.
• iOS runner commands serialize per device on that same lock, so overlapping commands queue rather than colliding on the single-command-at-a-time runner.

Internal

• New FileMutex lock primitive (utils/file_mutex.dart), used by the session store and the iOS runner launch/command paths.
• CommandSessionRecord gained a devicePlatform field (remembered alongside deviceSerial).
• version.dart is regenerated from pubspec.yaml as part of make compile, so the embedded ad --version can no longer drift from the published version.

Bump version to 0.0.8 and add changelog covering the work since the
unpublished 0.0.7: gesture command coverage + iOS transform gestures,
precise location settings, platform alerts, scroll-to-edge, the iOS
runner content-fingerprint staleness cache, device-enumeration skip,
CLI negative-positional + clean --json fixes, and the npm head-to-head
benchmark harness.

Co-Authored-By: Claude Opus 4.8 (1M context) [email protected]

New Capabilities (ported from agent-device)

• iOS transform gesture support and full gesture command coverage — pan / pinch / rotate / fling / transform driven through real multi-finger XCUITest synthesis
• iOS repro-evidence capabilities (#573)
• Precise location settings, with setSetting wired through the full stack (iOS location + Android) (#491)
• Auto-reverse adb localhost port forwards when opening URLs on Android
• Platform alert handling (alert accept/dismiss/text)
• Scroll-to-edge support and iOS scroll support
• Bundled pre-built Android multitouch helper APK (no on-device build needed)

Performance

• iOS: rebuild the XCUITest runner only when its source actually changes, using a content-fingerprint staleness cache (runner_build_cache.dart). Mirrors upstream's distribution staleness detection and eliminates the stale-build slowdown that made snapshot ~4× slower; restores parity with upstream.
• Skip device enumeration when both --serial and --platform are pinned (~300ms faster per command on Android)
• Direct iOS selector tap/fill optimization and faster hot iOS taps
• Cache app resolution

Bug Fixes

• CLI: accept negative-number positionals (e.g. gesture pan offsets) instead of misparsing them as short flags; keep --json stdout clean of progress/info lines (now routed to stderr)
• Android: clarify gesture transform behavior (#584); improve snapshot fidelity (#580); recover from app-owned ANRs; preserve scoped snapshot refs (#456); finishSafely in the snapshot-helper APK
• iOS: simplify interactive snapshots; compact unchanged-snapshot output
• Report blockers on wait timeout

Tooling

• Added a head-to-head benchmark harness (benchmark/) that drives the same app on the same iOS simulator / Android emulator through both this port and the upstream npm agent-device, measuring latency, command-level accuracy, and feature parity.

chore: fix analysis issue

Features

• add slider increment and decrement support for ios
• ios: slider control with vertical + horizontal support - experimental
• android: slider support via adb input swipe - experimental

Refactor

• deduplicate native assets via symlinks

chore: update version to 0.0.5

Features

• add update command

Features

• add Android frame health perf metrics (port of 0c7e48d7)
• add iOS frame perf sampling (port of cff8bd81)

Features

• find: enhance find command to support selector DSL, locator tokens, and substring queries

Bug Fixes

• implement retry logic for Android UiAutomation conflicts

Refactor

• migrate to cli_logger

Initial Dart port of agent-device (TS upstream). There are some changes to the upstream API to adjust it to the Dart workflow. The main differences are:

• no device session
• slight differences in the API
• way of bundling the native executables with the package