feat: add gesture telemetry recording overlays

[thymikee]

Summary

Replace recording-time touch visualization with a telemetry-first pipeline.

• capture gesture telemetry as a sidecar artifact for every recording, even when overlays are hidden
• render semantic overlays from telemetry for iOS and Android instead of relying on platform-native touch dots
• keep native video capture as the source of truth and post-process overlays as a final export step

Improve recording reliability and timing.

• unify Android recording on screenrecord + overlay rendering and remove the old screenshot-encoder path
• trim iOS device recordings from target-app readiness instead of including runner startup
• anchor iOS simulator gesture timing to a runner uptime baseline captured at record start, so overlays track the actual injected gesture rather than command completion drift
• soften overlay-export failures so recordings remain usable and telemetry is still returned

Add a real verification harness for this feature.

• add a Swift inspection helper that samples frames around telemetry timestamps
• add live E2E tests for iOS tap/scroll/back-swipe and Android tap/scroll using video + telemetry + sampled-frame assertions
• keep telemetry and overlay verification separate from recording health so regressions localize cleanly

Validation

• pnpm typecheck
• pnpm test:unit
• pnpm test:smoke
• pnpm build:xcuitest
• AGENT_DEVICE_RECORDING_E2E=1 node --test test/integration/recording-overlay.test.ts

Known gap:

• Android tap verification currently runs as a short tap+scroll recording because the emulator's ultra-short tap-only screenrecord stop path is still less stable than longer recordings.

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://callstackincubator.github.io/agent-device/pr-preview/pr-245/
Built to branch gh-pages at 2026-03-23 15:47 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d352b5c7b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Avoid hard-requiring Swift for Android recording validation

Android record stop now depends on isPlayableVideo, which shells out to swift/AVFoundation; on non-macOS hosts this validator is unavailable, so valid pulled MP4s are treated as unplayable and copyAndroidRecordingWithValidation eventually fails with pulled file is not a playable MP4. This breaks Android recording stop in environments like the repo’s Ubuntu Android workflow, so the validation path should be cross-platform or gracefully skipped when the validator runtime is missing.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Keep telemetry artifact keyed to telemetryPath

The new telemetry artifact is emitted as field: 'gestureTelemetry' and without localPath, while the response key is telemetryPath; in remote-daemon flows artifact materialization keys off artifact.field and requires a client path, so this artifact is not materialized back and telemetryPath remains a daemon-local path. As a result remote clients cannot reliably download/use the telemetry sidecar from record stop.

Useful? React with 👍 / 👎.