Skip to content

Feature: window screenshot without overlay (takeScreenshotOfWindow) #1

Description

@andreanjulianp-cmd

Problem

When running automation from Termux with a floating terminal window, the and endpoints capture the floating overlay on top of the target app. This is because captures the full composited frame including all overlays.

Proposed Solution

Use Android's AccessibilityService.takeScreenshotOfWindow() API (available since Android 13 / API 33) to capture a specific window without overlays drawn on top.

Google's commit message for this API:

"This enables accessibility services to take a screenshot of a window without any overlay windows drawn on top, which is beneficial for ML-based understanding of the visual content in the window."

Use Case

AI agents running from Termux floating window need clean screenshots of target apps (e.g., TikTok Seller) without the Termux overlay appearing in the captured image.

Suggestion

Add a new endpoint like /window_screenshot or a query param ?window_id=<id> to /snapshot that uses takeScreenshotOfWindow() instead of takeScreenshot().

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions