How computer-use agents handle CAPTCHAs and logins

A computer use agent captchas interaction is the fastest way to discover that your automation isn’t as human as you thought. Most production browser agents break at login walls because they treat authentication as an afterthought instead of a first-class state machine. This guide gives an ordered path for detecting, handling, and surviving CAPTCHA and login flows without grinding your pipeline to a halt.

1. Separate authentication from agent logic

Build a dedicated auth module that produces a logged-in browser context and serializes it. Do not weave login clicks into the same loop that scrapes data or clicks buttons. When authentication is a side effect of the main agent, you cannot reuse sessions or recover from expiry without re-running the whole task.

Playwright’s storage_state is the cheapest way to capture cookies, localStorage, and session tokens:

from playwright.sync_api import sync_playwright

def login_and_save_state(email, password, state_file="auth.json"):
    with sync_playwright() as p:
        # Headless=false reduces bot scoring on many sites.
        browser = p.chromium.launch(headless=False)
        ctx = browser.new_context(viewport={"width": 1280, "height": 800})
        page = ctx.new_page()
        page.goto("https://example.com/login")
        page.fill("#email", email)
        page.fill("#password", password)
        page.click("button[type=submit]")
        page.wait_for_url("**/dashboard")
        ctx.storage_state(path=state_file)
        browser.close()

Later, the agent loads the context without touching the login form:

def load_agent_context(state_file):
    with sync_playwright() as p:
        browser = p.chromium.launch()
        ctx = browser.new_context(storage_state=state_file)
        return ctx

Run the login flow under a real display or a stealth patch only when the target is your own property or a sandbox with explicit permission. Headless Chrome still leaks enough signal that a computer use agent captchas challenge will fire on any protected endpoint.

2. Detect CAPTCHA and login walls programmatically

Before the agent executes any planned action, scan the page for anti-bot markers. Relying solely on the model’s screenshot interpretation wastes tokens and adds latency. A DOM pre-check catches 90% of cases:

def detect_captcha(page):
    iframe_srcs = page.eval_on_selector_all(
        "iframe", "els => els.map(e => e.src)"
    )
    markers = ["recaptcha", "hcaptcha", "px-captcha", "arkose", "funCaptcha"]
    return any(m in src for src in iframe_srcs for m in markers)

For canvas-based or text-only challenges, diff the current screenshot against a known clean template, or run a lightweight heuristic for “verify you are human” strings. A computer use agent captchas detector must combine DOM signals with pixel checks because modern widgets lazy-load inside shadow roots.

If a wall is detected, halt the main task and enter a dedicated auth sub-routine. Do not let the agent attempt to “click through” a puzzle blindly; that trains the site’s risk engine against your IP.

3. Choose a CAPTCHA handling strategy

3.1 Human-in-the-loop for third-party sites

If you do not own the domain, the only compliant path is to proxy the screenshot to a review queue where a person solves it. Tools like LabelStudio or a simple Slack bot with a reaction trigger work. The agent waits on a webhook, then injects the token.

3.2 Test keys and whitelists on owned infrastructure

On staging or first-party properties, use reCAPTCHA’s test sitekey (6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe) which always returns a pass token. For hCaptcha, set hcaptcha-test mode. IP-allowlist the agent egress so production risk scores stay low.

3.3 Vision models for legacy image CAPTCHAs

Distorted-text or simple math CAPTCHAs can be transcribed by a multimodal model. Route the screenshot through an OpenAI-compatible endpoint; for example, n4n.ai exposes one endpoint covering 240+ models with automatic fallback when a provider is degraded, so a single rate limit won’t stall your agent.

import requests, base64

def read_text_captcha(img_path, api_key):
    b64 = base64.b64encode(open(img_path, "rb").read()).decode()
    r = requests.post(
        "https://api.n4n.ai/v1/chat/completions",
        headers={"Authorization": f"Bearer {api_key}"},
        json={"model": "openai/gpt-4o",
              "messages": [{"role": "user", "content": [
                  {"type": "text", "text": "Transcribe the characters in this CAPTCHA."},
                  {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}}]}]})
    return r.json()["choices"][0]["message"]["content"]

Tradeoff: accuracy is rarely 100%, and behavioral CAPTCHAs (reCAPTCHA v3, Kasada) ignore the image entirely. Solving a CAPTCHA on a site you don’t control violates most ToS and may carry legal risk. Use this only on owned test surfaces.

4. Handle MFA and one-time passwords

Modern logins add TOTP, SMS, or WebAuthn. For TOTP, pull the shared secret from a vault and generate the code in-process:

import pyotp
code = pyotp.TOTP("BASE32SECRET").now()
page.fill("#otp", code)

For SMS/email OTP, poll a mailbox API or a Twilio webhook. Never embed secrets in agent source. WebAuthn and passkeys are harder: a computer use agent captchas flow cannot satisfy a hardware-bound assertion unless you run a cloud HSM or use a delegated auth proxy that issues session cookies out-of-band. Design for that limitation rather than fighting it.

5. Persist sessions to avoid repeated challenges

Re-authenticating every run is the surest way to get fingerprinted. After a successful login, store the storage_state and reuse it for days. Check cookie expiry (expires field) and refresh proactively via a silent token endpoint if the site offers one.

Rotate the user-agent and viewport to match a consistent real device profile across runs. If you must use proxies, prefer static residential egress with documented consent. Jumping IPs mid-session triggers step-up challenges.

6. Common pitfalls and tradeoffs

• Linear mouse paths: Agents that move the cursor in straight lines fail behavioral checks. Inject randomized delays and quadratic Bézier curves between points.
• Token burn: Vision-based CAPTCHA solving costs per screenshot. Cache the image and retry with the same bytes instead of re-capturing.
• Canvas-only widgets: DOM scans miss them. Add a screenshot classifier that flags unusual full-screen overlays.
• Session rot: A 403 after hours of work usually means the auth cookie died. Wrap actions with a re-auth trigger that reloads state and retries once.
• Compliance blindness: A computer use agent captchas bypass on third-party sites can breach CFAA-style statutes. Keep a human approval gate for any non-owned target.

7. Operational checklist

1. Auth module isolated; emits auth.json storage state.
2. Pre-action DOM + screenshot scan for captcha markers.
3. Branch: human queue / test key / vision model (owned only).
4. MFA injected from secret store, never hardcoded.
5. Context persisted; alert on auth regression.
6. Mouse/keyboard jitter enabled by default.

Follow that order and your agent will treat logins as a solved subproblem instead of a daily fire.