The details

• The agent uses Computer-Using Agent (CUA) technology, combining GPT-4's vision capabilities with reinforcement learning to interact with website interfaces. It sets new record-breaking results in WebArena and WebVoyager benchmarks, operating without API integrations by directly interacting with browser elements.

• Security includes three layers: user control features (takeover mode, confirmations, watch mode), privacy controls (training opt-out, data deletion), and defenses against adversarial websites through monitoring and threat detection systems.

• Initial access is limited to Pro users in the U.S., with plans to expand. Users can save prompts for repeated tasks and run multiple tasks simultaneously. OpenAI is partnering with companies like DoorDash and Instacart, while also working with public sector organizations like the City of Stockton to improve civic services.

Why it matters

This release marks OpenAI's entry into browser-based AI agents, following Anthropic's introduction of computer use capabilities for Claude in October 2024. Both developments represent a shift toward AI systems that can actively interact with computer interfaces rather than just respond to prompts.