01Computer-use & browser agents
Claude computer use
Anthropic's tool that lets Claude view a screen and control a mouse and keyboard to operate software.
Best for: Automating tasks across desktop applications.
Visit Claude computer use →Large Action Models (LAMs)
Models and agents that turn instructions into actions: computer-use and browser agents, open GUI-agent models, function-calling action models and the benchmarks that test them.
17models and tools
3Categories
5FAQs answered
02Computer-use & browser agents
OpenAI computer use
OpenAI's API guide to a model that looks at screenshots and proposes clicks, typing and scrolling to complete tasks.
Best for: Browser and UI automation through the OpenAI API.
Visit OpenAI computer use →03Computer-use & browser agents
Gemini computer use
Gemini API documentation for a model that interprets screenshots and returns UI actions.
Best for: Web and app automation with Gemini.
Visit Gemini computer use →04Computer-use & browser agents
Amazon Nova Act
Amazon's service and SDK for building agents that perform actions in a web browser.
Best for: Reliable browser workflows on AWS.
Visit Amazon Nova Act →05Computer-use & browser agents
Browser Use
Open-source library that lets language-model agents control a web browser.
Best for: Self-hosted browser agents.
Visit Browser Use →06Computer-use & browser agents
Skyvern
Platform that automates browser-based workflows by combining language models with computer vision.
Best for: Automating web forms and portals.
Visit Skyvern →07Open action & GUI models
Salesforce xLAM
Salesforce AI Research's family of large action models tuned for function calling and agentic tasks.
Best for: Tool-calling and multi-step agents.
Visit Salesforce xLAM →08Open action & GUI models
UI-TARS
ByteDance's open GUI-agent models that perceive screenshots and act on desktop, mobile and web interfaces.
Best for: Open end-to-end GUI agents.
Visit UI-TARS →09Open action & GUI models
CogAgent
Open visual language model for GUI agents from the Zhipu / THUDM team.
Best for: Understanding and operating screens.
Visit CogAgent →10Open action & GUI models
OS-Atlas
Open foundation action model for GUI agents, released with a large cross-platform GUI grounding dataset.
Best for: Grounding instructions to screen elements.
Visit OS-Atlas →11Open action & GUI models
ShowUI
Open vision-language-action model for GUI agents, from the Show Lab at the National University of Singapore.
Best for: Lightweight GUI automation research.
Visit ShowUI →12Open action & GUI models
Microsoft Magma
Microsoft's foundation model for multimodal AI agents, covering both UI navigation and robot manipulation.
Best for: Research spanning screens and robots.
Visit Microsoft Magma →13Open action & GUI models
Microsoft UFO
Microsoft's open-source UI-focused agent framework for automating Windows applications.
Best for: Windows desktop automation.
Visit Microsoft UFO →14Open action & GUI models
OmniParser
Microsoft's tool that parses screenshots into structured, interactable elements for GUI agents.
Best for: Helping models locate on-screen elements.
Visit OmniParser →15Devices & benchmarks
rabbit
Consumer AI device company whose r1 product is built around what it calls a large action model.
Best for: Seeing the consumer LAM idea in practice.
Visit rabbit →16Devices & benchmarks
OSWorld
Benchmark environment that tests multimodal agents on open-ended computer tasks across operating systems.
Best for: Measuring computer-use agents.
Visit OSWorld →17Devices & benchmarks
WebArena
Realistic web environment and benchmark for evaluating autonomous web agents.
Best for: Testing web agents on long tasks.
Visit WebArena →Choosing the right approach
Match your requirement to a starting point, then compare vendors.
| Automate a desktop or browser task | A computer-use API from Anthropic, OpenAI or Google, run in a sandbox |
|---|---|
| Browser workflows on a schedule | Nova Act, Browser Use or Skyvern |
| Self-hosted GUI agent | Open models such as UI-TARS, CogAgent or ShowUI |
| Reliable function calling | An action model such as xLAM, or a general LLM with tool use |
| Windows applications | Microsoft UFO |
| Compare agents | OSWorld and WebArena, then your own scenarios |
Related pages on PeterIndia.net
Neighbouring directories and hubs.
World Action ModelsPredicting before acting, for robots.Agentic AIThe agentic AI hub.Agentic AI BrowsersBrowsers with built-in agents.AI Agent FrameworksBuild agents on top of models.AI Model ArchitecturesGPT, MoE, LRM, SLM and LAM explained.Vision Language ModelsThe perception behind GUI agents.Large Reasoning ModelsPlanning and thinking models.Large Language ModelsThe directory of LLMs.
Frequently asked questions
Quick answers.
What is a large action model?
A LAM is a model designed to carry out tasks, not only produce text. It plans steps and performs actions such as calling functions, clicking and typing in interfaces, or operating a browser.
How is a LAM different from an LLM with tools?
Many agents today are a general LLM plus tools. The LAM term emphasises models trained specifically on action data, such as function-calling traces or GUI interaction logs. The boundary is blurry and the term is used loosely.
Is it safe to let a model control my computer?
Treat it carefully. Run agents in a sandbox or virtual machine, limit permissions and credentials, and require human approval for irreversible steps. Web pages can also contain instructions meant to mislead an agent.
How reliable are computer-use agents?
They are improving but still fail on long or unfamiliar tasks. Benchmarks such as OSWorld show a gap between agents and people, so test on your own workflows.
How does this relate to world action models?
World action models focus on robots that predict the effects of actions in the physical world. LAMs mostly act in software. See the World Action Models page.
Descriptions are summaries of each vendor's own positioning, checked against its website on 4 October 2026. Product names and offerings change often, so confirm details on the vendor site before deciding.