Anthropic Unveils AI Model Capable of Controlling Desktop Apps

• Anthropic has released an upgraded version of its Claude 3.5 Sonnet model that can interact with any desktop app.
• The AI model can emulate keystrokes, button clicks, and mouse gestures, essentially acting like a human using a PC.
• Developers can access this feature via Anthropic’s API, Amazon Bedrock, and Google Cloud’s Vertex AI platform.
• Despite its capabilities, the model struggles with basic actions like scrolling and zooming and often misses short-lived actions and notifications.
• Anthropic has taken steps to deter misuse, such as not training the model on users’ screenshots and prompts, and preventing the model from accessing the web during training.

Source: Techcrunch