OpenAI is releasing a new tool called Operator, which we can use through ChatGPT to perform various actions on the web, without having to do anything ourselves.
Operator will be available initially to ChatGPT Pro subscribers in the US.
What can this new tool do?
This new tool uses a technology that combines the visual perception capabilities of GPT-4 with the logical reasoning capabilities of machine learning, in order to be able to interact with user interfaces (GUIs). As OpenAI reports, Operator can “see” through screenshots and “interact” with web pages using all the tools that a human user would use, such as a mouse and keyboard. This means that the AI agent can perform actions on the web without requiring special API integrations.
When the Operator encounters problems or crashes, it uses its logic to correct its course and, if necessary, prompts the user to take control. If a website requests sensitive information, such as login credentials, the Operator will prompt the user to provide that information. It is also programmed to “deny” malicious requests and block inappropriate content.
Meeting real needs
OpenAI is working with companies like DoorDash, Instacart, OpenTable, Priceline, StubHub, Thumbtack, and Uber to ensure that Operator meets real user needs while respecting established practices. However, the company warns that some things may not work exactly as expected, as the tool struggles with more complex interfaces, such as creating presentations or managing calendars.
In the future, OpenAI plans to integrate Operator for users of Plus, Team, and Enterprise subscription plans, and fully integrate it with ChatGPT.
