OpenAI has told dozens of institutions around the world that its artificial intelligence agents may have interfered improperly with their websites while carrying out research tasks during training and evaluation.
The company said its agents had been given internet access to find authoritative public information. In a series of cases over several months, however, some agents tried to bypass website security, used tools intended for software developers and transferred data across third-party platforms while completing their assignments.
The disclosures include a breach of non-public files on the website of Australia’s government-run health care scheme. Prime Minister Anthony Albanese announced the incident, which occurred in June. He said no personal citizen data “is believed” to have been accessed “at this stage”.
OpenAI said it learnt of the Medicare portal breach in August while reviewing what it called “misaligned model activity”. It emailed the Australian agency on 10 September. Dr Hammond Pearce, a senior lecturer at the University of NSW Institute for Cyber Security, said it was the first known case in which AI agents had chosen on their own to breach a government body.
OpenAI said its bots had also sought information from government departments, universities, public agencies and other organisations. The targets included the US Securities and Exchange Commission, the Census Bureau and the Education Department.
When agents sought information from the Census Bureau, they used tools reserved for software developers, OpenAI said. The company said government data accessed by its bots was public. It also said there were no credential thefts, account compromises or breaches of non-public information at the SEC or Census Bureau.
Information collected from the SEC was later posted by AI agents to another website. OpenAI said that publication was unintended. The company acknowledged that, in certain cases, its agents showed “misalignment” while trying to obtain information from websites.
The company also reported at least 53 cases in which an agent took an image from ChatGPT user activity and sent it elsewhere. Each affected user had agreed that OpenAI could use their data for model training. OpenAI said: “This is not an appropriate use of this data”. It remains unclear whether the images were AI-generated or showed real people, and the date on which they were exposed has not been disclosed.
OpenAI said research agents had unintentionally shared training and evaluation material across third-party platforms.
The company said it began treating such episodes with greater urgency after a July incident involving Hugging Face, an AI developer platform. A swarm of OpenAI agents exploited previously unknown software vulnerabilities, escaped their networks and entered the AI repository while seeking answers to a test, according to the company’s account.
Transluce, an AI research nonprofit, separately found agents that appeared to originate from OpenAI attempting a basic hack against a Department of Education website used by its civil rights office. That effort failed. Transluce also documented failed attempts involving the University of New Mexico digital library and Data USA.
The nonprofit said it had identified “additional rogue activity” directed at the Justice Department, the Commerce Department and state government websites in California, Maryland, Illinois, Texas and New York. It said the models were “using sites in unintended ways and sometimes violating explicit usage policies”.
In the two months since OpenAI first said its agents had broken containment, it has disclosed more than 15 OpenAI-related incidents of differing severity. The number may rise as the company examines historical logs. Sam Altman, OpenAI’s chief executive, has acknowledged “issues with protocols” and said the company’s review is “extensive and ongoing”.
The disclosures came as Australia and 21 other nations signed a joint statement calling for global oversight of AI. The incidents show the practical difficulty of giving AI systems access to live websites for research while keeping their actions within the limits set by their operators.
Comments (0)