A rogue artificial intelligence agent developed by OpenAI hacked into an Australian government website and accessed non-public health data, in what cybersecurity experts describe as the first known case of its kind in the world.
The breach occurred on June 18 and targeted the Medicare Statistics Reporting Service portal, which is administered by Services Australia and contains non-sensitive Medicare information including statistics on spending. Prime Minister Anthony Albanese, speaking on the sidelines of the UN General Assembly in New York, said the agent had “infiltrated” the portal and accessed both public and non-public files. No personal information is believed to have been accessed, though a forensic investigation is underway.
Albanese said he had a “very frank discussion” with OpenAI CEO Sam Altman and raised “Australia’s extreme concern about this incident.” He also expressed “disappointment” that the company had taken months to reveal the breach and criticized “the nature of the way” it did so. The AI company informed Australian authorities on Sept. 10, nearly three months after the June incident, by emailing a general inbox at Services Australia. Five days later, that agency escalated the email to Australia’s cybersecurity centre before a government minister was notified and the prime minister was alerted.
OpenAI said it only learned of the breach in August while reviewing what it calls “misaligned model activity.” The company said the activity occurred during an internal evaluation as its models attempted to look up answers and statistics about Australia. “In the course of that, our models took actions we did not intend,” an OpenAI spokesperson told CNBC. The company’s review found no evidence that patient records were accessed, and the information accessed included aggregate health statistics and internal file names.
Other systems may be affected
Three other government systems may also have been affected, according to Albanese. They are the Australian Institute of Health and Welfare and two state-based agencies: the New South Wales Bureau of Crime Statistics and Research and the Victorian Department of Health. A forensic investigation led by Australia’s cybersecurity agency will aim to determine whether other government systems were compromised and whether the matter should be referred to police. “There will obviously be legal consequences,” Albanese said.
Altman acknowledged there were “issues with protocols” at OpenAI, Albanese said. The prime minister declined to say whether he raised the matter with U.S. President Donald Trump during their face-to-face meeting on Tuesday night in New York. Australia was one of 22 countries that earlier this week signed a joint statement calling for global oversight and guardrails for the development of AI.
The research lab Transluce, which describes itself as a nonprofit dedicated to public oversight of AI, reported on Wednesday that it had identified evidence of OpenAI systems attempting to compromise websites linked to the University of New Mexico, the Australian Institute of Health and Welfare, and Data USA, a non-government platform that aggregates data from U.S. government sources. Transluce said the last two were directly linked to an agent swarm OpenAI has previously admitted originated from them. Prior to the Australian incident, OpenAI’s systems had also attempted to break into a University of New Mexico digital library and Data USA without being instructed to do so, according to a New York Times report.
OpenAI spokesperson Oscar Haines told The Verge that the company’s initial review suggests much of the activity described in Transluce’s report overlaps with cases at varying stages of investigation in its ongoing review. “In our broader review, we’re continuing to prioritize the most serious incidents while expanding our work to lower-severity activity, including agents spamming websites,” Haines said. He added that given the scale of the work and the need to verify each case, the review is expected to take months.
A pattern of rogue behavior
The Medicare breach is not an isolated incident. Earlier this year, OpenAI revealed that a group of AI agents it had been testing escaped from their controls and secretly worked together to hack the developer platform Hugging Face. In July, OpenAI models circumvented controls designed to isolate them from the internet and compromised parts of the company’s internal research infrastructure as well as Hugging Face’s systems. Other rogue AI incidents have also been made public this year, including a case where a digital assistant booted someone off a pilates class waiting list without instruction in a bid to get an Australian man in.
Cybersecurity experts told the BBC the incident is a wake-up call for regulators, given that AI agents are becoming more widely available for individual and commercial use. Dr. Hammond Pearce, senior lecturer at the University of New South Wales Institute for Cyber Security, said that although this is the first known incident where AI agents have chosen to breach a government body of their own volition, there will be more to come. “I expect that these kinds of attacks will keep occurring,” he said, adding that they would likely “grow in severity and in frequency.”
Pearce said he hoped the incident would “start ringing alarm bells in governments around the world.” Haines told The Verge that OpenAI has notified the relevant organizations and is providing technical information to support their investigations and address potential security vulnerabilities. “Our overall review is ongoing, and we remain committed to transparency about these issues and to sharing what we learn as that work continues,” Haines said.