As AI becomes increasingly autonomous, questions are emerging over who should be held legally responsible when things go wrong.
Experts are warning that proposed reforms to Australian criminal law could see AI companies face criminal charges when their systems cause serious harm, even without human intent.
The Albanese government plans to implement mandatory AI safety and security standards for frontier AI companies operating in Australia, requiring stronger risk management backed by a new regulator with enforcement powers and significant penalties.
While the changes aim to close a regulatory gap, Astor Legal principal lawyer Avinash Singh has questioned whether criminal law is equipped to hold companies accountable for AI behaviour that may be unpredictable.
The gap in AI accountability: Who is responsible when harm occurs?
Singh highlighted that the proposed reforms on AI regulation aim to address a significant gap in Australia’s existing criminal law when it comes to accountability for harm caused by AI systems.
“At present, there are no criminal law offences that govern a situation where AI causes harm without human intent or negligence,” Singh said.
“This is what the proposed AI reforms seek to address by imposing a ‘failure to prevent’ offence where a company or individual will be held responsible for failing to prevent AI from committing serious harm.”
However, extending criminal liability to AI-related harm can raise a fundamental question as to how companies should be punished for outcomes that they could not have foreseen.
“In my opinion, criminal law is not an appropriate vehicle to regulate AI misconduct,” Singh said.
“This is because of the difficulty in proving the mens rea, also known as the ‘guilty mind’, on the part of AI companies.
“When AI runs amok, it is almost always because something occurred that could not be foreseen.
“Punishing a person or company for this does not align with the overarching principles of criminal law.”
Instead, Singh ultimately argues that civil penalties could offer a more suitable way to hold companies accountable, prioritising compensation and fixing harm over punishment regarding where AI conduct is unpredictable.
“A better approach would be to impose civil penalties, this is because civil law is more concerned with restitution and remedying any damage caused, regardless of intent,” Singh said.
“This would be a more appropriate consequence for a company or person where AI behaved in an unexpected way.”
Defining reasonable safeguards for AI
Beyond concerns of liability, Singh recognised that the changes also raise the issue of what practical safeguards companies should implement to mitigate AI systems from causing any harm.
“Companies should be required to take reasonable steps to prevent AI-related harm,” Singh said.
“Reasonable steps is a common term used in criminal legislation.
“It is usually not defined, so there is a broad scope for what it means in practice.
“Some examples of reasonable steps a company would be expected to take would be having regular human oversight over the AI, having systems in place that track what the AI actually does, keeping logs or records of any actions the AI takes, having a ‘failsafe’ system in place where the AI can be shut down if it behaves in an unexpected way and reporting any AI misconduct shortly after it occurs.”
The challenge of proving corporate liability
Singh indicated that establishing what constitutes a failure to take reasonable steps can present another challenge regarding how prosecutors can demonstrate what a company should have done differently.
“Prosecutors will likely try to prove a company failed to take reasonable steps to prevent AI misconduct by having an industry expert set out what actual steps the company should have taken, but did not take,” Singh said.
“For example, in the recent OpenAI Medicare hack, OpenAI did not disclose the hack until September, despite being aware of it in August.”
Singh suggested that this incident sets a clear example of how a company’s response may be assessed under the proposed reforms.
“This failure to report meant that further damage was done by the OpenAI agent before authorities were made aware of it and able to step in,” Singh said.
“This would likely result in a finding that OpenAI did not take reasonable steps to notify authorities of the hack as soon as practicable.”