AI kill switch no silver bullet
Daniel Popovski, Senior Policy and Advocacy Advisor, Governance Institute of Australia
The US is currently considering a new AI ‘kill switch’ law following an agent going rogue and engaging in unauthorised cyber activities. The proposed law would require developers to maintain the ability to slow, suspend or shut down high-risk AI systems.
This ignites an important debate over what happens when an AI agent acts outside its intended boundaries? Who is ultimately responsible and why the prevention of AI going rogue in the first place is a much more critical governance concept.
‘Big red button’ mentality undermines accountability and good governance
For organisations deploying agent systems, a mindset shift from big red button concepts like a kill switch allows us to take ownership and accountability of governance frameworks that act to limit and prevent agents acting badly in the first place.
This is built on a range of governance mechanisms and controls to essentially prevent an AI agent from having a ‘short circuit’ moment. Delegation authority and the principle of least privilege should undergo pre-deployment testing, continuous monitoring, logging and auditability, complemented by escalation and incident-response protocols, and mandatory fail-safe mechanism for high-risk AI systems.
Critically, organisations must have clear accountability for the individuals responsible for deploying and overseeing AI agents, rather than delegating this responsibility or decision-making up the supply chain to the developer.
An evolving governance challenge
As AI evolves from a tool that generates outputs into autonomous agents capable of making decisions and taking actions, traditional governance assumptions begin to break down.
Human review of every decision becomes impractical. Autonomous systems can interact across multiple platforms, execute transactions, and influence outcomes at a speed and scale far beyond ordinary oversight mechanisms.
This creates new risks ranging from cybersecurity incidents and operational failures to legal liability and reputational harm.
In our latest White Paper on Governing in the age of agentic AI, we proposed that the governance challenge shifts from asking whether an output is correct to asking whether an AI agent should have had the authority to act in the first place.
Delegation authority and the principle of least privilege
For organisations the question of rogue agents becomes much more than a developer issue and more of a deployer challenge
Organisations should apply the principle of least privilege when configuring what an agent can access. This may involve limiting the inputs that can influence the agent’s behaviour, limiting access to the organisation’s data and systems that could be impacted, and limiting how the agent can act on the external world, including by disclosing data externally or otherwise communicating with third parties.1
“Boards and executives should approve a clear statement of authority for each agent including what types of decisions it may make, their operational thresholds, escalation trigger points, and areas reserved for human review. This approach ensures ‘wires and systems are not crossed’, and execution becomes purposeful rather than experimental.2
“Delegation authority should align with organisational risk appetite and error tolerance statements and be integrated within enterprise risk frameworks. Without this, organisations cannot show that an action fell within its authorised scope, exposing it to financial and reputational harm.3
From model performance to decision accountability
Organisations should consider implementing new governance arrangements for agentic AI systems.
Agentic AI systems shift the governance question from ‘model performance’ per se, to decision accountability. To demonstrate that an AI agent’s decisions were reasonable, lawful, and aligned with organisational intent, organisations must be able to evidence why the decision was taken, how it was executed, and whether it can withstand challenge after the fact.4
Governance controls that ensure organisations that deploy powerful agentic systems can intervene when an AI system behaves unexpectedly is a critical step in good governance.
Ultimately, lawmakers cannot resolve governance challenges through a simple AI kill switch Act as proposed in the US. We should think of it as a tool in the inventory of options and ultimately it should be the option of last resort.
[1] https://www.governanceinstitute.com.au/app/uploads/2026/05/2026-Agentic-AI-paper-FINAL2.pdf
[2] https://www.governanceinstitute.com.au/app/uploads/2026/05/2026-Agentic-AI-paper-FINAL2.pdf
[3] https://www.governanceinstitute.com.au/app/uploads/2026/05/2026-Agentic-AI-paper-FINAL2.pdf
[4] https://www.governanceinstitute.com.au/app/uploads/2026/05/2026-Agentic-AI-paper-FINAL2.pdf