Skip to content
News update

AI Governance in 2026: From experimentation to maturity

by DANIEL POPOVSKI - SENIOR POLICY AND ADVOCACY ADVISER GOVERNANCE INSTITUTE OF AUSTRALIA -
As an increasing number of businesses gain confidence in adopting AI technologies, the effectiveness of their deployment in the workplace will be measured by the depth of understanding and robustness of governance surrounding their use.

AI law and AI ethics and legal concepts artificial intelligence law.

Governance of agentic AI systems will become top priority for companies integrating the latest wave of AI technologies. Increasingly dynamic and autonomous AI technologies are expected to shift and increase cyber-attack surfaces, with cybercriminals increasingly targeting non-human identities and decision-makers. Stronger human–AI connections are key to bridging the divide between indiscriminate adoption and outright resistance.

This article summarises key challenges of an evolving technology offering insights into the limitations of traditional governance models and looking to new systems of thinking and governance.

Agentic AI increases the cyber-attack surface of organisations

Increased AI autonomy and accessibility heighten risks if governance and security are lacking. Organisations are increasingly embracing automation to streamline processes and boost efficiency in the workplace.1 These are used by business systems and applications to authenticate, access resources and interact within the enterprise. Non-human identities (NHIs) are digital credentials such as bots, API keys, and service accounts.2 The ungoverned use of non-human identities creates heightened risks for organisations that may miss blind spots in their integration of AI systems in the workplace.

Cybercriminals will increasingly target non-human identities

Weak authentication, misconfigurations and poor monitoring of agents and AI systems will lead to breaches and system disruptions. This will require identification and security of AI agents through comprehensive governance of NHI that restrict access and monitoring permissions to prevent credential misuse and data exploitation. AI agents will amplify existing security challenges by operating at machine speed and scale, chaining unpredictable tools and permissions, running continuously without clear session limits, requiring broad system access, and introducing new attack vectors in multiagent setups.3

Governance of agentic AI will rise as a top priority for companies

A study of 300 tech leaders by API management and security platform found that while businesses are rapidly adopting agentic AI and LLMs for efficiency gains, governance has become a top priority, with over three quarters rating it ‘extremely important’ due to concerns over system integration, data security and managing LLM costs.4 The EU Act identifies agentic AI under high-risk categories particularly in areas such as customer service, finance and rights-based decision making. The Act requires clear disclosure when users interact with AI, and mandates understandable explanations for AI-driven decisions, such as dynamic pricing and credit assessments.5

Traditional AI governance practices may not suffice with agentic AI systems

Traditional AI governance practices, such as data governance, risk assessments, explainability and continuous monitoring will remain essential throughout 2026 but governing agentic systems requires going further to address their autonomy and dynamic behaviour.6

A key challenge will be in controlling what actions NHI’s can perform, such as sending or receiving data, data flow destinations, volumes and formats as well as restricting access to external or sensitive resources.7 Continuous verification of changes in any identity’s permissions or software since its last activity will require increased human oversight. Governance professionals set to become decision-making architects with autonomous models leading and executing decisions within discreet areas of the business.

Centralised governance oversight essential

The key benefit of centralised governance oversight enables the rapid detection, isolation and response to violations that reduce fragmentation and enhance security postures. Maintaining secure authentication of NHIs can be achieved through distinct management practices, such as credentialing, access monitoring and privilege controls. Recent studies suggest that fewer than 20% of organisations have formal processes to offboard and rotate application programming interface (API) keys, leaving identities exposed to significant security risks.

Governing through controlled agency and zero trust principles

Controlled agency is a model that enforces accountability while allowing agents to act independently within defined limits.8 Controlled agency in agentic automation refers to the combination of tools and practices that allow AI agents to operate independently while keeping their actions aligned with enterprise security standards.

Why controlled agency? Traditional identity systems, that we’ve grown to use and understand, such as OAuth and Security Assertion Markup Language (SAML) are widely adopted industry standards for secure identity and access management in web applications but were designed for static human users, not for dynamic, autonomous workflows of AI agents.9 These agents often shift from human to nonhuman identities to perform tasks, requiring adaptive access controls that maintain security, accountability and policy enforcement through centralised governance oversight.10

Zero Trust principles allow us to consider the essential need to securing AI agents operating in dynamic and autonomous environments. Zero Trust principles require us to consider explicit verification of all requests, limiting access based on least privilege and assuming breaches to minimise risk. This emphasises the need for continuous ‘real-time’ authentication, a more holistic understanding of data protection, and a proactive threat detection culture to safeguard whole of business systems, particularly where AI agents become more integrated.11

Governance Institute of Australia is currently undertaking further research on the future of Agentic AI governance. Stay tuned!

  1. Kshetri, N. (2025). Governing Agentic AI: Security, Identity, and Oversight in the Age of Autonomous Intelligent Systems. Computer (Long Beach, Calif.), 58(8), 123–129
  2. Kshetri, N. (2025). Governing Agentic AI: Security, Identity, and Oversight in the Age of Autonomous Intelligent Systems. Computer (Long Beach, Calif.), 58(8), 123–129
  3. Kshetri, N. (2025). Governing Agentic AI: Security, Identity, and Oversight in the Age of Autonomous Intelligent Systems. Computer (Long Beach, Calif.), 58(8), 123–129
  4. S. Evans, “Governance is top priority for companies using agentic AI: Survey,” AI Business, May 8, 2025.
  5. M. C. Borrelli and S. Musch, “How to use agentic AI in line with the EU AI Act,” CX Network, Feb. 11, 2025.
  6. www.ibm.com/think/insights/ai-agent-governance
  7. www.darkreading.com/cybersecurity-operations/non-human-identities-gain-momentum-requires-both-management-security
  8. www.darkreading.com/cybersecurity-operations/non-human-identities-gain-momentum-requires-both-management-security
  9. cloudsecurityalliance.org/blog/2025/03/11/agentic-ai-identity-management-approach
  10. cloudsecurityalliance.org/blog/2025/03/11/agentic-ai-identity-management-approach
  11. cloudsecurityalliance.org/blog/2025/03/11/agentic-ai-identity-management-approach

Surveillance, privacy, and governance: Navigating the new landscape in Australian workplaces

Next article