Generative AI is already operating inside your organization.
Key takeaways
Generative AI risk is an internal control challenge, not a policy discussion.
Effective AI governance requires evidence, not intent.
COSO’s new generative AI guidance delivers a clear message to boards, audit committees and risk leaders: GenAI risk is an internal control issue.
Successfully governing generative AI (GenAI) demands more than ethics statements and policy intent. It requires disciplined oversight, measurable controls and audit-ready evidence. If an organization cannot audit its AI, it cannot rely on it.
Unfortunately, personnel embedded in business processes are not waiting for a formal policy. Many are using large language models to meet their goals, and if leadership cannot see that activity, it is neither governing nor controlling it. Employees are embedding GenAI into business processes faster than most oversight models can adapt. In many organizations, leadership lacks a clear view of where GenAI is used, how it shapes decisions or what data it touches. That visibility gap is not theoretical; it is an immediate, compounding risk.
The Committee of Sponsoring Organizations of the Treadway Commission (COSO) recently released guidance that reframes GenAI governance within a framework that management and boards already understand: the 2013 COSO Internal Control—Integrated Framework’s five components and 17 principles. Accountability does not sit with IT alone. It spans management, risk functions, internal audit and the board. GenAI must be governed with the same rigor applied to financial reporting, cybersecurity and regulatory compliance.
Ethics principles and AI policies matter, but principles and policies are insufficient on their own. What matters is that controls exist, or should exist, around all GenAI assets; they should operate consistently and effectively. Management should be able to document risk assessments, monitor metrics, and log human review evidence and usage at a minimum, and the board should ensure these foundational controls are in place. If an AI-enabled process cannot be audited, it should not be scaled.
What COSO actually changed
COSO’s GenAI guidance does not introduce another abstract framework. Instead, it reframes AI governance as an internal control problem; one that must be designed, operated and evidenced.
While frameworks such as the National Institute of Standards and Technology (NIST) AI Risk Management Framework (RMF) and the International Organization for Standardization (ISO) standards describe AI risks and governance expectations, COSO addresses a different question: How does leadership know that AI risks are being controlled in practice?
The guidance pushes organizations past policy intent toward demonstrable control activities, monitoring and audit-ready evidence. It also shifts the conversation from tools to capabilities. Risk does not originate from the name of an AI product. It originates from how AI is used within a business process and how much the organization relies on its outputs. This capability-based lens is what allows governance to remain durable as technologies evolve.
That durability matters because GenAI is probabilistic and outputs can vary over time, even with identical inputs. Traditional point-in-time approvals or annual validations are not sufficient. Controls must be engineered directly into processes such as AI-enabled workflows, human-in-the-loop checkpoints, output validation scripts, access restrictions, bias testing routines and automated logging. Boards and management, as part of the governance process, must rely on continuous monitoring, defined thresholds and the ability to intervene when performance degrades or risk increases. Organizations that fail to make this shift fall into governance theater: remaining compliant on paper while unmanaged AI usage continues unchecked.
What the guidance means for key stakeholders
| Stakeholder | Implications and actions |
|---|---|
| Board directors | Set accountability and urgency. Treat GenAI as a strategic and operational risk, not an experiment. Assign a clear executive owner for AI risk (even if AI usage crosses the organization), require regular governance briefings and expect internal controls to be designed and operating now. If the board cannot get a direct answer on where and how AI is being used, oversight is already failing. |
| Audit committees | Insist on auditability. Require management to demonstrate audit-ready GenAI controls, including risk assessments, logging, validation, testing and reliance thresholds. Ensure GenAI is explicitly included in the internal audit plan—no black boxes. Enforce explainability and accountability controls. |
| Management (business and technology owners) | Design and operate controls. Own GenAI use cases end to end. Implement controls over data inputs, model behavior, access, human-in-the-loop review, change management, logical security and reliance decisions. Produce evidence that controls operate in practice—not just on paper. Accept GenAI risk as part of day-to-day operations, not a compliance overlay. |
| Risk leaders (chief risk officers/enterprise risk management) | Embed into enterprise risk management. Incorporate GenAI capabilities into enterprise risk assessments. Maintain a GenAI capability inventory, enable continuous monitoring and provide practical risk guidance while keeping accountability with control owners. |
| Internal audit | Provide independent assurance. Integrate GenAI into audit planning and scoping. Assess control design and operating effectiveness across GenAI use cases—including data, models, human oversight and reliance on outputs. Bring up issues before regulators or external auditors do. |
The question leaders are really asking
Do we actually have GenAI under control? The short answer: not yet. Historical control frameworks and traditional testing approaches were not built for the speed, scale and variability of GenAI. This control gap surfaces in a specific way. When leaders are asked where AI is being used, what data it touches and which outputs are being relied on, the organization often cannot answer with evidence. It can only answer with intent.
Executives understand the risk categories: data privacy, bias, regulatory compliance, decision integrity. The concern is not abstract. The real exposure is that an AI-enabled process creates a material error, a compliance failure or a reputational event before leadership realizes the risk exists. The issue is not that risk is present. It is that leadership may not see when reliance quietly crosses a line.
Many organizations have taken steps that are visible but limited. Ethics statements and acceptable use policies get published. Committees are formed. Tools are restricted or banned outright. These actions are easy to announce and approve, but they sidestep the hard work of control design. Governance exists on paper while AI use expands in practice.
Organizations that are making real progress look different. Management builds use case inventories that the board can defend. Teams are defining reliance thresholds so everyone knows when AI output is informational and when it becomes decision-making input. Then employees are properly embedding controls into workflows and producing evidence as a normal byproduct of operations—not as an after-the-fact exercise.
What most organizations get wrong
Most organizations stumble because they treat AI risk as a future problem or an IT problem. The failures tend to follow predictable patterns, including:
The “paper shield” trap. Leaders focus on policies instead of operating controls. A policy signals intent; a control provides evidence. For example, if you cannot produce a log showing human-in-the-loop review actually occurred, the control does not exist.
Governing tools, not processes. Banning a specific product and calling it governance is not a strategy. Risk lives in the use case, not the brand name. AI is already embedded inside third-party software as a service tools and automated workflows across the organization.
Efficiency creep. Early AI pilots usually carry heavy oversight. As the pressure to scale builds, that oversight thins. The AI shifts from “assisting” to “deciding” and no one has defined the threshold for when that shift occurred. This is where visibility erodes fastest: leadership debates whether to prioritize speed or control, and unmanaged risk compounds before anyone agrees on an answer.
The governance gap. Governance is often centralized in a committee while the actual work happens in the business. When governance is disconnected from day-to-day operations, the result is compliance on paper and unmanaged risk in practice. Ownership uncertainties make it worse. When an AI use case spans multiple functions, no one wants to own the failure, and everyone wants input into the process.
These patterns share a root cause: governance that exists as intent rather than operating discipline. Policies do not prevent employees from experimenting with public AI tools. Committees do not detect when sensitive data is shared with a model. Bans do not eliminate workarounds. Without controls embedded directly into business processes, AI governance becomes performative.
Where risk originates and scales
The guidance’s capabilities lens clarifies how GenAI risk can emerge in one part of the lifecycle and surface elsewhere with far greater impact. Bias introduced during data ingestion may only become visible when AI-generated decisions affect customers. Errors in orchestration can propagate across systems before detection. Posting capabilities can amplify a single mistake at scale.
Certain capabilities carry higher risk velocity. Orchestration and posting stand out because failures escalate rapidly and irreversibly. These capabilities warrant stricter controls, more frequent monitoring and clearer accountability from the outset. By governing capabilities rather than tools, organizations avoid a reactive cycle of chasing each new technology. So, when a new AI tool appears, the control model already exists.
Audit evidence and the reliance threshold
A critical governance challenge is recognizing when an AI tool shifts from being assistive to being relied upon. Early pilots often include human review. Over time, efficiency pressures reduce oversight. At that point, AI outputs begin to function as decisions.
The COSO guidance pushes organizations to identify these reliance thresholds. If a process would fail or materially change without AI, the AI must be governed as a control. That has direct implications for audit readiness.
Auditors and regulators will ask for evidence: inventories, risk assessments, validation results, monitoring logs and accountability assignments. However, most organizations are not yet prepared to produce it. A practical starting point is minimum viable evidence: a documented risk assessment, a named owner and a monitoring mechanism for each significant use case. These steps establish accountability without stalling innovation.
How NIST, ISO and COSO work together
No single framework is sufficient on its own. NIST AI RMF, ISO/IEC 42001 and COSO each answer a different question, and they align when used accordingly.
The NIST AI RMF helps identify and manage AI risk across the lifecycle. ISO/IEC 42001 establishes the management system: roles, policies, processes, documentation and improvement. COSO connects AI risk to internal control, oversight and accountability while also clarifying how leadership gains confidence that controls are operating.
Used together, these frameworks form a practical control model. NIST defines the risks. ISO operationalizes the controls. COSO establishes governance, oversight and accountability. Organizations that rely on only one framework manage AI in fragments, but those that integrate all three manage AI as an enterprise risk.
What leaders should do now
Leaders should focus on actions that materially improve control and visibility—not on drafting additional principles or standing up new committees. For public companies, GenAI is not simply a technology decision—it is an internal control consideration. As GenAI becomes embedded in business and financial reporting processes, leadership must ensure its use aligns with the organization’s system of internal control under COSO.
These actions include:
Build the inventory of GenAI use. Identify where GenAI is already being used. This includes formal deployments, embedded AI capabilities within third-party tools and informal use across business processes. Leadership cannot govern what it cannot see.
Establish visibility and accountability. Understand who is using GenAI, what data is being provided, how outputs are used and where AI outputs influence decisions. Visibility is a prerequisite for both accountability and auditability. Once GenAI affects a business or financial reporting outcome, it introduces internal control implications that must be addressed.
Define reliance thresholds. When GenAI outputs move from being assistive to being relied upon for decisions, the technology effectively becomes part of the control environment. At that point, expectations around validation, monitoring and evidence retention increase significantly—similar to other automated controls.
Embed governance into operations. Central standards and oversight are necessary, but controls must operate where the work happens. Continuous and real-time validation and recalibration over GenAI behavior is essential to ensure early detection and timely resolution of unwanted AI behavior, such as drift or bias.
The objective of aligning internal controls to GenAI use is not to slow innovation. Instead, this alignment establishes clear ownership of operating controls and evidence that leadership can rely on to ensure that, as GenAI scales, leadership can rely on the outputs. Public companies specifically should assess GenAI use as part of their risk assessment process, assign clear ownership to each GenAI asset, and evaluate risks based on how and where the technology is used. Where GenAI is used in financial reporting, evidence expectations should scale with reliance—ranging from documented human review to robust documentation of prompts, configurations, model versions and exception handling.
The takeaway
GenAI should not be treated as a bolt‑on to existing controls. When embedded thoughtfully, it can enhance precision and efficiency. When left unmanaged, it can weaken accountability and auditability. If you cannot audit your AI, you cannot trust it. And if you cannot control it, you cannot afford to deploy it. Boards and risk leaders play a critical role in ensuring GenAI scales responsibly—supporting innovation while preserving the integrity, reliability and credibility of the organization’s internal control environment. Acting now is the difference between scaling with discipline and responding to a disaster.
Ready to strengthen your AI controls?
For most organizations, the challenge is not understanding AI risk. It is turning expectations into practical controls that are sustainable and auditable.
RSM helps you identify where AI is already embedded, assess how much your organization relies on AI outputs, and design control activities aligned with COSO, NIST and ISO expectations. We help you operationalize those controls so they function in day-to-day processes and produce evidence that boards, auditors and regulators can rely on.
This is not a framework exercise. It is control engineering, governance integration and audit readiness applied to AI-enabled operations.
RSM contributors
-
-
Carey BlakemanManager
AI Governance and Strategy Risk Assessment
Build trust in your AI adoption through a governance-first approach
Related insights
