Before an AI Agent Acts, Decide Who Is Accountable

A sales team connects an AI assistant to its customer database. The pilot works well: it summarizes conversations and suggests follow-ups. Someone then enables email sending. Another person adds access to pricing information. Within weeks, a writing assistant has become a system that can make commitments to customers.

Who approved that change in responsibility?

This hypothetical situation illustrates a governance problem: an AI system's capabilities can expand faster than the organization's decisions about its use. Approval for one task should not silently become approval for every future capability.

The first control is a clear description of the intended use. Document the business objective, the people affected, the information involved, and the actions the system may perform. Distinguish between producing a recommendation and executing a transaction. A draft discount proposal and an automatically issued discount carry different consequences.

Assign responsibilities that people can actually exercise. A business owner defines acceptable outcomes and accepts residual business risk. A technical owner maintains the integration. Security and privacy specialists assess relevant exposure. An operational owner can suspend the service and coordinate recovery. One person may hold several roles, but an ownership label is useful only if it comes with authority, time, and an escalation path.

The NIST AI Risk Management Framework offers a voluntary structure for managing AI risk through Govern, Map, Measure, and Manage. These functions support continuing risk management; adopting the framework does not certify that a particular system is safe. NIST AI RMF

For implementation, I suggest a short deployment record that follows the system through its lifecycle. It should identify the approved purpose, model and tool configuration, accessible data, prohibited actions, necessary approvals, evaluation results, known limitations, and review date. Record who can approve a material change, including a new data source or broader tool permissions.

Consider the sales assistant again. Its initial approval could permit reading assigned customer records and preparing email drafts. Sending an external message would require a separate decision, supported by tests of recipient selection, factual accuracy, confidential information handling, and escalation when the assistant lacks evidence.

Governance also extends beyond cybersecurity. A system can protect credentials and still generate misleading advice, treat customers inconsistently, or create an inaccessible service. Review the foreseeable effects on people, the reliability needed for the task, and how someone can question or correct an outcome.

Human oversight should have a defined purpose. Give reviewers the proposed action, supporting information, uncertainty, and likely consequences. If a reviewer sees only an approval button, the organization has created a checkpoint without providing the means to make an informed decision.

A practical starting point is one consequential workflow. Map its decisions, assign owners, define the operating limits, and specify what evidence would justify expansion. Connect that record to change management so that adding a tool or replacing a model triggers the appropriate review.

The test of governance is whether an organization can explain why an AI system is allowed to act, within which boundaries, and who can intervene when those boundaries no longer fit.

Before expanding your next pilot, ask: which capability would require a new approval, and who has the authority to give it?