AI Agent governance should not begin with another “responsible AI use” policy. Begin by mapping what each actor can read, which tools it can call, when a person must approve, and which evidence remains afterward. Then bind each policy statement to scope, approval, telemetry, recovery, and a named owner. That is how governance enters the product and daily operations.
Why principles alone cannot stop a tool call
Chat tools primarily produce text. Agents may repeatedly read data, invoke tools, create files, send results, or modify settings. Risk comes not only from malicious behavior but also from ambiguous objectives, misread data, contaminated tool descriptions, or a model mistaking “can do” for “should do.”
If a policy says only “human oversight is required,” implementers still do not know which queries can run automatically, who approves them, which parameters approval displays, or whether a mistake can be reversed. Likewise, “retain an audit trail” does not mean logging the full text of every prompt and dataset. That could create another copy of sensitive data.
The NIST AI RMF Core divides AI risk management into Govern, Map, Measure, and Manage, and calls for clearly defined human and AI roles, continuous monitoring, and periodic review. AI RMF 1.0 is currently being revised, so it is most useful as a shared organizational risk vocabulary rather than an immutable compliance checklist.
A six-part capability map: turn abstract governance into testable questions
Complete these six sections for one specific use case. That is more effective than beginning with “all AI in the company.”
1. Scope: where does it work?
List data sources, workspaces, projects, user groups, and environments. Do not write only “company data.” Specify the schema, folder, or customer boundary.
Test question: Can the agent cross workspaces without an explicit selection? Can it see sensitive fields unrelated to the task?
2. Capability: what may it do?
Separate read, analyze, write artifact, export, external action, and admin. The execution layer should enforce permissions rather than asking the model to restrain itself through a prompt.
Test question: Does the backend reject a write-tool call from a read-only role? Can formal-analysis tools run before approval?
3. Approval: who approves which decisions?
Approval should match impact. Field exploration may not require confirmation for every call. Formal numbers used in finance, customer communications, or executive meetings need definition review first. External transmission, administrative settings, and irreversible actions need stronger confirmation.
Test question: Can reviewers see the purpose, data scope, parameters, and side effects? Does a rejection fail closed?
4. Telemetry: how do you know what actually happened?
Record at least the actor, time, workspace, tool, policy/approval outcome, affected artifact, and execution status. An output summary should link to formal evidence instead of preserving only the agent's account of its actions.
Test question: Three days later, can you identify the query that produced a number? Are credentials and sensitive full text redacted?
5. Recovery: how do you stop and restore after failure?
Recovery includes terminating a session, revoking credentials, reverting reversible content, rerunning work, and reporting an incident. Distinguish text edits from real external side effects; not every action can be undone.
Test question: Can an abnormal loop be stopped? If a person has made a later edit, does rollback detect the conflict and refuse to overwrite it?
6. Accountability: who owns the outcome and controls?
Assign a business owner, data owner, technical owner, and incident owner to the use case. Human approval does not magically transfer all responsibility to the last person who clicked a button. Approval itself can fail when the interface or information is inadequate.
Test question: Who signs off on metric definitions, who grants access, who handles incidents, and how often is the system reviewed?
Example: split “analyze last month's conversion” into three levels
The same request can be graded by impact rather than always automated or always blocked:
| Risk level | Task | Automated scope | Human control |
|---|---|---|---|
| Low | Inspect schema, nulls, and date ranges | Automatic exploration in one workspace | Can stop at any time; query activity retained |
| Medium | Calculate the funnel for an internal weekly meeting | Agent drafts Plan and SQL | Review denominator, time window, and exclusions before Execute |
| High | Disclose externally or modify data/settings | No direct authorization | Separate confirmation, restricted destination, additional owner approval |
The decisive factors are impact and recoverability, not SQL length. A ten-line query can produce a number used in financial reporting. A dozens-of-steps schema inventory can be low risk when it is read-only and never leaves the environment.
Three designs that look governed but have little effect
Asking a person to click Allow at every step
Confirmation fatigue turns approval into a pacing obstacle. Combine traceable queries within one Plan scope into one meaningful authorization, then confirm high-impact actions separately when they exceed that scope.
Calling any log an audit
A log without the workspace, tool parameters, and artifact is hard to investigate when it contains only a natural-language summary. At the other extreme, logging every token and complete dataset is not a secure audit either.
Using Admin as the convenient default
Admin does not mean “less likely to get blocked.” It permits higher-trust organizational and configuration actions. A first trial should begin with minimum capability and use observed blockers to prove that elevation is necessary.
The OWASP AI Agent Security Cheat Sheet treats tool misuse, privilege escalation, memory poisoning, excessive autonomy, and sensitive-data exposure as distinct risks. Agent governance therefore cannot stop at prompt injection. Identity, tools, memory, and data flows all need controls.
How Lantide Data embeds governance in the analysis workflow
Lantide Data is a local-first SQL analysis IDE and Agent runtime. Formal Project Analysis puts governance into Plan → review/annotation → user-approved Execute → Report. The Plan captures the question, denominator, grain, and checkpoints before execution. Formal evidence remains in execution progress, and the Report links findings to that evidence and its limitations. See the Plan → Execute → Report guide.
When an external Agent connects through the local MCP Server, Observe/Execute/Admin is the connection's capability ceiling. Plan approval authorizes only that Plan and does not upgrade the connection. External MCP Activity in the GUI lets users inspect tool calls, policy/review outcomes, and artifact changes. Supported rollback of text edits still performs a hash-conflict check. Connection, recovery, and client compatibility must be verified for the actual version. See External Agent integration.
These mechanisms reduce the risk of invisible capability and evidence. They do not guarantee a correct model conclusion or replace organizational data classification, legal judgment, and incident response. Local-first also does not mean that every external model and connector keeps data on the device.
Conclusion: complete the first map with one real use case
Choose an analysis with clear data that will face questions but has manageable consequences if wrong. Fill one page with Scope, Capability, Approval, Telemetry, Recovery, and Accountability. Then design one rejection test and one recovery test for each relevant control. A policy becomes governance only when the team can prove that prohibited actions cannot happen, completed actions can be traced, and failures can be stopped.