A formal analysis starts with a Plan to turn an ambiguous request into a reviewable analysis contract before AI retrieves data. Execute is the authorization event after a person confirms definitions and scope. It adds a step, but prevents a wrong assumption from first appearing as a set of seemingly definitive numbers.
How “Run First, Ask Later” Hardens Errors
Suppose a product manager says, “Compare conversion before and after the new checkout page launched.” If AI executes immediately, it might choose to:
- use sessions that entered checkout as the denominator;
- use payment events as the numerator;
- compare 14 days on each side of launch;
- exclude test accounts but not refunds;
- count multiple payments in one session only once.
Each choice may be reasonable, yet the combination may not answer the team's intended question. Worse, once “conversion increased 3.2%” enters a presentation, discussion often shifts to why it rose instead of asking what denominator produced the 3.2%.
Human-in-the-loop does not mean asking someone to click “reviewed” at the very end. It means returning decisions to accountable people at moments of high impact, low reversibility, or unclear intent. OpenAI recommends risk-based guardrails and human intervention for high-risk tools, while Anthropic notes that only users can resolve some gaps involving preference or intent. See the OpenAI agent guide and Anthropic: Trustworthy agents in practice.
What Belongs in an Executable Plan
A Plan should not be a task title such as “analyze conversion and recommend improvements.” A minimal version can use this template:
Decision question: Should the new checkout page roll out to everyone?
Population: Eligible sessions entering checkout for the first time
Numerator/denominator: Sessions paying within 14 days / sessions entering checkout
Grain: One row per session; deduplicate multiple payments in one session
Time range: 14 complete days before and after launch, Asia/Taipei
Exclusions: Employees, test traffic, cancellations, and fully refunded orders
Join key: checkout.session_id = payment.session_id
Checkpoints: Daily sample size, distinct sessions before/after join, device segments
Limitations: Concurrent campaigns and weekday mix may confound results; do not claim causality
This Plan serves three purposes: nontechnical owners can understand it, analysts can translate it into SQL, and the completed Report can be checked for drift from the original contract.
Execute Is Authorization, Not Just a Play Button
A mature workflow separates exploration from formal execution. Exploration can inspect schemas, confirm columns, and sample data. Formal Execute produces the complete result from the approved Plan. This boundary lets the team answer, “Who agreed to run this analysis, under which assumptions?”
The U.S. NIST generative AI risk framework emphasizes risk management across design, use, and ongoing monitoring, along with clear roles for people and AI. It does not require human approval for every action; controls should match the context. See NIST AI 600-1. Step-by-step approval may be excessive for a low-risk column preview, but explicit authorization is valuable for results that will enter a financial forecast or product launch decision.
Six Questions for Reviewing a Plan Without SQL
- Which decision will this result support?
- Who is included, and who is excluded?
- What are the numerator and denominator?
- Does one row represent a person, order, session, or event?
- What are the time range, time zone, and data cutoff?
- Which checkpoint will prove that execution did not duplicate or omit records?
If any answer is missing, improve the Plan rather than hoping AI guesses correctly.
How Lantide Data Implements This Boundary
In Lantide Data Project Analysis, an agent can explore data and draft a Plan, while the user reviews, comments, and requests revisions. Only when the user selects Approve & Execute does the Plan enter execution; the agent then creates SQL steps and produces a Report from the approved content. See the Plan-state user guide and the rationale in the agentic analysis workflow.
This mechanism suits formal analysis whose definitions still require collaboration and whose result will be cited. It does not mean every simple question needs a project: a one-off column lookup or syntax check can use Quick Analysis. Nor does a Plan make the business strategy decision for you. It makes assumptions and responsibility boundaries explicit; people still own the decision.
Conclusion
The value of Plan before Execute is not the appearance of rigor. It lets a team fix the cheapest and most consequential error before any number appears: defining the wrong question. Before your next formal analysis, review the Plan with these six questions and only then decide whether to authorize execution.