Reproducible AI analysis means more than saving the final answer. Another person must be able to locate the same inputs, rerun the same transformations, verify the execution conditions, and understand why the result is the same or different. A minimal evidence chain includes sources, version or time, SQL, dependencies, execution records, and limitations.
Saving the Chat Is Not Yet Reproducibility
The U.S. National Academies defines computational reproducibility as obtaining consistent results with the same input data, computational steps, methods, code, and analysis conditions. See Reproducibility and Replicability in Science. This definition is just as useful for business analysis.
A conversation may remember “calculate monthly revenue” without preserving which file was used, the data cutoff, when refunds were backfilled, the join key, intermediate tables created by the model, or the SQL behind the final number. Exporting the entire chat may still preserve a story about the work rather than executable evidence.
Six Links in a Minimal Evidence Chain
1. Sources
Record table or file names, worksheets, external connection aliases, and the relevant data owner. If the source changes, a filename is insufficient; retain a snapshot, query time, or data cutoff.
2. Version and Time
Answer both “when was it retrieved?” and “through what date does it cover?” The same SQL can legitimately produce a different answer tomorrow after orders, refunds, or exchange rates are updated.
3. Transformation SQL
Preserve cleaning, joins, filtering, and aggregation logic. Natural language explains purpose, while SQL reveals exactly which conditions changed. For statistical tools, retain the input tables and parameters as well.
4. Intermediate Dependencies
Monthly revenue may depend on clean_orders, which depends on raw order and refund tables. If this relationship exists only in the author's memory, reruns easily omit upstream work. W3C PROV supplies a standard model for entities, activities, and responsibility, illustrating that provenance is not only for large platforms. See the W3C PROV Overview.
5. Execution Records
Record which steps succeeded, failed, or were retried, and which queries actually ran. Planned steps do not prove execution; results alone do not explain their origin.
6. Limitations
Reproducible does not mean correct, and it does not establish causality. Missing cancellation reasons, stale segment labels, or cost data finalized a week late must travel with the result.
Rehearsing a Rerun of “Monthly Gross Margin Fell”
Suppose a Report says June gross margin fell. A valid rerun path looks like this:
orders.csv + refunds.csv + product_cost.xlsx
→ clean_orders SQL
→ order_margin SQL
→ monthly_margin SQL
→ June gross margin and limitations in the Report
Before rerunning, check whether the source is the same snapshot, whether product_cost.xlsx added retroactive costs, and whether the refund cutoff changed. Execute in dependency order rather than running only the last query. Finally, compare old and new numerators, denominators, and distinct order counts. Even when the result changes, this process locates the difference in data, logic, or execution conditions.
Reproducible Does Not Mean “Identical Forever”
New results are expected when a live source changes. To verify an old conclusion, use the same snapshot and conditions. To refresh an operational result, use new data but retain both cutoffs and any SQL differences. Queries involving sampling, machine learning, or external APIs also require random seeds, tool versions, or response versions.
Reproducing a result only proves that the same inputs and method produce a consistent output. A wrong denominator can be reproduced perfectly. Reproducibility must therefore work alongside Plan review, data-quality checks, and Report limitations.
How Lantide Data Builds the Evidence Chain
In Lantide Data, persistent SQL tabs preserve transformation logic. Local caches can feed downstream SQL, lineage can connect persistent tabs, and Source Run recomputes upstream work in DAG order. Project Analysis links the approved Plan, execution evidence, and Report. See the unified query layer and Source Run and the cache user guide.
One boundary matters: Lantide's DuckDB TEMP caches are intermediate results in the current workspace and may be cleared after switching workspaces or restarting. They are neither permanent tables nor a warehouse. Reproducing a historical state precisely still requires source snapshots or versions and, where relevant, external environment and statistical tool versions. Lantide can preserve logic and dependencies; it cannot create an upstream history that never existed.
One-Minute Reproducibility Check
- Can I identify the source and cutoff for every key number?
- Can I find the SQL or statistical parameters that produced it?
- Do I know the upstream dependencies and execution order?
- Can I distinguish planned work from actual execution?
- If updated data changes the result, can I locate why?
- Does the Report preserve definitions and limitations?
If two answers are “no,” you have probably preserved an answer rather than an evidence chain.
Conclusion
The goal of reproducible analysis is not to return the same number forever; it is to make differences explainable. Choose a frequently questioned monthly report and trace backward through sources, SQL, dependencies, execution, and limitations. Wherever that path breaks is the analysis infrastructure most worth improving first.