Lantide Data
Back to blog
Agent Governance

Why Does AI Still Calculate the Wrong Answer When It Has the Schema? Data Analysis Needs Business Semantics

Table names and field types describe the shape of data, not the definitions of valid customers, revenue, conversion, or time windows. Learn which business semantics AI analysis needs and how to turn definitions into a reviewable contract.

A schema can tell AI which tables, fields, and types exist. It cannot tell AI how the company defines a valid customer, whether refunds should be deducted, or which denominator belongs in a conversion rate. Reliable access to enterprise data requires more than understanding structure. AI also needs business semantics that are traceable, reviewable, and clearly scoped.

A schema describes the shape of data, not how a company makes decisions

Suppose a database has these fields:

accounts(account_id, created_at, status)
subscriptions(account_id, started_at, cancelled_at, plan)
events(account_id, event_name, occurred_at)
invoices(account_id, amount, paid_at, refunded_at)

From names alone, a model can infer that paid_at is a payment time and amount is a monetary value, then assemble syntactically valid SQL. But when a user asks, “What was new-customer conversion last month?” the schema answers none of these questions:

  • Is a “new customer” defined by account creation, first activation, or first payment?
  • If one company opens three accounts, does the denominator count one or three?
  • Does conversion from a free plan to a paid plan qualify?
  • Is the conversion window seven days, 30 days, or the same calendar month?
  • Should test accounts, internal accounts, and cancellations followed by resubscription be excluded?
  • Which reporting period receives late-arriving data and refunds?

These are business rules, not field-discovery questions. A model can write flawless SQL syntax yet choose the wrong one among multiple plausible definitions and return a precise but unusable number.

The 2026 enterprise Text-to-SQL benchmark EntSQL was designed to measure this gap. It contains 1,066 semantically aligned Chinese-English cases across five business domains. The paper reports that most cases require internal metrics, reporting conventions, or organizational rules in addition to the question and schema. Under its English-input setting with long enterprise documents, the best evaluated system achieved 15.9%. This is not a universal accuracy rate for every AI analytics product. It demonstrates that placing a schema in a prompt does not mean the model understands the enterprise. See the EntSQL paper abstract and revision history.

AI analysis needs six kinds of semantics, not simply more fields

A “semantic layer” generally means a model between raw data and consumers that centrally describes business concepts. Implementations differ by product, but common elements include metrics, dimensions, relationships, and calculation rules. Salesforce's Tableau Semantics, for example, identifies dimensions, measures, relationships, and metrics as major definition types. See Tableau Semantics concepts.

When the objective is AI-assisted analysis, avoid beginning with a debate over product names. First verify that six kinds of semantics exist:

Semantic Question that must be answered Typical error when missing
Entity Which keys identify customers, accounts, and orders? A company with multiple accounts is counted repeatedly
Grain Does each row represent an event, day, order, or customer? Amounts or counts inflate after a JOIN
Metric What are the formula, numerator, denominator, and aggregation? Multiple versions of the same KPI appear
Time Should the query use event time, ingestion time, or a financial period? Cross-month results cannot be reconciled
Scope Which statuses, plans, regions, and test data must be excluded? Inapplicable samples enter the result
Authority Who approved it, which project uses it, and when is it updated? Expired rules remain in use

A complete semantic layer can let multiple tools reuse metrics and dimensions and may translate definitions into queries. The official Cube documentation, for example, shows how measures, dimensions, filters, and calculated measures become a reusable model from which SQL is generated. See Cube Data Modeling.

But “we have a metric dictionary” and “the system enforces the same calculation” are still different claims. Markdown, a wiki, or a catalog can provide context. An executable semantic layer can apply definitions at query time. Teams should state which layer they actually have rather than assuming that every AI query automatically follows a definition merely because documentation exists.

Turn one metric definition into a contract people can review

“Paid conversion = paying customers / new customers” sounds clear, but it still omits execution conditions. A usable definition should look more like this:

Metric: 30-day paid conversion for new customers
Scope: Taiwan self-service signups; exclude employee, test, and partner accounts
Entity: company_id; count each company once
Denominator: Companies that first completed email verification during the observation month
Numerator: Denominator companies whose first successful payment occurred within 30 days of verification
Time fields: verified_at and paid_at, both calculated in Asia/Taipei
Resubscription: A company that paid before the observation month is not a new customer
Data sources: accounts, company_members, invoices
Validation: Reconcile denominator with the CRM monthly new-customer report; sample payments crossing month-end
Owner: Growth Ops
Version: Effective 2026-07-01

The value of this format is not Markdown. It lets people raise precise objections. Why use email verification rather than account creation? Why exclude customers sourced through enterprise sales? How should a 30-day window that crosses reporting months be displayed?

AI can help translate the contract into SQL, but it should not silently guess the contract itself. If entity, grain, time window, or exclusions lack agreement, mark them as assumptions requiring confirmation instead of substituting the most likely field name for a decision.

You may not need to buy a semantic layer first; identify the gap

Adoption can grow with reuse needs:

  1. One-off exploration: Specify grain, filters, joins, and time window in the analysis Plan; preserve SQL for review.
  2. Reuse within a project: Create metric cards, field mappings, and status-code References with an owner and version.
  3. Reuse across tools: Put stable metrics, dimensions, and relationships in an executable semantic or metrics layer.
  4. Organization-wide governance: Add change review, permissions, tests, lineage, and retirement.

If a company has only a few stable reports, defining the ten most disputed metrics is usually more practical than modeling all data at once. Conversely, if the same metric feeds BI, spreadsheets, APIs, and several AI Agents, attaching a document to each prompt invites version drift. An executable, centrally governed semantic layer then becomes more important.

How Lantide Data brings business semantics into a governed workflow

Lantide Data does not position itself as an enterprise semantic layer or metric service. It addresses a different part of the risk: when analysts and AI Agents actually query data, how can business semantics be read, reviewed, and retained in a traceable delivery process?

  • Reference docs can store field mappings, status codes, join explanations, and complete metric definitions. Project Knowledge keeps only their purpose and retrieval guidance so the Agent loads them for the relevant task instead of permanently filling the prompt with large documents.
  • Reusable definitions that persist across conversations can be proposed by the Agent as Queued Knowledge with verifiable evidence. They enter later context only after the user applies them to the User scope or a specific Project.
  • Formal analysis records entity, grain, filters, time windows, and validation methods in a Plan. The Agent executes against that contract only after user review and Execute, then forms a Report from SQL evidence and limitations.

These layers address large reference material, approved long-term knowledge, and the current analysis contract respectively. They cannot replace one another. Reference docs and Agent Memory provide context; they do not automatically enforce metric definitions like an executable semantic layer. Plan review also cannot prove that source data is correct. Teams must still maintain upstream models, test definitions, and leave the final decision to someone accountable for the result. See Agent analysis workflow, governed Agent Memory, and Reference docs for analysts.

Conclusion: identify the layer where the model is guessing

The next time AI produces SQL that runs, do not review syntax alone. Ask whether entity, grain, metric, time, scope, and authority each have an explicit answer. Wherever the answer is missing, the model is guessing.

A schema is the entry point to analysis, not a business contract. Reliable AI analysis is not simply a matter of sending more documentation into context. Definitions need a source, scope, and approver, and every execution should leave reviewable SQL and limitations.

References