Prompt and Context Engineering: What the Model Sees and How It Thinks

Companion ①: The Prompt is a layered structure reassembled on every turn, not a hard-coded system string. This article targets engineers; for methodology, see Workflow and Memory. For usage, see User Guide §12.11.

Series Position

For the full introduction, see Series Introduction and Product Positioning.

1. The Prompt Is a Layered Structure Reassembled on Every Turn

In Lantide Data, the Prompt refers to the system prompt (Core + State + conditional sections + knowledge) reassembled on each ReAct (reasoning–acting loop) iteration, along with the Context Summary, tool schemas, condensed history, and the active tab's Detail (including annotation-Optimizer output). This differs from the old approach of "write one system string and call it final": the goal is to make the model know which product state it is in, what it can do, and what it cannot do within a limited context window.

2. Overall Data Flow

User message
    │
    ├─► Context Summary (fixed each turn, deliberately small)
    ├─► get_app_context / tools ─► Detail (on demand, can be large)
    ├─► System: core_system + state_*.md + knowledge injection
    ├─► Function schemas (filtered by state)
    ├─► History (truncation strategy)
    └─► Active tab payload (Markdown with annotation optimization)
            │
            ▼
        LLM → tool calls → observation → next turn (may refresh state)

3. Context Summary: Injected Every Turn, but Deliberately Small

The Summary lists, as structured text:

• Current time (a concise one-liner: YYYY-MM-DD HH:MM (UTC±N), injected each turn)
• Workspace, Focused Project (including Plan/Report filenames and status; archived files are skipped)
• Active Tab type and whether it is dirty
• Number of Open Tabs, count of Cached Tables, external libraries, and MCP summary

It does not include the full text of each tab or complete schemas. If the model needs details, it should call get_app_context or show_tables / read_schema.

4. Context Detail: Full Content Fetched On Demand via Tools

The Detail channel carries:

• ANSI DDL for a specified table (read_schema)
• Application context JSON (list of open tabs, project file metadata)
• Active Markdown tab: the full text of the Plan/Report (via the annotation Optimizer)

When the user says "handle the annotations on the current Plan," this relies on the active tab Detail, rather than stuffing the entire workspace into the Summary.

5. Why Split into Two Channels, Summary and Detail

A single channel that stuffs full text into every turn's prompt quickly explodes tokens in a multi-tab IDE, and it is hard to purge stale content. Two channels keep the Summary stable and small, while Detail is pulled on demand by tools:

6. Layered Design of the System Prompt

1. Core system — product role, DuckDB rules, general tool-usage principles, the Query Execution Model, and the rule that statistical analysis requires ask_user + activate_analysis.
2. State prompt — switches workflow sections based on NoProject / ProjectFocused / Planning (PlanPlanning) / Executing (PlanExecuting), etc.; it does not repeatedly enumerate the tool catalog (schemas are authoritative).
3. Conditional sections — for example, the expand hint under catalog mode; the HTML editing section appended when html_report_active.
4. Knowledge — see §10.
5. Recent Query Steps (Layer 3.5) — when a conversation_id exists, the ledger summary is injected; for the authoritative implementation, see AI Agent Architecture §6.

State files record only methodology and state constraints (e.g., no statistics during the Planning phase); the exploration budget sentence count is defined only in core, to avoid duplicating and wasting tokens.

Assistant protocol: after each successful run_query / run_sql_tab, emit a one-line [[QUERY_STEP]] summary in the body (including purpose, etc.); for parsing and patch_step, see AI Agent Architecture §6.2.

Core also carries a cross-state analysis-operation loop: first confirm the question, denominator, granularity, columns, and JOIN key; then run the smallest verifiable SQL; and only then consider cache, DAG, or Source Run. This ensures that across states, the model treats "analytical credibility" as higher priority than "pipeline shape."

7. Query Execution Model (Semantic Contract for Query Execution)

Chat-style analysis tools often hide Run, materialization, and multi-step dependencies inside generated Python/SQL, leaving the user to see only charts. Lantide's Query Execution Model (QEM, the semantic contract for query execution) writes the semantics of validate / run_query / run_sql_tab / Source Run into the prompt, making the work of the "hidden DA" visible and auditable within the IDE state—the same thread as Workflow §3 SQL-first.

A dedicated core section serves as the cross-state authority, defining:

Hints returned by tools come in two kinds:

State files reference this with "see Query Execution Model," without repeating lengthy text. For the materialization reason_code and API details, see Unified Query Layer §8.1.

8. Core System: SQL Rules and Safety Boundaries

Beyond QEM, Core also carries the following safety and formatting constraints (do not inject sensitive settings such as API keys into the prompt; knowledge and user messages are still subject to length limits; System instructions and user bubbles are distinguished in the UI to prevent the model from confusing roles):

• Double-quote table names, two-part Excel references, MCP table-name format.
• Exploration phase: validate_query returns only ok/error, with no data rows; it must not be used for analytical conclusions.
• Formal data retrieval: run_query (preview limit of 200 rows) or run_sql_tab.
• External library ATTACH is managed by the UI; do not write ATTACH inside chat SQL.

9. State Prompt: State Decides "How to Think"

Example differences:

• NoProject: can do Quick analysis or guide focus_project.
• ProjectFocused + Planning: edit the Plan, handle annotations; do not Execute.
• Executing: run SQL in foreground/background mode; can source_run_sql_tab.
• html_report_active: appends the HTML editing workflow (co-present with chunk tools).

ProjectFocused HTML: the Generating section is always present; the Editing section is injected only when html_report_active, to avoid a prompt that encourages patching when no tools are available.

10. Knowledge Injection

For the governance narrative, see Governable Agent Memory. This article covers only engineering behavior:

• Only Enabled and approved User / Project files are injected.
• Queued items are never injected.
• Knowledge files use an mtime cache; they are re-read after changes (including migration from the old four-section layout).
• flat (User < 2,500, Project < 5,000 chars): full text of Rules + Info.
• catalog (at threshold): a catalog index + a hint to call expand_knowledge_catalog; truncated if it exceeds the injection budget.
• pre-inject Reorganize: when at catalog word count but without a catalog structure, an automatic reorganization may occur before the first run of a new conversation (see Governable Agent Memory §6.3).

11. Annotation Sidecar and AI Context

This echoes the annotation product story in Data Analysis Workflow in the Agent Era §6.

Plan / Report annotations are stored in a *.annotations.json sidecar; the Markdown body contains id-only <mark data-annotation-id> tags. read_plan / read_report and the Context Engine Active Tab return:

1. Disk body content (with id-only <mark data-annotation-id> tags)
2. Annotation sidecar annotations (open / resolved status, quote, history; the stale anchor.span is ignored)
3. open_annotation_count and annotation_summary (the Action Required task list)
4. patch_guidance (resolve modes A/B, mark boundaries; the Agent ignores the stale sidecar span)

Resolve path: the toolbar Resolve (N) or a chat command → resolve_annotations atomically updates the sidecar and the Markdown mark → sync_tab_content refreshes the editor; if the sidecar has already been updated externally on disk, the main window may prompt a reload (dirty tab toast).

For usage, see §11.8–11.9.

12. History Message Truncation

Not "the last N messages," but rather preserving the analytical context: recent user/assistant messages, paired tool traces, and key system events. The goal is to keep the Plan-execution context continuable in long conversations while controlling tokens.

13. Mid-turn Compaction

Within the ReAct loop, if tokens approach the limit, the tool results already accumulated in the current turn can be compacted, to avoid a single turn with many tools blowing the window. This complements the cross-turn truncation of §12.

When compacting run_query / query_result, it preserves step_id, cached_table_name, materialized, materialize_reason, and warnings, and truncates sql into sql_preview (first 200 characters), to avoid a multi-step chain losing memory over long turns.

14. Context Window Discovery

With multiple profiles and multiple models, probe the available context limit at runtime to pragmatically decide truncation and whether to auto-continue (see AI Agent Architecture). Avoid hard-coding assumptions for a single model.

15. Engineering Decision Overview

16. Conclusion

Context engineering determines whether the Agent's "perceived world" matches the IDE's real state—the Summary must be small, the Detail fresh, the State correct, and knowledge injected only after approval.

How these prompts turn into streamable tool behavior is the subject of AI Agent Architecture; query semantics are then landed on the DuckDB pipeline by the Unified Query Layer.