MCP AI dataset generation

Under **Ask Your Data → AI generate**, ScalerDeep reads your data source schema, combines rule templates with LLM output, and batch-produces semantic datasets ready to import.

Workflow (four steps)

1. **Select data source** — must be configured with an allowed-tables whitelist.

2. **Select AI provider and model** — generation still runs without a key, but only rule templates apply (narrower coverage).

3. **Configure analysis directions** — add manually or accept AI suggestions; pick business themes for this batch.

4. **Read schema and generate** — review results, SQL test-run if needed, then import selected rows.

Results show `generation_source` (e.g. `template+llm`) and validation pass counts. Only **validated** rows can be imported.

Coverage modes

Coverage mode controls **how templates and LLM output are combined during generation**. It is separate from whether import overwrites an existing `dataset_key` (see **Import and overwrite** below).

Broad coverage (recommended, `broad`)

A **three-layer merge** for a fuller first batch:

1. **Heuristic templates** — preset SQL for common commerce tables (orders, products, support, etc.).

2. **Schema-driven** — time-column tables get basic daily stats (only when no analysis direction is selected).

3. **LLM supplement** — at least 8 datasets from the model where templates do not cover (6 when directions are selected).

**Best for:**

• First connection to a new data source.

• Teams that want broad out-of-the-box coverage before tuning.

**Note:** Without an AI key, broad mode still uses templates + schema-driven rules, but skips LLM supplement. The UI will show “rule template generation only”.

Standard (LLM-first, `standard`)

**LLM output is preferred**; schema-driven auto-datasets are not merged. Templates are fallback when the LLM is unavailable or fails.

• LLM available: mainly model-generated datasets (≥5, or ≥4 with directions).

• LLM unavailable: falls back to heuristic templates.

**Best for:**

• You already have template datasets and want differentiated AI-generated analyses.

• Smaller import batches with less duplication.

• Unusual schemas where templates match poorly.

Analysis directions

When **analysis directions** are selected (manual or AI-recommended), generation enters **direction-focused** mode:

• The LLM generates around selected themes; prompts include direction notes and `focus_topics`.

• **Schema-driven** auto-datasets are **not** appended.

• Coverage mode still affects minimum LLM counts (broad ≥6, standard ≥4). If the LLM fails with no fallback, check **model call logs**.

Generation source labels

• **template** — heuristic rule templates included

• **llm** — model returned valid JSON

• **template+llm** — merged in broad mode

**Archetype** and **derived mode** in the result list are inferred by the platform; you do not set `query_mode` manually.

Import and overwrite

Before import

• Select validated datasets; use **SQL test-run** to verify again.

• Enable **Create suggested intent rules** to import bundled `suggested_intent_rules` (keywords → dataset routing).

Same `dataset_key`

If a **custom dataset with the same `dataset_key` already exists**, import **updates** it (name, description, SQL, fields, synonyms, sample questions, query_modes, etc.) instead of creating a duplicate. Built-in seed datasets cannot be overwritten and are skipped on conflict.

vs coverage mode

• **Broad / standard** — how templates, schema rules, and LLM combine at **generation** time.

• **Same-key overwrite** — at **import** time, refresh an existing `custom_xxx` dataset with new content.

FAQ

**Q: Zero results or many validation failures?**

• Check that allowed tables include real business tables.

• Open **model call logs** and confirm the LLM returned valid JSON.

• Click **Validate** on failed rows for SQL errors (e.g. ONLY_FULL_GROUP_BY, missing columns).

**Q: Broad or standard?**

• New tenant / new source: **broad**.

• Existing templates, AI top-up only: **standard**, with clear notes in **additional instructions**.

**Q: Intent routing still wrong after import?**

• Add **sample questions** and **synonyms** on the Datasets tab.

• Add keyword rules under Intent rules, or use **Create suggested intent rules** on import.