AI schema generation

There are two AI-powered schema-generation flows. Both observe your content, infer a working schema, and commit it to git.

• Full investigator — bootstraps a brand-new .norcube/cms/ directory layout from a complete repo scan. Use when you connect a site with existing content and don't want to author 100+ lines of YAML by hand.
• Single-block generation — drafts the schema for one block by name. Triggered from the editor's "missing block" affordance when you open a page that uses a block whose schema isn't defined yet.

When to use which

The full investigator is destructive — it wipes everything under .norcube/cms/collections/ and .norcube/cms/blocks/ and writes the freshly-generated files in the same commit. Use it for greenfield bootstrap or deliberate full-refresh.

The single-block flow is additive — it writes one file under .norcube/cms/blocks/<key>.yml and touches nothing else.

What it observes

Whichever flow you trigger, the CMS does the same observation first: walk your repo, sample your Markdown files, parse their frontmatter, and build a picture of what fields exist and what shapes they take. The result is what the LLM works from.

File discovery

The observer looks for .md files. It auto-detects your content root by trying common conventions — src/content/pages, src/content, content/pages, content, pages — and falls back to whatever directory contains .md files.

For large repos it samples up to roughly 120 files, spread evenly across subdirectories so no single deep folder (blog/posts/2024/01, …/02, etc) dominates the sample. Small repos get all files.

Only the YAML frontmatter is parsed — the Markdown body is ignored. Field types are inferred from frontmatter values exclusively.

Type inference

Each observed field gets classified before the LLM ever sees it:

Enum detection. A field becomes a select (enum) when:

• All observed values are short (< 80 chars) and not prose-shaped.
• There are a small number of distinct values across the sample.
• The field's name is the kind that typically holds enum values (status, theme, role, etc.) AND its name isn't one that typically holds free text (title, description, body, etc.).

When flagged, the LLM is hinted to emit type: select with the distinct values as options.

Block detection

The observer recognises blocks by looking for a _type: or _component: discriminator key on nested objects. The three patterns it picks up:

• blocks: [{_component: "hero", ...}] — array of blocks.
• sections: [{_component: "cta", ...}] — array under any field name.
• _component: "case_study" at page root — the page itself is a block instance.

If your content uses a different discriminator key (kind:, block:, …) the observer won't pick it up. Hand-author the schema with array.idKey pointing at your key.

Progress

Both flows go through the same phases — discover files, observe shapes, run the LLM, commit. The streaming variant surfaces each phase to the UI as it happens, so you see real-time progress ("Reading the block summary…", "Validating the draft against your content…", "Writing schema files to the repo…") instead of a spinner. The synchronous variant runs the same phases to completion and returns one final response.

Full investigator

Triggered from "Analyze repo" on the empty schema screen of a newly-connected site.

What it commits

One commit containing:

• .norcube/cms/config.yml — root config (pagesDir, collections reference, site metadata).
• .norcube/cms/collections/<key>.yml — one file per top-level collection (typically pages).
• .norcube/cms/blocks/<key>.yml — one file per observed block (hero, media-text, stats-row, …).

Clean-slate semantics

Before writing, the agent wipes every existing schema file in .norcube/cms/collections/, .norcube/cms/blocks/, and .norcube/cms/inputs/ — in the same commit. This is intentional:

• Without the wipe, re-running on an existing schema would leave orphan files whenever the LLM renamed or removed a block.
• With the wipe, every run produces exactly what the agent generated and nothing else. The git history captures both states, so nothing's lost.

If you've hand-edited a block file and re-run the investigator, your edits get overwritten. Treat the full investigator as a bootstrap or refresh tool, not an iteration tool. For incremental block additions, use the single-block flow instead.

Self-correction

The agent doesn't just guess once. It proposes a schema, runs it through an internal validator (which checks every observed block has a schema entry, every observed key appears in its block's fields, types line up), and then iterates if the validator finds mismatches. You see the iteration in the streaming UI as messages like "Reading the block summary…", "Finding shared sub-shapes…", "Validating the draft against your content…". Most runs converge in a few iterations.

If the agent runs out of iterations without a clean schema, the run fails with the unresolved problems surfaced in the error message. Rare in practice.

What can stop it

• AI not configured for your deployment. Talk to your platform admin.
• Site isn't connected to a repo yet. Run the connect-repo flow first.
• No Markdown files anywhere in the repo. The investigator needs something to observe. Check your repo layout (does the content directory exist? does it have .md files?).

Single-block generation

Triggered from the editor's missing-block affordance — the inline card that surfaces when a page uses a block whose schema isn't defined yet. Click the "✨ Generate from samples" button and wait.

What it commits

One file: .norcube/cms/blocks/<key>.yml.

When the block hasn't been used yet

If you ask for a block whose key isn't in any frontmatter on the site, the request fails: the agent can't infer a schema for a block it's never seen.

The remedy: either place the block in at least one content file (its discriminator + a few fields) and re-run, or hand-author the schema yourself in .norcube/cms/blocks/<key>.yml.

When the AI struggles

If the LLM call fails on the single-block flow — timeout, rate limit, content-policy refusal — the agent falls back to a deterministic stub: a name-only block with the fields straight out of what was observed, no labels or previews or validation. You get something working immediately and polish it by hand. The editor surfaces "This is a generated stub — please refine" inline on the block card so you know it's not a polished output.

The full investigator does not fall back to stubs — if the LLM fails there, the run fails. The full investigator is supposed to be a polished bootstrap; the single-block flow is supposed to be a fast unblocker.

What you see

When the run finishes, the missing-block card in the editor populates immediately with the generated schema — no manual refresh needed.

Limitations to know

• Re-runs are not incremental for the full investigator. It wipes and regenerates. If you've hand-edited schemas, those edits are lost. Use single-block for incremental work.
• Sampling bias. With a 120-file cap, large repos miss rare blocks or under-weight niche variations. Round-robin sampling across subdirectories helps but doesn't fully eliminate the bias.
• No body content is observed. Field types and counts come from frontmatter values only. Markdown body is ignored.
• Discriminator key is hardcoded. The observer looks for _type: or _component: keys. Content using a different key (kind:, block:) won't be recognised as block variants — hand-author with idKey:.
• The LLM can hallucinate. Despite validation, it sometimes emits type: select on free-text fields, or invents fields not in the shape report. The validator loop catches most; some slip through. Review the generated schema after running.
• No cross-repo conventions. The investigator works purely from observed content. It doesn't know "you're using Astro" or "this looks like Hugo frontmatter" — won't suggest framework-idiomatic fields unless you have them in your content already.

What's not there yet

Refinements on the list but not shipped:

• Incremental full-investigator (union mode) — re-run as additive rather than wipe-and-replace. Wanted for sites that want periodic shape refreshes without losing manual edits.
• Body-content observation — read prose for context (e.g. infer richtext for long unstructured fields).
• Framework presets — "scaffold for Astro / Hugo / Eleventy" picker that pre-loads the idiomatic frontmatter shape.