The grilling-session narrative that produced this design. Captured so the why survives ephemeral cloud sessions, context compaction, and future agents landing cold.
The session opened with a question about whether Vercel AI SDK was the right front-end choice. The operator confirmed the path. Then the conversation pivoted to the bigger question — was V2's vision actually being captured by what was on disk?
First grilling round surfaced that the existing V2 spec had baselined against the wrong layer— it treated V1-original's 42-agent complexity as the migration source, when collapsed (the actual production system) had already simplified that. The simplification commit 0e0703d that deleted the LLM Evaluator was solving a problem that didn't exist against the correct baseline.
From there, 17 more rounds nailed: the milestone shape (M1–M4 + Publish, locked from 08-milestone-architecture.md), the dispatch pattern (pipelined-sequential, APPROVED threshold), the agent layer (prompts-as-agents — “what would Claude Code do?”), the substrate (git in Supabase Storage, per-course repos), the review surface (the existing universal review system with workspace projection), the Knowledge Base (Karpathy's LLM Wiki pattern), and the UI shape (three-zone workspace, left-sidebar 3 sections, chat dock with context band).
The final move was consolidation: the conversation crystallised into a 44-decision spec, 5 ADRs, and the doc updates you're reading.
Is the Vercel AI SDK the right choice for V2's front-end AI calls, or should we look at alternatives?
“We can continue down the current path.”
Vercel AI SDK stays; allow narrow escape hatch for Anthropic-only features.
Set the technical baseline before the architecture grilling started.
Want to spend a real LLM call against the Generator (the handoff's ranked-1 next move)?
“I will run locally later. Let's spend this time ensuring the vision of this new branch is understood.”
Pivot from execution to architecture grilling.
Reframe the session: ensure vision is captured before any real spending happens.
Identify gaps between vision and current V2-P1 state — grill on the differences.
“The existing codebase is the problem. There is V1, then the collapsed architecture. My intention was to cull V1 references. The 12 HITL checkpoints I cited are V1 — not collapsed.”
V2 baseline = collapsed architecture, NOT V1-original.
Re-grounded the entire grilling — collapsed is the real baseline. ADR 0004 captures this.
Is V2 the next migration after collapsed (α), or does V2 jump over collapsed (β)?
“For question 5, a. The goal is to base this new version off the collapsed architecture.”
V2 IS the next-gen migration from collapsed. Cutover at V2-P6 via default-branch swap.
Locked the migration path. V2 inherits collapsed's 8-agent contract on a new substrate.
Q-A: Bring back the Evaluator? The 0e0703d simplification deleted it citing 'Claude Code uses one LLM.'
“Yes, it was a mistake. Bring back the evaluator.”
Restore the Evaluator capability. ADR 0007.
Identified that the simplification was justified against the wrong baseline. Reverting in spirit, restored as prompts.
What counts as a milestone? Three plausible models — phase-boundary, per-artifact, per-unit-spec-chain — with very different harness implications.
“(a) phase-boundary milestones BUT 'each milestone has an opportunity to finalize content and all finalized content that is student facing can be published separately.' M1 strategy → M2 structure → M3 skeleton → M4 substance.”
4 milestones with strict gating BETWEEN milestones, per-artifact finalize WITHIN milestones. (D6, D7)
Hybrid model nailed — coarse milestones, granular finalize within them.
What lives in M3? Per-unit only, per-module only, or both?
“M3 is per-module. Once a module is approved by the teacher, it can move past that gate. Other modules within that unit can still be in draft.”
M3 = per-module skeletons. Per-module finalize. (D10 — strict mirror dispatch from M2 unit's module subsections.)
Defined the M3 layer concretely. Per-module sub-agents become the M3 dispatch pattern.
Strict milestone gate (a), parent-gated cascade (b), or parent-gated + auto-dispatch (c)?
“(c) — yes, modules are nested (as will lessons and assignments under each module).”
Parent-gated cascade with auto-dispatch. Workspace tree is nested (units/unit-N/modules/module-M/...). (D9)
Locked the dispatch model. Generator auto-dispatches downstream when parent finalizes.
One sub-agent per unit-spec-lock (operator's instinct), or per-module parallel sub-agents (push-back)?
“Agreed (per-module parallel sub-agents with bound tools).”
Per-module parallel sub-agents with bound tools (writeFile scoped to single target file). (D14)
Aligned with Claude Code's bound-tools pattern. Mechanical safety, not prompt discipline.
Multiple decisions confirmed: per-artifact finalize, parent-gated cascade, student-facing taxonomy, targeted-eval + LLM judge for edit/regen, Orchestrator LLM-driven.
“1: per-artifact. 2: only student-facing get published. 3: targeted eval, then the LLM can judge to edit or Regen. 4: Orchestrator LLM-driven. 5: collapsed system breaks continuously. Grill me on individual questions one by one moving forward.”
(D5) Pure-workspace projection. (D8) 4-state lifecycle. (D9) Parent-gated cascade. (D15) LLM-driven Orchestrator. (D24) Targeted re-eval flow.
Six decisions confirmed simultaneously. Switched to one-question-at-a-time grilling.
How does the teacher enter M1? Form, chat, or markdown? What's in M1 concretely?
“(b) chat-based intake + (iii) hybrid M1. Initial intake is a form for basic info, then a markdown with questions and articles that need to be satisfied via chat before generation.”
M1 entry = form + MarkdownContract chat. M1 artifacts: course-outline + pedagogy (LLM, finalizable) + preferences-*.md + non-negotiables-*.md (deterministic, inputs).
Locked M1 entry flow. MarkdownContract pattern from collapsed reused for intake.
Auto on finalize, explicit per-artifact, or batch?
“Per-artifact publish OR 'Publish all' batch button — existing code. No unpublish currently, but there should be one.”
(b) Explicit per-artifact + batch publish. Unpublish added to backlog. (D16 implicit)
Publishing decoupled from finalizing. Module-or-larger subtree granularity.
Section structure: fixed minimal, conditional (V1), or floor+ceiling? Sub-agents: per-lesson or per-section?
“(c) floor + ceiling (Generator discretion above required floor). Same pattern probably for eval. (α) per-lesson sub-agent.”
(D17 corrected from b to c) Floor + ceiling sections. (D19) Per-lesson sub-agents.
M4 contract finalized. Generator has bounded pedagogical discretion.
How should reviewed edits land? Digest-checked linear apply, then readiness reports and teacher review — no automatic repair action.
“Yes. Look at the universal review system — this surface allows teachers to comment on markdowns and answer questions raised by the LLM. Aligns with git and our design well, akin to a PR queue.”
(B) Trigger on finalize-after-edit. (I) Verdict + diff_proposal output. Universal review surface is the V2 review UX home. (D24, D21)
Discovered existing review-panel surface that aligns perfectly with V2's needs. Just swap projection layer.
(a) pure-workspace projection, (b) hybrid (comments in Supabase), or (c) shadow rows?
“(a) — let's remove options for the system to break.”
Pure-workspace projection. Comments live as sidecar markdown files; no Supabase comments during generation. (D5, D23)
Confirmed substrate purity. Supabase only holds published rows; everything else is workspace markdown.
Where do workspaces live? (A) hosted git, (B) Supabase Storage, (C) attached volume. Per-course (α), per-teacher (β), or per-school (γ)?
“(B) and (α), but teachers should be able to host multiple repos in their dashboard.”
(D3, D4) Supabase Storage + per-course repos + multi-repo teacher dashboard.
Substrate location nailed. New course = new bucket folder + git init.
(I) inline-blocking, (II) full async, or (III) hybrid (Orchestrator inline, sub-agents async)?
“(III). We have a plan/progress tab as well to provide UI affordances. Chat stays minimal (just a working progress spinner).”
(D16) Hybrid async execution. Plan/progress tab is the sub-agent visibility surface; chat minimal during work.
Discovered the existing Progress tab (#1217) is already wired for this; only data source needs to change.
Evaluator mode structure — (a) single agent with mode param, (b) four specialized files, or (c) two evaluators + judge?
“The collapsed architecture might have the wrong call. What would Claude Code do?”
(d) Prompts-as-agents pattern. One Task primitive + N markdown prompts. (D17 — ADR 0005)
Critical reframe: Claude Code's pattern isn't 'one LLM' — it's 'one Task primitive + many prompts.' Architecture collapses to a single TS file + ~30 markdown prompts.
(B + ii) Per-teacher KB repo + Task-based query — recommended. Does Karpathy's LLM Wiki pattern fit?
“Confirm your recommendations. Research Karpathy's LLM wiki to ensure the model fits with our refactor. It should be a perfect fit.”
Per-teacher KB repo following Karpathy's LLM Wiki pattern. Three additions: kb/AGENTS.md schema file, kb/linter.md prompt, 'File as KB page' chat-dock affordance. Rename manifest.md → index.md. (D36–D40 — ADR 0006)
Researched Karpathy's actual gist. Fit was exact. KB structure aligned at two scales (course workspace + KB repo, same pattern).
The spec describes what was decided. The ADRs describe why each decision was taken over alternatives. This page describes how the decisions emerged — the question-answer arc that produced the architecture.
That arc is normally lost. Sessions end; context compacts; future agents land cold into the spec and have to reverse-engineer the reasoning. Capturing the arc means someone (or some agent) can read it, recognise the pattern of grilling that worked, and apply it to the next set of decisions.
Specifically: this page tells future readers that the V2 baseline was once misidentified, that the “simplification” in 0e0703dwas a direction error, and that “what would Claude Code do?” was the question that produced the prompts-as-agents architecture. Without this page, those are just artifacts in git log.