Slide 1: Title (1 min)

Speaker notes

• Set the frame: this is a build workshop, not a survey. Everyone changes a real repository in the first forty minutes.
• Say up front that Larkspur is invented and the shape of the problem is not. It buys credibility now and saves the question later.

Slide 2: One line (3 min)

@@ export function roundHalfEven(x: number): number
-  return roundHalfEvenImpl(x);
+  return Math.round(x);

Speaker notes

• Let the diff sit for a few seconds before you speak.
• Ask the room: who would have caught this in a 600-line PR? Hands go up slowly. That is the point.
• The agent did what it was asked. The test was wrong, and it made the code agree with the test.

Slide 3: The cost (2 min)

Speaker notes

• The amount is small on purpose. In regulated work the cost is never the amount.
• The real cost was trust: agent use frozen, pilot cancelled, and a memo from Model Risk.

Slide 4: The memo (3 min)

“I am not going to ask how often the agent gets it right. I am going to ask what it cannot do, how you know, and where I can see that for myself. Instructions are not controls.”Ruth Achterberg, Head of Model Risk Management

Speaker notes

• This is the design brief for the whole day. Read it slowly.
• Three conditions follow from it: protected code protected by mechanism, every change traceable, controls inspectable.
• Ask who in the room has a Ruth. Everyone in a regulated company does.

Slide 5: The harness (4 min)

Speaker notes

• Walk the layers top to bottom. Orange dots are the ones that enforce; the rest ask or add a checkpoint. MCP supplies facts; the Lab 8 hook that checks the ticket is what enforces.
• Everything here is a file in the repository. That is what makes it inspectable, which is Ruth’s third condition.

Slide 6: Measure first (4 min)

Speaker notes

• Lab 0 exists because “it seems better” does not survive a risk review.
• Three runs because agents are not deterministic. A task that passes once has not passed.
• All figures in this deck are illustrative until the pilot. Say so.

Slide 7: Prose moves most of it (3 min)

Speaker notes

• This is the most cost-effective hour in the workshop. Do not skip it.
• But the remaining failure is the one that matters. Set up the question: what turns a request into a control?

Slide 8: Request vs control (3 min)

Never edit src/money/ without
a human decision.

"PreToolUse": [{
  "matcher": "Create|Edit|ApplyPatch",
  … exit 2 on src/money/
}]

Speaker notes

• The model can argue with an instruction. It cannot argue with an exit code.
• Exit 2 blocks. Any other non-zero exit, like 1, is a non-blocking error: the hook looks right and stops nothing.

Slide 9: The hole (4 min)

Edit     src/money/rounding.ts        ✕ blocked by hook
Execute  sed -i … src/money/rounding.ts  ✓ ran

Speaker notes

• Demo this live if you can: /playground/hooks, config 1, step 3.
• The lesson generalizes: guard the outcome, not just one path to it.

Slide 10: Three layers (3 min)

Speaker notes

• Each layer covers what the one before it cannot see. None of them is complete alone.
• Ruth will ask “is it complete?” The honest answer is no, and here is what catches the rest.

Slide 11: False positives (3 min)

Speaker notes

• This slide earns trust with skeptical engineers. Admit the cost of controls.
• Feedback hooks and blocking hooks have different false-positive budgets.

Slide 12: Too easy (2 min)

Speaker notes

• A perfect score on eight tasks is a floor, not proof. The scoreboard is a living thing.

Slide 13: Ruth signs (3 min)

Speaker notes

• End on evidence, not a claim. Offer to let the room try to break a hook in the break.

Slide 14: Start here (2 min)

Speaker notes

• Point at the three doors on the home page. Most people start with the hook stepper.
• For a mixed room, point at /paths instead: four paths by role, two with nothing to install. Send the sponsor /paths?path=leader and the risk reviewer /paths?path=risk.