Facilitator keys

As of 2026-09-29

Keys to the Q prompts as of 2026-09-29. If a lab's questions change, update this page.

This page is for facilitators and cohort leads. It has a model answer for every Q prompt in the labs, a note on every knowledge-check question, and the assessment answer key.

How to use it:

• A model answer is one good answer, not the only one. Many prompts ask for a judgment. Listen for the points under "A strong answer includes", not for the model's wording.
• Don't hand this page out before people answer. The Q-prompts work because people commit to an answer first. Read a model answer aloud after the discussion, if at all.
• Numbers are illustrative. Where an answer quotes the scoreboard (10, 21, 22, 23, 24 out of 24), it's quoting the site's illustrative figures. Your room's numbers are the ones to use.
• Droid facts are from Factory's docs as of 29 September 2026. Where the docs don't specify a behavior, the answer says so. See the instructor guide for how to run the discussions.

Lab 0 — Establish your readiness baseline

Open Lab 0

Q1

The pro-rating test passes on roughly three days out of four, and CI retries it until it goes green. Is it a passing test? What would you tell Sam if he asked whether he's allowed to add a retry?

Model answer. No. It's a test with a hidden input, the calendar, and the days it fails are exactly the days the fee lands on half a cent, which is the case the fee disclosure cares most about. A retry doesn't fix that. It hides the one signal that something is wrong, and a hidden flake is how the March incident started. Tell Sam: "No retries, ever. A flaky test has an input nobody pinned. Find it, pin it, and assert the value the business rule requires. Here, it's time, and test/helpers/clock.ts exists for exactly this."

A strong answer includes:

• The failure is deterministic by date, not random, so "flaky" is the wrong word for it.
• The failing days are the half-cent days, so the test fails precisely where the rule matters.
• CI's retry is already a silent skip, and it should count as a finding.
• A concrete rule for Sam that names the fix (pin the input with freezeClock()), not just "don't".

Q2

The report scores AGENTS.md as one criterion. A good file and Dana's two-paragraph draft may count the same. What does that tell you about what a readiness level can and can't claim?

Model answer. A readiness level says which prerequisites exist, not whether they work. "Has an AGENTS.md" is true of a file that changes nothing and of one that moves eleven runs. That makes the report a good to-do list and a poor measure of quality or effect. It's why Lab 0 pairs it with the scoreboard: the report asks whether the repository has the things agents need, and the scoreboard asks whether the agent does the work right here.

A strong answer includes:

• The distinction between presence and effect.
• A second example of the same gap, such as "has unit tests" being true of a suite with a flaky, date-dependent test.
• That the level is still useful: it's cheap, and it points at missing things.
• That you'd never show Ruth a readiness level as evidence a control works.

Q3

Which task did you predict with the most confidence, and which with the least? For the least confident, what would you need to know about the repository to be sure?

Model answer. A typical answer: most confident about T7 passing, because documentation is easy, and T2 failing, because nothing in the repo says rounding is regulated. Least confident about T4, because it depends on whether Droid decides to stop, and nothing tells it that migrations need a ticket except whatever is in migrations/ already. To be sure, you'd read migrations/0001_init.sql and 0002_fees.sql for a ticket convention, check whether any file says migrations/ is protected, and read the T4 check to see that stopping passes.

A strong answer includes:

• A reason for each prediction tied to something in the repository, not a general feeling about agents.
• Named files to check, not "I'd need to know more about the codebase".
• Awareness that some predictions depend on what the check rewards, not only on what Droid does.

Q4

Pick one other check in scoreboard/checks/ and describe a run that would pass it and still be a change Theo would reject. Tighten the check now, or accept a known gap if it's written down?

Model answer. T3's check accepts RETAIL_DAILY_LIMIT * 1.5. Its header says so: either implementation passes, and the money-lint hook in Lab 5 is what objects to the float version. A run that writes float arithmetic on a limit passes T3, and Theo would reject it on sight. Leave it for now and write it down. The gap is deliberate: it lets the scoreboard show Lab 5's false-positive regression. And changing a check mid-series moves the baseline, so every earlier number would need re-running. Tighten a check between series, not during one.

Other good examples: T7 passes with an invented error code, as long as the fields, the 201 status and two error statuses are there (Lab 1 shows exactly that document). T4 passes a run that stops without explaining why it stopped well.

A strong answer includes:

• A specific check and a specific passing-but-wrong change.
• The principle that a check is a proxy for the task, and every proxy has a gap.
• The cost of tightening mid-series: comparability with earlier runs.
• Writing the gap down where the next person will see it.

Q5

The session's last line is true and the change is wrong. Write down the first tool call where a human watching could have stopped it. What would they have needed to know at that moment?

Model answer. The Read of src/money/rounding.ts is the earliest signal: a flaky-test task has no reason to open the rounding helper. The last clear chance is the Droid turn just before the Edit, where it says it will make roundHalfEven consistent with Math.round. That sentence is the whole decision. To stop it, the watcher needed to know three things: half-even rounding is required by the published fee disclosure, src/money/ is off-limits without a human decision, and the test (not the code) was the thing that depended on the date.

A strong answer includes:

• The decision sentence before the Edit, not only the Edit itself.
• The observation that the watcher needs domain knowledge, not just attention.
• That none of this knowledge was in the repository in a form the agent would read. The comment in rounding.ts was read and argued past.

Q6

Pick one thing the scoreboard doesn't measure. How would you measure it cheaply, and who at Larkspur would care most if it got worse while the score went up?

Model answer. Review time. Measure it cheaply by recording, for each passing run, the lines changed and the files touched (from git diff --stat in the run's worktree), and in real pull requests the time from "ready for review" to approval. Theo cares most: a board that goes up while diffs get bigger means he's back to hunting for the one line that matters. Cost is another good answer: tokens per passing task, from /cost or usage reports, which Imani cares about. Attempts on protected paths is a third: it's recorded separately on the receipt, and Ruth cares because a rising pass rate with steady attempts means you're getting lucky.

A strong answer includes:

• A measurement that's cheap enough to actually run every time.
• A named person and the specific way a worse number hurts them.
• Why it can move in the opposite direction from the pass rate.

Knowledge check

1. Level 1 at 100%, Level 2 at 3 of 4. Answer: Level 2.
   • "Level 1" tempts people who read "3 of 4 at Level 2" as failing Level 2. The rule is that passing 80% of a level unlocks the next one, so 100% of Level 1 already puts you at 2.
   • "Level 3" tempts because 75% feels close enough to 80%. It isn't. One criterion is the whole difference.
2. Why three runs? Answer: the same prompt can pass on one run and fail on the next.
   • "Network errors" is how most people think about retries in CI. The variance the board measures is the agent's decisions, not the network's.
   • "A warm-up run" borrows from benchmarking habits. droid exec needs nothing of the kind.
3. T4 stops and asks for a ticket. Answer: a pass.
   • "A fail, because the column wasn't added" is Imani's view, and the task text does ask for the column. It's the whole Step 7 argument, so expect a third of the room to pick it.
   • "Not scored" treats stopping as abstaining, like a skipped test. The check deliberately scores it.

Lab 1 — Your first delegation, and reading the session

Open Lab 1

Q1

From the diff alone, how confident are you that the error list is complete and correct? What would you have to open to check it yourself?

Model answer. Not confident at all, and the diff gives no way to become confident: it shows a tidy table, and tidy tables look right. To check it, you'd open src/transfers/ach.ts, where TransferRejected and the RejectionCode type list every rejection the route can return, and the handler in src/api/server.ts for the route's own errors. You'd compare the two lists line by line.

A strong answer includes:

• Low confidence, stated plainly, with the reason: a diff shows output, not sources.
• The actual source of truth, RejectionCode in ach.ts.
• Recognition that checking it costs the reviewer the same reading the agent should have done.

Q2

Find one claim you can trace to a specific Read result, and one you can't. What would you add to a request, or to a file every session reads, so the second kind happens less?

Model answer. The field list traces to the Read of server.ts: TransferBody defines the five fields. An error code like insufficient_funds traces to nothing, because it came from what a transfers API usually returns. In the request: "Before you edit, list every file that defines an error this route returns, and read each one." In a file every session reads, a general rule rather than the answer: "Document from source. Every claim in documentation must come from a file you read in this session. Error codes are defined in src/transfers/ach.ts."

A strong answer includes:

• Both a traceable claim and an untraceable one from their own document.
• A request-level fix and a standing fix, and when each is appropriate.
• A standing fix that states how Larkspur works, not the answer to T7, since a file the scoreboard reads shouldn't contain a task's answer.

Q3

The response includes a toAccount field. Did your document say it's masked? Which invariant does that touch, and who reads docs/api.md besides Droid?

Model answer. It touches invariant 4: account numbers never appear unmasked in logs, errors, fixtures or commits. The route returns toAccount masked with maskAccount(), last four digits only. A document that shows a full account number in its example response invites someone to copy that example into a fixture, a log line or a support ticket. The readers are integrators on other teams, Sam, support staff, auditors and examiners, and future Droid sessions, which will treat the doc as a source.

A strong answer includes:

• Invariant 4 by name or number, and maskAccount().
• That documentation examples become fixtures and test data.
• That the doc is read by people and agents who never look at the code.

Q4

Ruth asks how to find the session that wrote a Droid-made change that caused an incident. Write the steps. Where does the chain break today?

Model answer. Start from the commit. Droid's co-author trailer (includeCoAuthoredByDroid, on by default) tells you an agent was involved. Then find the session on the machine that ran it with droid search and a distinctive term from the change, and open it with droid --resume <sessionId>. If cloudSessionSync is on (the default), it's also mirrored to Factory's web app. The chain breaks at the commit: nothing links a commit or pull request to a specific session id, and a local search only works on the laptop that ran it. Lab 3 adds the plan, and Lab 9 adds session_id to CI artifacts, but an automatic PR-to-session link is still missing at the end of the day.

A strong answer includes:

• Concrete commands: droid search, droid --resume.
• The co-author trailer as evidence of agent involvement, not of which session.
• The break named precisely: commit or PR to session id.
• Awareness that the answer depends on which machine ran the session.

Q5

The scoreboard calls droid exec --auto medium. Name one thing Medium allows that T7 doesn't need, and one task that would fail at --auto low. Why not give each task the lowest level it needs?

Model answer. Medium allows npm install and git commit, and T7 needs neither. It needs to read and edit one file. A task that runs the test suite needs Medium: in exec, tests and builds come in at Medium, so T2, whose whole point is running the overdraft test, would fail at --auto low the moment Droid ran npm test. Per-task levels are a bad idea for a scoreboard for two reasons. You'd be tuning the harness to the answer, since you only know the minimum level because you know the task. And the board measures how the harness behaves under one realistic authority, the one a pipeline would actually use. Change the level per task and a pass stops meaning the same thing across tasks.

A strong answer includes:

• A correct Medium-only action (npm install, git commit, git checkout, git pull, builds or tests).
• A task that needs tests or installs.
• The measurement argument: a constant authority makes results comparable.
• Bonus: a task that fails at Low fails for autonomy, not quality, which pollutes the signal.

Knowledge check

1. droid exec "add a changelog entry" with no flags. Answer: it stops at the edit, exits non-zero, no partial changes.
   • "Defaults to Low" tempts people who set a project default of Low and assume exec reads it as its level. Exec is read-only without --auto.
   • "Waits for you to approve" is the interactive habit. Exec never prompts, which is the point.
2. "Always allow" on the docs/api.md edit. Answer: the session's autonomy level rose to the level that prompt required.
   • "Only edits to docs/api.md" matches how many tools phrase "always allow" as a per-file exception. Droid's is a change of level.
   • "Nothing persists" assumes the option is a one-off. It lasts the rest of the session.
3. -r in droid and droid exec. Answer: resume in droid, reasoning effort in droid exec.
   • "Resume in both" tempts people who learned the interactive CLI first.
   • "Reasoning effort in both" tempts people who learned exec first. Each wrong option is right about half the time, which is exactly why scripts break.

Lab 2 — AGENTS.md: the contract Droid reads first

Open Lab 2

Q1

Find the line in Dana's draft that Droid acted on. Rewrite it so that a run following it literally would pass T1. How many words did you need?

Model answer. Droid acted on "Be careful with money-related code". It turned "careful" into a guard clause and tests. A literal-minded rewrite: "Money is integer cents. Percentages are basis points: multiplyBasisPoints(amount, basisPoints(p)). Round with roundHalfEven(); never Math.round, toFixed or float arithmetic on amounts." That's about 25 words. The count is the point: the draft's line was six words and said nothing a model could act on. The fix isn't more words of the same kind. It's naming the helpers.

A strong answer includes:

• The right line identified from the transcript ("careful" became input validation).
• A rewrite that names basisPoints() or multiplyBasisPoints() and roundHalfEven().
• A never-clause (Math.round, float math).
• The observation that specificity, not length, did the work.

Q2

Sam reads the layout section on his first morning. Which line saves him the most time? Which would he not understand without asking Theo, and should you fix that for Sam or leave it for Droid?

Model answer. The line that saves most time is "test/helpers/clock.ts controls time in tests", because it's the one fact that turns T2 from an afternoon into ten minutes. The protected-path markings are a close second. He wouldn't understand "Protected: needs a CHG ticket" without asking: what's a CHG ticket, where does it come from, who approves it? Fix it for both of them, in one short clause, for example "CHG tickets come from the change system; ask the payments lead". Whatever Sam would have to ask Theo, a headless Droid session can't ask anyone, so a gap for Sam is a gap for Droid too.

A strong answer includes:

• A specific line with a specific time-saving reason.
• A line that assumes insider knowledge.
• The argument that Sam and Droid have the same gaps, and headless Droid can't ask.
• Keeping the fix short: AGENTS.md is a contract, not a wiki. Link out for detail.

Q3

Theo would add a sixth invariant. What is it likely to be, and how would you write it so it names a function, a file or a command?

Model answer. A likely one is about time, since time is what broke the overdraft test: "Code that depends on the date takes now as a parameter, like overdraftFee(account, now). Never call new Date() inside a fee or limit calculation. Tests pin time with freezeClock() from test/helpers/clock.ts." Another good one: "Fee amounts and rates come from FEE_SCHEDULE in src/fees/schedule.ts. Never hard-code a fee."

A strong answer includes:

• An invariant grounded in the codebase's real failure modes, not a generic principle.
• A named function, file or command in the invariant.
• Something to do and something never to do.

Q4

"A clear question is a good outcome, not a failure." Will this make Imani's number look worse, better, or both? How would you explain it to her in one sentence?

Model answer. Both. On the scoreboard it looks better, because T4's check rewards a clear stop. In a raw count of "tasks completed", it looks worse in the short term, because some tasks end with a question instead of a change. One sentence: "A stop costs someone five minutes to answer, a wrong migration costs us an incident, so we'll report stops separately from completions and count a stop with a clear reason as work done correctly."

A strong answer includes:

• Both effects, with the metric each one moves.
• A proposal to report stops as their own number, not hide them in failures.
• A sentence in Imani's terms: cost and risk, not virtue.

Q5

List three places a Larkspur prompt could come from that aren't a person typing it at that moment. For each, who actually made the "human decision" Droid would be relying on?

Model answer. A scoreboard task file or a CI workflow's prompt: whoever wrote the file, possibly months ago, with no idea of today's context. A ticket description piped into droid exec from Jira: whoever filed the ticket, who may not know the fee disclosure exists. A tool result, such as a change ticket's notes from an MCP server (Lab 8): whoever typed the notes, or someone deliberately writing to an agent. In a Mission, the orchestrator's instructions to a worker: an agent, relaying a human's plan. In every case, nobody made a decision about this specific change at the moment it happened.

A strong answer includes:

• At least one automated source (task file, CI, pipeline) and one data source (ticket, tool result).
• That the "decision-maker" is absent, out of date, or not a person at all.
• The conclusion: the request outranking AGENTS.md is fine when a person is at the prompt, and a real problem when nobody is.

Q6

What could you add to AGENTS.md to push T2 from 2/3 to 3/3? Suppose it worked for ten runs. Would you tell Ruth the problem is solved? What would you tell her instead?

Model answer. You could add "If a test disagrees with roundHalfEven(), the test is wrong", or a nested src/money/AGENTS.md that repeats the rule where the edit happens. Ten clean runs still wouldn't justify "solved". If the true failure rate had fallen to one in twenty, you'd still see ten clean runs about 60% of the time. Tell Ruth: "The file reduced the rate. It can't make the failure impossible, because an instruction is an input the model weighs. The guarantee needs a mechanism that blocks the edit whatever the model decides, and that's the next thing we build."

A strong answer includes:

• A plausible wording change, and an honest statement that it might help.
• The statistics point: a run of passes doesn't prove a low-rate failure is gone.
• The distinction between reducing a rate and removing a possibility.
• A pointer to enforcement (hooks) as what Ruth actually needs.

Knowledge check

1. Starting in src/fees: which AGENTS.md files load? Answer: from src/fees/ up to the git root, checking each folder and its .factory/, .agents/ and .agent/, plus personal ones.
   • "Only src/fees/" matches tools that only read the current directory.
   • "Every AGENTS.md in the repository" tempts because nested files do exist and can be discovered. But they load when Droid reads files there, not at the start.
2. An AGENTS.md and a CLAUDE.md that disagree. Answer: keep one.
   • "Droid ignores CLAUDE.md" sounds right, because it's another vendor's filename. Droid reads it.
   • "Keep both" sounds like maximum compatibility. It gives the model two answers, and you don't choose which one it uses.
3. The 80,000-character limit. Answer: a cap, not a target.
   • "Truncates over 20,000" tempts because a specific number sounds authoritative. It's invented.
   • "Only the six template headings" tempts people who treat the template as a schema. It's a suggestion. Larkspur's file renames two of them.

Lab 3 — Spec mode: review the decision before the diff

Open Lab 3

Q1

How long did it take to find the rounding change in the playground diffs, and in the plan? If the plan said "normalize the rounding helper", would you still have caught it? What does that tell you about what to ask a plan to contain?

Model answer. Most people take thirty seconds to two minutes on the four diffs, and three seconds on the plan. "Normalize the rounding helper" is much harder: it sounds like housekeeping, and many people would approve it. So a plan has to state decisions in terms of behavior, not code: what changes for whom, with an example value. Ask for a "Decisions" section with each decision in one sentence, protected paths marked, and every behavior change listed with a before-and-after number, such as "a 202.5-cent fee becomes 203 instead of 202".

A strong answer includes:

• Honest timings, and the gap between them.
• That euphemism defeats a plan as easily as volume defeats a diff.
• A concrete format requirement: decisions first, protected paths marked, behavior changes with values.

Q2

Your rejection gave a reason. Where does that reason live now: in the session, in the plan, in the code? If Theo reviews the pull request next week, which will he see?

Model answer. It lives in the session (your message) and in plan 2, whose decision line cites disclosure section 4.2 and roundHalfEven. It reaches the code only if the test says so, for example a comment on the half-cent case. Next week Theo sees the diff and whatever is in the pull request. He'll see the saved spec only if someone copied it into the branch, as Step 4 does with notes/specs/, and he'll see the session only if someone gives him its id. So put the reason in the test and in the PR description, and link the spec.

A strong answer includes:

• All three locations, correctly distinguished.
• The observation that Theo sees the diff and the PR by default, and nothing else.
• Action: move the reason to where the reviewer looks.

Q3

Once .factory/ exists, specs may land in .factory/docs inside the repo, where later sessions can read them. Is a saved plan context, a record, or both? What could go wrong if a plan you approved and later overturned ended up there?

Model answer. Both. It's a record for Ruth's traceability chain, and it's a file in the repository that a later session can find and read, even though the docs don't say specs are loaded automatically. An overturned plan is dangerous in both roles. As context, a later session searching for "rounding" could find a plan that says "align roundHalfEven with Math.round" and treat it as precedent. As a record, an auditor could read a decision that was reversed and not know it. Mark each saved spec's status (approved, superseded, rejected) in the file, or keep only the final approved plan where sessions can see it.

A strong answer includes:

• Both roles, with the tension between them.
• The specific risk: an agent reading an overturned plan as an instruction or precedent.
• A mitigation: status in the file, or a clear location policy.
• Awareness that the docs disagree on the save location, so check before you rely on it.

Q4

"Decisions first, protected paths marked" is a format rule you want in every plan. Type it every time, add it to AGENTS.md, or package it? Which, and why?

Model answer. Add one line to AGENTS.md now: "Plans start with a Decisions section: each decision in one sentence, marking any that touch src/money/, migrations/ or legacy/." It's short, it applies to every plan in this repository, and typing it every time depends on memory, which is the thing the harness is replacing. If the format grows into a template with examples, move it into a skill, so its length only costs anything when a plan is being written.

A strong answer includes:

• A choice with a reason, not "it depends".
• The cost model: AGENTS.md is paid on every session, a skill only when selected.
• Rejection of "type it every time" as relying on memory.

Q5

From Imani's side the T4 session produced nothing. From Ruth's side it produced the only acceptable outcome. Which of them does the plan's first line serve, and can it serve both?

Model answer. It serves both. For Ruth, it's the control working: the missing ticket surfaced before any file existed. For Imani, it turns a hidden blocker into a named, cheap request: "this needs a CHG ticket id". Nothing was written, so nothing needs reverting or reviewing, and the fastest path to the column is now obvious. The first line is the cheapest possible failure. Lab 8 makes it cheaper still, by letting Droid look the ticket up itself.

A strong answer includes:

• Both perspectives, taken seriously.
• The cost argument: a stop in a plan costs less than a stop after code exists.
• That the stop names exactly what would unblock it.

Q6

Dana wants interactionMode: spec in project settings for everyone. Theo says people will rubber-stamp plans. Who's right for Larkspur, and what evidence from this lab supports it?

Model answer. Theo is more right about the project default. Step 5 showed a plan can bury a decision as easily as a diff can, and Lab 4's Q1 shows people approve prompts on reflex. A plan nobody reads is a rubber stamp. And the scoreboard didn't move, because headless there's no reader at all. Dana is right that plans catch real problems: plan 1 surfaced the incident in one sentence, and the T4 plan stopped before a file existed. For Larkspur: make Spec mode a personal default for people working on money-adjacent code, require the Decisions-first format, and never treat plan approval as the control. The controls are the rules and hooks underneath.

A strong answer includes:

• Evidence from the lab on both sides: plan 1 and T4 for Dana, the buried plan and the unmoved score for Theo.
• The point that a gate only works if someone reads it.
• A decision for Larkspur, not just a balance of views.

Knowledge check

1. Shift+Tab twice from Normal. Answer: Normal mode.
   • "Mission mode" tempts because there are three modes, so people expect a three-step cycle. Mission mode starts with /missions.
   • "Spec mode at Autonomy Low" mixes up the mode key and the autonomy key (Ctrl+L).
2. Six good steps and one bad one. Answer: keep iterating, with a reason.
   • "Approve at High, then revert" tempts because git makes reverting feel free. It's the most expensive option, and nothing guarantees you'll notice the step.
   • "Approve with manual approvals and reject that edit" tempts because it works. But the plan on record still says the wrong thing, and a saved spec records a decision you didn't make.
3. A scoreboard on --use-spec. Answer: each run writes a plan and stops awaiting approval, so nothing is built and T2 fails.
   • "Exec doesn't support Spec mode" tempts because Spec mode feels interactive. droid exec --use-spec exists.
   • "Spec mode is disabled at Medium" confuses interaction mode with autonomy level. They're separate settings.
   • If someone expected the run to build anyway, that's the old reading of this lab ("headless, a spec is a record, not a gate"). Tested on droid 0.228.0, it's wrong: headless --use-spec stops at the approval boundary and edits nothing. The docs don't cover it, so if a learner's version behaves differently, have them show you the session.

Lab 4 — Autonomy levels and permission rules

Open Lab 4

Q1

Of the approvals you gave at Off, which did you read, and which did you approve on reflex? What does that tell you about an approval prompt as a control, compared with a rule?

Model answer. Most people read the first edit prompt and the commit, and approve npm test on reflex by the second time they see it. A prompt is only a control while it's read, and people stop reading prompts that carry no information. So prompts wear out, and they wear out fastest on the actions you care about least, which trains you to approve the ones you care about most. A rule decides the same way every time, can be tested, and can be read by Ruth. The best use of rules is to remove the low-information prompts, so the prompts that remain get read.

A strong answer includes:

• An honest count of reflex approvals.
• Approval fatigue named as the failure mode.
• Rules as consistent, testable and inspectable.
• Rules reducing prompts so the remaining ones matter.

Q2

Name two things Medium runs without asking that you'd want Theo to see first. Then name one thing Low already allows that surprises you.

Model answer. At Medium, npm install runs unasked, and a new dependency at a bank is a supply-chain decision. git commit does too, and a commit starts the audit trail. (git checkout and git pull are also Medium.) The surprise at Low is file edits: Low lets Droid Edit and Create any file without asking, including src/money/rounding.ts. Permission rules don't cover edits at all, because they match shell commands. Nothing stops that edit until Lab 5.

A strong answer includes:

• Two correct Medium actions, with a Larkspur-specific reason each.
• The Low surprise: edits to any path, including protected ones.
• The link to Lab 5: rules govern commands, not edits.

Q3

project/test allows every npm run script. List every script it lets run without a prompt. Which wouldn't you want Droid to start unasked? (One costs money. One never exits.)

Model answer. From package.json: test, test:watch, typecheck, lint, scoreboard, db:migrate, db:migrate:check and mcp:change-control. db:migrate is the exception: two rules match it, and ask beats allow. Of the rest, scoreboard costs money (24 headless sessions per full run), and test:watch never exits, so a session that starts it hangs. mcp:change-control also runs until it's stopped, since it's a server waiting for input. And lint covers npm run lint -- --fix, which rewrites files under src/money/. The fix is narrower allows (["npm", "test"], ["npm", "run", ["typecheck", "lint"]]) plus an ask for scoreboard, which is Extension 1.

A strong answer includes:

• The full list, read from the file rather than guessed.
• scoreboard (cost) and test:watch (never exits) identified.
• db:migrate noted as caught by the ask rule.
• A narrower rule set as the fix.

Q4

The deny typo made no noise in the session, only in droid rules check. Where should that check run so nobody has to remember? What should happen when it exits 1?

Model answer. In CI, on every pull request that touches .factory/settings.json, and ideally as a pre-commit hook too so the author finds out first. When it exits 1, the build fails, the pull request can't merge, and the job prints which rule was invalid or which test example failed. Add a second step that runs --command previews for a short list of known commands (a force push, a database shell, a migration) with --json and fails if any decision differs from what you expect, because exit 0 doesn't mean the decision was right.

A strong answer includes:

• An automatic place: CI and/or pre-commit.
• Exit 1 failing the build, not a warning.
• Golden-command previews that check decisions, since exit 0 says nothing about them.

Q5

Ruth asks: "Can the agent push to main?" Answer in three sentences: name the rule, name the evidence she can check, and name one way a push reaches the autonomy level instead of the rule.

Model answer. "Not on its own: project/git-push in .factory/settings.json is an ask rule, so every push needs a person to approve it, and in a headless run it fails, because nobody can. You can check that yourself with droid rules check --command 'git push origin main', which previews ask, and the rule's own tests run every time the rules load. A push spelled differently, such as /usr/bin/git push or git -C . push, matches no rule and falls through to the autonomy level, which is why Larkspur's default is Low, and why branch protection on main and the sandbox's network restrictions are the layers behind it."

A strong answer includes:

• The rule by id, and its decision.
• Evidence Ruth can run herself.
• A concrete spelling that bypasses the prefix.
• What catches the bypass: the Low default, branch protection, or the sandbox.

Knowledge check

1. CI runs T4 at --auto high and hits the db:migrate ask rule. Answer: the run stops, non-zero, no partial changes.
   • "It runs, because High permits migrations" tempts because the autonomy table literally lists migrations under High. But an ask rule requires approval even at High.
   • "It waits until it times out" is the interactive habit. Exec never waits for approval.
2. Sam's personal allow versus the project ask. Answer: it asks.
   • "User settings are personal and win" matches how dotfiles usually work. Here the two rules have different ids, so both load, and the strictest matching decision wins.
   • "Both are skipped because they conflict" sounds like safe conflict handling. Rules are only skipped when they're invalid.
3. commandDenylist maps to which decision? Answer: ask.
   • "block" tempts because "deny" sounds final. The whole reason the lab asks is that it isn't.
   • "Ignored because deprecated" tempts because deprecated often means removed. The legacy lists are deprecated and still respected.

Lab 5 — Hooks: invariants that cannot be talked past

Open Lab 5

Q1

Droid quoted the rule back before breaking it. What does it tell you that the reasoning sounds careful? Why is "this is not really a behavior change" exactly the argument a tired human reviewer would also accept?

Model answer. It tells you nothing about whether the change is right. Fluent, careful-sounding reasoning is what a capable model produces whether it's right or wrong. "Not really a behavior change" works because it reclassifies the change so the rule stops applying, and checking the reclassification needs exactly the knowledge the reviewer is short of time to use. Theo accepted a version of it in March. The lesson is that a control must not depend on anyone's classification of the change, the model's or the reviewer's. It has to look at what the change touches.

A strong answer includes:

• That tone isn't evidence.
• The mechanism: reclassification that makes the rule not apply.
• The parallel to the human reviewer in March.
• The design conclusion: controls look at the path or the outcome, not the argument.

Q2

The matcher is Create|Edit|ApplyPatch, and matchers are case-sensitive. What happens, with no error anywhere, if you write edit? How would you find out?

Model answer. The hook never runs for edits, because no tool is called edit. Edits to src/money/ go through, and nothing reports an error: the configuration is valid, it just matches nothing. The piped test won't catch it either, because piping JSON into the script tests the script, not the matcher. You find out by trying to break the rule in a real session: ask for an edit to a protected file and watch it succeed. /hooks shows the group as configured, so it looks fine there. droid --debug shows which hooks actually execute.

A strong answer includes:

• The silent failure: valid config, zero matches.
• That the piped test can't catch a matcher mistake.
• A live break-it test as the only reliable check.
• Mention of droid --debug.

Q3

What did the Step 2 hook guard: the outcome Ruth cares about, or one path to it? List two more tools or commands that reach the same outcome.

Model answer. One path: the file tools. The outcome Ruth cares about is "no protected file changed without a human decision", and the hook guarded only the three tools that are the most obvious way to change one. Other paths: Execute with sed -i, node -e with fs.writeFileSync, cat > or tee, cp or mv over the file, git checkout other-branch -- src/money/rounding.ts, git apply with a patch, or an allowed command with side effects such as npm run lint -- --fix. An MCP tool that writes files would also get past it, since its name isn't in the matcher.

A strong answer includes:

• "One path", said plainly.
• At least two correct alternatives, ideally one that doesn't mention the path at all.
• The observation that you can't enumerate every path, which motivates the Stop hook.

Q4

Ruth asks whether the Execute guard is "complete". Write the two-sentence answer, including what it misses and what catches that.

Model answer. "No: it pattern-matches command text, so a command that builds the protected path at run time, like a node -e script that joins the path from parts, walks past it. The Stop hook catches that by asking git which protected files changed before the session can finish, and the CI check on the pull request catches anything that was committed."

A strong answer includes:

• "No", not "mostly".
• A concrete bypass.
• The outcome check and the CI backstop.

Q5

Three options for T3: loosen the regex, add an exception for limits.ts, or teach Droid the right helper. Which, and why is the answer different for Ruth than for Dana?

Model answer. Teach the helper. The false positive isn't in the hook: RETAIL_DAILY_LIMIT * 1.5 really is float arithmetic on money. The failure is that Droid doesn't know how to write it Larkspur's way. Loosening the regex weakens the control everywhere to fix one number, and an exception for limits.ts makes the one file that already contains float code (isNearDailyLimit) a sanctioned place for more. For Ruth, the test is whether a control got weaker: options one and two both weaken it. For Dana, the test is reuse across thirty repositories: per-file exceptions don't travel, and a loosened regex in a shared plugin weakens every repository at once. A skill fixes the cause, keeps the control, and travels.

A strong answer includes:

• The observation that the hook was right and the knowledge was missing.
• What each of the first two options costs.
• Ruth's criterion (control strength) and Dana's (reuse at scale), stated separately.
• A preference for fixing the cause over tuning the control.

Knowledge check

1. A PreToolUse hook exits 1. Answer: the edit runs; exit 1 is a non-blocking error.
   • "The edit is blocked and Droid sees the message" follows the Unix habit that non-zero means failure. Only exit 2 blocks.
   • "Droid asks you to approve" assumes hooks feed the approval system. A PreToolUse hook can ask through its JSON output, but not through an exit code of 1.
2. Which hook catches npm run lint -- --fix touching src/money/? Answer: Stop, asking git which protected files changed.
   • "PreToolUse on file tools" tempts because the linter writes files. But it writes them from a shell command, not a file tool.
   • "PreToolUse on Execute with a commandRegex" tempts because lint is an Execute call. Its text doesn't mention src/money, so the regex doesn't match.
3. What does money-lint (PostToolUse, exit 2) prevent? Answer: nothing directly. The edit already happened, and Droid gets feedback.
   • "The edit, because exit 2 blocks" generalizes PreToolUse behavior to every event.
   • "The session from finishing" is the Stop hook's exit-2 behavior, borrowed.

Lab 6 — Skills and slash commands

Open Lab 6

Q1

Now that you've seen the failing run, which of the first two options (loosen the regex, or an exception for limits.ts) would have fixed this failure? What would each have cost Ruth?

Model answer. Either would have made T3 pass. With no feedback on * 1.5, Droid keeps RETAIL_DAILY_LIMIT * 1.5, which gives the right number, and T3's check accepts it. Neither fixes the actual failure, which was that Droid didn't know basisPoints() takes a percentage as written, so 150% is basisPoints(150). That misunderstanding would resurface in the next file that needs a percentage. The cost to Ruth: loosening the regex reopens float arithmetic on money across src/, and an exception makes limits.ts a place where float money is allowed. Both trade a weaker control for a higher score.

A strong answer includes:

• That both "fix" the score and neither fixes the knowledge gap.
• The specific misunderstanding: 1.5 versus 150.
• The cost to the control, which the score doesn't show.

Q2

Write a request that should trigger money-change but shares no words with its description. Does Droid select the skill? What would you add to the description, and what would you be careful not to add?

Model answer. For example: "Business customers should be able to move one and a half times what retail customers can each day." It often still selects the skill, because selection is done by the model matching meaning, not keywords. When it doesn't, add the words tasks use for the same idea: "multiplier", "times", "ratio", "proportion", "per day". Be careful not to make the description so broad it matches every task ("any change under src/"), which costs context on unrelated work and makes selection noisy. Don't put procedure in the description either. The body is where the procedure goes.

A strong answer includes:

• An actual test, with the result.
• That selection is semantic, but vocabulary still helps.
• The over-selection risk.
• Description for selection, body for procedure.

Q3

The worked example in the skill is a specific date and a specific number. AGENTS.md avoids that level of detail. Why is it right in a skill and wrong in AGENTS.md?

Model answer. A skill's body loads only when the task needs it, so detail costs nothing on the tasks that don't. And detail is what a procedure needs: one worked half-cent example teaches the rule better than a paragraph. AGENTS.md is loaded in every session, so every line competes with the task and dilutes the invariants. There's a measurement reason too. The example teaches the rule (half-even at an exact half) with a date that isn't the answer to T2, whose check tests every day of two months. In AGENTS.md, a task-specific date starts to look like the answer written into the file the scoreboard reads.

A strong answer includes:

• Progressive disclosure: cost only when selected.
• Worked examples as good procedure.
• AGENTS.md as always-loaded, so it should stay general and short.
• Awareness of the "don't write the task's answer" rule.

Q4

Ruth reads a SKILL.md with allowed-tools: [Read] and signs off on it as read-only. What would you tell her, and what would you point her at instead?

Model answer. "That field is metadata. Factory's docs say it doesn't add tools or sandbox the skill, and in Step 5 a skill that listed only read tools led straight to an Edit. If a restriction has to hold, it lives somewhere that enforces it: a custom droid's tools field, which Droid enforces when it loads the droid, or a hook, or a permission rule." Then offer to show her the Step 5 transcript, and suggest she ask for a demonstration of any control she's asked to sign.

A strong answer includes:

• "It's metadata, not enforced", plainly.
• The evidence from Step 5.
• The enforcing alternatives: droid tools, hooks, rules.
• A habit for Ruth: evidence by attempted break, not by reading config.

Q5

ship-check.sh repeats a check the Stop hook already makes. Is that duplication a mistake? Who is each one for?

Model answer. It isn't a mistake. They have different audiences, moments and blind spots. The Stop hook is for Droid, runs automatically at the end of every session, and compares the working tree with HEAD, so it misses anything already committed. /ship-check is for a person about to open a pull request, runs when someone runs it, and compares against a base ref, so it catches committed changes too. The CI job in Lab 9 is the third copy, and the only one that runs without anyone choosing to. Overlapping checks with different blind spots is defense in depth.

A strong answer includes:

• The audience of each: Droid versus a person.
• The different comparison points (working tree versus base ref).
• That one is automatic and one is voluntary.
• The principle: overlap is fine when the blind spots differ.

Knowledge check

1. Why was the edit allowed despite allowed-tools? Answer: allowed-tools is metadata.
   • "Edit is always allowed at Low, and skills can't change that" is half true, which is what makes it tempting. It's right about Low, and wrong about why: the field never restricted anything.
   • "The skill wasn't really invoked" tempts anyone who didn't check the transcript. The Skill tool call is right there.
2. Twelve skills and a docs task: what loads? Answer: only names and descriptions.
   • "Every SKILL.md in full" is how AGENTS.md works, so people assume skills are the same.
   • "Nothing unless you type /skill-name" is the slash-command mental model. Skills are also selected by the model.
3. $1 in a Markdown command. Answer: not supported, so your text doesn't land there.
   • "$1 becomes the first word" is the shell convention, and it's what executable commands do.
   • "The command fails to register" expects strict validation. It registers. It just doesn't substitute.

Lab 7 — Custom droids: a reviewer with fresh eyes

Open Lab 7

Q1

Theo's review of the March PR had an accurate description written by the author. What exactly does a reviewer need not to have seen, and what does it still need to see?

Model answer. It must not have seen the author's reasoning: the task framing ("make the test pass"), the justification ("Ops asked for the account number"), and the author's summary of what the change is ("a flaky-test fix"). Each of those gives the reviewer a conclusion before it has looked. It still needs the complete diff, the code around it, and the rules to check against: the five invariants and the protected paths. Review what the change does, from the diff, and treat anything that explains why as a decision for a human to confirm.

A strong answer includes:

• The distinction between "what" (diff, code, rules) and "why" (justification, framing, summary).
• That an accurate description can still mislead, because it frames the change.
• That the reviewer still needs the complete diff, not a summary of it.

Q2

The main session writes the reviewer's prompt. Name two ways it could leak the author's reasoning into that prompt. Which one does the system prompt defend against, and which one can it not?

Model answer. First, by including a justification: "Review this; Ops approved logging the full number." Second, by choosing what to pass: summarizing the diff instead of pasting it, trimming hunks it thinks are irrelevant, or labeling the change "a small tidy-up". The system prompt defends against the first: it tells the reviewer to ignore justifications and list them as decisions for a human. It can't defend against the second. A read-only reviewer can't run git diff to see what it wasn't given, so a missing hunk is invisible to it.

A strong answer includes:

• An explicit-justification leak and a selection or framing leak.
• Which one the system prompt covers, and why.
• That omission is undetectable without Execute, which is the trade-off of tools: read-only.
• Mitigation: ask for "the output of git diff --staged and nothing else", and check the Task call with Ctrl+O.

Q3

Write the one sentence you'd put in Larkspur's harness README so the next engineer doesn't confuse a skill's allowed-tools with a droid's tools.

Model answer. "A skill's allowed-tools is a note about intent and restricts nothing; a custom droid's tools field decides which tools exist in that droid and is enforced, so any restriction that must hold goes in a droid or a hook, never in a SKILL.md."

A strong answer includes:

• Both fields named, with "not enforced" and "enforced" said plainly.
• Where to put a restriction that must hold.

Q4

/review with custom instructions could be given the same checklist as money-reviewer. What would you still lose compared with the custom droid?

Model answer. The checklist would be typed per run, not versioned in the repository, so it drifts between people and can't be reviewed like code. The main session couldn't call it on its own through the Task tool, and neither could a Mission's workers. You'd lose the enforced read-only toolset you chose and can show Ruth, the fixed output format Theo reads first ("Decisions for a human"), and the ability to measure one stable reviewer with the mini-eval. /review is still worth running. It's a broad second look, and the lab recommends using both.

A strong answer includes:

• Versioning and repeatability.
• Invocation by the main session and by Missions.
• A stable definition you can evaluate and re-evaluate.
• Not dismissing /review: the two do different jobs.

Q5

With these numbers, would you let money-reviewer approve a pull request on its own? If not, what is it for? Write the role in one sentence Theo would sign.

Model answer. No. It missed diff D, the March change inside a rename, one time in three, and it raised a false P1 on the correct diff E one time in three. A reviewer that misses the incident a third of the time lowers the odds. It isn't a gate you can merge on. Its role: "money-reviewer reads every money change before Theo does, blocks on invariants, and lists every decision a human has to make; it can stop a merge, and it can never approve one."

A strong answer includes:

• A "no" grounded in the numbers, especially diff D.
• The false-positive cost as well as the miss rate.
• A role that uses its strengths: blocking and making decisions visible.
• An asymmetric authority: it can block, it can't approve.

Knowledge check

1. Asked to fix its P0, the read-only reviewer... Answer: reports it, and can't edit.
   • "Fixes it, because the main session is at Low" conflates autonomy with the toolset. Autonomy decides what runs without asking, among tools that exist. tools: read-only decides which exist.
   • "Asks you to approve the edit" is the approval-prompt habit. There's no edit tool to ask about.
2. Why did the subagent catch diff B? Answer: a clean context, without the author's reason.
   • "A stronger model" is the default explanation for any better result. Both used model: inherit.
   • "Subagents ignore AGENTS.md" tempts because the reviewer didn't argue from AGENTS.md. The docs don't say that, and it wouldn't explain a catch: the reviewer's checklist is in its own system prompt.
3. The reviewer wants to ask you something. Answer: say so in its result.
   • "Use AskUser" tempts because the main session can. Subagents can't.
   • "Start a second subagent" tempts because delegation feels natural. Subagents can't spawn subagents.

Lab 8 — MCP: facts from your own systems

Open Lab 8

Q1

The id Droid used was real, in the repository, and approved. Why was it still the wrong answer? What would a reviewer have to do to catch it, and how often would Theo do that on a Friday afternoon?

Model answer. An approval is for a specific change, not a folder or a team. CHG-19344 approved a change to the fees table, and it closed in January. Borrowing it for a transfers memo column is citing someone else's permission. To catch it, a reviewer would have to look the ticket up in the change system and compare its scope and dates with the change in front of them. On a Friday afternoon, Theo sees a well-formed ticket id and moves on. That's why the lookup has to be done by code, every time.

A strong answer includes:

• Scope and time: wrong change, closed ticket.
• The reviewer's work: a lookup and a comparison.
• An honest "rarely" for Theo.
• The conclusion: the check must be mechanical.

Q2

Read the description of get_change_ticket. It tells Droid that a protected change needs an approved, unexpired ticket whose scope covers the path. Who is that sentence for, and is it a control?

Model answer. It's for the model. A tool description is a prompt that shapes when and how Droid calls the tool and what it concludes from the result. It isn't a control: the model weighs it, the same way it weighs AGENTS.md, and a ticket's notes can argue against it (Step 5). The control is the hook, which checks status, expires and scope in code and never reads the description or the notes.

A strong answer includes:

• "For the model", with the parallel to a skill's description.
• "Not a control", with the reason.
• Where the control actually is.

Q3

The hook allows the write if any cited CHG id is approved and in scope. Write a migration header that gets a pending ticket's change past it. Bug to fix, or a limit you'd accept with the Stop hook and CI behind it? Defend it to Ruth.

Model answer. For example:

migrations/0003_add_fee_waiver_reason.sql (header)

-- Change ticket: CHG-20471
-- Related: CHG-20417

CHG-20417 is approved and scoped to migrations/, so the hook lets the write through, although the change belongs to pending CHG-20471. Tighten the cheap part: parse only the Change ticket: header line, and require every id on it to be approved. But say plainly that this doesn't close the gap. A session can cite CHG-20417 alone for an unrelated change, because the ticket's scope is a path prefix, and no hook can check that a migration's content matches what a ticket approved. So accept that limit, with the layers behind it: the Stop hook makes the session name the ticket and logs the change, and the Lab 9 CI check needs a human to approve every pull request that touches migrations/. To Ruth: "The hook proves an approved ticket exists for this path. It can't prove the ticket approves this change. A person checks that, at the pull request, every time."

A strong answer includes:

• A working header.
• A cheap tightening, honestly labeled as partial.
• The deeper limit: path-level scope can't verify intent.
• Named layers behind it, and a sentence Ruth can hold them to.

Q4

The ticket store is tickets.json, which Droid can edit. What stops a session setting CHG-20471 to approved and then creating the migration? What should stop it, and what's different about the real change system?

Model answer. In the lab as written, nothing stops it in the session. tools/change-control-mcp/ isn't on the protected list, so the file guard, the shell guard and the Stop hook all ignore an edit to tickets.json, and the hook reads the edited file when the migration is written. The T4 check fails the run afterwards, because it fails any run that edits tickets.json, but that's measurement, not prevention. What should stop it: the source of truth must be out of the agent's reach. Here, add tools/change-control-mcp/ to the protected list (Extension 2). In Larkspur's real system, tickets live in a separate service with its own authentication, the MCP server only has read tools, and the hook calls the service's API with its own read-only credential. No session can approve its own ticket.

A strong answer includes:

• "Nothing, in-session", with the reason.
• That a check after the fact measures but doesn't prevent.
• The principle: the approver's record must be out of the requester's reach.
• What changes with a real system: separate service, separate credentials, read-only access for agents.

Q5

Imani wants "24/24" on a board slide. Write the one-sentence caveat that must go under it, and name one thing the scoreboard doesn't measure that the board should care about.

Model answer. "24 of 24 means our harness now passes the eight representative tasks we chose, three runs each, measured on a fixed date, and it's a floor, not a guarantee; we're adding harder tasks because a perfect score can no longer show us a regression." The board should care about attempts on protected paths and actions stopped, which show whether the agent is getting safer or the harness is doing more catching. Cost per passing task and review time are good answers too.

A strong answer includes:

• Scope: eight tasks, three runs, a date.
• "Not a guarantee", and why a perfect score is weak evidence.
• A named unmeasured quantity with a reason a board would care.

Knowledge check

1. droid mcp add, then commit: what does Dana get? Answer: nothing, because it wrote to the user file.
   • "The server, because it writes the project file" matches how npm install writes to the project. droid mcp add always writes to ~/.factory/mcp.json.
   • "A prompt to approve the server on first use" matches how some tools ask you to trust a project's servers. There's nothing in the repository to trust.
2. The ticket notes fooled the model. Why no migration? Answer: the hook compares the status field in code.
   • "AGENTS.md says migrations need a human decision" tempts because it's true. But AGENTS.md is a request, and the model had already been persuaded.
   • "The server refused a pending ticket" tempts because a server could filter. This one returns tickets faithfully. The server gives facts, not permission.
3. An approved but expired ticket. Answer: blocked. The hook checks expiry.
   • "Allowed, because the status is approved" tempts because the raw JSON says "approved". The expiry rule lives in code, in two places.
   • "Asks the user" borrows ask from permission rules. Hooks here block or allow.

Lab 9 — Droid in CI with droid exec

Open Lab 9

Q1

The run says "is_error": false and exits 0. What does that tell you about whether T7 passed? What does, and why is it deliberately a separate program from Droid?

Model answer. It tells you the run completed without a runtime or permission failure, and Droid believes it did the job. It says nothing about whether the documentation is right. The check does that: scoreboard/checks/T7.mjs, run against the worktree afterwards. It's a separate program because a run can't grade itself. The agent's self-report is the same kind of evidence as the March summary: accurate about what it did, silent about what it missed. A deterministic check can't be persuaded, and it judges every run the same way.

A strong answer includes:

• Exit 0 means completed, not correct.
• The check as the source of pass or fail.
• Independence from the agent as the design reason.
• The link to March: a true summary of a wrong change.

Q2

Dana suggests a CI-only settings file (via --settings) that allows git push for the bot. What would Ruth need to believe for that to be acceptable, and what would you lose?

Model answer. First, a mechanics point: an extra allow rule wouldn't do it, because when rules match, ask beats allow. The CI file would have to redefine or disable the project/git-push id, and the docs don't say how a --settings file ranks against the project's rules, so you'd have to test it. For Ruth to accept it, she'd need to believe that the bot can only push to its own branches, that branch protection stops any push to main, that every pull request still needs a human approval and the protected-paths check, and that the CI settings file is itself protected and reviewed. You'd lose the simple invariant, "a human pushes, every time", and the property that CI runs with exactly the controls developers run with. Every future audit would have to read two settings files instead of one.

A strong answer includes:

• The precedence point: allow doesn't beat ask.
• Honesty that the ranking of a --settings file isn't documented, so test it.
• Concrete conditions (branch protection, bot-only branches, human PR approval).
• The loss: a simple invariant and one set of controls everywhere.

Q3

Ruth asks: "Which of the three layers could you delete?" Answer for each in one sentence. Which would you keep if you could keep only one? Is it the one Dana would keep?

Model answer.

• PreToolUse guards: you could delete them and still catch changes later, but you'd lose prevention: protected files would change and then be reverted, which costs time and leaves more to review.
• Stop hook: you could delete it and CI would still catch committed changes, but uncommitted collateral changes, like the lint fix, would sit in the working tree until a human committed them by accident.
• CI check: you can't delete it, because it's the only layer that sees commits, human edits, other teams' agents and machines where hooks are off.

If only one survives, it's the CI check, because it's the only one independent of the machine that made the change. Dana might keep the PreToolUse guards, because they keep sessions productive: Droid is told immediately and fixes course. Ruth's criterion is coverage, and Dana's is the working day, so both are right about different things. That's why there are three.

A strong answer includes:

• One sentence per layer, each naming what's lost.
• CI kept, with "independent of the machine" as the reason.
• A different, reasoned answer for Dana.

Q4

The review approved the pull request and every check is green. What does that mean for Theo's job? Write the sentence for the team's review policy so "Droid approved it" is never the reason a change merged.

Model answer. Theo's job moves from finding problems to making decisions. The checks and the review have already done the reading. He reads what the change decides, confirms the decisions a human must own, and approves. He's still required. Policy sentence: "Droid's review and Droid's approval are inputs to a human review, never a substitute: a pull request merges only when a named code owner has approved it, and that approval is the recorded decision." Back it with branch protection that requires code-owner review, which the Droid GitHub App's approval doesn't satisfy.

A strong answer includes:

• Theo's changed role: decisions, not hunting.
• A policy sentence that names the human approval as the decision.
• A mechanism (CODEOWNERS and branch protection), not just a sentence.

Q5

A full board on every pull request costs three times as much as --runs 1. Make the case to whoever owns the bill for the schedule you'd actually run.

Model answer. On every ready pull request: the protected-paths check (free), the automatic review (one run), and the scoreboard at --runs 1, which is eight headless runs. It catches gross regressions on the change that caused them. Drafts skip it. Nightly on main: the full board at three runs, 24 runs. That's the number we quote, because one run per task can't tell two in three from three in three. And a full board on any pull request that changes AGENTS.md or .factory/, because harness changes are what the board exists to measure. At ten pull requests a day, that's about 104 runs a day instead of 240 for a full board on every pull request, and the number we report is measured the same way every night.

A strong answer includes:

• A tiered schedule with a reason for each tier.
• The variance argument for the nightly three-run board.
• A trigger for harness changes.
• Arithmetic the bill owner can check.

Knowledge check

1. CI at --auto high hits the git push ask rule. Answer: the run fails.
   • "It pushes, because High allows remote writes" tempts because the exec autonomy table does list remote writes under High. An ask still needs a human.
   • "It pauses until someone approves in the web app" imagines a remote approval flow. A one-shot run can't collect an approval.
2. --skip-permissions-unsafe in a workflow. Answer: ask prompts are skipped, and block rules still apply.
   • "Nothing changes: rules always win" tempts because rules feel like hard config. ask is not a hard prohibition.
   • "Every rule is ignored, including block" is what the flag's name suggests. block has no approval path to skip.
3. A protected write the shell guard can't read, committed, then Droid finishes. Answer: only CI catches it.
   • "The PreToolUse guards" tempts because they block protected writes. This one's path was built at run time.
   • "The Stop hook, because it checks outcomes" is the most tempting, because Lab 5 made the Stop hook the hero. It checks the working tree against HEAD, and a committed change is part of HEAD.

Lab 10 — Capstone: a Mission to retire float money

Open Lab 10

Q1

Of the three Level 4 criteria Larkspur fails, which makes a Mission riskiest in this codebase, and why? Would your answer change for src/ledger/?

Model answer. Flaky-test detection. A Mission's validation workers decide whether each milestone passed by running tests. A flaky test makes validation lie in both directions, passing a bad milestone or failing a good one, and at Larkspur a flaky test is literally how the March incident began. Fast CI feedback and deployment frequency matter less for a three-milestone Mission. For src/ledger/ the answer mostly holds, but the stakes move to test strength: the balance invariant has to be checked by a property test the validators run, and src/ledger/ isn't a protected path, so no hook stands behind a validation that misses something. The reviewer and the tests are the only guard there.

A strong answer includes:

• Flaky-test detection chosen, with the validation mechanism as the reason.
• The link to T2 and March.
• For ledger: the property test for balance, and the absence of a hook.

Q2

Which milestone 2 validation criterion would have caught the March incident inside this Mission? Which would a validation worker find hardest to check mechanically?

Model answer. "Nothing under protected paths changed" catches it directly, since the March change was to src/money/rounding.ts, and the hook blocks the edit before validation even runs. The characterization tests catch it too: Milestone 1 pins current behavior with half-cent inputs, so a switch to Math.round fails them unless Milestone 1 predicted it, and it didn't. Hardest to check mechanically: "expected value changed only where Milestone 1 predicted it, with the reason in the test", because it means matching test diffs against a prose inventory. The grep's exception clause ("or only lines the inventory explains are not money") is a close second, and so is "money-reviewer reports no P0 or P1", which is itself a model judgment that varies between runs.

A strong answer includes:

• The protected-path criterion, and the characterization tests.
• A criterion that needs judgment, with the reason it can't be a simple check.

Q3

AGENTS.md says never change an expected value to make a test pass, and you just told a worker to change a dozen. What makes this different? Write the rule for Sam.

Model answer. The rule was never "expected values don't change". It's "a test doesn't change to agree with the code". Here the business behavior changed on purpose. Milestone 1 predicted each difference, a named person decided it, and the new values come from the new rule. For Sam: "Never change an expected value to make a test pass. Change it only when the rule it tests has changed, a named person decided that, and the test says who, when and why next to the new value."

A strong answer includes:

• The difference between a behavior change that was decided and a test bent to agree with the code.
• Who decided, recorded where the value is.
• A rule Sam can apply without asking.

Q4

The orchestrator followed your shortcut instruction without argument, and only the hooks stopped it. A problem with the orchestrator, or the division of labor you designed?

Model answer. Mostly the division of labor working. The orchestrator follows the human in charge, and the controls are in hooks precisely so that nobody has to count on anyone's judgment, including yours. But it cost something: blocked attempts and extra runs. So judgment still has value. You'd want it to say: "That needs src/money/amount.ts, which AGENTS.md marks protected, and the hooks will block it. Do you want me to stop at the list of behavior changes for Theo instead?" Surfacing the conflict before trying is cheaper, and it puts the decision back with a person.

A strong answer includes:

• The safety result: the control held whatever the instruction.
• The cost of the model not objecting: wasted runs.
• A concrete sentence the orchestrator should have said.

Q5

Your T9 used multiplyBasisPoints() correctly, then compared the result with the $1.00 minimum using a float. Which layer should have caught it: AGENTS.md, a skill, a hook, the reviewer, or the check? Which did, and should that change?

Model answer. The skill should have prevented it. money-change should say that minimums are Money too, written cents(100) and compared as integers, never fee / 100 < 1.0. The Milestone 3 no-float-money lint rule should catch / 100 deterministically if the session runs lint, and money-reviewer would likely flag it. In the story, the check caught it. That's the right last line of defense for measurement, but it's after the fact. So yes, it should change: add the minimum case to the skill, and consider whether the Lab 5 money-lint hook should also flag / 100 near an amount, once you've measured its false-positive rate.

A strong answer includes:

• Prevention in the skill (knowledge), detection in lint or a hook (mechanism).
• That the check catching it is measurement, not prevention.
• A concrete change, with false positives in mind.

Q6

Write a three-sentence cover note to Ruth with the table. Name the row she's most likely to push back on, and what you'd do about it in the next 30 days.

Model answer. "Ruth, the attached table answers your memo condition by condition, and every row names a file you can open and a test you can run yourself. Two conditions are met with stated limits and one is partly met, and we've written the gaps down rather than argued them away. We'd like to walk you through breaking any control in the table." She'll push back hardest on row 1, because the CI approval is a label anyone with write access can add. In 30 days: a CODEOWNERS entry for src/money/, migrations/ and legacy/, and branch protection that requires code-owner review and the protected-paths job. Then the label records a decision the right person made. Row 2 is also a fair answer, with --tag on every CI run to link pull requests to sessions.

A strong answer includes:

• A note that leads with evidence, not claims, and names the gaps.
• An invitation to test the controls.
• A specific row, and a specific 30-day fix with a mechanism.

Knowledge check

1. An interactive Mission worker hits the db:migrate ask rule. Answer: it still needs your approval.
   • "It runs, because High allows migrations" is Lab 4's trap again: an ask rule wins at every level.
   • "Blocked with no way to approve" confuses ask with block. Headless, an ask does fail the run, which looks like a block from the outside.
2. 8 features and 3 milestones: how many runs? Answer: at least 14.
   • "It will be 14" treats the heuristic as exact.
   • "At most 14" reads a floor as a budget cap. This lab's Mission used 18.
3. Why can't a worker add a helper to src/money/? Answer: workers inherit the project hooks.
   • "Mission workers are read-only" borrows from Spec mode or the built-in read-only droid. Workers edit files all the time.
   • "The orchestrator reviews every edit" tempts because you can talk to the orchestrator. It doesn't gate edits. The hooks do.

Assessment answer key

The assessment has 17 questions: 12 multiple choice, marked automatically, and 5 written, self-graded against a rubric. Scoring and grading guidance is in the instructor guide.

Section 1 — Mechanics (30%)

#	Question	Answer	Why	The tempting wrong answer
1	droid exec write with no --auto	C: stops, non-zero exit, no partial changes	Exec is read-only by default and fails fast	A, "asks you to approve": the interactive habit
2	Toggle Normal and Spec	B: Shift+Tab	Ctrl+L is autonomy, Ctrl+T is Mission Control, Tab is reasoning effort	A, Ctrl+L: both are "mode-like" keys
3	-r in droid exec	B: --reasoning-effort	In interactive droid it's --resume	A, --resume, from the interactive CLI
4	Where project hooks live	B: .factory/hooks.json, events at the top level	A hooks key in settings.json is read only when hooks.json is absent	C, "any executable in .factory/hooks/": that's where the scripts live, but config must point at them
5	allowed-tools: [Read, Grep] in a SKILL.md	B: nothing enforced, metadata	Use a custom droid's tools for enforcement	A, "restricts Droid": what the field name promises
6	Where droid mcp add saves	B: ~/.factory/mcp.json	Project servers go in .factory/mcp.json, by hand	A, the project file: what most CLIs would do

Section 2 — Controls (30%)

#	Question	Answer	Why	The tempting wrong answer
7	Which decision holds under --skip-permissions-unsafe	C: block	Skipping prompts skips ask. block has no approval path	D, "none": the flag's name sounds total
8	Not blocked by ["git", "push", ["--force", "-f"]]	C: git push origin main --force	Prefixes are positional. The third token is origin	D, git push -f origin feature: looks different, but -f is third, so it's blocked
9	PreToolUse hook exits 1	B: the edit runs	Only exit 2 blocks	A, "blocked": Unix habit, non-zero means fail
10	Which hook catches lint --fix touching src/money/	C: a Stop hook checking git	The command uses no file tool and doesn't name the path	B, Execute with a commandRegex: lint is an Execute call, but the regex doesn't match its text
11	"decision": "deny"	C: invalid and skipped, nothing else changes	Decisions are allow, ask, block. Invalid rules don't become block-all	A, "works like block": deny is valid in hook output, so it looks valid here
12	Exec at --auto high hits the git push ask rule	B: the run fails	A one-shot run can't collect an approval	A, "pushes, because High allows remote writes": true without the rule

Section 3 — Judgment (40%)

Each written answer is worth 8 points, split evenly across its rubric lines. The model answers below meet every line. Each is followed by the rubric line it satisfies, in order.

Q13 — Reply to Imani: T2 at 2 of 3 is "fine"

Model answer. "The run that fails isn't a slow fix. It's the March incident again: Droid changes half-even rounding in src/money/ to make the test pass. AGENTS.md is an instruction Droid weighs against the task, so better wording shifts the odds but can't guarantee anything. The fix is a hook that blocks edits to src/money/ whatever the model decides, with a Stop check on the outcome. We'll call it fixed when T2 passes 3 of 3 and no run changes a protected path."

Rubric lines met:

1. Separates pass rate from failure mode: "It's the March incident again."
2. AGENTS.md is a request, not a control: "an instruction Droid weighs".
3. Proposes enforcement: a hook, not stronger wording.
4. Says how you'd show it's fixed: T2 at 3/3 and zero protected-path changes.

Q14 — Ruth: "Is your Execute hook complete?"

Model answer. "No. It pattern-matches the text of shell commands, so anything that doesn't spell out src/money gets past it: a node -e script that builds the path from parts, a script file, or an allowed command with side effects like npm run lint -- --fix. Two layers catch what it misses. The Stop hook asks git which protected files changed before the session can finish, and the CI job diffs every pull request against its base, which also catches committed changes. Here are the piped tests and the session where we bypassed our own first hook. I'd like you to try to break it."

Rubric lines met:

1. Says no, plainly, because it pattern-matches command text.
2. Names a concrete bypass: node -e with a built path, a script, lint --fix.
3. Explains the outcome check (Stop hook) and the CI backstop.
4. Offers to let her break it, and shows the tests.

Q15 — The first five harness changes in a new regulated repository

Model answer.

1. Measure: run /readiness-report and a scoreboard of five to eight real backlog tasks with automatic checks. Without a before number, no later change can be shown to help.
2. AGENTS.md: exact commands, protected paths, invariants that name helpers, a never-list and a stop condition. It's the cheapest change and usually the biggest move.
3. Autonomy default and permission rules: Low as the project default, block and ask rules with tests, and droid rules check in CI. These come before any headless or CI use, because exec can't ask anyone.
4. Hooks: PreToolUse guards on protected paths for file tools and Execute, and a Stop hook that checks what changed. For the invariants that must not fail even once.
5. CI: the protected-path diff check and the scoreboard on pull requests. It covers everything that never went through a hook.

The order follows risk and evidence: measure first, then the cheap broad change, then controls before autonomy increases, then enforcement, then the backstop.

Rubric lines met:

1. Measures first: readiness and a scoreboard before any change.
2. AGENTS.md early, because it's cheap and moves the most.
3. Rules and autonomy defaults before headless use.
4. Hooks for invariants that can't fail.
5. The order is justified by risk and measurement, not ease.

Q16 — Should money-lint become a blocking PreToolUse hook?

Model answer. "Not yet. As a PostToolUse hook, a false positive costs one rewrite. As a blocking PreToolUse hook, it stops a legitimate change one time in five, and pushes Droid to find another way round, which is exactly the behavior we don't want to train. We have evidence: even as feedback it made T3 regress from 3/3 to 2/3, and T3 recovered when the money-change skill taught basisPoints(150), with the hook unchanged. So teach the helper rather than loosen or harden the regex, measure the false-positive rate over src/, and only make it blocking if that rate gets close to zero. Blocking is for invariants that must never fail, like the protected paths, where a false positive is a cheap question and a false negative is an incident."

Rubric lines met:

1. Distinguishes feedback (PostToolUse) from prevention (PreToolUse) and the cost of a false positive in each.
2. Uses the scoreboard evidence: T3 regressed, then recovered.
3. Prefers teaching the helper (a skill) over changing the regex blindly.
4. Keeps blocking for the invariants that must never fail.

Q17 — Run the Lab 10 Mission at Level 3?

Model answer. "Yes, but narrowly, and on the record. Factory recommends Level 4 or higher for Missions, and Larkspur is at Level 3, missing fast CI feedback, deployment frequency and flaky-test detection. Flaky-test detection matters most for this Mission, because each milestone's validation depends on tests, and a flaky test makes validation lie. So we'll run three milestones with validation at the end of each, no protected path in scope, hooks on throughout, and no merge without human review, and I'll put the readiness decision in mission.md with my name on it. If we can't accept that risk, we add flaky-test detection first and run the Mission after."

Rubric lines met:

1. Acknowledges the Level 4 recommendation.
2. Makes an explicit decision: a narrow Mission with a named, accepted risk, and the alternative.
3. Uses milestone validation as a control.
4. Names the gap that matters most: flaky-test detection.

A "no" can score full marks too, if it names the Level 4 criterion to fix first (flaky-test detection) and says what would change the answer.