The Claude Architect – Foundations (CCAR-F) exam is 60 items, closed-book, proctored by Pearson VUE, with a 120-minute time limit. It costs $125 and uses a scaled score out of 1000, with 720 needed to pass. Partners who design Claude solutions end to end. This page's question bank and flashcards are free and domain-weighted to match the breakdown below; the timed mock exam reproduces the full exam length and scoring.
Exam domains & weighting
Domain
Weight
Your progress
Agentic Architecture & Orchestration
27%
0/54
Claude Code Configuration & Workflows
20%
0/40
Prompt Engineering & Structured Output
20%
0/40
Tool Design & MCP Integration
18%
0/36
Context Management & Reliability
15%
0/30
Published scenarios
Anthropic publishes these scenario names in advance for CCAR-F — a subset appears in any given sitting.
Support Resolution Agent
Claude Code for Development
Multi-Agent Research
Developer Productivity Tooling
Claude Code in CI/CD
Structured Data Extraction
About these questions
Every question below is written from scratch by GetDev, grounded in Anthropic's published exam domains and weightings plus general Claude, Claude Code and MCP product knowledge. None are reproduced from any other prep site. Each is tagged with its domain, subtopic and difficulty so you can target weak spots — use the filters in the Question bank tab.
200 of 200 questions · 0 answered · 0 correct so far
Q1
Agent loop design & termination
Applied
Support Resolution Agent
Your support agent resolves most tickets but occasionally loops, re-calling lookup_order on the same order ID eight times in a row before timing out. What is the correct structural fix?
Not quite.
Agent loops need structural, not persuasive, termination conditions. Prompt instructions (A) are probabilistic and the model already 'knows' not to loop pointlessly — it's stuck, not ignoring guidance. Temperature (B) and token budget (C) don't address a model that's confidently repeating the same unproductive action. The fix is engineering: cap iterations, detect identical repeated calls, and route to a human when the budget is exhausted.
Q2
Agent loop design & termination
Foundational
In the standard agentic loop, what signals the model has finished its turn and tool-calling should stop?
Not quite.
The loop naturally terminates when the model's response contains no tool_use blocks and stop_reason is end_turn — the model has decided it has enough to respond directly. Timeouts and context limits (A, B) are safety nets, not the primary termination signal, and there's no built-in finish() tool (D); some implementations add one, but it isn't the mechanism the loop relies on by default.
Q3
Agent loop design & termination
Applied
A coding agent is given a budget of 40 tool calls per task. On a genuinely hard refactor, it hits the budget and stops mid-task with no explanation to the user. What is missing from the design?
Not quite.
Iteration budgets are a necessary safety mechanism — removing them (A) reopens the unbounded-loop risk. The real gap is that hitting the budget is treated as a hard stop instead of a trigger for graceful degradation: warn the agent near the limit, have it summarize state and next steps, and hand off to a human or a fresh session with that summary as context. A faster model (C) just delays the same problem; avoiding tool use (D) defeats the agent's purpose.
Q4
Agent loop design & termination
Advanced
You're designing repeated-call detection to catch loops like the one in question 1. Which detection strategy is most robust against an agent that loops with slightly varying arguments each time (e.g., lookup_order with a trailing space added, then removed)?
Not quite.
Exact match (B) is defeated by trivial variation. Ignoring arguments (C) would false-positive on an agent legitimately calling the same tool with different real inputs. The model's stop_reason (D) doesn't know it's looping — from its perspective each call looks like reasonable progress. Normalizing before comparison catches near-identical calls while a per-pair threshold (tighter than the global budget) surfaces the loop before the whole budget is burned.
Q5
Agent loop design & termination
Foundational
Why is an unbounded agentic loop a cost risk as well as a reliability risk?
Not quite.
Every turn in an agentic loop sends the accumulated conversation history back to the model, so a loop isn't just N extra calls — it's N calls against an ever-growing context, compounding cost. This is a second, independent reason (beyond reliability) that iteration budgets and loop detection matter architecturally, not just as a UX nicety.
Q6
Agent loop design & termination
Applied
Claude Code in CI/CD
A Claude Code agent running headless in a CI pipeline is given a task to fix a failing test. It needs a termination condition appropriate for an unattended environment. Which is the best design?
Not quite.
Unattended CI runs need a definite, checkable success condition (not just 'time ran out') and a budget, so failure is explicit and debuggable rather than silent (A, D assume a human is watching, which defeats the point of CI automation). Pure wall-clock timeout (B) doesn't distinguish 'about to succeed' from 'hopelessly stuck' — checking the actual success condition (tests passing) after each attempt does.
Q7
Agent loop design & termination
Advanced
An agent's loop includes a step where it asks itself 'am I making progress?' before continuing. What is the main architectural risk of relying on this kind of model self-assessment as the sole termination mechanism?
Not quite.
Self-assessment is a reasonable supplementary signal but a weak sole mechanism: a model that's confidently looping is, by definition, not recognizing the loop — asking it to judge its own progress uses the same faculties that produced the stuck behavior. This is why engineered, external checks (iteration counts, repeated-call detection, objective success criteria) are the load-bearing termination mechanism, with model self-assessment as a secondary signal at most.
Q8
Agent loop design & termination
Foundational
What distinguishes a 'soft' iteration limit from a 'hard' iteration limit in agent loop design?
Not quite.
A soft limit is a signal to the agent — often injected as a system reminder near the budget's edge — nudging it to converge, summarize, or ask for help. A hard limit is the actual backstop: the loop is terminated by the orchestrator regardless of the agent's internal state. Using only a hard limit means no graceful degradation (see question 3); using only a soft limit means no real guarantee against runaway cost.
Q9
Single vs multi-agent decision
Applied
A client wants a document-processing pipeline that extracts five distinct fields from invoices using five different extraction logics. They propose five separate agents, one per field. What should you evaluate before agreeing?
Not quite.
More agents is not automatically better (A) or impossible to avoid (B). The architectural question is whether the task decomposes into genuinely independent units that benefit from isolation or parallelism, weighed against the real overhead (latency, cost, orchestration complexity, merge logic) of multiple agents. For five independent fields with no cross-field reasoning needed, one structured-output call is often simpler and just as reliable.
Q10
Single vs multi-agent decision
Foundational
Which scenario most clearly justifies a multi-agent architecture over a single agent?
Not quite.
Multi-agent architectures earn their complexity when subtasks need genuinely different capabilities, privileges, or isolation — splitting by privilege (e.g., a write-capable agent kept separate from a read-only research agent) is a real security and architecture win, not just a performance trick. Repetitive-but-simple tasks (A) and raw speed (C) don't inherently need multiple agents, and 'more than one tool call' (D) describes almost every useful single agent.
Q11
Single vs multi-agent decision
Advanced
What is the main hidden cost of a multi-agent system that a single-agent system doesn't have?
Not quite.
The defining hidden cost of multi-agent architectures is coordination overhead: explicit handoff design, failure propagation across boundaries, and multi-transcript debugging. Raw token cost (A) isn't always higher — parallel agents can sometimes be cheaper in wall-clock time even if token totals rise, and the comparison is situational, not absolute. Tool use (B) and model choice (D) aren't restricted by agent count.
Q12
Single vs multi-agent decision
Applied
Multi-Agent Research
A research assistant needs to search five independent topics and synthesize a combined report. Why might a multi-agent design (one searcher per topic, fanning in to a synthesizer) outperform a single agent doing all five searches sequentially?
Not quite.
Independent, parallelizable subtasks are the textbook case for fan-out: wall-clock latency drops because searches run concurrently, and each searcher's context stays focused on its own topic instead of one agent's context window filling up with five topics' worth of search results competing for attention. This isn't an API requirement (B), and it doesn't make cost zero (C) — token cost may even rise slightly due to the synthesis step, which is an accepted tradeoff for latency and focus.
Q13
Single vs multi-agent decision
Foundational
A team wants to add a second agent 'just in case it helps' without a specific decomposition reason. What is the architectural risk?
Not quite.
Multi-agent architecture is a cost (coordination, debugging surface, latency from handoffs) paid in exchange for a specific benefit (parallelism, specialization, isolation). Adding agents without a decomposition reason pays the cost without the benefit. It doesn't automatically breach rate limits (C) or improve accuracy (D) — in fact it can introduce new failure modes like the provenance loss discussed elsewhere in this domain.
Q14
Single vs multi-agent decision
Applied
Which heuristic best captures when to prefer a single agent with more tools over splitting into multiple specialized agents?
Not quite.
The decision hinges on whether subtasks are genuinely independent/parallelizable or need isolated privileges — if they're sequential, share the same context, and don't need separation, a single agent with multiple tools avoids the coordination tax entirely. Client phrasing (C) and token count (D) aren't architectural signals.
Q15
Single vs multi-agent decision
Advanced
A single agent handling customer support has grown to 40 tools (billing, shipping, refunds, account management, escalation). Response quality has degraded. What is the most likely architectural cause, and the fix?
Not quite.
Tool selection accuracy tends to degrade as the tool count grows and tool purposes start to overlap or become harder to discriminate between — this is a real, documented failure mode, not a model-quality issue (A) to be fixed by swapping models, and it isn't 'normal' to ignore (B). Splitting into domain-scoped agents each with a smaller, sharply-defined tool set is the standard architectural response; removing tools (D) would remove the agent's capability entirely.
Q16
Orchestrator/coordinator patterns
Foundational
In a coordinator-worker pattern, what is the coordinator's primary responsibility?
Not quite.
The coordinator's job is decomposition, dispatch and assembly — it generally should not do the deep domain work itself (A), which is what the workers specialize in. It's an active decision-maker, not a passive logger (B), and its existence doesn't make workers redundant (C); the whole pattern depends on workers doing the specialized labor.
Q17
Orchestrator/coordinator patterns
Applied
Multi-Agent Research
In a fan-out/fan-in orchestration pattern, three research subagents run in parallel and report back to a coordinator. What should the coordinator do if one of the three subagents fails?
Not quite.
Partial failure in a fan-out pattern should be surfaced, not hidden (C) or treated as total failure (B) — discarding good results because one branch failed wastes the successful work. Retrying everything including successes (D) is wasteful; a well-designed coordinator retries only the failed branch (within policy) and, if it still fails, reports the gap explicitly so downstream consumers know the result is partial rather than assuming it's complete.
Q18
Orchestrator/coordinator patterns
Advanced
A planner-executor split separates 'decide what to do' from 'do it.' What architectural benefit does this separation provide over a single agent that plans and executes in one undifferentiated loop?
Not quite.
Separating planning from execution produces a concrete, reviewable plan as an intermediate artifact — valuable exactly when execution has consequences worth gating (a human or a validation step can approve the plan before irreversible actions happen). It isn't primarily a speed optimization (C) and the executor still needs tools (D) — the split is about risk and inspectability, not performance.
Q19
Orchestrator/coordinator patterns
Foundational
What is a 'supervisor' pattern in agent orchestration, as distinct from a simple one-shot coordinator?
Not quite.
A supervisor pattern adds an ongoing monitoring and re-planning loop — it can notice a worker is stuck or off-track and intervene (re-dispatch, adjust the plan, escalate) rather than firing off a task and passively waiting. A plain coordinator in its simplest form may just dispatch-and-collect. This is a meaningful architectural distinction (not just a naming difference, A), applies to agent systems directly (B), and doesn't eliminate workers (D).
Q20
Orchestrator/coordinator patterns
Applied
You're building a system where incoming requests must be routed to one of four specialized agents (billing, technical, sales, general) based on content. What is the most robust way to implement this routing?
Not quite.
Keyword matching (A) is brittle against paraphrase and mixed-topic requests. Broadcasting to all four and racing (B) wastes cost and risks a wrong agent 'winning' on speed rather than correctness. Letting each agent self-select (D) has no accountable decision point and can lead to a request being handled by none or several agents inconsistently. A dedicated, cheap classification step gives an explicit, auditable routing decision before any expensive specialized work begins.
Q21
Orchestrator/coordinator patterns
Advanced
In a sequential pipeline pattern (agent A's output feeds agent B, whose output feeds agent C), what is the primary architectural risk compared to fan-out/fan-in?
Not quite.
Sequential pipelines have additive latency (unlike parallel fan-out, where wall-clock time is closer to the slowest single branch) and error compounding — if stage A produces a subtly wrong result, B and C build on that error without any natural checkpoint to catch it unless one is deliberately added (validation steps between stages). This is independent of tool use (A) or Claude Code compatibility (D), and token cost (C) isn't reliably lower — it depends on the task.
Q22
Orchestrator/coordinator patterns
Foundational
Why might a coordinator deliberately run a 'reduce' or aggregation step after fan-out, rather than just concatenating worker outputs?
Not quite.
Raw concatenation (B) works only when worker outputs are already independent and non-overlapping — often they aren't: two research agents may surface conflicting facts, or redundant ones. A deliberate aggregation/reduce step is where conflicts get resolved and the final output is made coherent, rather than leaving the user to reconcile several disjointed pieces. This applies regardless of worker count (C) and is a separate step from fan-out (D), not a replacement for it.
Q23
Context passing between agents
Advanced
Multi-Agent Research
A coordinator delegates to search, analysis, and report-writing subagents in sequence. The final report cites sources that don't actually support its claims, even though each subagent behaves sensibly in isolation. What is the most likely architectural cause?
Not quite.
This is a classic provenance-loss failure: each stage looks locally correct, but if the handoff format summarizes conclusions without carrying the claim-to-source linkage through, the next stage has no way to reattach correct citations — it may confidently pair a true claim with the wrong source, or vice versa. The fix is architectural (redesign the handoff schema to carry claim+source pairs), not a hallucination problem to dismiss (A), a context-size problem (B), or a cross-model issue (C).
Q24
Context passing between agents
Foundational
What is the main tradeoff between passing full conversation history versus a structured summary when handing off from one agent to another?
Not quite.
This is a genuine architectural tradeoff, not a dominated choice (A isn't right). Full history preserves everything but costs tokens and can bury the signal the next agent needs in noise. A structured summary is cheap and focused but is lossy by design — if the summary schema omits something the next stage needs (as in the citation example), failures appear downstream. The right choice depends on what the next agent actually needs to do its job.
Q25
Context passing between agents
Applied
You're designing the handoff schema from a 'triage' agent to a 'resolution' agent in a support pipeline. What should that schema minimally include, beyond the raw customer message?
Not quite.
The resolution agent needs the decision plus enough evidence to act on it sensibly (and to flag inconsistency if something looks off) — passing only the raw message (A) forces redundant re-triage, and a bare confidence score (D) without the reasoning gives no way to sanity-check it. Full verbatim chain-of-thought (C) is usually excessive and can be unreliable as a literal audit trail; a structured summary of the decision and its key evidence is the right granularity.
Q26
Context passing between agents
Advanced
Two parallel subagents each produce a structured result referencing 'the customer' with slightly different normalized details (one has a middle name, one doesn't). The coordinator merges both into a single record. What design prevents silent data corruption here?
Not quite.
Implicit merge behavior (first-write-wins or last-write-wins, option B) hides real conflicts and risks silently picking the wrong value. The robust design makes conflict resolution explicit — defined precedence rules per field, or routing genuine conflicts to a human — so the system's behavior under disagreement is a deliberate choice, not an accident of call ordering. Preventing all overlap (C) isn't realistic for independent subagents working on related data, and temperature (D) has nothing to do with structured merge logic.
Q27
Context passing between agents
Foundational
Why is a shared, mutable scratchpad (all agents read and write the same memory store) generally riskier than explicit message-passing between agents?
Not quite.
Shared mutable state is a classic concurrency hazard: when multiple agents can write to the same store, races and accidental overwrites become possible, and tracing which agent wrote what becomes harder than with explicit, directional message-passing (which naturally preserves order and origin). This is a real risk (not purely theoretical, A), it's fully implementable with the API (B), and it is a meaningful architectural difference (C is wrong).
Q28
Context passing between agents
Applied
A coordinator passes each subagent's entire raw tool output (including verbose API responses) directly to the next stage. What problem does this likely cause as the pipeline grows to five or six stages?
Not quite.
Passing raw, unfiltered output forward at every stage compounds: cost grows with every hop, and signal-to-noise degrades as each downstream agent has to extract what's relevant from an increasingly bloated context. More raw data is not unconditionally better (A) — the fix is to have each stage extract and pass forward only what the next stage actually needs, not the full raw payload. This is implementable regardless of context window size differences (D), and it's a real, not hypothetical, problem (B is wrong).
Q29
Context passing between agents
Advanced
What is the architectural purpose of including a 'confidence' or 'certainty' field in the structured handoff from one agent to the next, rather than just the conclusion?
Not quite.
A confidence signal in the handoff enables differentiated downstream handling — high-confidence results flow through automatically, low-confidence ones get extra scrutiny or human review. This directly supports risk-appropriate automation rather than treating every output as equally trustworthy (A undersells its value). It isn't an API requirement (C), and it complements rather than replaces an escalation path (D) — it's often the signal that triggers escalation.
Q30
Subagent delegation & isolation
Foundational
What is the primary architectural benefit of giving a subagent its own isolated context window rather than appending its work to the main agent's conversation?
Not quite.
Context isolation is primarily about keeping the main agent's context clean — a subagent can churn through many exploratory tool calls and dead ends, but only its distilled result needs to return to the parent, rather than every intermediate step. This isn't guaranteed to be cheaper (B depends on the task), isn't an API mandate (B), and doesn't remove the subagent's own need for tools (C).
Q31
Subagent delegation & isolation
Applied
A coordinator delegates a code-search task to a subagent with full read/write filesystem access, even though the subagent only needs to search and report findings. What is the architectural problem?
Not quite.
This is a direct least-privilege violation. Tools and access should be distributed by what a given subagent's role actually requires — a search-and-report subagent needs read access, not write. Scoping access tightly limits the damage if the subagent reasons incorrectly or is adversarially misled by content it reads; relying on correct behavior alone (B) is not a security boundary. Access absolutely can be scoped per subagent (C), and the risk applies in any environment, not just CI (D).
Q32
Subagent delegation & isolation
Advanced
When running several subagents in parallel, each with its own isolated context, what should the coordinator do before merging their results if the subagents' tasks had any possibility of overlap?
Not quite.
Context isolation solves one problem (cross-contamination) but not another (independent agents reaching genuinely inconsistent conclusions about overlapping scope) — isolation is not the same as consistency (A overclaims what isolation guarantees). The coordinator needs an explicit validation/reconciliation step for the overlap case. Blanket re-running (C) is wasteful when most runs will be consistent, and discarding dissenting results without investigation (D) risks throwing away the correct one.
Q33
Subagent delegation & isolation
Foundational
Why might an architecture use specialized subagents (e.g., a 'security review' subagent, a 'style review' subagent) rather than one general-purpose code review agent?
Not quite.
Specialization allows each subagent's prompt, tools and criteria to focus tightly on one concern, generally producing more thorough coverage than a single generalist agent trying to hold security, style, performance and correctness all at once in one pass. This isn't a hard requirement (B) or a capability gap in general-purpose agents (A) — it's a design choice that trades more orchestration for depth per concern, and the benefit scales down to much smaller codebases too (D is an arbitrary threshold).
Q34
Subagent delegation & isolation
Applied
Developer Productivity Tooling
A coordinator spins up a new subagent for every single file in a 200-file codebase review, each with its own context setup overhead. What is the likely architectural problem, and better approach?
Not quite.
Subagent creation has real overhead (context setup, coordination), and at 200 instances that overhead compounds significantly for what may be small per-file tasks. The better design batches logically related files into fewer subagent invocations (e.g., by module or directory) rather than one-per-file, balancing isolation benefits against spin-up cost. There's no such 'one subagent per file' convention (B), and overhead isn't negligible at this scale (A is wrong); model choice (C) doesn't address the structural issue.
Q35
Subagent delegation & isolation
Advanced
A subagent is given a narrow, well-defined task but ends up calling a tool outside its intended scope because the tool happened to be available in its toolset. What does this reveal about the delegation design?
Not quite.
This is the same least-privilege lesson as question 31, applied to tool availability specifically: if a tool is in the toolset, a model may reasonably decide to use it when it looks helpful for the task at hand, even if that's outside the intended scope. The robust fix is to not grant tools the subtask doesn't need — enforcing scope structurally rather than relying on prompt instructions alone (D) to suppress an available capability. Disabling tool use (C) defeats the subagent's purpose, and this isn't a model reliability failure to write off (B) — it's a predictable consequence of over-provisioning access.
Q36
Subagent delegation & isolation
Foundational
What should a coordinator do if a delegated subagent returns a result but the coordinator cannot verify whether the subagent actually completed its assigned task correctly?
Not quite.
A subagent asserting 'done' is not verification — the handoff contract should require evidence the coordinator (or a check) can actually confirm, such as a diff, a test result, or a specific artifact, rather than trusting a bare claim (A). Defaulting to assuming failure (C) wastes the (likely correct) successful work most of the time. The answer is a better output contract, not eliminating delegation (D).
Q37
Escalation & human-in-the-loop design
Applied
Support Resolution Agent
Support policy caps agent-initiated refunds at $200; larger refunds require a human. Where should that $200 rule be enforced?
Not quite.
Hard policy limits with real financial consequences should be enforced deterministically at the tool boundary, not requested of the model. Prompt phrasing (A) and model judgment (B) are probabilistic where the requirement is absolute — a model can misread context or be manipulated into exceeding the limit. Asking the user (D) is not a control at all, since the user has no authority over the policy. The tool itself should structurally refuse or redirect out-of-policy amounts.
Q38
Escalation & human-in-the-loop design
Foundational
What is the architectural difference between a 'hard gate' (action blocked until a human explicitly approves) and a 'soft gate' (action proceeds but is flagged for human review after the fact) in escalation design?
Not quite.
The choice between hard and soft gating is a direct function of how risky and reversible the action is. Hard gates suit irreversible, high-stakes actions (large refunds, production deployments) where prevention beats remediation. Soft gates suit lower-risk, easily-reversible actions, where blocking on human approval would add unnecessary friction — post-hoc audit is sufficient. This is a meaningful distinction (A is wrong), unrelated to transaction value specifically (B), and both are fully implementable (C is wrong).
Q39
Escalation & human-in-the-loop design
Advanced
An agent escalates to a human queue whenever its confidence drops below a fixed threshold, but the queue has grown unmanageable because the threshold is miscalibrated. What architectural approach addresses this better than picking a single static number?
Not quite.
A static, uncalibrated threshold is a common and fixable mistake — the right approach treats the threshold as a parameter to tune against real outcome data (how often 'high confidence' decisions were actually right), adjusting it as the system and its failure modes evolve. Removing escalation (B) reintroduces unchecked risk; escalating everything (C) is what caused the unmanageable queue in the first place; letting each instance improvise its own threshold (D) produces inconsistent, unauditable behavior.
Q40
Escalation & human-in-the-loop design
Applied
What should an escalation to a human reviewer include, at minimum, to be actionable rather than just a bare alert?
Not quite.
An actionable escalation gives the human what they need to make the specific decision being asked of them — the decision point, supporting evidence, and a clear ask — without forcing them to either work from nothing (B) or wade through an unsummarized full transcript (C). Asking the human to restart from scratch (D) wastes the agent's prior work entirely and defeats the purpose of partial automation.
Q41
Escalation & human-in-the-loop design
Foundational
Why is an audit trail important specifically for escalated decisions, beyond general logging?
Not quite.
Escalated cases are exactly the highest-value data for improving the system — they're where the agent itself flagged uncertainty, so tracking what was escalated, the context, and the human's resolution is what enables threshold recalibration (as in question 39) and reveals systemic issues (e.g., one subtopic escalating constantly, suggesting a tool or prompt gap). This value isn't limited to financial contexts (B), and the audit trail is a complement to escalation, not a replacement for it (C).
Q42
Escalation & human-in-the-loop design
Advanced
A system grants agents 'partial autonomy' — full autonomy for low-risk actions, approval-required for high-risk ones. What is the architectural risk if 'risk level' is determined by the same agent that's about to take the action?
Not quite.
Letting the acting agent self-classify risk undermines the gate's purpose — the whole point of an approval gate is an independent check, and if the gate's trigger condition is itself decided by the party being gated, errors or incentive misalignment can bypass it silently. The more robust design determines risk level from structural properties of the action (which tool, what parameters, what thresholds) rather than the acting agent's self-report — this is often hardcoded per tool/action type (so D is too absolute but gestures at something real), not dynamically self-assessed by the same agent.
Q43
Error handling & retry strategy in agent loops
Foundational
Why should transient errors (e.g., a network timeout calling an external API) and permanent errors (e.g., invalid parameters, a resource that doesn't exist) be handled differently in an agent's tool-calling loop?
Not quite.
This distinction is fundamental to sane retry logic. A timeout might succeed on retry as conditions change; a 'resource not found' or 'invalid parameter' error will fail the exact same way every time, so blind retrying wastes calls and time. The structured response design (category, retryable flag) mentioned in MCP tool design directly supports this: the agent or orchestrator can check the retryable flag and behave accordingly, rather than treating every error identically (A, C are both wrong, and D ignores that tool error semantics are meaningfully different from raw API-level errors).
Q44
Error handling & retry strategy in agent loops
Applied
An agent calls a payment-processing tool that times out. The agent, unsure if the payment succeeded, retries the call. What architectural safeguard prevents this from double-charging the customer?
Not quite.
This is exactly the scenario idempotency keys exist to solve — a timeout is genuinely ambiguous (the request may have succeeded server-side even though the response didn't arrive), so the safe design makes the operation itself idempotent: retrying with the same key is a no-op if it already happened. Prompt instructions not to retry (C) just trades one risk (double-charge) for another (a legitimately failed payment never getting retried), and disabling all retries (D) is far too blunt an instrument for one specific risk.
Q45
Error handling & retry strategy in agent loops
Advanced
What is the purpose of exponential backoff in an agent's retry strategy, as opposed to immediate fixed-interval retries?
Not quite.
Exponential backoff exists to avoid compounding a transient failure into a bigger one — if a downstream service is struggling, immediate rapid-fire retries from many callers can turn a brief hiccup into a full outage (a 'retry storm'). Spacing retries with growing delays reduces that load while still allowing recovery. It doesn't guarantee success (B) — it's about not making things worse while retrying — and the pattern applies to any external call an agent makes, not just the Claude API itself (C).
Q46
Error handling & retry strategy in agent loops
Foundational
What is a circuit breaker pattern, applied to an agent repeatedly calling a failing external tool?
Not quite.
A circuit breaker is distinct from backoff: rather than spacing out retries indefinitely, it 'opens' after sustained failure and stops trying for a cooldown period, failing fast rather than continuing to hammer a dependency that's clearly down — then periodically allows a probe call to see if it's safe to 'close' the circuit again. It's not permanent (B) and is a different, complementary mechanism to backoff (D is wrong) — backoff controls retry spacing within attempts, a circuit breaker controls whether to attempt at all.
Q47
Error handling & retry strategy in agent loops
Applied
An agent receives a tool error with a generic message: 'An error occurred.' What architectural problem does this create for the agent's ability to recover?
Not quite.
This connects directly to the MCP tool design principle that errors should return structured responses with category, retryable flag, and actionable messages — a bare 'an error occurred' gives the agent nothing to reason from, forcing a guess between retry, alternate approach, or escalation. Vague errors aren't more secure (C) — if anything they push the agent toward uninformed, potentially unsafe retries. This applies to any tool error the agent has to reason about, not just Claude API errors specifically (D).
Q48
Error handling & retry strategy in agent loops
Advanced
An agent's retry logic for a flaky tool is capped at 3 attempts, after which it falls back to a simpler, lower-fidelity method of accomplishing the same subtask. What is this fallback pattern called, and why is it architecturally valuable?
Not quite.
This is graceful degradation: rather than letting one failing subtask halt the entire pipeline, the system falls back to an alternative (lower-fidelity but still useful) path. It's a different mechanism from a circuit breaker (C) — which governs whether to keep attempting a specific dependency — and is often genuinely preferable to a hard failure, since partial progress usually beats total failure (D overstates the risk; the tradeoff is often worth it when clearly communicated as degraded).
Q49
Agent state & memory management
Foundational
What is the fundamental difference between a 'stateless' agent and a 'stateful' agent across multiple user interactions?
Not quite.
This is a core architectural choice. Stateless agents start fresh each time — simpler, more predictable, but unable to build on prior context without it being re-supplied. Stateful agents persist and retrieve some memory across sessions, enabling continuity (e.g., remembering a user's preferences) at the cost of added complexity (storage, retrieval, staleness management). Tool use (B) is independent of statefulness, and stateful agents still operate within a context window (C is wrong) — persisted memory still has to be loaded into context when used.
Q50
Agent state & memory management
Applied
A long-running agent session has accumulated 180,000 tokens of conversation history, approaching the context window limit. What is the most architecturally sound response?
Not quite.
Proactive compaction — summarizing older, resolved context into a condensed form before the limit forces an emergency response — is the standard pattern for long-running sessions. Letting it fail outright (A) is avoidable. Raw truncation with no summarization (D) risks silently losing decisions or constraints established earlier that are still relevant. Switching models mid-session (C) treats a design problem as a model-selection problem and doesn't address what happens when the larger window also eventually fills.
Q51
Agent state & memory management
Advanced
What is the risk of summarizing conversation history too aggressively to save context space?
Not quite.
Summarization is inherently lossy, and the risk compounds silently — information discarded as 'not important enough to keep' can turn out to matter for a later decision, and unlike raw truncation (where you at least know something was cut), overly confident summarization can make the agent believe it has the full picture when it doesn't. This argues for conservative summarization that errs toward preserving specifics (decisions, constraints, exact values) over narrative color. It's unrelated to extended thinking specifically (D) and is a practical, well-known risk (B is wrong, A overstates the safety of aggressive summarization).
Q52
Agent state & memory management
Foundational
What is the distinction between 'working memory' (current task context) and 'long-term memory' (persisted knowledge across sessions) in an agent architecture?
Not quite.
This is a standard architectural split: working memory is the live, in-context state for the task at hand, while long-term memory is durable storage that outlives any single session and gets selectively pulled into working memory (context) when it's relevant to a new interaction. Long-term memory doesn't eliminate the context window (B) — it still needs to be loaded into context to be used — and working memory is relevant across multi-turn tasks generally, not just single turns (C).
Q53
Agent state & memory management
Applied
An agent persists 'facts it has learned about the user' to a long-term store, but the store has no mechanism for the user to correct or delete incorrect entries. What architectural and product risk does this create?
Not quite.
Persisted memory without a correction or deletion mechanism is a real architectural gap — facts can become stale or were simply wrong to begin with, and without user control, errors compound silently across future sessions. This is also frequently a user expectation or compliance requirement (users often have a right to see and correct what's stored about them), not just a technical nicety. This risk is unrelated to tool use (C), and the answer isn't to avoid user-facts memory altogether (D is too absolute) — it's to build in correction and deletion as first-class capabilities.
Q54
Agent state & memory management
Advanced
Why might an architecture deliberately choose NOT to give an agent persistent memory across sessions, even when the technology to do so is available?
Not quite.
Statelessness is often the right architectural default, not a missing feature — it avoids an entire category of obligations (storage security, staleness, correction mechanisms, data retention/privacy compliance) that are simply unnecessary overhead if the use case is, for example, a stateless API serving independent one-off requests. This is a genuine design tradeoff (A overclaims persistent memory's universal benefit), it's entirely feasible to implement when needed (C is wrong), and the consideration applies broadly, not just to Claude Code specifically (D).
Q55
CLAUDE.md hierarchy & scope
Foundational
What is the purpose of a CLAUDE.md file in a project directory?
Not quite.
CLAUDE.md is a standing-instructions file: project conventions, architecture context, and guidance that would otherwise need repeating every session. It has nothing to do with compilation (B) or secrets storage (D, which would be a serious security anti-pattern), and it isn't auto-generated (A) — it's authored and maintained deliberately.
Q56
CLAUDE.md hierarchy & scope
Applied
A user-level CLAUDE.md says 'always write terse commit messages.' A project-level CLAUDE.md in a specific repo says 'commit messages must follow the Conventional Commits spec with a full description.' Which takes precedence when working in that repo?
Not quite.
The hierarchy runs broad-to-narrow (user, then project, then directory), with more specific scope overriding more general scope when they conflict for the context at hand. A user-wide preference is a sensible default but a project's explicit, specific requirement for that repo should take precedence. The hierarchy doesn't skip user-level entirely (B), and precedence isn't about ease (C) — it's about specificity of scope.
Q57
CLAUDE.md hierarchy & scope
Advanced
A monorepo has a root CLAUDE.md with general conventions and a CLAUDE.md inside packages/payments/ with stricter rules (no direct database writes outside a specific module). What architectural purpose does this directory-level override serve?
Not quite.
Directory-scoped CLAUDE.md files let a large, heterogeneous codebase carry context that's actually relevant to the area being worked on — a payments module's stricter constraints don't need to pollute the root file and apply confusingly to unrelated packages. This is explicitly supported (C is wrong) and it layers with, rather than replacing, broader scopes (D is wrong) — the directory file adds specificity, consistent with the general-to-specific hierarchy.
Q58
CLAUDE.md hierarchy & scope
Foundational
Which of these is most appropriate content for a project-level CLAUDE.md?
Not quite.
Good CLAUDE.md content is exactly the kind of context that prevents wrong assumptions and repeated explanation — framework identity (especially when it could be confused with something similar), how to run tests, and non-obvious decisions. A full file listing (C) is redundant with the agent's own tools, license text (A) isn't actionable project context, and a personal task list (D) isn't project-level information at all.
Q59
CLAUDE.md hierarchy & scope
Applied
A team's CLAUDE.md has grown to several thousand words covering every edge case anyone has ever hit. What is the architectural downside of an overly long CLAUDE.md?
Not quite.
CLAUDE.md content is loaded into context on every session, so an overly long file is a standing tax on context budget — space that could go toward the actual task — and in practice important instructions can get diluted among less relevant ones, lowering the chance any specific rule is actually followed. There's no fixed word limit (B) and it doesn't affect raw inference speed (D) — the cost is context budget and attention, not latency per token.
Q60
CLAUDE.md hierarchy & scope
Advanced
What is the architectural argument for keeping CLAUDE.md focused on 'what would surprise a competent engineer joining this project' rather than documenting everything?
Not quite.
This mirrors a well-known documentation principle, extended to apply to Claude Code too: a capable reader (human or model) can derive the obvious from the code itself, so the highest-value content is what can't be inferred — intentional constraints, historical context, non-obvious gotchas. This isn't exclusively a human-docs principle (B is wrong) and overlap with code comments isn't inherently forbidden (C) — the point is about where marginal documentation effort pays off most, not a strict non-overlap rule.
Q61
CLAUDE.md hierarchy & scope
Foundational
If a CLAUDE.md instruction conflicts with an explicit instruction given directly by the user in the current conversation, which generally takes precedence?
Not quite.
CLAUDE.md establishes standing defaults and context, but an explicit, specific instruction from the user in the live conversation is the most direct and current signal of intent and generally takes precedence over a standing file written in advance for the general case. CLAUDE.md isn't meant to be an immutable override the user can't adjust (B), nor does disagreement halt the session (D) — and the model doesn't ignore both signals (C), it reconciles them with the live instruction weighted more heavily.
Q62
Custom commands
Foundational
What problem do custom slash commands in .claude/commands/ solve?
Not quite.
Custom commands package a frequently-used prompt or workflow (e.g., '/review-pr', '/write-tests') into a short, reusable invocation instead of re-explaining the same multi-step instruction every time. They aren't required for basic operation (A) and serve a different purpose than CLAUDE.md's standing context (C) — commands are invoked workflows, CLAUDE.md is passive background context. Commands aren't restricted to git operations (D); they can encode any repeatable workflow.
Q63
Custom commands
Applied
A team wants a command that always runs the same five-step code review checklist against a diff. What is the architectural advantage of encoding this as a custom command over asking for it in natural language each time?
Not quite.
The value is consistency — a saved, versioned command template guarantees the same checklist every time, independent of how well any individual team member phrases their request on a given day. This isn't an API-level speed difference (B); it's a reliability and repeatability argument. Commands do support arguments (D is wrong) — that's part of what makes them reusable across different inputs.
Q64
Custom commands
Advanced
What is the architectural tradeoff of making a custom command very rigid (highly specific, many fixed steps) versus very flexible (a loose template with room for judgment)?
Not quite.
This is a genuine design tension. A rigid command is reliable for the exact repeated case it was built for but brittle when reality doesn't match the assumed pattern. A flexible one adapts better but reintroduces some of the variance that commands exist to reduce. The right balance depends on how uniform the real-world invocations actually are. Version control applies regardless of flexibility (C is wrong), and the tradeoff matters in any context, not just CI (D).
Q65
Custom commands
Foundational
Where are custom slash commands typically defined for a project, making them shareable with the whole team via version control?
Not quite.
Project-scoped commands living in .claude/commands/ within the repo means they're versioned and shared exactly like code — everyone who clones the repo gets the same commands. This is the mechanism that makes team-wide consistency (as in question 9) actually achievable, rather than each person maintaining private, divergent command sets (A), which wouldn't be shareable (B is wrong) or centrally dashboard-managed outside the codebase (C).
Q66
Custom commands
Applied
A custom command is designed to take an argument (e.g., a file path or PR number). What architectural benefit does parameterization give over a set of separate, hardcoded commands for each specific case?
Not quite.
Parameterization is the standard way to avoid command sprawl — one '/review-pr {number}' generalizes across every PR instead of needing a separate command per PR number, and if the review workflow changes, there's one place to update instead of many near-duplicates drifting out of sync. This is fully invokable within sessions (B is wrong) and isn't limited to numeric args (D) — file paths, strings, and other argument types work too.
Q67
Custom commands
Advanced
A custom command encodes a destructive workflow (e.g., force-pushing after a rebase). What architectural safeguard should accompany a command like this?
Not quite.
Convenience (a short command invocation) and risk (what the command actually does) are independent — a command is just a packaged prompt, not a different trust tier, so a destructive action triggered via a command still warrants the same safeguards (confirmation, review) it would need if typed out manually. Commands don't auto-disable destructive operations (D), and this isn't a CLAUDE.md-vs-commands distinction (C) — it's about matching safeguards to actual risk regardless of how the instruction was packaged.
Q68
Skills
Foundational
What is the architectural purpose of a 'skill' in Claude Code, as distinct from a custom command?
Not quite.
Skills are a heavier-weight unit than commands — they can bundle instructions alongside supporting scripts or reference files for a whole category of task, and are typically loaded based on relevance to what's being worked on rather than requiring an explicit, exact invocation. This is a meaningfully different mechanism from commands (B is wrong), available to any user with access to Claude Code (C is wrong), and skills complement tool use rather than replacing it (D).
Q69
Skills
Applied
A team builds a skill for 'writing database migrations following our schema conventions,' including example migrations and a validation script. What architectural advantage does bundling the script with the skill provide, versus just describing the conventions in prose?
Not quite.
Bundling an executable check with the skill adds a deterministic verification layer on top of the model's prose-guided generation — the script can catch violations objectively rather than relying purely on the model correctly interpreting and following written conventions every single time. Scripts are a supported part of skills (B is wrong), and this value holds regardless of whether human reviewers also exist (C) — it's a complementary safeguard, not a replacement for review.
Q70
Skills
Advanced
What architectural risk exists if a skill's bundled reference material becomes outdated (e.g., it documents an API that has since changed)?
Not quite.
A skill carries the same staleness risk as any documentation — because it's presented as deliberate, authoritative guidance, the model has every reason to trust and follow it, so an outdated skill can actively mislead rather than simply being unhelpful. This isn't an automatically-solved problem (A) and doesn't depend on whether the skill includes scripts (B); skills, like CLAUDE.md, require upkeep. Outdated skills don't cause a hard failure (D) — the risk is silent, confident incorrectness, which is arguably worse.
Q71
Skills
Foundational
Which scenario is a better fit for a skill than for inline CLAUDE.md instructions?
Not quite.
Skills earn their complexity for substantial, self-contained capabilities with real supporting material — a one-line fact (A) is exactly what belongs in lightweight CLAUDE.md context instead, not a skill. Skills don't universally replace CLAUDE.md (C) — they serve different weights of content. There's no 'read once per week' cadence concept for either mechanism (D).
Q72
Skills
Applied
A skill is scoped to activate only when working with files matching a specific pattern (e.g., files under /migrations/). What architectural benefit does this scoping provide?
Not quite.
Relevance-scoped activation is a context-budget and focus optimization, similar in spirit to the directory-scoped CLAUDE.md discussion — loading a migrations-specific skill while editing an unrelated frontend component would be both wasted context and a potential source of confused, misapplied guidance. This scoping is a real, supported capability (C is wrong) and isn't restricted by implementation language (D) — it's about which files are being worked on, not the skill's own language.
Q73
Skills
Advanced
How should an architect decide whether a piece of reusable project knowledge belongs in CLAUDE.md, a custom command, or a skill?
Not quite.
Each mechanism fits a different content shape and invocation model: CLAUDE.md is ambient and passive, commands are explicit and lightweight, skills are substantial and relevance-triggered. Treating them as interchangeable (A) or always defaulting to one regardless of fit (B, C) wastes each mechanism's actual strength — a one-line fact doesn't need a skill's overhead, and a multi-file capability doesn't belong crammed into CLAUDE.md.
Q74
Plan mode vs execution mode
Foundational
What is the architectural purpose of plan mode in Claude Code?
Not quite.
Plan mode's value is the same as the planner-executor separation discussed in agentic architecture: it surfaces an inspectable, reviewable plan before anything happens, so a human can catch a misunderstanding or wrong approach before irreversible actions are taken. It isn't about speed (A), isn't mandatory for every task (B), and doesn't disable tool use for the rest of the session (C) — it gates the transition into execution, not tool use generally.
Q75
Plan mode vs execution mode
Applied
For which kind of task is plan mode most architecturally valuable?
Not quite.
Plan mode's value scales with the stakes and ambiguity of the task — for a large, multi-file refactor where the approach is itself a real decision, catching a wrong direction before execution saves significant wasted work. For a trivial fix (B) or a read-only action (D), the review overhead doesn't buy much, since there's little ambiguity or risk to catch. Value is not uniform across task types (C is wrong).
Q76
Plan mode vs execution mode
Advanced
A team routinely accepts plans without actually reading them, just to get past the plan-mode checkpoint faster. What does this reveal about the gate's design or usage?
Not quite.
A review gate only provides value if the review is real — rubber-stamping defeats the entire purpose while preserving the appearance of oversight, which can be worse than no gate at all (false confidence). This is a genuine failure mode to watch for, not evidence the gate works fine regardless (A) or evidence it should be scrapped (C, which conflates a usage problem with a design flaw) — nor is unread approval the intended behavior (D). The fix is addressing why reviewers aren't engaging (task too frequent, plans too long, stakes perceived as low), not removing the safeguard.
Q77
Plan mode vs execution mode
Foundational
What happens to tool calls that would modify files or execute commands while plan mode is active?
Not quite.
Plan mode's mechanism is specifically to allow investigation (reading files, searching, understanding) while withholding side-effecting actions until the plan is approved — this is what makes the checkpoint meaningful rather than just descriptive. Full normal execution (A) would defeat the purpose entirely, and silent discarding (B) would lose the agent's proposed actions rather than holding them for approval. The restriction applies to side effects generally, not just network calls specifically (D).
Q78
Plan mode vs execution mode
Applied
Claude Code for Development
An engineer wants quick, low-friction iteration for routine changes (adding a test case, fixing a lint error) but also wants a review checkpoint for larger changes. How should this be architected rather than picking one mode globally?
Not quite.
This mirrors the hard-gate/soft-gate tradeoff from agentic architecture: the review checkpoint should be proportional to risk and stakes, not applied uniformly regardless of task size. Routine, low-risk changes benefit from low-friction direct execution; larger or ambiguous changes benefit from the plan-review checkpoint. Forcing plan mode universally (A) adds friction with no benefit for trivial changes; never using it (B) loses the safeguard exactly when it matters most; random alternation (C) has no principled basis at all.
Q79
Plan mode vs execution mode
Advanced
In a CI/CD context where no human is present to review a plan interactively, what is the architectural equivalent of plan-mode's safety benefit?
Not quite.
The underlying need plan mode serves — a checkpoint before consequential action — doesn't disappear in CI just because a human isn't watching interactively; it gets implemented as automated, programmatic checks instead (test gates, diff-size limits, protected-path restrictions) that serve the same risk-reduction purpose without requiring a human in the loop for every run. Skipping the safeguard entirely (B) or restricting CI to only trivial tasks (D) both abandon the goal rather than adapting the mechanism; running the same unattended step twice (C) doesn't add a genuine independent check.
Q80
Hooks
Foundational
What is the architectural role of hooks in Claude Code?
Not quite.
Hooks are deterministic intercept points in the loop — they let you attach custom logic (validation, logging, blocking) to specific lifecycle events, inserting reliable, non-probabilistic control into a system that is otherwise driven by model judgment. They're functional, not cosmetic (A), they complement rather than replace permissions (C), and their use cases go well beyond notifications (D) — validation and enforcement are central use cases.
Q81
Hooks
Applied
A team wants to guarantee that no commit ever includes a hardcoded API key, regardless of what the model decides. What is the most architecturally sound way to enforce this?
Not quite.
This is the same principle as the refund-cap tool-boundary question: a hard requirement with real consequences belongs in a deterministic control, not a prompt-level request. A pre-commit hook scanning for secret patterns blocks the action structurally regardless of the model's judgment on a given run. A CLAUDE.md instruction (A) is probabilistic, relying on training behavior alone (B) provides no guarantee, and after-the-fact manual review (D) doesn't prevent the leak from happening and being briefly exposed in the first place.
Q82
Hooks
Advanced
A pre-tool-call hook is designed to validate every proposed file edit against a style guide before allowing it to proceed. What is the architectural risk if this hook's validation logic itself has a bug that occasionally lets invalid edits through?
Not quite.
Hooks are custom code and inherit all the usual risks of custom code, including bugs — and a safeguard with a quiet bug is particularly dangerous because it creates false confidence: the team believes the check is catching problems when it sometimes isn't. This argues for testing hook logic itself, not just trusting it by construction (A is wrong). It isn't a language-specific risk (B), and hooks absolutely can validate and block, not just log (C) — blocking invalid actions is a core use case.
Q83
Hooks
Foundational
What is the difference between a hook that runs 'before' an action versus one that runs 'after' an action?
Not quite.
Timing determines what's architecturally possible: a before-hook sits in the causal path and can prevent the action (as in the secret-scanning example), while an after-hook observes something that has already happened and can react to it (logging, notifications, triggering a follow-up) but has already lost the chance to block the original action. This is a meaningful distinction (B is wrong), unrelated to raw execution speed (C), and before-hooks are explicitly suited to blocking, not just logging (D is backwards).
Q84
Hooks
Applied
A team adds a hook that runs an expensive, slow validation (a full test suite) before every single file edit, not just before commits. What architectural problem does this likely cause?
Not quite.
This is a granularity mismatch: matching the cost of a check to the frequency of the event it's attached to matters architecturally, not just practically. An expensive check firing on every micro-edit creates enough friction that teams often end up disabling the safeguard altogether — a slower but more frequently-triggered hook can paradoxically provide less real protection than a well-placed one at a more natural checkpoint (like pre-commit). Hooks can run arbitrary logic including test suites (B is wrong), and the core problem exists independent of flakiness (D) — it's about frequency versus cost, not reliability of the check itself.
Q85
Hooks
Advanced
What is the architectural distinction between using a hook to enforce a rule versus using a CLAUDE.md instruction to request the same behavior?
Not quite.
This is the probabilistic-versus-deterministic distinction that recurs throughout tool and policy design in this domain: a hook is code that runs regardless of what the model 'decided,' while a CLAUDE.md instruction is guidance the model is likely to follow but isn't guaranteed to, especially under edge cases or long sessions where earlier instructions compete for attention. The two aren't interchangeable (A) or reversed in reliability (C) — and they're commonly used together (D is wrong), with hooks handling hard requirements and CLAUDE.md handling softer guidance and context.
Q86
Permissions & tool access control
Foundational
What is the architectural purpose of Claude Code's permission system for tool and command access?
Not quite.
The permission system is a genuine control surface — it defines what the agent can do autonomously versus what needs a human nod first, directly shaping blast radius and risk exposure. This has real security function (B is wrong), applies to writes and commands specifically, often more than reads (C is backwards — reads are typically lower-risk and less gated), and is a distinct mechanism from CLAUDE.md's passive context (D) — permissions are an enforcement layer, not guidance.
Q87
Permissions & tool access control
Applied
A project grants blanket approval for all bash commands to speed up iteration. What architectural risk does this introduce?
Not quite.
Removing the approval gate for an entire broad category (all bash commands) removes the checkpoint for the dangerous subset along with the convenient subset — there's no way to get the speed benefit for safe commands without also losing protection against destructive ones, unless the approval is scoped more narrowly. Bash commands are a primary, well-supported target for permission gating (B is wrong), and the risk is real in local development too (C) — accidental destructive commands aren't limited to production.
Q88
Permissions & tool access control
Advanced
What is the architectural advantage of scoping permissions narrowly (e.g., auto-approve `npm test` and `npm run lint` specifically, but require approval for all other bash commands) rather than choosing between 'approve everything' and 'approve nothing'?
Not quite.
Fine-grained, allow-listed permissions avoid the false binary between full friction and full exposure — known-safe, frequently-needed commands run without interrupting flow, while the approval gate stays in place for anything outside that explicit list, including commands nobody anticipated needing to gate individually. This is a different (and generally better) risk profile than either extreme (A is wrong), applies to more than read-only commands (C — test and lint commands can have side effects), and is independent of hook configuration (D).
Q89
Permissions & tool access control
Foundational
Why might network access (e.g., making arbitrary HTTP requests) warrant stricter default permissions than local file reads within the project directory?
Not quite.
Risk-appropriate default permissions track the actual blast radius of an action category. Local file reads within the project are contained within an already-trusted boundary; network requests can reach arbitrary external systems, potentially exfiltrating data or triggering real side effects well outside that boundary. Treating all tool categories identically (A) ignores this real difference, network access can absolutely be gated (C is wrong), and the risk ordering isn't reversed (D) — reads of local, trusted content are typically lower risk than arbitrary outbound requests.
Q90
Permissions & tool access control
Applied
A permission policy is defined once at the start of a project and never revisited, even as the codebase and its risk profile evolve. What architectural problem does this create?
Not quite.
Permissions, like the confidence thresholds discussed in agentic architecture, aren't a set-once artifact — they should be revisited as the system they govern changes. A codebase that grows a new high-stakes area (like payments) after the original policy was set needs that policy to evolve too, or the access controls quietly stop matching the real risk surface. Policies are fully mutable (C is wrong), and this consideration applies to any project with evolving scope, not just open-source ones (D).
Q91
Headless/CI execution modes
Foundational
What does 'headless' execution mean in the context of running Claude Code in a CI/CD pipeline?
Not quite.
Headless mode is about the absence of an interactive human-in-the-loop UI, not about hardware (B, which is an unrelated meaning of 'headless' from browser automation) or the absence of a model (A). It runs autonomously end to end based on whatever permissions and success criteria were configured in advance — which is exactly why those criteria need to be well-defined upfront, as discussed in the CI termination-condition question. Logging is typically still present and often more important in headless mode, not disabled (C).
Q92
Headless/CI execution modes
Applied
Claude Code in CI/CD
A headless Claude Code run in CI is given the same broad tool permissions it would have in an interactive local session. What architectural adjustment should be made for the unattended context?
Not quite.
The presence or absence of a human-in-the-loop materially changes the right permission posture — actions that relied on a human catching problems interactively need either tighter restriction or an automated substitute (as discussed in the plan-mode-in-CI question) when no human is present to intervene in real time. Loosening permissions further in an unattended context (B) is backwards — less oversight should generally mean more caution, not less. Permission systems are fully usable in CI (D is wrong), and treating interactive and headless contexts identically (A) ignores a real difference in risk.
Q93
Headless/CI execution modes
Advanced
A headless CI run completes 'successfully' according to its own exit code, but the change it made is actually wrong in a way the automated success criteria didn't catch. What does this reveal about designing success criteria for unattended runs?
Not quite.
This surfaces the gap between 'the automated check passed' and 'the result is actually correct' — success criteria are a proxy for correctness, and a proxy is only as good as its coverage. If tests don't exercise the specific failure mode, passing tests doesn't guarantee correctness. The architectural response is improving criteria coverage proportional to task risk, or layering in human review for higher-stakes headless changes — not abandoning headless execution altogether (C) or distrusting the exit code mechanism itself (D), which is a reliable signal for what it actually measures.
Q94
Headless/CI execution modes
Foundational
Why is detailed logging particularly important in headless execution, more so than might be strictly necessary in an interactive session?
Not quite.
Without a human watching in real time, the log becomes the only window into what the agent actually did and why — there's no other way to reconstruct the session afterward for debugging or audit. This makes logging quality materially more load-bearing in headless contexts than in interactive ones, where a human's live observation provides a redundant source of insight. Headless mode fully supports logging (C is wrong), and its value goes well beyond billing (D) — it's the primary debugging and audit tool for unattended runs.
Q95
Tool-use/schema structured output
Foundational
Structured Data Extraction
You need Claude to reliably return JSON with a fixed set of fields for downstream parsing. What is the more reliable mechanism: asking for JSON in the prompt, or defining a tool with an input schema and forcing a tool call?
Not quite.
Tool-use with a defined input schema constrains output structurally — the model is filling in a schema's slots rather than free-form generating text that happens to look like JSON, which is far more reliable, especially for nested or strict structures. A prose request for 'please return JSON' (A) is easier to write but more prone to formatting drift, extra commentary, or malformed output. Structured output is a well-supported, reliable pattern (C is wrong), and schema-based tool use handles nesting well (D is wrong).
Q96
Tool-use/schema structured output
Applied
A structured-output schema has a field `status` that must be exactly one of "approved", "rejected", or "pending". What is the best way to constrain this in the schema?
Not quite.
Schema-level enums are the correct mechanism for a closed, known set of valid values — it's structural, not advisory, the same principle as question 1 applied one level deeper. A free-text field (B) depends on the model happening to stick to the implied set. Three unrelated booleans (C) loses the mutual-exclusivity constraint entirely and allows invalid combinations (all three true, or all false). Schemas can absolutely constrain to enums (D is wrong) — this is a standard, well-supported capability.
Q97
Tool-use/schema structured output
Advanced
A schema requires an array of 'line items,' each with a required `amount` field, but the model occasionally returns an item with amount missing. What two complementary layers should handle this?
Not quite.
Schemas strongly bias output toward compliance but don't provide an absolute guarantee — the robust pattern is defense in depth: mark the field required in the schema (raising the odds it's included) and still validate programmatically on receipt (catching the cases where it wasn't), treating failures as a condition to handle rather than assuming success. A required field alone with no validation (C) trusts the schema too much; validation with no schema constraint (B) leaves the model under-guided in the first place; prompt reminders alone (A) are the weakest, most probabilistic layer.
Q98
Tool-use/schema structured output
Foundational
What is the architectural benefit of using descriptive field names and descriptions within a structured-output schema (e.g., a `description` property on each field), rather than terse, unexplained field names?
Not quite.
A schema's field names and descriptions are read by the model as part of understanding what's being asked — this is directly analogous to the MCP tool-description-as-interface-contract principle: clear, well-described fields reduce ambiguity about what should go where, the same way a terse, unexplained API parameter is more likely to be misused than a well-documented one. This value isn't cosmetic (A) or size-gated (B), and well-described schemas don't meaningfully affect processing speed (D) — the benefit is accuracy, not performance.
Q99
Tool-use/schema structured output
Applied
A schema includes an optional `notes` field with no description, intended for free-form commentary. The model frequently leaves it populated with content that duplicates other structured fields. What is the likely cause and fix?
Not quite.
This is the same principle as question 4 applied to a concrete failure: an underspecified field invites the model to fill it with its own interpretation of 'reasonable content,' which often means restating what's already captured elsewhere. A tighter, more specific description narrows that interpretation. Optional fields are a legitimate and common schema pattern (C is wrong), and the fix is improving the description rather than removing the field outright (D) — the field's purpose is presumably still needed, just under-specified.
Q100
Tool-use/schema structured output
Advanced
Why might forcing structured tool-use output sometimes reduce output quality compared to allowing the model to reason in free text first, for a task requiring genuine multi-step analysis?
Not quite.
This is a real and important tradeoff: structured output's strength (constraining the final shape) can become a weakness if it forces the model to skip reasoning that genuinely complex analysis needs. The common mitigation is designing the schema or flow to allow reasoning first (e.g., an explicit `reasoning` field before the `conclusion` field, or a separate reasoning pass before the structured extraction call) — structure and reasoning aren't mutually exclusive, but a naively designed schema that only has final-answer fields can inadvertently suppress useful intermediate work. Structured output is not universally superior regardless of task (B), is used well beyond simple extractions (C), and this tradeoff isn't model-version-specific (D).
Q101
Tool-use/schema structured output
Foundational
What should happen when a received structured output fails schema validation (e.g., a required field is missing or has the wrong type)?
Not quite.
Schema validation failure is exactly the kind of structured error the earlier error-handling domain discussed — it should be treated as an actionable condition (retry, possibly with added context about what went wrong, or escalate) rather than silently patched over with defaults (A, which hides a real problem and can corrupt downstream logic) or allowed to crash the whole application ungracefully (B). Schemas reduce but don't eliminate the chance of invalid output (C is wrong, consistent with question 3) — validation is still a necessary runtime check.
Q102
Tool-use/schema structured output
Applied
A team is extracting structured data from unstructured customer emails using a tool-use schema. Some emails genuinely don't contain a value for an optional field (e.g., no phone number was mentioned). How should the schema and downstream logic handle this, as distinct from a case where the model failed to extract a value that was actually present?
Not quite.
These are semantically different situations that deserve different handling: a genuinely absent value in the source (correctly represented as null/not-present) is a successful extraction, while a missed value that was actually there is an extraction failure. Collapsing both into the same representation (A, D) loses information downstream consumers may need — e.g., for quality monitoring, you'd want to know the real miss rate, not inflate it with legitimately-absent data. Optional/nullable fields are the right schema tool for genuinely absent data (B is wrong) — removing them doesn't solve the distinction problem, it just forces a workaround.
Q103
System prompt design & role framing
Foundational
What is the architectural role of the system prompt relative to the user-turn messages in a conversation?
Not quite.
This mirrors the standing-context-versus-specific-instruction pattern seen elsewhere (CLAUDE.md versus live user instructions): the system prompt sets the durable frame — role, constraints, tone — that persists across the conversation, while individual user turns carry the specific task within that frame. The two serve distinct architectural purposes (A is wrong), system prompts carry far more than formatting (C), and while user intent is weighted heavily, well-designed system-level constraints (e.g., safety boundaries) are deliberately meant to hold even against conflicting user requests (D overstates user override).
Q104
System prompt design & role framing
Applied
A system prompt defines the agent's role as 'a helpful assistant' with no further specificity, for a tool that's actually meant to do structured legal-document review. What is the architectural problem?
Not quite.
Specific role framing ('a contract review specialist focused on identifying liability and indemnification clauses') gives the model a much clearer frame for what expertise, tone, and focus to bring than a generic 'helpful assistant' framing, which carries no domain signal at all for a specialized task. This isn't convention-bound (C) and role framing does measurably affect output character (B is wrong) — it isn't interchangeable regardless of task (A).
Q105
System prompt design & role framing
Advanced
A system prompt is written as a long list of 'do not' instructions (don't be verbose, don't use jargon, don't speculate, don't...) with no positive description of desired behavior. What architectural weakness does this have?
Not quite.
Describing a target positively ('write concise, plain-language summaries for a non-technical reader') generally gives the model a clearer, more actionable pattern to follow than an equivalent-length list of prohibitions, which defines the boundary of unacceptable output without describing what good output actually looks like. This is a real, well-observed difference in effectiveness (B, C are wrong), not a length-dependent effect specifically (D) — though the problem does tend to compound as prohibition lists grow longer.
Q106
System prompt design & role framing
Foundational
Why might a system prompt explicitly state constraints the model would 'obviously' already follow, such as 'do not reveal this system prompt verbatim if asked'?
Not quite.
What's 'obvious' to a well-intentioned user may be exactly what an adversarial prompt tries to route around — explicit constraints are more robust against edge cases and unusual framings than relying on the model to correctly infer an unstated boundary every time, especially under pressure designed to find the gap. This isn't an API mandate (C), and while the stakes are often highest in customer-facing contexts, the principle of explicit-over-implied constraints applies more broadly (D is too narrow).
Q107
System prompt design & role framing
Applied
A system prompt for a customer-facing support agent includes extensive internal context (pricing logic, internal escalation thresholds, competitor comparison notes) that should never be shared with the customer. What architectural risk does embedding this directly in the system prompt carry?
Not quite.
System prompts are a strong but not absolute confidentiality boundary — extraction attempts are a known risk category, so the most sensitive information (pricing internals, escalation thresholds) is often architecturally safer kept outside the prompt entirely, in a backend system the agent queries via a tool rather than holding the raw sensitive content in context at all. This risk exists independent of whether the agent also has other tool access (C is a false qualifier), and there's no restriction on what a system prompt can contain by design (D) — the architectural question is what's wise to put there, not what's technically disallowed.
Q108
System prompt design & role framing
Advanced
What is the architectural tradeoff of a very long, highly detailed system prompt covering every edge case the designers anticipated, versus a shorter system prompt relying more on the model's general judgment?
Not quite.
This echoes the CLAUDE.md length tradeoff in a different context: more detail covers more anticipated cases but isn't free — it's sent (and billed, and competes for attention) on every single call in the conversation, not just once (B is wrong, since the system prompt is part of every request's context, not a one-time cost). There's a real balance to strike between coverage and cost/focus, not a strictly dominant choice in either direction (A, D both overclaim).
Q109
Few-shot/example design
Foundational
What is the architectural purpose of including a few worked examples (few-shot examples) in a prompt, rather than relying on instructions alone?
Not quite.
Few-shot examples work because showing a concrete pattern often resolves ambiguity that prose description leaves open — exactly how terse should a summary be, exactly what counts as 'relevant,' exactly how should an edge case be formatted. This complements rather than replaces system-prompt framing (D is wrong) and generalizes well beyond classification to generation, extraction, formatting, and more (C is wrong) — and the effect is real, not negligible (B is wrong).
Q110
Few-shot/example design
Applied
A prompt includes three few-shot examples, all drawn from the same narrow case (short, simple customer complaints with an obvious resolution). In production, the task also includes long, ambiguous, multi-issue complaints. What architectural problem is likely?
Not quite.
Examples implicitly communicate 'this is the shape of input and output to expect,' so a narrow, unrepresentative set of examples can bias the model toward handling only that narrow shape well, under-preparing it for the actual range of inputs it will see in production. The fix is choosing examples that span the real difficulty and variety of the task (including at least one harder, more ambiguous case), not removing examples altogether (D) — well-chosen examples still provide real value per question 15.
Q111
Few-shot/example design
Advanced
Why might including one deliberately imperfect or tricky example (with a brief note on how to correctly handle it) be more valuable than three straightforward, easy examples?
Not quite.
Examples are most valuable where they resolve genuine ambiguity — three easy examples largely reinforce what the model would likely do correctly anyway, while one well-chosen hard example (with guidance on the right call) directly addresses the actual failure mode worth preventing. This isn't about avoiding difficulty (B is wrong, and overclaims risk) — it's about example selection being as deliberate as the rest of prompt design, and the principle isn't limited to extended-thinking contexts (D).
Q112
Few-shot/example design
Foundational
What risk does hardcoding a real customer's personal data into a few-shot example carry, beyond the immediate prompt design concern?
Not quite.
Real personal data in a prompt template isn't just a onetime input — it persists wherever that prompt lives (version control, logs, possibly eval sets), which is a genuine privacy and compliance exposure independent of how well-designed the example otherwise is. The standard mitigation is synthetic or clearly fictional data that still demonstrates the pattern. This risk is just as real in development and test environments (B is wrong) as production, and fictional-looking data that illustrates the pattern is entirely fine to use (C is wrong) — the problem is specifically real personal data, not personal-data-shaped examples in general.
Q113
Few-shot/example design
Applied
A team maintains few-shot examples as hardcoded strings directly inline in application code, scattered across several files. As the examples need updating, this becomes error-prone. What architectural improvement addresses this?
Not quite.
This is a standard content-management lesson applied to prompts: scattered, duplicated content is hard to maintain consistently, and the fix is centralizing it as organized, versioned data that the application references, rather than embedding copies throughout the codebase. Examples do need updating as task understanding evolves, often more than expected (B is wrong), reuse across related prompts is often beneficial, not forbidden (C), and this is squarely a code-organization problem with a code-organization fix (D is wrong).
Q114
Few-shot/example design
Advanced
In what situation might zero-shot prompting (no examples, just clear instructions) be architecturally preferable to few-shot prompting?
Not quite.
Few-shot isn't free — examples cost context budget and, if not carefully chosen, risk narrowing behavior as shown in question 16. For tasks that are simple enough to specify clearly in instructions, and varied enough that no small example set would safely represent the full range, zero-shot can be the better-calibrated choice, deliberately saving the context cost and avoiding the narrowing risk. Few-shot is not unconditionally superior (B), zero-shot works across many task types (C is wrong), and it's fully supported (D is wrong) — this is a genuine judgment call, not a fixed rule.
Q115
Chain-of-thought & extended thinking usage
Foundational
What architectural purpose does extended thinking (allowing the model dedicated space to reason before producing a final answer) serve for genuinely complex tasks?
Not quite.
Extended thinking gives the model deliberate space to reason step by step, explore an approach, and potentially self-correct before finalizing an answer — this matters most for tasks where the 'obvious first answer' risks being wrong in ways that careful reasoning would catch. It isn't mandatory for simple tasks (B is wrong, and would be wasteful) and it affects substantive reasoning, not just formatting (C is wrong) — the effect on quality for hard tasks is real, not negligible (A is wrong).
Q116
Chain-of-thought & extended thinking usage
Applied
A team enables extended thinking for every single API call in their application, including simple, high-volume classification calls with an obvious answer. What is the architectural downside?
Not quite.
Extended thinking has real cost and latency implications, so applying it unconditionally to simple, high-volume, obvious-answer tasks pays overhead without a corresponding quality benefit. The architecturally sound approach matches the setting to task complexity — reserving deeper reasoning budget for genuinely hard calls (echoing the proportional-overhead theme that recurs throughout this domain, e.g., permissions and plan mode). This isn't a one-call-per-session limitation (B), and the waste is real even without a strict SLA (D) — it's unnecessary cost regardless of whether latency is formally constrained.
Q117
Chain-of-thought & extended thinking usage
Advanced
A system displays the model's extended thinking output directly to end users as 'the AI's reasoning,' verbatim and unfiltered. What architectural and product risk does this carry?
Not quite.
Extended thinking is exploratory, working reasoning — it may include false starts, self-correction, or informal internal framing that's useful as process but confusing or alarming as a final-facing artifact, since users generally expect a considered answer, not a transcript of the model changing its mind mid-thought. This isn't universally safe to show raw (A overclaims), it's not a regulatory mandate (C), and it is technically possible to surface (D is wrong) — the question is whether doing so raw is wise, not whether it's possible.
Q118
Chain-of-thought & extended thinking usage
Foundational
What distinguishes a task well-suited to extended thinking from one that is not?
Not quite.
The right lens is reasoning complexity, not surface-level task length (B is wrong) or domain (C, D are both too narrow). Tasks where a snap judgment is likely wrong — because they require weighing several factors, multi-step logic, or careful analysis — benefit from deliberate reasoning space. Tasks with one clear, simple answer (a straightforward lookup or format conversion) don't gain much from extra reasoning, regardless of domain.
Q119
Chain-of-thought & extended thinking usage
Applied
A coding agent uses extended thinking before making significant architectural decisions in a codebase, but not before trivial changes like fixing a typo. What design principle does this reflect?
Not quite.
This is the same proportional-overhead principle recurring throughout the domain (plan mode for big changes, tight permissions for risky actions): investing more deliberate reasoning where stakes and complexity are higher, and skipping that overhead where it wouldn't change the outcome, is a coherent, well-justified policy, not arbitrary inconsistency (A). It applies to generation decisions directly, not just review (C is too narrow), and there's no technical restriction preventing extended thinking on trivial tasks (C) — it's simply not worth the cost there.
Q120
Chain-of-thought & extended thinking usage
Advanced
Why might a system use a structured-output call (final answer only) immediately after an extended-thinking pass, rather than trying to get both reasoning and final structured output from a single undifferentiated call?
Not quite.
This connects back to the tradeoff in question 6 (structured output can suppress useful reasoning if forced too early) — splitting into a reasoning pass followed by a structured extraction pass lets the model think freely first, unconstrained by schema, then commit to the final shape as a distinct step once the reasoning is done. This is a common, practical pattern (A is wrong), fully supported technically (B is wrong), and applies broadly, not just to math tasks (D).
Q121
Prompt chaining & decomposition
Foundational
What is prompt chaining, as an architectural technique?
Not quite.
Prompt chaining decomposes a complex task into a sequence of smaller, focused steps, each building on the last — this is conceptually the sequential-pipeline pattern from agentic architecture, applied at the prompt level even without full multi-agent orchestration. It isn't about repeating the same prompt (A) or merging unrelated tasks to save calls (D, which would hurt focus for both tasks), and it's a general prompting technique, not Batches-API-specific (C).
Q122
Prompt chaining & decomposition
Applied
A single prompt asks the model to simultaneously translate a document, summarize it, and extract key entities, all in one pass. Output quality on all three sub-tasks is mediocre. What architectural change is most likely to help?
Not quite.
Asking for three substantively different outputs simultaneously splits the model's effective focus and can degrade all three compared to giving full attention to one task at a time — decomposing into a chain, where each step is singularly focused, generally outperforms a crowded single-pass attempt. Adding more detail to the same crowded single prompt (B) doesn't address the fundamental issue of divided focus, and reducing to one sentence (C) would likely make things worse by removing necessary detail. This pattern is a known, addressable issue, not an immutable limitation (D is wrong).
Q123
Prompt chaining & decomposition
Advanced
What is the architectural cost of prompt chaining that must be weighed against its quality benefits?
Not quite.
This is the prompt-level analog of the sequential agent pipeline tradeoff: more steps means more latency (sequential, not parallel, so it adds up) and more cost, plus more opportunities for an early error to propagate uncorrected into later steps unless validation is deliberately added between them. It's not strictly free (A is wrong), doesn't require different models per step (C — the same model can handle each focused step), and definitely doesn't eliminate the need for error handling (D is backwards — it often increases the need for it, given more steps where things can go wrong).
Q124
Prompt chaining & decomposition
Foundational
In a prompt chain, how should the output of one step typically be passed to the next, to keep the chain reliable?
Not quite.
This mirrors the context-passing-between-agents principles from earlier in the exam domain structure: structured, purposeful handoffs between chain steps are more reliable than unstructured free text the next step has to parse loosely (A), or indiscriminately passing full raw history (C) regardless of relevance. Chained steps absolutely do exchange information by design — that's the entire point of a chain (D is wrong).
Q125
Prompt chaining & decomposition
Applied
A three-step chain (draft, critique, revise) is used to improve the quality of generated content. What architectural purpose does the middle 'critique' step serve specifically?
Not quite.
A dedicated critique step is valuable precisely because generation and evaluation are different cognitive modes — a model (like a person) often catches issues in a dedicated review pass that it didn't catch while initially generating content, since review involves actively checking against criteria rather than producing fresh content. This isn't limited to grammar (C is far too narrow — it can assess substance, logic, completeness, tone) and doesn't require different models for each step (D) — the same model performing each distinct role in sequence is the standard pattern.
Q126
Prompt chaining & decomposition
Advanced
When should a team choose prompt chaining (multiple sequential calls to the same model/agent) over spinning up a proper multi-agent architecture with separate specialized agents?
Not quite.
Chaining and multi-agent delegation sit on a complexity spectrum — chaining gets much of the decomposition and focus benefit (see question 28) without the overhead of full agent isolation, separate toolsets, and coordination logic. It's the right choice when steps are sequential and don't need the stronger isolation or specialized access that justifies true multi-agent architecture (echoing the single-vs-multi-agent decision domain). They are related but distinct techniques (B is wrong), and chaining is often the more cost-conscious choice, not a technique reserved for when cost doesn't matter (D is backwards).
Q127
Validation-retry loops
Foundational
What is a validation-retry loop, in the context of structured data extraction?
Not quite.
A validation-retry loop is the practical implementation of the 'don't trust output blindly, validate and act on failure' principle discussed in structured-output design — check the output, and if it fails, feed specific, actionable feedback back to the model for another attempt, rather than either accepting flawed output or requiring human intervention for every single failure (D, which doesn't scale). It isn't numeric-only (C) and doesn't guarantee first-attempt success by definition (B) — the whole point is handling the cases where it doesn't.
Q128
Validation-retry loops
Applied
A validation-retry loop gives the model generic feedback on failure: 'that was wrong, try again,' with no specifics. How does this compare to feedback that specifies exactly what validation rule failed and why?
Not quite.
This mirrors the tool-error-message principle from agentic architecture (structured error responses beat generic 'an error occurred'): specific, actionable feedback gives the model something concrete to fix, while vague feedback forces a guess, which is prone to missing the actual issue or introducing a new one. The marginal token cost of specific feedback (C) is almost always worth the improved retry success rate (D is wrong) — and the model generally cannot reliably infer the exact failure from no information (A is wrong).
Q129
Validation-retry loops
Advanced
A validation-retry loop has no maximum retry count, so a persistently malformed input can cause indefinite retries. How does this relate to the agent-loop termination principles discussed elsewhere in this exam's scope?
Not quite.
This is the same structural lesson from agent loop termination, applied specifically to validation-retry: any loop without an engineered stopping condition risks running indefinitely against an input that will never validate. The fix is identical in spirit — a retry cap, after which the system escalates or fails explicitly rather than looping forever. The loops aren't naturally self-limiting (B is wrong) and removing validation (D) would reintroduce the original problem the loop exists to solve.
Q130
Validation-retry loops
Foundational
What metric would be most useful for monitoring the health of a validation-retry system in production over time?
Not quite.
Tracking first-attempt pass rate and retry distribution over time surfaces regressions early — a declining pass rate or growing retry counts is an actionable signal that something changed (inputs, schema, or prompt) even while the overall final success rate might still look fine because retries are quietly compensating. Looking only at final success (C) or raw call volume (A) would miss this degradation until it got bad enough to actually fail outright. Monitoring remains valuable precisely because a loop can mask a growing underlying problem (B is wrong) — self-correction isn't the same as the system staying healthy.
Q131
Batch processing
Foundational
What is the primary architectural use case for the Message Batches API, as distinct from standard synchronous API calls?
Not quite.
Batches are designed for bulk, asynchronous workloads where throughput and cost efficiency at scale matter more than per-request latency — a backlog of documents to classify, not a live chat. Real-time interactive use (B) is exactly the wrong fit, since batches trade immediacy for efficiency. It's explicitly for high-volume requests, not single ones (C is backwards), and the same models are available via batch as via synchronous calls (D is wrong).
Q132
Batch processing
Applied
A team needs to classify 50,000 support tickets from the past year for a one-time analysis, with no urgency on when results arrive. What is the architectural case for using the Batches API over 50,000 individual synchronous calls?
Not quite.
This is close to the textbook batch use case: high volume, no urgency, matching the API's design intent exactly. Beyond cost savings, there's real architectural simplicity in submitting one batch job instead of building and managing rate-limiting, retry, and concurrency logic for 50,000 synchronous calls. The Batches API supports far larger volumes than 100 requests (C is wrong), and it works for any task type including classification (D is wrong) — synchronous calls aren't unconditionally preferable (A is wrong) when latency doesn't matter and volume is high.
Q133
Batch processing
Advanced
A team wants to use the Batches API for a workflow where each item's processing depends on the result of the previous item (a genuinely sequential dependency chain). What is the architectural mismatch?
Not quite.
Batch processing is built around independent items that don't need to be processed in a guaranteed order relative to each other — it's optimized for throughput on unrelated units of work, not sequential dependency chains where item N's input depends on item N-1's output. A genuinely sequential workflow is a poor fit for batching and calls for the chaining/pipeline patterns discussed earlier instead. There's no such ordering guarantee by design (D is wrong, which is part of why sequential dependencies don't fit), and the mismatch isn't a function of batch size (B is wrong) — it's about the independence assumption batches are built on.
Q134
Batch processing
Foundational
What tradeoff does a team accept when choosing the Batches API's cost efficiency over synchronous calls?
Not quite.
The core tradeoff is latency for cost efficiency — batch jobs process within a window of up to 24 hours, which is the deliberate exchange being made for the throughput and cost benefits at scale. This makes it unsuitable for latency-sensitive needs, which is exactly why the architectural decision in question 37-38 hinges on whether urgency matters. Batches don't sacrifice output quality (B is wrong) and fully support structured output (D is wrong) — the tradeoff is specifically about timing, not correctness or capability.
Q135
MCP tool description design
Foundational
Developer Productivity Tooling
What is the architectural role of a tool's description field in an MCP tool definition?
Not quite.
The description is the model's primary signal for understanding what a tool does, when to use it, and how to call it correctly — it functions as an interface contract, not passive documentation (B, C are wrong). A vague description directly causes real failures: wrong tool selection, malformed parameters, or missed use cases. Descriptions are a first-order reliability factor, not optional polish (D is wrong).
Q136
MCP tool description design
Applied
Two tools, `search_orders` and `lookup_order`, have nearly identical descriptions and overlapping apparent purpose. What problem does this create, and what's the fix?
Not quite.
Ambiguous boundaries between similar tools are a well-known source of selection errors — the model has to choose between genuinely confusable options with no reliable signal for which is correct. The standard fix is sharpening each description to make the distinction concrete and unambiguous (exact lookup vs. fuzzy search), not necessarily eliminating one tool (C) if both serve genuinely distinct, valid purposes — the problem is clarity, not redundancy per se.
Q137
MCP tool description design
Advanced
A tool description states what the tool does but omits any mention of its side effects (e.g., that calling it sends a real email to a customer). What architectural risk does this create?
Not quite.
A tool description's job includes communicating consequence, not just function — if a destructive or irreversible side effect isn't flagged, the model has no strong signal to treat the call with appropriate caution versus a safe, read-only operation, and may invoke it more casually than warranted. This isn't an inference the model can be trusted to make reliably without the signal (A is wrong), it's the opposite of a security best practice to hide side effects (B is wrong), and the risk generalizes well beyond financial tools to any consequential action (D is too narrow).
Q138
MCP tool description design
Foundational
Why should a tool's parameter descriptions specify expected formats explicitly (e.g., 'date in ISO 8601 format, YYYY-MM-DD') rather than leaving format implicit?
Not quite.
This extends the interface-contract principle down to individual parameters — an implicit or ambiguous format (is '03/04/2026' March 4th or April 3rd?) is exactly the kind of ambiguity that produces real errors. Explicit format specification closes that gap. This applies to any parameter type where format matters, including strings and dates, not just numeric values (C is too narrow), and while good practice, it isn't a hard spec requirement enforced by the protocol itself (D overstates it) — it's an architectural best practice.
Q139
MCP tool description design
Applied
A tool is named `process` with a one-line description: 'Processes the request.' What is the core architectural problem?
Not quite.
This is an extreme case of the interface-contract failure discussed throughout this subtopic — a name and description this vague gives the model essentially no signal for correct selection or invocation. Brevity isn't the problem (short descriptions can be excellent if precise); content-free vagueness is. This directly affects model behavior, not just human readability (D is wrong), and the severity isn't gated by parameter count (C) — a vague description is a problem even for a zero-parameter tool.
Q140
MCP tool description design
Advanced
A tool description includes an example of a typical call (e.g., a sample set of parameters and what the response looks like). What architectural benefit does this provide beyond the prose description alone?
Not quite.
This is the same show-versus-tell principle from few-shot prompt design, applied to tool definitions — a concrete example call resolves format and usage ambiguity especially well for complex, nested parameter structures where prose description alone tends to leave gaps. This is a real, supported technique (D is wrong) with no restriction to single-parameter tools (C is wrong) — the benefit scales with complexity, if anything.
Q141
MCP tool description design
Foundational
Who is the primary audience a tool description should be optimized for?
Not quite.
The model is the direct consumer of the tool description — it reads the description to decide when and how to call the tool, functioning analogously to how a careful developer reads API documentation before calling an endpoint. End users (A) typically never see the raw definition at all, and writing only for the original author's memory (B) ignores that the description's real job is guiding correct invocation by the model, every single call, not just serving as a note to self.
Q142
Tool error response design
Foundational
What should a well-designed tool error response include, beyond a bare failure indication?
Not quite.
This is the structured-error principle established in agentic architecture, specified concretely here: category, retryable flag, actionable message — information the agent can actually act on, not just a bare failure signal (A) which leaves it guessing. A raw stack trace (B) is typically unhelpful noise for the model and can leak implementation details inappropriately; errors shouldn't be undifferentiated (C) since different failures call for different agent responses.
Q143
Tool error response design
Applied
A tool's error response for 'insufficient permissions' and 'resource not found' both return the identical generic message 'operation failed.' Why is this problematic?
Not quite.
These failure modes call for genuinely different recovery strategies — a permissions problem needs a different human escalation path than a not-found problem, which might just need a corrected identifier. Collapsing them into one undifferentiated message removes the agent's ability to choose the right response (B is wrong — the correct next action does differ by cause). This applies regardless of how many agents consume the tool (C), and overly generic error messages reduce the agent's ability to recover correctly — that's a reliability cost, not a security benefit in most cases (D oversimplifies; specific error categories, as opposed to leaking raw internals, are generally fine and helpful).
Q144
Tool error response design
Advanced
A tool call fails with a 'rate limited, retryable' error. What should the retryable flag enable the calling agent (or its orchestration layer) to do differently than if the error were marked non-retryable?
Not quite.
This directly connects tool error design to the retry-strategy principles from agentic architecture: a retryable flag is the signal that tells the orchestration layer to apply backoff-and-retry logic rather than treating the failure as terminal. Treating retryable and non-retryable identically (A) wastes the signal's entire purpose, and immediately escalating regardless of the flag (B) defeats the point of distinguishing recoverable from unrecoverable failures. The principle applies to any tool, not just network-calling ones specifically (D) — rate limiting can occur in various contexts.
Q145
Tool error response design
Foundational
Why might a tool error response include a suggested next step (e.g., 'resource not found; verify the ID or use search_orders to find the correct identifier') rather than just stating what went wrong?
Not quite.
An actionable suggestion gives the agent a head start toward self-correction, improving the odds of successful recovery without requiring it to independently derive the best next move from limited information — this is a genuine reliability improvement, applicable in both interactive and headless contexts (C is too narrow). It's a strong hint, not a mandatory directive the agent must follow exactly (D overstates it) — the agent still exercises judgment about whether the suggestion fits the actual situation.
Q146
Tool error response design
Applied
A tool occasionally returns a successful-looking response (status 200, well-formed JSON) that actually represents a failure condition buried in a nested field the schema doesn't surface clearly. What architectural problem does this create?
Not quite.
A success-coded response that actually contains a buried failure is a genuine silent-failure trap — both the model and any automated validation checking the surface-level status code can be misled into treating it as success, letting a real failure propagate unnoticed. The fix is surfacing failure conditions unambiguously at the top level of the response, not buried in a field easy to miss. This is a real risk that doesn't reliably self-correct (B, D are both wrong), and it isn't protocol-specific to HTTP (C) — any tool response format can have this flaw if designed carelessly.
Q147
Tool error response design
Advanced
What is the architectural argument for using a consistent error response schema across all tools in a system, rather than letting each tool define its own ad hoc error format?
Not quite.
A shared error schema means generic error-handling logic (retry policy, escalation routing, logging) works uniformly across every tool, rather than requiring bespoke parsing logic per tool — this is a scalability argument that gets more valuable, not less, as the system grows (directly contradicting C's claim that it only matters for small systems). It's strong architectural practice rather than a hard protocol mandate with zero flexibility (C overstates it), and the benefit compounds specifically as tool count grows, the opposite of A's claim that consistency provides no benefit.
Q148
Tool scoping & least-privilege
Foundational
What does the principle of least-privilege mean when applied to agent tool access?
Not quite.
Least-privilege scopes an agent's access to exactly what its task requires, limiting blast radius if something goes wrong (a reasoning error, manipulation via adversarial content, or simply an unexpected edge case) — this is the same principle discussed for subagent delegation, generalized to all tool-access design. Granting maximal access by default (A) is the opposite of the principle, and it applies directly to AI agents, not just human accounts (B is wrong) — different agents with different roles should generally NOT have identical access (D is backwards).
Q149
Tool scoping & least-privilege
Applied
A single customer-support agent is given both a `read_customer_data` tool and a `delete_account` tool, even though its actual job is only answering questions, never deleting accounts. What is the architectural risk, and the fix?
Not quite.
This is a direct application of the MCP design principle from earlier material: distribute tools by role, and remove capabilities entirely rather than merely guard them via prompt instruction. If an agent's job never requires account deletion, the tool shouldn't be in its toolset at all — relying on the system prompt to suppress a technically-available destructive capability (A) is weaker than removing the capability structurally, especially under adversarial pressure designed to find exactly that gap. A longer confirmation phrase (C) is a weaker mitigation than removing access entirely, and the risk exists in any environment the agent operates in, not just production (D).
Q150
Tool scoping & least-privilege
Advanced
A team argues that giving every agent in their system the full set of available tools is simpler to maintain than carefully scoping access per agent. What is the counter-argument from an architecture perspective?
Not quite.
This is a real tradeoff, but not one that favors maximal access as the counter-argument claims: broad access does reduce per-agent configuration work, but it concentrates risk — any single agent's failure or manipulation now has system-wide reach instead of being contained to its actual job. The incident-severity cost of concentrated risk generally outweighs the maintenance savings, which is why least-privilege is the standard architectural recommendation despite the added configuration effort. This tradeoff applies broadly, not just to financial-data contexts (C), and granular access control is a well-supported MCP capability (D is wrong).
Q151
Tool scoping & least-privilege
Foundational
What is the difference between scoping tool access by agent role versus scoping it by individual task instance?
Not quite.
These are complementary layers of granularity, not identical or mutually exclusive (A, D are wrong) — role-based scoping sets a sensible baseline for a category of agent, while instance-based scoping can further restrict access for a specific invocation that doesn't need everything the role baseline allows, providing even tighter least-privilege than role-level scoping alone. Instance-based scoping is a real, used pattern for high-stakes or fine-grained control, not a theoretical-only concept (B is wrong).
Q152
Tool scoping & least-privilege
Applied
An MCP server exposes a single `database_query` tool that accepts arbitrary SQL, used by an agent that only ever needs to run simple SELECT queries against two specific tables. What is the architectural improvement here?
Not quite.
An arbitrary-SQL tool is about as broad a capability as it's possible to grant — it can read, write, or query any table, vastly exceeding what 'run simple SELECTs against two tables' actually needs. The least-privilege fix is replacing it with narrow, purpose-built tools that structurally cannot do anything beyond their intended scope, rather than relying on the agent to voluntarily restrict its own arbitrary-SQL usage. This isn't a database-interaction requirement (B is wrong — narrow, purpose-specific tools work perfectly well), and the risk exists regardless of whether the specific data is classified as sensitive (D is too narrow — overly broad access is a problem on principle).
Q153
Tool scoping & least-privilege
Advanced
A subagent delegated a narrow research task is given the same broad toolset as the parent coordinator 'just in case it needs something extra.' What two principles from this exam's scope does this violate simultaneously?
Not quite.
This connects two threads from across the exam's domains: the direct least-privilege violation (unnecessary access granted 'just in case'), and the specific mechanism by which that becomes a real problem rather than a theoretical one — a capable model may reasonably use an available tool when it looks locally useful for the task, even if it's outside the narrow intended scope, simply because the tool is present in its toolset. 'Just in case' provisioning is a common but genuinely risky anti-pattern, not standard best practice (A is wrong), and it's unrelated to error-handling design specifically (C) or extended thinking (D).
Q154
Tool boundary enforcement
Foundational
What is the general architectural principle for deciding whether a business rule should be enforced inside a tool (deterministically) versus requested of the model (via prompt instruction)?
Not quite.
This is the central lesson threading through the refund-cap and secret-scanning examples discussed earlier: the right location for a rule depends on its stakes. Absolute, consequential rules belong in deterministic code the model cannot reason its way around; softer, stylistic, or contextual preferences are appropriate for prompt-level guidance, where some flexibility is actually desirable. Treating all rules identically in either direction (A, C) ignores that different rules have genuinely different risk profiles, and the choice has very real reliability consequences (D is wrong) — it's the difference between a guarantee and a request.
Q155
Tool boundary enforcement
Applied
Structured Data Extraction
A tool that writes extracted data to a production database accepts any input the model provides with no server-side validation, trusting the model's structured-output schema to have already ensured correctness. What is the architectural flaw?
Not quite.
This is the defense-in-depth principle from structured-output design (question d3-003) applied at the tool boundary specifically: the schema strongly biases toward correct output but isn't an absolute guarantee, and the tool sits at the point of real-world consequence — it shouldn't assume the schema was perfectly honored upstream. Independent server-side validation at the boundary is the actual enforcement point, not a redundant afterthought to strip for efficiency (D is wrong) — and backups (B) address recovery after the fact, not prevention, which is a different and complementary concern.
Q156
Tool boundary enforcement
Advanced
A system enforces a hard rule deterministically in a tool (e.g., refunds capped at $200) but the model isn't told about this limit in its instructions at all. What UX and architectural problem does this create, even though the rule is safely enforced?
Not quite.
Safety and good UX are both needed, and they're not automatically the same thing — enforcing the limit deterministically prevents the bad outcome (an actual over-limit refund) but doesn't prevent the model from confidently promising something it then can't deliver, producing a jarring, broken-feeling interaction. The fix is informing the model of the constraint (even though it isn't the enforcement mechanism) so it can set accurate expectations from the start, while the tool boundary remains the actual safety guarantee. Removing deterministic enforcement (B) would reintroduce the original safety risk; this is a real behavioral/UX effect, not merely a documentation nicety (C is wrong).
Q157
Tool boundary enforcement
Foundational
Why is relying solely on a prompt instruction like 'never approve refunds over $200' insufficient as the only safeguard, even for a well-behaved, capable model?
Not quite.
This is the core probabilistic-versus-deterministic distinction underlying this whole subtopic: instructions genuinely do influence behavior (D is wrong) and capable models follow them well most of the time, but 'most of the time' isn't the same as 'guaranteed,' especially under adversarial pressure, long context competing for attention, or genuinely novel edge cases. A deterministic check has no such failure mode — it enforces the rule regardless of context. This isn't a model-version-specific limitation (C) — it's an inherent property of instruction-following versus code-level enforcement.
Q158
Tool boundary enforcement
Applied
A tool enforces a hard boundary (e.g., rejecting any write to a protected configuration file) but gives no explanation in its error response for why the write was rejected. What should be improved?
Not quite.
This connects the tool-boundary enforcement principle to the error-response design subtopic covered earlier — a deterministic rejection is the right safety behavior, but it should still be paired with an actionable, explanatory error (not silent rejection, C) so the agent understands why and can respond appropriately (inform the user, suggest the correct process) rather than hitting an opaque, unexplained wall. Removing the boundary (D) would reintroduce the safety risk the rule exists to prevent — the fix is better communication about the enforcement, not weaker enforcement.
Q159
Tool boundary enforcement
Advanced
A team is deciding where to enforce a complex business rule that depends on several factors (customer tier, order history, current promotional status) to determine refund eligibility. Unlike a simple fixed dollar cap, this rule has real nuance. How does this change the deterministic-vs-model-judgment calculus?
Not quite.
Complexity doesn't argue against deterministic enforcement — if anything, a nuanced multi-factor eligibility rule is exactly the kind of logic that benefits from being encoded once, correctly, in code (where it can be tested and audited) rather than re-derived by the model's reasoning on every single call, which introduces exactly the inconsistency determinism is meant to prevent. The model's appropriate role shifts to gathering inputs and interpreting results, not performing the calculation itself (A, C are both wrong) — this is the same principle as the fixed dollar cap (D is wrong), just applied to more complex business logic.
Q160
MCP server architecture & transport
Foundational
What is the architectural purpose of the Model Context Protocol (MCP) as a standard?
Not quite.
MCP's core value proposition is standardization — a tool or data source exposed via an MCP server can be consumed by any compatible client without custom, one-off integration work per application, similar in spirit to how a standard protocol like HTTP lets many different clients talk to many different servers. It isn't model-specific or proprietary in that narrow sense (A is wrong), it supports actions with real side effects, not just reads (B is wrong), and it's a protocol for tool/data connectivity that complements rather than replaces the underlying model API (C is wrong).
Q161
MCP server architecture & transport
Applied
A team is deciding whether an MCP server should run as a local process (communicating over stdio) or as a remote, network-accessible service (communicating over HTTP/SSE). What is the primary architectural factor driving this choice?
Not quite.
The right transport follows from the resource's actual location and intended consumers — a tool wrapping local filesystem access naturally fits a local process model, while a tool meant to be shared across a team or accessed by multiple remote clients needs network accessibility. This isn't an arbitrary, freely interchangeable choice (B) and 'local is always more secure' (C) is too broad a generalization — a poorly secured local process has its own risks, and a well-secured remote service can be quite safe. Volume (D) isn't the primary driver of this particular decision.
Q162
MCP server architecture & transport
Advanced
An MCP server exposes tools that connect to a company's internal database. What architectural consideration is critical for this server that wouldn't apply to a purely local, single-user tool?
Not quite.
Once a server is reachable by more than a single trusted local process, the threat model changes fundamentally — it needs real authentication (who is this caller) and authorization (what are they allowed to do) rather than the implicit trust reasonable for a local-only tool talking to its one user. This is a materially different, and more demanding, security posture (A is wrong to treat as identical), it's a concern tied to accessibility and sensitivity of the data, not transport mechanism specifically (C is wrong), and such servers absolutely can be secured properly with the right controls (D is overly defeatist).
Q163
MCP server architecture & transport
Foundational
What is the benefit of an MCP server exposing multiple small, focused tools rather than one large, multi-purpose tool with many optional parameters controlling different behaviors?
Not quite.
This connects to the tool-selection-degrades-with-count/ambiguity principle from agentic architecture, but from the opposite angle: a single tool with many optional parameters controlling wildly different behaviors creates its own ambiguity, since the model has to correctly configure the right combination of flags rather than simply selecting the clearly-named, focused tool for its actual intent. MCP fully supports exposing many tools per server (B is wrong), and the benefit of focused tool design applies well below fifty tools (D is an arbitrary threshold) — it matters even with just a handful of tools whose purposes could otherwise blur together.
Q164
MCP server architecture & transport
Applied
An MCP server's tools occasionally take several seconds to respond due to slow upstream API calls. What architectural consideration does this introduce for the agent calling it?
Not quite.
Per-call latency compounds across a tool-calling chain, the same additive-latency concern as sequential multi-agent pipelines — several seconds per call, chained across multiple calls in one task, can add up to an experience that feels broken or unresponsive to a waiting user. This motivates real architectural choices: async patterns, progress communication, or simply being conscious of how many slow calls a single task realistically chains together. Latency can't be eliminated outright (C is an unrealistic goal) and absolutely does affect the broader system's responsiveness, not just the server in isolation (D is wrong) — the agent loop needs deliberate handling, not a one-size-fits-all default (B is wrong).
Q165
MCP server architecture & transport
Advanced
A company builds one MCP server exposing all of its internal tools (HR, finance, engineering, support) rather than separate servers per domain. What architectural downside does this consolidated approach have as the company scales?
Not quite.
This is the least-privilege and blast-radius argument applied at the infrastructure level: one monolithic server covering every domain means every connected agent potentially has a path to every domain's tools unless carefully gated within that single server, and any issue with the server affects all domains simultaneously. Domain-separated servers allow tighter, more naturally-scoped access boundaries and independent operational characteristics (deployment, scaling, incident containment). This isn't a protocol restriction (B is wrong — MCP allows any grouping), and the architectural concern applies well below massive company scale (D is an arbitrary and unrealistic threshold) — it matters for any organization with meaningfully distinct domains and access needs.
Q166
Tool selection & discovery at scale
Foundational
As the number of tools available to an agent grows into the hundreds, what architectural problem tends to emerge even if each individual tool is well-designed?
Not quite.
This is the same tool-selection-degrades-with-count principle introduced in agentic architecture, here framed explicitly as a scaling concern — even a set of individually excellent, well-described tools can produce worse selection accuracy in aggregate once the model has to discriminate among hundreds of options for every call. It isn't solely a naming problem (B) — it's a volume problem that good naming alone doesn't fully solve, and more raw options isn't unconditionally better (D is wrong) — curation matters as much as individual tool quality.
Q167
Tool selection & discovery at scale
Applied
A system with 300 available tools implements dynamic tool filtering — only presenting the ~15 tools relevant to the current task context to the model at any given time, rather than all 300 every call. What architectural benefit does this provide?
Not quite.
Dynamic, context-aware filtering is a direct architectural response to the scaling problem in question 32 — narrowing the presented set to what's actually relevant for the current task meaningfully reduces the discrimination burden per call, improving selection accuracy, not just reducing the token overhead of listing every tool. This is a real, implemented pattern in production agent systems (C is wrong), and while it may have secondary latency/cost benefits too, the primary architectural win discussed here is accuracy (D undersells the benefit).
Q168
Tool selection & discovery at scale
Advanced
A dynamic tool-filtering system occasionally filters out a tool the model actually needed for an edge-case task, because the filtering heuristic didn't anticipate that use case. What tradeoff does this reveal about dynamic filtering as a mitigation?
Not quite.
This is a genuine, important tradeoff: filtering improves the common case (reducing noise among many options) at the cost of occasionally excluding something genuinely needed for an edge case the filtering logic didn't anticipate — trading perfect recall for better average precision. The fix usually isn't abandoning filtering altogether (C, which reintroduces the original scaling problem) but improving the filtering heuristic's coverage, adding a fallback/escape-hatch mechanism, or accepting the tradeoff as a reasonable cost for the aggregate benefit. The tradeoff exists at any scale where filtering is applied, not just above an arbitrary tool-count threshold (D).
Q169
Tool selection & discovery at scale
Foundational
What role can tool categorization or namespacing (e.g., grouping tools under prefixes like `billing_*`, `shipping_*`) play in tool selection at scale?
Not quite.
Logical grouping is a genuine organizational aid, for models navigating a large tool surface in much the same way it helps human developers navigate a large API — it provides an additional structural signal (this is a billing-related action) that complements individual tool descriptions. This has a real, positive effect on navigability (A is wrong), isn't restricted by parameter count (B), and isn't prohibited by the protocol (C is wrong) — it's a widely-used best practice for large tool surfaces.
Q170
Tool selection & discovery at scale
Applied
A team notices that tool selection accuracy has degraded over the past six months as they've steadily added new tools to their agent's toolset, with no single large change responsible. What monitoring approach would have caught this degradation earlier?
Not quite.
Gradual degradation from many small additions is exactly the kind of drift that's invisible without a tracked metric and obvious only once it's severe — the fix is treating tool-selection accuracy (or a workable proxy for it) as an ongoing monitored metric, the same discipline recommended for validation-retry pass rates earlier in this exam's scope, so degradation is caught as a trend rather than discovered as a crisis. This is a genuinely measurable problem, not purely a hindsight-only observation (A is wrong), doesn't require exhaustive manual review of every call (C — sampling or proxy metrics work), and there's no universal fixed tool-count ceiling that prevents this (D is an oversimplified rule that ignores how well-organized and filtered a larger toolset can still perform).
Q171
Context window management / compaction
Foundational
What problem does context compaction address in a long-running agent session?
Not quite.
The context window is finite, and a long-running agentic session (many tool calls, long conversations) will eventually approach it — compaction is the mitigation, condensing earlier content down to its essential state so the session can continue productively rather than hitting a hard wall. This isn't primarily a storage-cost concern (A) or image-specific (B), and it manages the limit rather than removing it outright (C is wrong — the window is still finite, just used more efficiently).
Q172
Context window management / compaction
Applied
Claude Code for Development
During a long coding session, context is compacted and a summary replaces much of the earlier conversation. Afterward, the agent seems to have 'forgotten' a specific constraint the user mentioned 40 turns ago that was never restated. What does this reveal about compaction as an architectural technique?
Not quite.
Compaction trades perfect recall for continued usability within a finite window — it is fundamentally lossy, and a detail mentioned exactly once, long ago, with no reinforcement, is a realistic casualty of summarization. The architectural lesson is that truly critical constraints shouldn't rely solely on being remembered from deep conversational history — persisting them somewhere more durable (a CLAUDE.md-style file, a pinned note, a scratchpad) is more reliable than hoping compaction preserves them. This is expected lossy behavior, not a bug (D is wrong), and avoiding compaction altogether (B) isn't practical for genuinely long sessions — it would just hit the hard window limit instead.
Q173
Context window management / compaction
Advanced
What is the architectural tradeoff between compacting context aggressively (summarizing frequently, keeping context small) versus conservatively (summarizing rarely, keeping more raw detail)?
Not quite.
This is a genuine tradeoff without a universally correct answer — aggressive compaction buys headroom and efficiency at the cost of fidelity, while conservative compaction preserves detail at the cost of running out of room sooner and higher ongoing cost. Neither extreme is strictly superior (A, C are both wrong) — the right point on that spectrum depends on how much the task's correctness depends on fine-grained historical detail versus how long the session realistically needs to run. This tradeoff is relevant at any session length where compaction applies, not just extremely long ones (D is an arbitrary threshold).
Q174
Context window management / compaction
Foundational
Why might an architect deliberately design a long-running task to periodically write its state to an external file or scratchpad, rather than relying entirely on the conversation's context window?
Not quite.
This directly previews the long-running-task-scratchpad subtopic covered next: state that lives only in the conversation's context is vulnerable to compaction loss and vanishes if the session ends, while externalized state (a file, a database record, a scratchpad) persists independent of context window pressure. This is a genuinely valuable reliability pattern for critical state, not busywork (B is wrong), applicable to any long-running task, not just financial ones (C is too narrow), and the reliability benefit generally outweighs the modest overhead of writing to a file (D overstates the cost).
Q175
Context window management / compaction
Applied
A multi-agent pipeline passes a full, uncompacted conversation history from one subagent to the next at every handoff. As the pipeline progresses through many stages, what problem is likely to emerge?
Not quite.
This connects directly back to the context-passing-between-agents subtopic in agentic architecture (curated handoffs, not full history dumps) and shows why that principle matters here specifically: uncurated full-history handoffs accumulate across every stage of a pipeline, consuming context window budget on each successive subagent and forcing compaction (with its lossy tradeoffs) sooner than a curated approach would. This is a real, common failure mode, not a false concern (A is wrong), it doesn't require an extreme pipeline length to manifest (C is an arbitrary threshold), and it's directly related to compaction, not unrelated (C is wrong) — curated handoffs are effectively a form of proactive compaction at each stage.
Q176
Context window management / compaction
Advanced
What is the architectural argument for summarizing and discarding a completed subtask's detailed working context (intermediate tool calls, exploratory reasoning) while retaining only its final result, once that subtask is done?
Not quite.
This is a practical application of compaction's core logic to subtask boundaries specifically: once a subtask has reached a settled conclusion, the exploratory path that led there (false starts, intermediate tool calls, discarded approaches) has generally served its purpose, and retaining only the distilled result is a reasonable trade that frees budget for remaining work. This isn't a blanket 'always keep everything' stance (A is wrong — that's exactly the pressure compaction exists to relieve), isn't gated by an arbitrary call-count threshold (B), and framing it as pure data loss with no benefit (D) ignores the real context-budget benefit it buys for the rest of the session — the tradeoff from question 3 applies here too.
Q177
Long-running task scratchpads
Foundational
What is the architectural purpose of a 'scratchpad' pattern in a long-running agentic task?
Not quite.
A scratchpad externalizes working state (current plan, findings so far, what's been tried) to something durable — a file, typically — so the task's progress survives exactly the risks context-window-only state is vulnerable to: compaction loss, session interruption, or needing to resume fresh later. This has real runtime function, not just debugging value (A is wrong), applies to any complex long-running task regardless of domain (C is too narrow), and complements rather than replaces tool use (D is wrong) — the agent still needs tools to do the actual work; the scratchpad just tracks where things stand.
Q178
Long-running task scratchpads
Applied
Multi-Agent Research
A research agent conducting a long investigation writes its working notes and intermediate findings to a markdown file as it goes, rather than keeping everything only in its conversational context. If the session is interrupted and resumed later, what architectural benefit does this provide?
Not quite.
This is the resumability payoff of externalizing state: a fresh session has no conversational memory of the original work, but if the working state lives in a durable file, that fresh session can read the file and reconstruct context — effectively picking up where the interrupted session left off rather than restarting the whole investigation from scratch. The benefit doesn't depend on interruption duration (C is an arbitrary and incorrect constraint), and scratchpad files are ordinary durable storage readable by any session with access to them, not session-bound (D is wrong) — that portability is the entire point.
Q179
Long-running task scratchpads
Advanced
What should a well-designed scratchpad entry prioritize recording, to be maximally useful for a future resumption (by the same or a different session)?
Not quite.
A scratchpad's value is proportional to how quickly it lets a reader (human or a resuming model) reconstruct actionable state — goal, done, in-progress, ruled-out-and-why, next-step is the kind of structure that supports fast, accurate resumption. A verbatim, unstructured transcript (A) technically contains everything but costs a huge amount of re-reading and re-deriving to extract the useful signal, largely defeating the purpose; recording only the final answer (B) loses exactly the 'what's in progress and what's been ruled out' context a genuinely interrupted task needs. Some structure consistently beats none for this purpose (C is wrong).
Q180
Long-running task scratchpads
Foundational
In a multi-agent architecture, how does a shared scratchpad differ in purpose from the curated context handoff passed directly between a coordinator and a subagent?
Not quite.
These are complementary, not identical or mutually exclusive (A, C are wrong) — a direct handoff is scoped and point-in-time (exactly what this subagent needs for this task), while a shared scratchpad is a more persistent, broadly-accessible record suited to state that outlives any single handoff (overall progress, accumulated findings, cross-cutting decisions). Many real systems use both together. This pattern is especially relevant in multi-agent systems specifically because of the coordination need across participants, not limited to single-agent contexts (D is backwards).
Q181
Long-running task scratchpads
Applied
A long-running agent updates its scratchpad file only once, at the very start of the task, and never again as work progresses. What limits this places on the scratchpad's usefulness?
Not quite.
A scratchpad's entire value proposition (questions 7-10) depends on it reflecting actual current state — a write-once-at-the-start scratchpad is stale the moment real progress begins, and resuming from it would wrongly assume the original plan still fully describes reality, potentially redoing completed work or missing what's already been ruled out. The benefit is tied to currency, not mere existence (A is wrong). This isn't duration-gated (C is an arbitrary threshold) — even a short task benefits from an accurate scratchpad if it might be interrupted — and periodic updates are the correct practice, not a resource-wasting anti-pattern (D is backwards).
Q182
Citation / provenance preservation
Foundational
Why does preserving citations or source provenance matter architecturally in a system where Claude synthesizes information from multiple retrieved documents?
Not quite.
Provenance is what makes a synthesized claim auditable — without a traceable link back to source material, a well-grounded answer and a fabricated-but-confident one look identical on the surface, which is a real architectural problem for any system where correctness matters and needs to be verifiable, not just a nice-to-have presentation detail (B is wrong). This verification need generalizes well beyond academic/legal contexts to essentially any business application where decisions get made based on the output (D is too narrow) — trustworthiness without traceability is architecturally fragile (A is wrong).
Q183
Citation / provenance preservation
Applied
Structured Data Extraction
A document-extraction pipeline outputs structured fields (e.g., contract term, dollar amount) but doesn't retain any link back to where in the source document each field was found. A reviewer later needs to spot-check the extraction for accuracy. What problem does the missing provenance create?
Not quite.
Missing provenance turns verification from a quick, targeted check (jump to the cited location, confirm the field) into either blind trust or a full manual re-read per field — neither scales, and both undermine the actual purpose of having a reviewer in the loop for high-stakes extraction. Re-reading the whole document per field (B) is exactly the scaling failure this causes, not a reasonable workaround. The problem exists at any document length, worsening with size rather than only appearing past a threshold (C), and provenance tracking (page/span offsets, source quotes) is a well-established, implementable pattern, not infeasible (D is wrong).
Q184
Citation / provenance preservation
Advanced
A system preserves citations, but they're generated by the model recalling 'roughly where' information came from after the fact, rather than being mechanically tied to exact source spans during retrieval. What architectural weakness does this introduce?
Not quite.
This is a subtle but important distinction: a citation is only as trustworthy as its generation mechanism, and a model asserting 'this came from source X' after the fact is itself a generative claim that can be wrong or fabricated — exactly the kind of ungrounded-but-confident output provenance was meant to guard against in the first place. Mechanically binding citations to exact spans at retrieval time (rather than relying on post-hoc recall) is architecturally stronger because it doesn't depend on the model accurately remembering its own sourcing. This isn't equally reliable either way (A is wrong), isn't gated by how often source documents change (C), and models are very capable of generating plausible-sounding but inaccurate post-hoc citations — that's exactly the failure mode described (D is wrong).
Q185
Citation / provenance preservation
Foundational
In an agentic research workflow where Claude gathers information across many tool calls before producing a final synthesis, what does 'preserving provenance through the pipeline' require architecturally?
Not quite.
Provenance has to survive every hop in the pipeline deliberately — it's exactly the kind of detail that's easy to silently drop during summarization or a context handoff (connecting back to the compaction and context-passing subtopics) unless each stage is designed to carry the source association forward, not just the content itself. It doesn't happen automatically (A is wrong) and isn't sufficient to track only at the first step (B is wrong) — any later summarization or handoff is a point where the source link can be severed if not deliberately preserved. This applies regardless of how many tool calls are involved, and if anything matters more as the pipeline gets longer, not less (D is backwards).
Q186
Citation / provenance preservation
Applied
A team is deciding between two designs: (A) the model generates a final answer and then a separate pass attempts to retroactively find supporting citations for each claim, versus (B) the model is required to cite its source inline as part of generating each claim, with retrieval results passed as structured, taggable context. Which is architecturally stronger for provenance reliability, and why?
Not quite.
This applies the lesson from question 14 directly to a design decision: design B keeps sourcing bound to content at the moment it's generated, while design A's retroactive approach reopens the exact post-hoc rationalization risk already identified — the model searching for a plausible-sounding citation after the answer already exists, rather than the citation reflecting what was actually used to produce that claim. The two designs are not equivalent (B is wrong), separating the steps doesn't inherently improve accuracy and in fact introduces this specific risk (A is backwards), and the weakness in design A isn't language-dependent (D is an irrelevant constraint) — it's structural to doing citation after the fact.
Q187
Reliability patterns — idempotency, retries
Foundational
What does it mean for a tool (such as 'charge customer $50') to be idempotent, and why does this matter for agentic retry logic?
Not quite.
Idempotency directly enables safe retries: if a call might have succeeded despite an ambiguous failure signal (a timeout, a dropped connection), retrying is only safe if doing so twice has the same effect as doing it once — an idempotency key on a charge operation is the classic example, ensuring a retried charge request doesn't double-charge. This is a performance-unrelated correctness property (B is wrong), and it matters specifically for operations with real side effects, not read-only ones where repeated calls are naturally safe anyway (C is backwards — idempotency is least interesting for read-only tools precisely because they're already safe to repeat). Idempotent tools are exactly the ones that CAN be retried safely (D is backwards).
Q188
Reliability patterns — idempotency, retries
Applied
A payment tool call times out — the agent doesn't know if the charge actually succeeded on the server before the timeout, or failed entirely. The tool was not designed with idempotency keys. What is the architectural risk if the agent naively retries?
Not quite.
This is the concrete failure case question 17 describes in the abstract: an ambiguous timeout combined with a non-idempotent operation means a retry could duplicate a side effect that actually already happened. The risk is real, not absent (A is wrong), and it's not transport-specific (C is an irrelevant distinction). The fix isn't to abandon retries altogether (D overcorrects — that sacrifices reliability for the many operations where retry is actually safe) — it's to make the operation idempotent (or at minimum, verify state before retrying) so retries become safe rather than avoiding them entirely.
Q189
Reliability patterns — idempotency, retries
Advanced
What is the architectural distinction between a transient failure (worth retrying) and a persistent failure (not worth retrying), and why does an agent's retry logic need to distinguish them?
Not quite.
This connects to the retryable-flag concept from tool error design: the reason the distinction matters is that retrying only helps when the underlying cause might actually resolve (transient) — retrying a persistent failure (bad credentials, malformed input) can't possibly succeed no matter how many attempts are made, so it just burns time and cost while delaying the escalation that's actually needed. A fixed blanket retry policy ignoring this distinction (A) is wasteful at best and misleading at worst. This distinction is relevant for any tool with failure modes, not solely external-API-connected ones (B), and while not always trivial, failure responses can often be designed (per the error-response subtopic) to make the distinction explicit rather than requiring guesswork (D overstates the difficulty).
Q190
Reliability patterns — idempotency, retries
Foundational
Why is exponential backoff generally preferred over immediate, repeated retries for a transient failure like rate limiting?
Not quite.
This reinforces the backoff concept referenced earlier in the context of agentic retry strategy: immediate repeated retries against a rate-limited or overloaded endpoint can actively worsen the condition (adding more load to an already-struggling system), while backoff gives the condition room to resolve and reduces the chance of compounding the problem. This is a materially different strategy from naive immediate retry (A is wrong), applies to any operation type experiencing transient failure, not just read-only ones (C is too narrow), and doesn't guarantee eventual success — a persistent failure will still fail regardless of backoff (D overstates the guarantee).
Q191
Reliability patterns — idempotency, retries
Applied
An architecture caps retries at a maximum of 3 attempts with exponential backoff, after which the failure is escalated rather than retried indefinitely. What is the architectural reasoning for having a cap at all, rather than retrying forever until success?
Not quite.
An unbounded retry loop has an unbounded worst case — if the failure genuinely never resolves, the task hangs indefinitely, consuming resources with zero forward progress and no path to resolution. A cap bounds that worst case and guarantees eventual escalation to a point (human review, an alternate strategy, a clear failure state) where the situation can actually be addressed, rather than silently stalling forever. The specific number isn't a universal constant — it should be tuned to the failure type and system context, not fixed at exactly 3 always (C is wrong), and this concern applies to any long-running system, not just CI/CD specifically (D is too narrow).
Q192
Evaluation & monitoring of agent behavior
Foundational
Why does an agentic system in production need ongoing evaluation and monitoring, beyond the pre-deployment testing done before launch?
Not quite.
Pre-deployment testing validates behavior against a necessarily limited set of anticipated scenarios — it cannot see real-world input distribution, future upstream changes (a tool's API changing, a model update shifting behavior), or slow drift that only shows up as a trend over time (the tool-selection-accuracy degradation example from d4). Ongoing monitoring catches exactly these categories of problem that pre-deployment testing structurally cannot, by design (B is wrong, overclaiming what one-time testing guarantees). This is standard practice for any production system, not contingent on having failed earlier tests (C is wrong), and the two activities serve genuinely different purposes despite surface similarity (D understates the distinction).
Q193
Evaluation & monitoring of agent behavior
Applied
Claude Code in CI/CD
A team wants to monitor an agentic system's behavior in production but is unsure what to actually measure. What is a reasonable starting set of signals for an agent that completes multi-step tasks using tools?
Not quite.
A single metric tells an incomplete story — task completion alone misses reliability issues (frequent retries papering over flakiness) and cost problems; error rates alone miss whether the system is actually accomplishing its goal. A reasonable monitoring set covers multiple dimensions together: is it working (completion), is it reliable (tool error/retry rates), is it escalating appropriately (not too often, not too rarely), and what does it cost (latency/cost per task). Reusing a static pre-deployment test set unchanged (A) misses exactly the real-world-distribution gap from question 22. User satisfaction (B) is a valuable signal but not the only one worth tracking, and meaningful measurement is absolutely possible here, not an unsolvable problem (C is wrong).
Q194
Evaluation & monitoring of agent behavior
Advanced
A monitoring dashboard shows an agent's overall task-completion rate holding steady, but a closer look reveals the escalation-to-human rate has crept up significantly over the same period. What does this combination suggest, and why does it matter?
Not quite.
This is exactly the kind of insight a multi-signal monitoring approach (question 23) is designed to surface — a single headline metric (completion rate) can mask a real underlying shift: if completion is only holding steady because escalation is quietly picking up the slack, the system's genuine autonomous capability may be degrading even while the surface-level number looks fine. This is a meaningful, investigation-worthy trend (A dismisses it incorrectly), a perfectly normal (if concerning) combination rather than a monitoring bug (B is wrong), and while escalation as a safety mechanism is good when appropriate, a rising trend still deserves scrutiny into *why*, not automatic praise (D oversimplifies).
Q195
Evaluation & monitoring of agent behavior
Foundational
What is the architectural value of logging an agent's intermediate reasoning and tool calls (not just final outputs) for later review?
Not quite.
Final-output-only logging tells you *what* happened but not *why*, which is a real limitation: debugging a failure, auditing a sensitive decision, or catching a 'right answer, wrong reasoning' case (which may look fine today but is likely to fail unpredictably on a slightly different input tomorrow) all require visibility into the intermediate path, not just the endpoint. This has broad value well beyond regulated industries specifically (D is too narrow), and while logging should of course respect data retention and privacy policies (what's logged and for how long), that's a design constraint on *how* to log intermediate reasoning responsibly, not a prohibition on doing so (C overstates it).
Q196
Evaluation & monitoring of agent behavior
Applied
A team runs their evaluation suite once before launch and considers monitoring 'done.' Six months later, the underlying model is upgraded to a newer version by the provider, and the team doesn't re-run evaluations. What risk does this create?
Not quite.
This is the connection back to question 22's point made concrete: an upstream model update is exactly the kind of change pre-deployment-only testing cannot see, because it happens after deployment. A general model improvement doesn't guarantee this specific system's particular evaluated behaviors (tone, exact tool-calling patterns, edge-case handling this system depends on) remain identical — re-evaluation after any meaningful upstream change is the ongoing practice that catches drift before it becomes a production incident instead of after. This risk applies to any system built on the model, tool-using or not (C is too narrow), and passing the original suite says nothing about compatibility with a model version that didn't exist at the time it was run (D is wrong).
Q197
Cost / latency tradeoffs
Foundational
What is the architectural tradeoff involved in choosing a more capable (and typically slower, costlier) model for every step of an agentic workflow versus reserving it for only the steps that need that capability?
Not quite.
This is the model-selection-per-step principle: a uniformly maximal-capability approach guarantees high per-step quality but compounds cost and latency across every step of a multi-step workflow, often including steps (simple classification, basic extraction) that don't actually need top-tier capability to get right. Matching capability to each step's real difficulty — lighter/faster models for simpler steps, reserving the most capable model for genuinely hard steps — is typically the more efficient overall architecture. More capable models are generally slower and costlier, not cheaper and faster (C is factually backwards), and this tradeoff is relevant at any workflow length with more than one step, not gated at 50 (D is an arbitrary threshold).
Q198
Cost / latency tradeoffs
Applied
A workflow includes a simple classification step (routing a request into one of five categories) immediately followed by a complex reasoning step (drafting a nuanced response based on that category). What is the architecturally sound approach to model selection across these two steps?
Not quite.
This is the concrete application of question 27's principle: a five-way classification is a comparatively simple task well within a lighter/faster model's capability, while nuanced drafting genuinely benefits from a more capable model's reasoning — matching each step to its real requirement is more efficient than forcing uniform configuration across structurally different work. Blanket consistency (A) sacrifices real efficiency gains for a consistency that isn't actually load-bearing here, and blanket cost-minimization (B) risks under-serving the step that genuinely needs more capability, which is just as much a design mistake in the other direction. This absolutely has measurable effect on cost and latency — that's the entire premise of the tradeoff (D is wrong).
Q199
Cost / latency tradeoffs
Advanced
A team reduces cost by aggressively caching and reusing model outputs across similar requests. What architectural risk does this introduce if the caching logic doesn't carefully account for context that makes seemingly similar requests actually different?
Not quite.
Caching is a genuine, common cost optimization, but aggressive caching risks a specific failure mode: similarity in the cache key doesn't guarantee true equivalence in the underlying request, and a cached response served for a superficially-similar-but-actually-different request can be wrong in a way that's invisible until it causes a real problem (stale data, wrong-account information, an outdated time-sensitive fact). This makes caching a real tradeoff between cost savings and correctness risk, not a free win (A is wrong), and the risk is about key design/granularity, not retention duration specifically (C is the wrong axis). Caching model outputs is a well-established, implementable pattern in agentic systems, not infeasible (D is wrong) — it just needs careful design.
Q200
Cost / latency tradeoffs
Foundational
Why might an architect choose to run independent subtasks in parallel rather than sequentially, and what does this trade off against?
Not quite.
This closes the loop back to the parallel-vs-sequential multi-agent pipeline discussion, framed specifically through the cost/latency lens of this subtopic: parallelizing genuinely independent subtasks cuts wall-clock time since they're not waiting on each other, but it isn't free — it trades for higher peak resource usage (more simultaneous model calls) and added coordination complexity (merging results, handling a partial failure among the parallel branches). Parallel execution typically costs about the same or more in total compute, not less (C is usually wrong — you're not reducing total work, just its wall-clock arrangement), and the tradeoff is relevant regardless of individual subtask duration — it's about dependency structure, not a time threshold (D is the wrong variable).