The Claude Developer – Foundations (CCDV-F) exam is 53 items, closed-book, proctored by Pearson VUE, with a 120-minute time limit. It costs $125 and uses a scaled score out of 1000, with 720 needed to pass. Engineers shipping Claude-powered applications. This page's question bank and flashcards are free and domain-weighted to match the breakdown below; the timed mock exam reproduces the full exam length and scoring.
Exam domains & weighting
Domain
Weight
Your progress
Applications & Integration
33.1%
0/66
Model Selection & Optimization
16.8%
0/34
Agents & Workflows
14.7%
0/29
Prompt & Context Engineering
11%
0/22
Tools & MCPs
10.6%
0/21
Security & Safety
8.1%
0/16
Claude Code
3.1%
0/6
Evaluation, Testing & Debugging
2.6%
0/6
About these questions
Every question below is written from scratch by GetDev, grounded in Anthropic's published exam domains and weightings plus general Claude, Claude Code and MCP product knowledge. None are reproduced from any other prep site. Each is tagged with its domain, subtopic and difficulty so you can target weak spots — use the filters in the Question bank tab.
200 of 200 questions · 0 answered · 0 correct so far
Q1
Claude API fundamentals
Foundational
What is the core unit of interaction when calling the Claude Messages API?
Not quite.
The Messages API is stateless per call: the client sends the full list of role-tagged messages (conversation so far) plus parameters, and gets back one new assistant message. There's no server-side session (C) — the client is responsible for resending history each call — and it's structured, not a raw string (B) or a query language (D).
Q2
Claude API fundamentals
Foundational
Why does the Messages API require the client to resend the entire conversation history on every call, rather than just the newest message?
Not quite.
Statelessness is an intentional design property, not a bug (A is wrong) — the server holds no session memory between calls, so whatever context the model should use must be present in that call's request. It isn't a billing trick (C), though it does mean cost scales with resent history length as a natural consequence. This applies from the very first follow-up turn, not only past 10 turns (D is wrong).
Q3
Claude API fundamentals
Applied
An application sets `max_tokens` very low (e.g., 50) for a task that occasionally needs longer answers. What is the architectural consequence when the model's natural response would exceed that limit?
Not quite.
`max_tokens` is a hard cap — generation simply stops once it's hit, which can truncate output mid-sentence or mid-structure (a serious problem if the output is meant to be parsed, e.g. JSON). The API does not auto-adjust it (A is wrong) and a hit limit is reported via a stop reason, not a thrown error (B is wrong) — the content up to that point is still returned, just incomplete. It directly affects content, not just latency (D is wrong).
Q4
Claude API fundamentals
Applied
What does the `stop_reason` field in an API response communicate, and why should a production application check it rather than assuming every response completed naturally?
Not quite.
`stop_reason` is the signal that distinguishes a naturally completed response from one cut off by a limit or redirected into a tool call — treating all responses as uniformly complete regardless of this field risks silently processing truncated or special-cased output as if it were a normal finished answer. It's populated on every response, not just errors (B is wrong), and matters for both streaming and non-streaming calls alike (C is wrong).
Q5
Claude API fundamentals
Foundational
What is the role of the `system` parameter versus a message with role `user` in the Messages API?
Not quite.
`system` is a distinct, top-level parameter for framing and standing instructions, kept separate from the turn-by-turn `user`/`assistant` exchange rather than being just another message in that list. It supports arbitrary length content (C is wrong), and a typical request includes both a `system` parameter and at least one `user` message together — they aren't mutually exclusive (D is wrong).
Q6
Claude API fundamentals
Advanced
An application needs the model to continue generating past where a previous response was cut off by `max_tokens`. What is the correct approach, given the API's stateless design?
Not quite.
Because the API is stateless (per the earlier fundamentals questions), continuing a truncated response requires explicitly including that truncated content back in the next call's message history, so the model has the context of what it already generated and can pick up from there — simply recalling the API with no changes (A) wouldn't give it any memory of the prior output, since nothing is retained server-side. This is a well-supported, common pattern, not impossible (C), and omitting the truncated response (D) would lose exactly the continuation point needed.
Q7
Claude API fundamentals
Foundational
Why do official Claude SDKs (Python, TypeScript, etc.) exist rather than requiring every application to construct raw HTTP requests?
Not quite.
SDKs wrap the same underlying HTTP API with convenience: typed request/response objects, built-in retry/backoff, streaming helpers — saving every application from reimplementing that plumbing. The underlying HTTP API remains directly callable (A, D are wrong — SDKs are optional, not mandatory or exclusive), and SDKs don't unlock model capabilities unavailable via raw HTTP (B is wrong) — they're an ergonomics layer, not a capability gate.
Q8
Claude API fundamentals
Applied
An application pins its integration to a specific, dated model version string rather than a generic 'latest' alias. What is the architectural tradeoff this represents?
Not quite.
Pinning is a genuine stability-versus-currency tradeoff: a pinned version behaves consistently over time (important for an application whose prompts and logic were tuned against specific behavior) but doesn't automatically benefit from newer model improvements until someone deliberately upgrades and retests — versus a 'latest' alias that evolves automatically but risks unannounced behavioral drift. Pinning is well-supported (B is wrong), and the tradeoff is relevant at any scale, not gated by user count (C is wrong).
Q9
Claude API fundamentals
Advanced
An application logs full request and response payloads for debugging, including conversation content that may contain customer PII. What architectural concern does this raise, independent of the Claude API itself?
Not quite.
Transport encryption (HTTPS) protects data in transit but says nothing about what happens to data once logged at rest — an application choosing to log full payloads takes on the responsibility of protecting that stored data per whatever regulations and policies apply to it (C conflates two different concerns). Blanket full-payload logging isn't automatically the safest choice (B is wrong) — it often means more sensitive surface area to secure. The API itself doesn't block PII from appearing in requests (D is wrong) — that's an application-level data-handling decision, not an API constraint.
Q10
Streaming responses
Foundational
What problem does streaming (`stream: true`) solve for a Claude-powered application compared to waiting for the complete non-streamed response?
Not quite.
Streaming is a user-experience and perceived-latency improvement — content appears incrementally rather than all at once after a potentially long wait, which matters a lot for longer responses. It doesn't change total cost (A is wrong — token billing is the same either way) or the content generated (D is wrong), and it's optional, not required past any length threshold (B is wrong).
Q11
Streaming responses
Applied
An application streams a response and needs to know when generation is fully complete versus just receiving another incremental chunk. What should it rely on?
Not quite.
The streaming protocol emits explicit, typed events marking structural milestones (content blocks starting/stopping, the message itself stopping) — these are the reliable signal for completion, not an inferred timeout (A) which is fragile and arbitrary, or raw token counting against max_tokens (D) which doesn't account for natural early completion. Completion detection is fully supported and reliable via these events, not impossible (C is wrong).
Q12
Streaming responses
Advanced
An application needs the model to produce structured JSON output but also wants to stream it for responsiveness. What complication does streaming introduce for structured output specifically?
Not quite.
A chunk of streamed JSON is, by construction, usually an incomplete fragment (an unclosed brace or array) that a standard JSON.parse would reject — naive per-chunk parsing (B) breaks for exactly this reason. The real options are using a streaming-tolerant/incremental JSON parser designed for partial input, or buffering until the stream completes before parsing normally. The two features can be combined (C is wrong), and the complication applies to any structured shape, not just arrays specifically (D is wrong).
Q13
Streaming responses
Foundational
What happens to tool use in a streamed response — can a tool call be identified before the full response finishes?
Not quite.
Streaming and tool use are fully compatible — the event stream includes structured signals for tool-use content blocks, including which tool is being invoked, as that part of the response is generated, rather than withholding that information until the stream fully ends (B, D are both wrong) or being incompatible altogether (A is wrong).
Q14
Streaming responses
Applied
A streaming connection drops partway through a response due to a network interruption. What should a robust application do?
Not quite.
A network interruption mid-stream is a real, distinguishable failure condition a robust application should handle deliberately — either retrying the whole request from scratch (since there's no standard resume-from-partial-stream capability) or using the partial content already received if that's acceptable for the use case, rather than letting the failure go unhandled (A overstates the loss as totally unrecoverable with no options, and D wrongly assumes automatic resumption that doesn't exist). This is a real operational scenario worth planning for, not a near-impossibility to ignore (B is wrong).
Q15
Streaming responses
Advanced
Why might an application choose non-streaming mode even for a reasonably long response, despite streaming's UX benefits?
Not quite.
Streaming's value is specifically about incremental display — if the application's actual need is a complete, validated answer before doing anything with it (no UI benefits from word-by-word display, or validation must happen on the whole response anyway), the added implementation complexity of stream handling isn't buying anything, making simple non-streaming the more sensible choice. Tool use works fine with either mode, not requiring non-streaming (C is wrong), and streaming doesn't carry a cost premium (D is wrong) — the choice is about UX fit and complexity, not price.
Q16
Streaming responses
Foundational
In a chat UI backed by streaming, what is the practical benefit users actually perceive, even though the total time to full completion is roughly the same as non-streaming?
Not quite.
The real win is time-to-first-content, not total completion time — users see the response starting to appear almost immediately rather than facing a blank wait followed by an instant full dump, which measurably changes perceived responsiveness even when the underlying total generation time is the same (C is wrong, conflating perceived and actual total latency). This is a noticeable UX difference, not nothing (B is wrong), and it's about latency perception, not data volume (D is wrong) — streamed and non-streamed responses transfer essentially the same total data.
Q17
Streaming responses
Applied
An application streams model output directly into a webpage via `innerHTML` without any sanitization. What security risk does this introduce?
Not quite.
Model output is not inherently safe to render as raw HTML — if it can be influenced by untrusted upstream input (a user message, retrieved content) and contains markup-like text, inserting it via `innerHTML` without sanitization is a real XSS vector, no different from any other untrusted-content-rendering scenario regardless of whether it arrived via streaming or not (B is wrong). `innerHTML` performs no sanitization on its own (C is wrong) — the application must sanitize or use safer rendering (e.g., textContent, a sanitizing library) itself.
Q18
Authentication, API keys & key management
Foundational
Why should a Claude API key never be embedded directly in client-side (browser or mobile app) code?
Not quite.
Anything shipped to a client is inspectable — a key embedded in browser or mobile code is effectively public, and any user could extract and reuse it however they like, at the key owner's cost and outside any usage control the developer intended. This is a real, serious security problem, not a style nicety (D is wrong). It's unrelated to execution speed (A) or IP addressing (B) — the issue is exposure, not networking.
Q19
Authentication, API keys & key management
Applied
What is the standard architectural pattern for letting a client-side application use Claude without exposing the API key?
Not quite.
A backend proxy is the standard fix: the key lives only on infrastructure the developer controls, never reaching the client at all, and the client talks only to that trusted backend. Obfuscating or encrypting a key shipped to the client (A) doesn't actually solve the problem — the client still needs to decrypt it to use it, so a sufficiently motivated user can extract it the same way. Frequent rotation (D) reduces but doesn't eliminate the exposure window and adds operational overhead without addressing the root cause; client-side use is entirely possible via the proxy pattern (C is wrong).
Q20
Authentication, API keys & key management
Foundational
What is a key architectural reason to use separate API keys for development, staging, and production environments rather than one shared key everywhere?
Not quite.
Per-environment keys give real operational isolation: a compromised or runaway development key can be revoked without any production impact, and cost/usage attribution per environment becomes meaningful rather than conflated. A single key technically works across environments (A overstates a requirement that doesn't exist), but the isolation benefit is real and valuable at any team size, not gated by headcount (C is wrong) — even a solo developer benefits from not being able to accidentally blow through production budget from a dev script.
Q21
Authentication, API keys & key management
Advanced
An API key is accidentally committed to a public GitHub repository and the commit is later removed. What is the architecturally correct response?
Not quite.
Removing a commit from the current branch does nothing about the key's actual exposure — the value was already public and may persist in forks, clones, cached views, or scrapers that indexed it before removal, for an indefinite time. The only response that actually closes the exposure is revoking/rotating the key itself; history rewriting (D) addresses the repository's appearance but not the already-leaked credential. Waiting to see if abuse appears (B) accepts avoidable risk during the window before any damage becomes visible — the correct move is immediate, not reactive.
Q22
Authentication, API keys & key management
Applied
A team stores its API key directly in application source code as a string literal, rather than in an environment variable or a secrets manager. What is the architectural problem beyond the risk of accidental public exposure?
Not quite.
Beyond the leak risk, hardcoding couples a credential's lifecycle to the code's: rotating the key means a code change and deploy instead of a config update, every source-access developer implicitly has the key regardless of whether they need it, and per-environment key separation (from the earlier question) becomes awkward without code duplication. This is a real architectural cost, not just style (A is wrong), and the access-control concern is just as real in a private codebase as a public one (B is wrong) — not every developer with repo access necessarily needs the production credential. There's no performance difference between a hardcoded value and an environment-loaded one (C is wrong).
Q23
Authentication, API keys & key management
Foundational
What is the purpose of scoping or limiting what a given API key or workspace can do (e.g., spend limits, restricted model access) in a production deployment?
Not quite.
This is least-privilege applied to credentials specifically: a key with a spend limit or restricted scope bounds the worst case if it's leaked or a bug causes a runaway loop, compared to an unrestricted key where the same incident could be far more costly. This benefit applies even to a key used by a single application (C is wrong — blast-radius containment matters regardless of how many consumers a key has), and granular per-key controls (not just org-wide) are the mechanism that makes this practical (D is wrong).
Q24
Authentication, API keys & key management
Applied
A backend service calls the Claude API on behalf of many different end users of an application. Should each end user have their own API key?
Not quite.
The typical, sound pattern is an application-level key (or a few) held by the backend, with the application's own authentication/authorization system managing individual end users — API keys authenticate the application to Anthropic, not individual end users within that application, which is a separate concern the application itself is responsible for. Requiring every end user to hold a personal Anthropic API key (A, D) is both unnecessary and a poor user experience for most consumer-facing products, and backend-mediated multi-user access is a completely standard, well-supported pattern (C is wrong).
Q25
Rate limits, retries & error handling
Foundational
What is the purpose of API rate limits from the provider's perspective, and what should an application do when it receives a rate-limit error?
Not quite.
Rate limits are a capacity-management and fairness mechanism, not an arbitrary obstacle (A is wrong) — the correct response is backing off and retrying with increasing delay, which gives the condition room to clear rather than compounding it with immediate retries. A rate-limit error is a temporary, recoverable condition, not a permanent revocation (B is wrong), and limits apply continuously, not on a time-of-day schedule (D is wrong).
Q26
Rate limits, retries & error handling
Applied
An application receives a 5xx server error from the API. Is this generally safe to retry, and why?
Not quite.
This is the transient-versus-persistent distinction applied to HTTP status codes specifically: a 5xx reflects a server-side issue that's often temporary, making backoff-and-retry sensible, whereas a 4xx (bad request, invalid parameters) reflects a client-side problem that will fail identically on every retry until the request itself is corrected — retrying a 4xx blindly (implied by treating all errors the same, A) wastes calls on a failure that can't resolve itself. Retrying as fast and often as possible with no backoff (B) risks compounding server-side strain; 5xx errors are specifically about server-side conditions, not application bugs (C is wrong).
Q27
Rate limits, retries & error handling
Advanced
An application implements retry logic but uses a fixed, short retry interval (e.g., always exactly 1 second) rather than exponential backoff. Under sustained rate limiting across many concurrent requests, what problem can this cause?
Not quite.
This is the thundering-herd problem: many clients retrying on the same fixed cadence tend to synchronize, producing bursts that can re-trigger the very rate limit they're retrying from, in a repeating pattern. Exponential backoff with jitter (randomized variation) spreads retries out over time and across clients, reducing this synchronization. The API doesn't reject fixed-interval retries outright (C is wrong) — the problem is an emergent systemic effect, not a protocol violation — and it can manifest at much smaller concurrency than 10,000 (D is an arbitrary and overly high threshold).
Q28
Rate limits, retries & error handling
Foundational
What does a 429 status code specifically indicate, as distinct from a 500-series error?
Not quite.
429 ('Too Many Requests') specifically signals the client has exceeded some rate or usage threshold — a condition the client caused by its own call volume — distinct from a 5xx, which reflects something going wrong on the server's side independent of how much the client has been calling. The practical response differs: 429 calls for slowing down the request rate itself, while 5xx calls for retrying a possibly-transient server issue. 429 is unrelated to key validity (C is a different error code/condition entirely) and is a protocol-level status code, not SDK-specific (D is wrong).
Q29
Rate limits, retries & error handling
Applied
An application sets an unbounded retry loop for rate-limit errors, planning to 'just keep trying until it works.' What operational risk does this create for an end user waiting on a response?
Not quite.
An unbounded retry loop has an unbounded worst case: if the condition doesn't clear quickly, the user-facing request hangs with no end in sight and no feedback, while also potentially holding open connections or threads that could otherwise serve other requests — this is the same retry-cap reasoning from reliability patterns (bound the worst case, escalate instead of hanging forever), just applied at the API-call level rather than the agent-loop level. This risk is just as real server-to-server as it is for a mobile client (B is wrong), and the API itself doesn't enforce a retry cap on the caller's behalf (D is wrong) — that discipline is the calling application's responsibility.
Q30
Rate limits, retries & error handling
Advanced
A production application wants to avoid ever hitting rate limits in the first place, rather than just handling them reactively with retries. What proactive architectural approach addresses this?
Not quite.
A client-side throttle or request queue that deliberately paces outgoing calls to stay under the known limit — informed by monitoring actual usage against that limit — shifts the strategy from reactive (fail, then retry) to proactive (avoid hitting the limit at all), which is generally a better experience for latency-sensitive applications. This is a real, implementable pattern, not a non-option (B is wrong), doesn't require per-request provider contact (C is wrong — that would defeat the purpose of an API in the first place), and is actually more valuable at higher request volumes, not restricted to trivially low ones (D is backwards).
Q31
Rate limits, retries & error handling
Foundational
Why is it important for an application's error handling to distinguish between different 4xx error codes (e.g., 400 bad request vs. 401 unauthorized vs. 429 rate limited) rather than treating all 4xx errors identically?
Not quite.
This is the same error-differentiation principle from structured tool-error design, applied to HTTP status codes: 400, 401, and 429 are symptomatically similar (all 4xx) but have entirely different root causes and correct next actions — a generic one-size-fits-all handler can't route to the right fix for any of them. Error codes are a standard part of the HTTP protocol itself, fully inspectable via raw HTTP, not an SDK-exclusive feature (B is wrong). While 4xx broadly points to client-side issues and 5xx to server-side ones, treating this as an absolute rule with literally no exceptions (C) overstates a useful heuristic into an ironclad guarantee.
Q32
Rate limits, retries & error handling
Applied
An application wraps every API call in a try/catch that silently swallows all errors and returns a generic fallback message, with no logging. What operational problem does this create?
Not quite.
This connects to the intermediate-reasoning-logging and monitoring principles discussed for agentic systems, applied to API error handling: silently swallowing errors with no logging might smooth over the immediate user experience, but it also means the team loses all visibility into error frequency, type, and trend — exactly the signal needed to catch a developing systemic issue before it becomes a crisis. This visibility gap matters for any team maintaining the application, not just ones with a dedicated ops function (B is wrong), and try/catch itself is a perfectly appropriate mechanism — the problem is what happens (or doesn't) inside the catch block, not the construct itself (D is wrong).
Q33
Rate limits, retries & error handling
Advanced
A team wants to implement a circuit breaker pattern around their Claude API calls. What problem does a circuit breaker solve that simple per-request retry logic does not?
Not quite.
Per-request retries operate locally, one request at a time, with no memory of how other concurrent requests are faring — a circuit breaker adds a system-wide view, tracking aggregate failure rate and tripping to fail-fast (reject immediately rather than retry) once failures cross a threshold, which protects the application's own resources and avoids every request independently hammering a dependency that's clearly degraded. This is a general resilience pattern applicable to any external dependency, not limited to databases (C is wrong), and it doesn't guarantee zero failures (D overstates it) — it bounds how the application behaves during a failure period, which is a different goal than preventing failures from ever occurring.
Q34
Multi-turn conversation & context management in apps
Foundational
In a chat application, what is responsible for maintaining the conversation history across multiple user messages, given the API's stateless design?
Not quite.
Given the API's statelessness (from the fundamentals subtopic), the application itself owns history management — typically persisting messages in a database or session store and resending the relevant portion with each new call. The API has no server-side memory to lean on (B is wrong). Browser cache is one possible client-side storage choice but not the only or required mechanism (C is wrong), and there's no 2-turn ceiling — conversations can run arbitrarily long, limited only by context window and application design (D is wrong).
Q35
Multi-turn conversation & context management in apps
Applied
A chat application's conversation grows long enough that resending the full history on every call starts approaching the context window limit. What are reasonable application-level strategies to address this?
Not quite.
This is the compaction tradeoff from context management, implemented at the chat-application layer: summarizing or truncating older history before resending it buys headroom at some fidelity cost, the exact same tradeoff discussed for agentic sessions generally. This is a solvable, common problem with established patterns, not a dead end (A is wrong) or something requiring manual user intervention (B is wrong). A larger context window model is one possible lever, but application-level compaction is a real, often-preferred alternative that doesn't require a model change (C is wrong).
Q36
Multi-turn conversation & context management in apps
Advanced
An application summarizes older conversation turns to save context budget, replacing them with a condensed system-level summary. A user later references something specific they said many turns ago, before the summarization point. What architectural risk does this illustrate?
Not quite.
This is a direct, concrete instance of the compaction-is-lossy principle: a detail that didn't make it into (or was under-weighted in) the summary is a real casualty, and because the chat UI presents an illusion of continuous memory, the user has no way to know a specific detail might have been dropped — this is exactly the user-visible gap the lossy tradeoff predicts. This is unrelated to SDK choice (B is wrong) — it's a consequence of the summarization strategy itself, applicable regardless of implementation details. Never summarizing (D) just reintroduces the original context-budget problem the strategy was adopted to solve; the realistic fix is being thoughtful about what gets preserved, not avoiding summarization altogether.
Q37
Multi-turn conversation & context management in apps
Foundational
What is a key reason a chat application might store conversation history with role-tagged messages (user/assistant) exactly as the API expects, rather than as a single flattened text blob?
Not quite.
Storing history in (or close to) the API's own expected shape minimizes transformation work when resending it, and critically, the role tags themselves (user vs. assistant) are meaningful structure the model uses to understand who said what — flattening into one blob would obscure or lose that turn-taking information entirely, which isn't equivalent (A is wrong). This isn't a regulatory requirement (C is an irrelevant and incorrect claim) and matters from the very first multi-turn exchange, not only past some length threshold (D is wrong).
Q38
Multi-turn conversation & context management in apps
Applied
A support chatbot needs to maintain context about which product a customer is asking about across many turns, even as the conversation drifts to cover several sub-questions. What is a robust application-level approach, beyond relying purely on the raw conversation history?
Not quite.
Explicit structured state tracked by the application (not solely embedded in free-text history) is a more reliable way to carry a specific, important fact across a drifting conversation than hoping the model successfully re-derives it from scattered historical mentions every time — injecting that state explicitly (e.g., into the system prompt) removes the guesswork. Relying purely on raw-history inference (B) is strictly weaker and more fragile. This is a solvable, common pattern (C is wrong), and it doesn't require burdening the user with repeating themselves (D is an unnecessarily poor UX workaround).
Q39
Multi-turn conversation & context management in apps
Advanced
An application allows a user to 'branch' a conversation — editing an earlier message and getting a new response from that point, while keeping the original branch accessible. What does this require architecturally, beyond a simple linear message list?
Not quite.
Branching breaks the simple-linear-list assumption — once an edit can create a new path while the original remains accessible, the underlying structure is naturally tree-shaped (or equivalent), and the application needs to track which branch is currently active to know what history to resend on the next call. A fully linear list (A) can't represent multiple diverging paths from the same point. This is an application-level data modeling choice, not something the API itself restricts (B is wrong) — the API only ever sees whatever single linear history the application chooses to send for a given call. Treating branches as fully independent conversations with zero shared structure (C) ignores that they share history up to the branch point, which is exactly what makes it a branch rather than two unrelated conversations.
Q40
Multi-turn conversation & context management in apps
Foundational
Why might an application cap the number of historical turns resent on each call, even before hitting the hard context window limit?
Not quite.
Context length has a cost and latency cost on every single call, independent of the hard context-window ceiling — resending a very long history by default, even when much of it is no longer relevant to the current turn, adds expense and processing time without necessarily improving response quality. This is an application-level design choice the API doesn't mandate (B is wrong), and it's a sensible default-efficiency consideration at any usage scale, not just at extreme volume (C is wrong) — a single long-running conversation for one user already pays this cost on every turn.
Q41
Multi-turn conversation & context management in apps
Applied
A multi-turn conversation's history contains an early assistant turn where the model made a mistake that the user then corrected. The application keeps resending the original mistaken turn verbatim in all future calls. What problem can this create?
Not quite.
Context that's technically 'in the conversation' still actively influences the model's behavior on later turns, including content that's since been superseded by a correction — the model doesn't automatically discount an earlier mistaken statement just because a correction came later, since both are present as equally legitimate-looking history. Depending on how consequential the mistake is, deliberately pruning or clearly annotating superseded content is a reasonable mitigation, rather than assuming the model will reliably sort it out on its own (B is wrong). This isn't limited to numerical mistakes — it applies to any kind of incorrect prior content (C is wrong), and starting an entirely new conversation (D) is a heavier-handed fix than necessary when targeted pruning or annotation would do.
Q42
Vision / multimodal & document (PDF) input
Foundational
How is an image included in a request to the Messages API?
Not quite.
Image input is a content block type within the same message structure as text — a message can mix text and image blocks together, with the image typically provided as base64-encoded data tagged with its media type. This is a core supported capability, not unavailable (A is wrong), uses the same Messages API rather than a separate endpoint (C is wrong), and multiple images (including across multiple turns) are supported, not capped at one per conversation (D is wrong).
Q43
Vision / multimodal & document (PDF) input
Applied
An application needs to process a scanned, multi-page PDF document (e.g., extracting data from an invoice). What is an architectural consideration for how to handle the PDF?
Not quite.
PDF/document input is natively supported as a content type in the Claude API (contradicting A and D's claims it is unavailable or requires a separate product); the architectural considerations are about how multi-page content scales (token/size budget across many pages) and whether layout/order carries meaning the application needs to preserve — not that PDFs are impossible or unusably opaque. Multi-page documents can often be sent as a single request rather than being forced into one call per page (C overstates a limitation that doesn't generally apply).
Q44
Vision / multimodal & document (PDF) input
Advanced
An application sends a very high-resolution image to the API for a task that doesn't need fine visual detail (e.g., just classifying the image's general subject). What is the architectural tradeoff of sending the image at full resolution versus downscaling it first?
Not quite.
Image size affects token/context consumption and processing cost, so sending unnecessarily high resolution for a coarse task (general subject classification, not fine detail reading) spends budget without buying proportional accuracy — downscaling to a task-appropriate resolution is a real, available cost/latency optimization with little accuracy cost for tasks that genuinely don't need fine detail. Resolution does affect cost (B is wrong), and downscaling before sending is an application-level preprocessing step fully within the developer's control (C is wrong) — nothing prevents it.
Q45
Vision / multimodal & document (PDF) input
Foundational
Why might an application need to handle multiple image formats (JPEG, PNG, WebP, etc.) rather than assuming a single format?
Not quite.
Real-world image sources are heterogeneous — user uploads, scans, and screenshots naturally arrive in different formats — so an application needs to either accept the range of formats the API supports or convert non-supported formats before sending, rather than assuming a single uniform input format. This requires deliberate handling, not zero consideration (C is wrong), and the concern applies regardless of file size, not gated by a size threshold (D is wrong) — a small image in an unsupported format is just as much a problem as a large one.
Q46
Vision / multimodal & document (PDF) input
Applied
An application extracts structured data from invoice images and needs the output tied back to which part of the image supported each extracted value. How does this connect to the citation/provenance principle discussed for context management and reliability?
Not quite.
The provenance principle is source-type agnostic — whether the underlying source is a text document or an image, the same verification problem exists: without a traceable link from an extracted claim back to where it came from, a reviewer faces the identical blind-trust-or-exhaustive-manual-search tradeoff. This generalizes rather than being text-only (A is wrong), and while precise pixel-level provenance is harder than text-span provenance, useful approximations (e.g., bounding boxes, cropped region references) are implementable, not impossible (C is wrong). The concern doesn't scale with dollar amount — it's about verifiability in principle, for any extracted value (D is wrong).
Q47
Vision / multimodal & document (PDF) input
Advanced
An application sends a chart/graph image and asks the model to extract the exact underlying data values. What is a realistic limitation to design around?
Not quite.
Visually reading exact values off an unlabeled chart is fundamentally an estimation task, not a lossless data-recovery one — the image itself may not encode the precision the underlying dataset had, so treating extracted values as approximate (and adding validation or human review where precision genuinely matters) is the realistic design posture, rather than assuming image-based extraction is always as exact as the original data (A is wrong). Chart images are fully processable as visual input (B is wrong), and the estimation limitation applies across chart types generally, not uniquely to pie charts (D is wrong).
Q48
Vision / multimodal & document (PDF) input
Foundational
Can a single message include multiple images alongside text, and what is a typical use case for doing so?
Not quite.
A single message can combine multiple image blocks with text, supporting use cases like direct comparison between images or reconstructing information spread across multiple photos or pages of the same source. There's no same-resolution requirement across images in one message (B is wrong) and no special tier gating this capability (C is wrong) — it's part of standard multimodal support, contradicting the one-image-only claim (A is wrong).
Q49
Vision / multimodal & document (PDF) input
Applied
An application sends user-uploaded images directly to the API without checking their dimensions or file size first, and some uploads are unusually large (very high resolution, many megabytes). What should the application account for, distinct from the earlier resolution-vs-accuracy tradeoff?
Not quite.
This is a distinct concern from question 44's resolution-for-accuracy tradeoff: here the issue is validity, not optimization — the API has concrete limits on image size/dimensions, and an application accepting unpredictable user uploads needs its own validation/resizing step before sending, or a too-large image simply fails the call outright. This has nothing to do with the upload device (B is wrong) and isn't something the application can assume the API silently handles on its behalf (C is wrong) — assuming no limits exist at all (A) risks exactly the failures this validation step is meant to prevent.
Q50
Prompt caching for applications
Foundational
What problem does prompt caching solve for an application that sends largely the same long context (e.g., a large system prompt or reference document) on every call?
Not quite.
Prompt caching targets exactly the scenario of a large, repeated, unchanged prefix (a long system prompt, a reference document) — reusing a cached version of that content across calls saves the cost and latency of reprocessing it every single time, for the portion that stays identical. It doesn't change response content/accuracy (B is wrong) and operates on input processing, not output length (C is wrong), nor does it change the underlying context window ceiling itself (D is wrong) — it makes using a given amount of repeated context cheaper, not larger.
Q51
Prompt caching for applications
Applied
An application's cached prefix includes content that changes on every single call (e.g., a timestamp embedded near the start of the prompt). What happens to caching effectiveness in this situation?
Not quite.
Caching relies on exact prefix matching — content that changes every call, especially placed early, breaks the match for everything that follows it in the prefix, effectively losing the cache benefit for that portion. The fix is structural: order the prompt so genuinely stable content (that can be cached) comes first, and frequently-changing content (timestamps, per-request specifics) comes after the cacheable boundary, not disabling caching altogether (D is an overcorrection that throws away the benefit for genuinely stable content too). This isn't a function of the changing content's length (B is wrong) — even a short but early-placed variable value breaks the match for everything after it.
Q52
Prompt caching for applications
Advanced
A chatbot application with a large, static system prompt serves many different users concurrently. How does prompt caching's benefit scale with the number of users sharing that same static prompt?
Not quite.
Because many users' calls share the exact same static prefix, the caching benefit isn't a per-user, isolated win — it compounds across every call from every user that hits the same cached content, making the aggregate savings scale with concurrent shared usage rather than staying flat. Concurrent use doesn't invalidate the cache for other users (C is wrong) — shared, identical prefixes are exactly the scenario caching is designed to benefit, and this has no geographic dependency (D is an irrelevant and incorrect constraint).
Q53
Prompt caching for applications
Foundational
Why would an application deliberately structure its prompt with stable instructions first and highly variable, request-specific content last, independent of any other performance consideration?
Not quite.
This is the direct practical implication of how prompt caching works via prefix matching: putting stable content first maximizes the portion eligible for a cache hit, leaving only the genuinely variable tail to be freshly processed each call. This ordering recommendation is specifically a caching optimization, not a model-comprehension requirement unrelated to caching (C is wrong) — the model can understand prompts in various orderings; it's the caching mechanism specifically that benefits from this structure. The benefit applies to any cacheable prefix length that's worth caching, not gated at a specific high token threshold (D is wrong).
Q54
Prompt caching for applications
Applied
A cached prompt prefix has a limited cache lifetime (it expires after a period of inactivity). What does this mean architecturally for an application with sporadic, infrequent traffic to a given cached prefix?
Not quite.
Cache lifetime is a real constraint — if the time between calls sharing a prefix exceeds that lifetime, the cache has already expired by the next call, so the application pays full cost and latency again, getting none of the caching benefit for that call. This means caching's value is concentrated in access patterns frequent enough to stay within the cache lifetime, not in sparse, occasional reuse of the same prefix. It affects both cost and latency together, not just one (B is wrong), and traffic frequency directly determines whether the benefit is realized at all (C is wrong, contradicting the core mechanism just described).
Q55
Prompt caching for applications
Foundational
What kind of content is a good candidate for prompt caching, and what kind is a poor candidate?
Not quite.
Caching value is about repetition, not just size — stable content reused unchanged across many calls (a system prompt, a reference document, a fixed set of few-shot examples) is exactly where caching pays off, while content that's unique per call (the user's specific question) has nothing to reuse and gains nothing from caching regardless of its length. Multimodal content (images, documents) can also be part of a cacheable prefix, not excluded (B is wrong) — and length alone, divorced from repetition frequency, isn't the deciding factor (D is wrong) — a large but never-repeated block of content is still a poor caching candidate.
Q56
Prompt caching for applications
Advanced
An application caches a large reference document as part of its system prompt. The underlying document is later updated. What architectural responsibility does the application have regarding the cache?
Not quite.
Caching matches on the literal content sent in the request — it has no awareness of some external 'source of truth' document changing elsewhere; the application itself must send the updated content for that update to take effect, at which point it's simply a different prefix that gets its own fresh cache entry going forward (not literally invalidating the old one, just no longer being requested). There's no automatic detection of external document changes (A is wrong), no manual provider-side invalidation step required or available (C is wrong), and updating is entirely possible — it just requires the application to actually send new content rather than continuing to send the old version (D is wrong).
Q57
Prompt caching for applications
Applied
A team is deciding whether prompt caching is worth implementing for their application. What question should drive that decision?
Not quite.
The deciding question is squarely about the shape of the application's traffic and prompts: is there large, stable, frequently-repeated content worth caching, and is call frequency to that content high enough to stay within the cache lifetime? If not, the implementation effort may not be worth it regardless of other factors. Caching and streaming are independent, fully compatible features, not mutually exclusive (B is wrong), the decision has nothing to do with organization headcount (C is wrong), and caching applies to any cacheable content type, not exclusively image-only applications (D is wrong).
Q58
Prompt caching for applications
Foundational
How does prompt caching's cost model typically differ between writing to the cache versus reading from an existing cache hit?
Not quite.
The typical cost asymmetry (a relatively higher one-time write, a substantially cheaper subsequent read) is exactly what makes caching an economically sound choice specifically for content reused many times — the write cost gets amortized across many cheap reads, so the more reuse, the better the economics. If reads and writes cost the same (A) or reads cost more than writes (C), caching wouldn't offer the same incentive structure. There is a real, non-zero cost model governing both operations, not a free feature (B is wrong) — understanding that cost model is part of deciding whether caching pays off for a given usage pattern.
Q59
Embeddings & RAG integration patterns
Foundational
What problem does a RAG (retrieval-augmented generation) architecture solve for an application that needs the model to answer questions about a large, private knowledge base?
Not quite.
RAG's core value is letting an application query a knowledge base far larger than any single context window by retrieving just the relevant slice for a given question and including only that in the prompt — the model still processes context (A is wrong, it still needs the retrieved portion), but never the entire knowledge base at once. It doesn't require fine-tuning a dedicated model per knowledge base (B is wrong) — retrieval plus a general-purpose model is the whole point. RAG is specifically valuable BECAUSE the knowledge base exceeds the context window, the opposite of D's claim that this makes it redundant for exactly that case.
Q60
Embeddings & RAG integration patterns
Applied
What is the role of embeddings in a typical RAG pipeline?
Not quite.
Embeddings are the mechanism that makes semantic (meaning-based) retrieval possible — converting both stored content and incoming queries into vectors where semantic similarity corresponds to vector closeness, enabling a similarity search to surface relevant chunks even when they don't share exact keywords with the query. This is specifically a retrieval-stage technique, not something used for final text generation itself (A is wrong), and it's central to retrieval quality, not cosmetic (C is wrong) — poor embeddings mean poor retrieval. Embeddings can represent arbitrarily long text spans (sentences, paragraphs, documents), not just single words (D is wrong).
Q61
Embeddings & RAG integration patterns
Advanced
A RAG application chunks its knowledge base documents into fixed-size pieces (e.g., exactly 500 characters each) without regard for sentence or section boundaries. What problem can this introduce?
Not quite.
Naive fixed-size chunking with no regard for natural boundaries risks cutting a sentence or idea in half, producing chunks that embed ambiguously and may retrieve as individually incoherent fragments missing surrounding context needed to interpret them. This is a real, common RAG pitfall, not a non-issue (A is wrong), and it's not language-specific — any language with structural units (sentences, sections) can be disrupted by arbitrary fixed-length cuts (C is wrong). Alternatives like semantic or boundary-aware chunking (splitting at sentence/section breaks) are well-established alternatives, not nonexistent (D is wrong).
Q62
Embeddings & RAG integration patterns
Foundational
After retrieving relevant chunks via similarity search, how are they typically used in a RAG pipeline's final answer-generation step?
Not quite.
The standard pattern is augmentation, not replacement — retrieved chunks are added into the prompt (the 'augmented generation' in RAG) alongside the original user query, giving the model grounded content to draw from while still knowing what was actually asked (A is wrong to discard the query). The chunks are the actual content the model reads and reasons over, not merely a hidden relevance signal (B is wrong). The model consumes the chunks as already-retrieved text; the embedding step happened earlier during retrieval, not as something the model repeats on receipt (D is wrong).
Q63
Embeddings & RAG integration patterns
Applied
A RAG application's similarity search returns chunks that are topically related to the query but don't actually contain the specific fact needed to answer it. What does this illustrate about retrieval quality?
Not quite.
Topical similarity and 'contains the needed answer' are related but distinct properties — a chunk can be semantically close to a query's subject matter without actually containing the specific fact needed, which is a known, common RAG limitation rather than an impossibility (A is wrong). This is exactly why retrieval tuning (adjusting chunk size, how many chunks are retrieved, adding a re-ranking step) is an ongoing, real practice rather than a solved problem once embeddings exist. It's not necessarily a sign the embedding model itself is broken (D overreacts) — it's often a chunking, retrieval-count, or ranking issue, and it doesn't require abandoning RAG altogether (C is an overcorrection) — it requires tuning the pipeline.
Q64
Embeddings & RAG integration patterns
Advanced
How does RAG's citation/provenance story compare to the general citation-preservation principle discussed for context management and reliability?
Not quite.
RAG is actually a natural fit for the stronger provenance pattern: since retrieved chunks are known, specific, already-identified pieces of source content (not the model's free recall), the application can bind citations directly to exactly which chunks were used — the mechanically-bound-at-retrieval-time approach identified as more reliable than a model guessing its sources after the fact. This directly connects to, rather than being unrelated to, the citation-preservation principle (B is wrong). If an application fails to track and surface this available information, it would be forgoing a natural advantage, but the capability is there architecturally, which is the opposite of saying RAG citations are inherently the weaker post-hoc pattern (C is wrong). This value is independent of how often the knowledge base updates (D is an irrelevant and incorrect constraint).
Q65
Embeddings & RAG integration patterns
Foundational
Why might an application use a vector database rather than a general-purpose relational database to store and query embeddings at scale?
Not quite.
Vector databases exist specifically because efficient similarity search over large collections of high-dimensional vectors needs specialized indexing (approximate nearest-neighbor structures) that general relational databases aren't built for — at small scale naive comparison works fine, but it degrades badly as the collection grows, which is exactly the opposite of D's claim that this only matters for small knowledge bases. Relational databases can store numeric/vector data just fine (A overstates a limitation that doesn't really exist at the storage level) — the issue is query-time search efficiency at scale, not storage capability. The Claude API itself has no dependency on or requirement for a specific vector database (C is wrong) — this is purely an application-architecture choice.
Q66
Embeddings & RAG integration patterns
Applied
A RAG application's knowledge base is updated frequently (new documents added daily). What architectural implication does this have for the embedding/indexing pipeline, compared to a knowledge base that's static?
Not quite.
A frequently updated knowledge base needs embedding/indexing to be an ongoing, likely automated pipeline process — new or changed content must be embedded and indexed incrementally to stay current — whereas a static knowledge base can reasonably use a single one-time batch embedding pass and be done. Treating embedding as inherently one-time regardless of update frequency (A) ignores this real operational difference. This doesn't make RAG infeasible (B is wrong) — it's a completely standard, well-supported pattern (continuous ingestion pipelines) — and the implication kicks in well below once-per-minute frequency; daily updates already require it (D is an arbitrary and unrealistically high threshold).
Q67
Model family tradeoffs (capability vs cost vs speed)
Foundational
Why does Anthropic offer multiple model sizes/tiers rather than a single model for every use case?
Not quite.
Tiered model families exist because real workloads vary widely in what they need — a high-volume, simple classification task is well-served by a fast, cheap model, while a task needing deep multi-step reasoning justifies a larger model's added cost and latency. Smaller models aren't strictly worse (C is wrong) — they're a better fit for a large class of tasks specifically because of their speed/cost profile. This is a genuine capability/cost/speed engineering tradeoff, not a marketing artifact (A) or a UI-only distinction (D).
Q68
Model family tradeoffs (capability vs cost vs speed)
Applied
An application uses the largest, most capable model for a simple, well-defined classification task (sorting support tickets into 5 categories) that a smaller model could likely handle just as well. What is the architectural cost of this choice?
Not quite.
This is the model-to-task-matching principle in its simplest, single-endpoint form: a simple, well-defined classification task is unlikely to see a meaningful accuracy gain from a much larger model, so paying that model's higher cost and latency on every single call is a real, avoidable ongoing expense. It doesn't make results less accurate (D is wrong — if anything, the larger model's floor is usually at least as good), and the cost compounds with volume at any user scale, not just gated above some threshold (C is wrong).
Q69
Model family tradeoffs (capability vs cost vs speed)
Advanced
A team benchmarks a smaller model against a larger one on their specific task and finds the smaller model performs nearly as well, at a fraction of the cost. What should inform the final decision beyond the headline accuracy numbers?
Not quite.
Average-case benchmark parity doesn't guarantee parity on the tail — a smaller model might match a larger one on typical inputs but diverge more on harder or unusual edge cases, and whether that matters depends heavily on how costly a wrong answer is in that specific application (a miscategorized low-stakes ticket vs. a missed safety-critical signal). Reducing the decision to only the headline number (A) or only speed (B) ignores this real risk dimension. Smaller models are absolutely usable in production when they clear the bar for the task, including its tail risk (C is wrong) — that's exactly the judgment call this question describes making well.
Q70
Model family tradeoffs (capability vs cost vs speed)
Foundational
In a multi-step application workflow, why might different steps reasonably use different model tiers rather than one tier for the whole workflow?
Not quite.
This is the per-step model-matching principle at workflow scale: individual steps within one pipeline often have very different real difficulty, so matching each step's model tier to its actual need is typically more efficient overall than forcing one tier across every step regardless of fit. Mixing model tiers within a workflow is a completely standard, well-supported pattern (A is wrong), relevant at any workflow length with more than one step (C is an arbitrary threshold), and output format compatibility is a matter of how each step's output is structured/parsed, not an inherent cross-model incompatibility (D is wrong).
Q71
Model family tradeoffs (capability vs cost vs speed)
Applied
A latency-sensitive, real-time application (e.g., a live voice assistant) needs to choose a model primarily optimized for response speed. What should the team be aware of when trading capability for speed in this way?
Not quite.
Speed and capability are generally in tension — a model chosen primarily for low latency is a real tradeoff, not a free lunch (A is wrong to claim no tradeoff exists), and understanding where that model's capability boundary actually sits for the specific use case (and planning for what happens when a request exceeds it — graceful degradation, escalation, or a fallback path) is the responsible way to adopt a speed-optimized choice. Such models are used successfully in many conversational applications (C is wrong), and this consideration matters regardless of request volume — even a low-volume real-time application faces the same per-request tradeoff (D is wrong).
Q72
Model family tradeoffs (capability vs cost vs speed)
Advanced
A team initially picks the most capable model 'to be safe' during prototyping, planning to 'optimize down to a cheaper model later' before launch. What risk does this plan carry?
Not quite.
This is a realistic, common trap: a prompt and set of expectations shaped around a highly capable model's behavior may not carry over cleanly to a cheaper model (different strengths, different failure modes), so 'optimize later' isn't a free deferred action — it's real re-testing and possibly re-tuning work, which is easy to never get around to once the (expensive but working) version has already shipped and other priorities take over. Changing model tier post-launch is technically straightforward (C is wrong) — the risk is organizational/prioritization, not technical feasibility — and it's a risk regardless of how long the prototype phase specifically runs (D is an arbitrary and irrelevant constraint).
Q73
Model family tradeoffs (capability vs cost vs speed)
Foundational
What is a sound default strategy for choosing a model tier when starting a new feature, before extensive benchmarking data exists?
Not quite.
A sensible default treats tier selection as an evidence-driven decision over time: starting capable enough to validate the feature is viable at all (rather than risking an underpowered model masking whether the feature idea even works), then revisiting with real usage data once it exists to see if a cheaper/faster tier still clears the bar. Defaulting to cheapest-always (A) without regard for task fit risks masking real feature viability with model limitations, while never reconsidering a more capable starting choice (B) leaves cost savings on the table once real data could justify a change. Tier choice doesn't need to be random or deferred all the way to full launch (C) — meaningful evaluation can and should happen earlier, incrementally.
Q74
Model family tradeoffs (capability vs cost vs speed)
Applied
Two different model tiers are benchmarked on the same task and produce nearly identical accuracy, but one has notably lower per-request latency. All else being equal, why might the faster model still not be the automatic right choice?
Not quite.
A benchmark result is only as good as how representative it is — matching accuracy and better latency on a specific, possibly narrow test set doesn't automatically mean cost is equivalent, or that the benchmark covers the real diversity of production traffic; a model that looks equivalent on a limited benchmark can still diverge once exposed to the full range of real inputs. This connects to the evaluation-drift and benchmark-representativeness concerns discussed for monitoring generally. Latency is a real, legitimate factor in model choice (B is wrong to dismiss it), and there's no law guaranteeing faster models are always proportionally less accurate (D is wrong) — ties absolutely happen and are exactly what makes this decision non-trivial.
Q75
Choosing model per task/step
Foundational
What is the first question to ask when selecting which model to use for a specific application feature?
Not quite.
Sound model selection starts from the task's actual requirements — reasoning depth needed, acceptable latency, volume and cost sensitivity — rather than defaulting to novelty (A), lowest price regardless of fit (C), or personal preference (D), all of which skip the actual fit analysis. A model that's cheapest but can't reliably complete the task isn't actually a cost saving — it's a correctness failure with a lower price tag.
Q76
Choosing model per task/step
Applied
A document-processing pipeline has a classification step (route to one of 4 document types) followed by an extraction step (pull out specific fields with careful reasoning about ambiguous cases). How should model selection likely differ between these two steps?
Not quite.
This is a direct, concrete application of per-step model matching: a 4-way classification is comparatively simple and well-bounded, a good fit for a lighter/faster model, while extraction requiring judgment about ambiguous cases benefits more from a more capable model's reasoning — matching each step's choice to its real difficulty rather than forcing uniformity (A, D are both wrong to insist on blanket consistency) or picking based on pipeline position rather than actual difficulty (B is wrong — order in the pipeline has no bearing on which step needs more capability).
Q77
Choosing model per task/step
Advanced
A team is unsure whether a specific step in their pipeline truly needs the more capable (and costlier) model tier, or whether a lighter one would suffice. What is a rigorous way to answer this, rather than guessing?
Not quite.
This is the evaluation discipline from monitoring and reliability applied pre-launch: a representative-sample, head-to-head comparison against a real success criterion for that specific step turns a guess into an evidence-based decision, which generalizes the same principle of measuring rather than assuming used throughout agentic reliability design. Guessing (B) or defaulting to the cheaper option without testing (C) both skip exactly the step that would reveal whether the fit assumption is actually correct. This evaluation is something the application team can and should run themselves against their own task and data (D is wrong) — it doesn't require provider involvement.
Q78
Choosing model per task/step
Foundational
Why might a task involving creative or open-ended writing (e.g., drafting marketing copy) generally warrant a more capable model than a task involving strict data extraction from a well-structured source?
Not quite.
The breadth and subjectivity of 'reasonable' output differs a lot between these task types — open-ended creative writing has a wide quality range where a more capable model's nuance and judgment tend to pay off more, while narrow, mechanical extraction from well-structured input has less room for a capability ceiling to actually bind, since the task is comparatively well-defined. This isn't a binary capability/incapability distinction — smaller models can attempt creative tasks, just potentially with a lower quality ceiling (C overstates it as impossible), and the reasoning generalizes to any open-ended-vs-narrow task pairing, not just marketing specifically (D is too narrow).
Q79
Choosing model per task/step
Applied
A step in a pipeline is currently using a capable model but the team observes its output quality barely changes when they experimentally swap in a lighter model. What does this observation suggest about their original model choice for that step?
Not quite.
This is exactly the outcome the rigorous, evidence-based evaluation approach is meant to surface: output quality staying roughly flat across a model swap for a given step is a real signal that the step's actual difficulty doesn't require the more expensive model's extra capability, which should prompt switching that specific step to realize a cost/latency win with little to no quality cost. Dismissing this evidence (B) defeats the entire purpose of running the comparison in the first place. The finding is scoped to this specific step's demands — it says nothing about other steps with different requirements in the same application (D overgeneralizes).
Q80
Choosing model per task/step
Advanced
A step's task difficulty varies significantly by input — most inputs are simple, but a meaningful minority are genuinely hard. What architectural pattern addresses this better than picking one fixed model tier for the step?
Not quite.
When difficulty varies meaningfully by input rather than being uniform across the whole step, a fixed single-tier choice forces a tradeoff between overpaying for the easy majority or underserving the hard minority — a routing/escalation pattern (try cheap first, escalate on a signal of difficulty or low confidence) matches cost to actual per-input need instead. This is a real, implementable architecture (A is wrong to claim no better option exists), and while always using the most capable model (B) avoids routing complexity, it reintroduces exactly the unnecessary-cost-for-easy-inputs problem the routing pattern is designed to solve. The pattern's value scales with volume, and is relevant well below an arbitrary low daily threshold (C is wrong).
Q81
Choosing model per task/step
Foundational
In an escalation-based model-routing pattern (light model first, escalate on failure/low-confidence), what determines whether the lighter model's attempt should be escalated?
Not quite.
The escalation decision needs to be driven by some concrete signal that the lighter model's specific attempt is unreliable — low confidence, a failed validation check, an unmet success criterion — so that escalation happens precisely when needed rather than arbitrarily (B, which defeats the whole cost-matching purpose) or requiring manual intervention on every request (C, which defeats the automation purpose). This is a genuinely two-tier pattern where the lighter model handles the (typically larger) share of inputs it can manage fine, with escalation reserved for the subset that actually need it — not a disguised way of always using the expensive model (D is wrong).
Q82
Token usage optimization
Foundational
Why does reducing unnecessary tokens in a prompt matter beyond just the direct cost savings?
Not quite.
Token count affects more than the bill — fewer unnecessary tokens generally means faster processing (lower latency) and leaves more context-window headroom for content that genuinely matters to the task, making this a multi-dimensional efficiency concern rather than a narrow cost-accounting one (D is too narrow a framing). It's not an automatic quality tradeoff (C is wrong) — the entire point of 'unnecessary' tokens is that they weren't contributing to quality in the first place, so removing them is close to a free efficiency win, not a quality sacrifice.
Q83
Token usage optimization
Applied
A system prompt includes several paragraphs of flowery, conversational framing before getting to the actual instructions. What token-optimization opportunity does this represent?
Not quite.
Verbose phrasing that doesn't carry additional information content is a genuinely low-risk place to optimize — tightening it to be concise while keeping the actual substantive instructions intact doesn't meaningfully risk quality, unlike trimming something that's actually doing real work (a necessary example, an important constraint). This isn't true for content that IS doing work, which is why 'proportional to length' (A) is the wrong mental model — it's about information density, not raw length. System prompts remain fully editable in production (B is wrong), and this optimization is worth doing regardless of absolute token count, not gated to extremely long prompts only (D is wrong) — it compounds with every single call that prompt is used in.
Q84
Token usage optimization
Advanced
A team wants to reduce token usage in their RAG application by retrieving fewer chunks per query. What tradeoff does this introduce?
Not quite.
This is a genuine recall-versus-efficiency tradeoff: fewer retrieved chunks means less token cost and latency, but also a higher chance of missing a chunk that actually contained the answer, especially when the needed information happens to be spread across more sources than the reduced retrieval count covers — it's not automatically quality-neutral or quality-improving (A is wrong). Chunk count directly affects total tokens sent, not just latency (C is wrong) — more chunks means more content in the prompt. This tradeoff is about the query/knowledge-base's actual information distribution, not knowledge-base size specifically (C is the wrong variable).
Q85
Token usage optimization
Foundational
How does output token usage factor into total cost differently from input token usage, and why does this matter for optimization priorities?
Not quite.
Output tokens are commonly priced at a different (often higher) rate than input tokens, and critically, output length isn't directly set by the application the way input content is — it emerges from the model's response, making instructing toward appropriately concise, well-scoped answers (or bounding with max_tokens) a distinct and meaningful optimization lever from trimming input. Output tokens are real, billed usage, not free (C is wrong), and this cost applies identically whether the response is streamed or not — streaming changes delivery, not the underlying token accounting (D is wrong).
Q86
Token usage optimization
Applied
An application asks the model to always restate the full original question before answering it, as part of every response. What token-optimization opportunity does this represent, and what should replace it if the restatement isn't actually needed downstream?
Not quite.
If the restated question isn't actually used by anything downstream (not displayed, not parsed, not needed for clarity in context), it's output-token overhead paid on every single call for no real benefit — removing the instruction to restate and letting the model answer directly is a straightforward, no-loss optimization in that case. This is a prompt-level instruction change, not something requiring model retraining (C is wrong), and the savings scale with call volume regardless of the original question's length — even a short restated question adds up across many calls (D is the wrong variable).
Q87
Token usage optimization
Advanced
A team wants to optimize token usage but is unsure which part of their pipeline (prompt, retrieved context, conversation history, or output) contributes most to total cost. What is the correct first step?
Not quite.
This is the same measure-before-optimizing discipline that runs throughout reliability and cost engineering: without breaking down actual token usage by component across real traffic, optimization effort is a guess (A) rather than being directed at whatever is actually driving the majority of cost — which component dominates varies a lot by application (a RAG-heavy app might be retrieval-context-dominated; a long-chat app might be history-dominated). Equal effort across all four regardless of contribution (B) wastes effort on low-impact areas. Token usage is fully measurable and attributable by component via logging/instrumentation, not an opaque black box (C is wrong).
Q88
Latency optimization techniques
Foundational
Beyond choosing a faster model tier, what is another application-level lever for reducing perceived latency in a user-facing feature?
Not quite.
This connects back to the streaming subtopic: independent of model tier, streaming is a real, application-level lever for perceived latency — content appears incrementally rather than after a full wait, which users experience as more responsive even though total generation time is unchanged. This directly contradicts the claim that model tier is the only lever (A) or that no application-level design choice matters (B). Reducing max_tokens (D) can shorten worst-case generation time but is a narrower, more blunt lever than streaming, and isn't the only other option available.
Q89
Latency optimization techniques
Applied
An application makes several sequential, independent API calls as part of handling one user request (e.g., three separate classification calls on unrelated pieces of the input). How can latency be improved here, assuming the calls don't depend on each other's output?
Not quite.
This is the parallel-vs-sequential tradeoff applied directly to independent API calls within a single request: since the three classification calls don't depend on each other's results, there's no architectural reason to force them sequentially — running them concurrently cuts wall-clock latency to roughly the slowest single call instead of their sum. Parallel API calls are fully supported, not prohibited (C is wrong), and the benefit applies starting from just two independent calls — no minimum count like 50 is required to see a latency win (D is wrong).
Q90
Latency optimization techniques
Advanced
An application wants to reduce latency for a specific, frequently-asked type of query by pre-computing and caching the full response ahead of time, rather than calling the model live each time. What is the key risk of this approach?
Not quite.
This is the same cache-correctness tradeoff discussed earlier (aggressive caching risking stale or contextually wrong responses) applied to a specific latency-optimization technique: pre-computing and serving a cached answer is only as good as the assumption that the query is truly uniform across users and time — if underlying facts change, or if what looks like 'the same query' actually has meaningful per-user variation, the cached response can be wrong in a way that's invisible until it causes a problem. Pre-computing responses is a standard, implementable pattern (C is wrong), and the risk is about correctness/staleness, not response length (D is the wrong variable).
Q91
Latency optimization techniques
Foundational
How does prompt caching (discussed earlier for applications generally) specifically function as a latency optimization, not just a cost one?
Not quite.
The same underlying mechanism (reusing a previously processed prefix instead of reprocessing it) delivers both benefits together — a cache hit skips fresh processing of that portion, which reduces both cost and processing time/latency simultaneously, not just one or the other (A is wrong). Cache-checking overhead doesn't outweigh the saved reprocessing time, so it doesn't net-increase latency (B is wrong) — and the latency benefit applies to every subsequent cache hit, not merely the first call (which is actually the write, not a hit at all — D has it backwards).
Q92
Latency optimization techniques
Applied
A multi-step agentic workflow's total latency is dominated by waiting on several sequential tool calls, each of which depends on the previous step's output. What realistic latency optimization remains available here, given the calls cannot be parallelized due to their dependency chain?
Not quite.
When steps are genuinely dependent, cross-step parallelization is off the table, but that doesn't mean latency optimization is exhausted — each individual step still has its own latency footprint (model tier choice, prompt size, unnecessary extra round-trips) that can be optimized independently of the sequential structure, so total latency still improves even though the dependency chain itself remains. Arbitrarily removing a dependency that the task logically requires (B) risks correctness for a latency gain that isn't actually sound, and collapsing a genuinely multi-step task into one call (D) may not be feasible if the steps truly need to happen in sequence with intermediate results — that's a different, more drastic redesign, not a guaranteed-available option.
Q93
Latency optimization techniques
Advanced
A team wants to optimize for latency but hasn't measured where time is actually being spent across their pipeline (model inference, network round-trips, application-side processing, etc.). What is the risk of optimizing without this breakdown?
Not quite.
This is the same measure-before-optimizing principle from token-cost attribution, applied to latency: without knowing where time is actually going, effort can easily land on a component that isn't the real bottleneck (trimming prompt tokens when the actual slow part is an unrelated downstream database call), producing real engineering effort with little to no measurable improvement where it matters. This is a real, avoidable risk, not a non-issue (A is wrong), applies regardless of pipeline length (C is an arbitrary threshold), and component-level latency is very much measurable via standard instrumentation/tracing (D is wrong).
Q94
Cost monitoring & budgeting
Foundational
Why should a production application implement cost monitoring (tracking actual API spend over time) rather than only estimating cost once during initial design?
Not quite.
This is the same ongoing-monitoring-versus-static-pre-launch-estimate principle discussed for evaluation generally, applied to cost specifically: real production traffic, prompt lengths, and usage patterns routinely diverge from initial design-time estimates and can drift further as the application evolves, so only ongoing monitoring can catch a cost surprise (or a newly available optimization) as it develops rather than discovering it well after the fact. The two concerns are directly related, not separate (D is wrong), and the discipline is valuable at any spend level, not gated to a specific large monthly threshold (C is wrong) — a small application can still blow its budget by proportion just as easily.
Q95
Cost monitoring & budgeting
Applied
An application's cost monitoring shows a sudden, unexplained spike in API spend. What should this trigger, architecturally?
Not quite.
An unexplained cost spike is exactly the kind of monitored-metric anomaly that warrants investigation, not dismissal (A is wrong) — the root cause could be anything from a genuine bug (a runaway retry loop, an unbounded agentic loop) to legitimate growth, and only investigation reveals which. An immediate, permanent full shutdown (B) is a drastic overreaction compared to a measured investigation, especially if the cause turns out to be legitimate. Spikes can absolutely originate from the application's own behavior (a bug, a usage pattern shift), not only from external pricing changes (C is wrong and too narrow).
Q96
Cost monitoring & budgeting
Advanced
A team sets a hard spend cap on their API usage to avoid runaway cost, such that requests are rejected once the cap is hit within a billing period. What user-experience consideration does this introduce, paralleling a lesson from tool-boundary enforcement?
Not quite.
This directly parallels the tool-boundary-enforcement lesson (a safe, deterministic limit should still be paired with good communication, not just silent enforcement): a hard spend cap is a sound safety mechanism against runaway cost, but if it causes requests to silently fail once hit, users face a confusing, unexplained broken experience — the fix is pairing the cap with graceful degradation and clear user-facing messaging, not removing the cap. Spend caps are a real, implementable application-level control (C is wrong — they're enforced by how the application manages its own usage, not something the API prohibits), and the UX consideration scales with however many users are affected, mattering even for a single-user application facing an unexplained failure (D is wrong).
Q97
Cost monitoring & budgeting
Foundational
What is the benefit of attributing API cost to specific features or user segments, rather than only tracking total aggregate spend?
Not quite.
This is the same measure-the-actual-breakdown principle applied to cost attribution specifically: knowing total spend alone gives no way to target optimization effort, while per-feature or per-segment breakdown reveals exactly where cost concentrates, enabling focused rather than guessed-at optimization (directly connecting to the earlier token-usage-attribution question). This is measurable via standard request tagging/logging practices (B is wrong), and the value of knowing where cost concentrates applies at any scale, not gated to massive user bases only (D is wrong) — even a modest application benefits from knowing which feature is actually expensive.
Q98
Cost monitoring & budgeting
Applied
An agentic feature in an application has no bound on how many tool calls or iterations a single task can consume. What cost-monitoring and budgeting risk does this create, connecting to the retry-cap and iteration-budget principles discussed for agentic reliability?
Not quite.
This directly connects iteration budgets (originally framed around preventing a hung task from running forever) to their cost dimension: every additional iteration or tool call is also additional spend, so an unbounded loop isn't just a time risk, it's an unbounded cost risk too — the same budget cap that bounds worst-case duration also bounds worst-case cost for that task. This is a real, not theoretical, risk (A is wrong to assume natural quick termination is guaranteed), and while tool-calling loops are a particularly visible case, any iterative generation process (even pure text, if allowed to loop or retry unboundedly) carries an analogous risk (C is too narrow). Iteration/cost caps are a standard, implementable architectural mitigation (C is wrong).
Q99
Cost monitoring & budgeting
Advanced
A team wants to forecast future API cost as their application scales, rather than only reacting to current spend. What makes this forecasting non-trivial, beyond simple linear extrapolation of current spend?
Not quite.
Real cost drivers rarely scale as a clean linear function of user count — conversation lengths can grow as a product matures and users engage more deeply, feature mix can shift toward more or less expensive capabilities, and optimization efforts (caching adoption, better model-tier matching) can offset raw volume growth — any of which makes naive linear extrapolation wrong, potentially significantly so in either direction. This nuance matters well before an application reaches massive scale — early, accurate forecasting is often most valuable precisely because it informs decisions before costs materialize (C is backwards), and forecasting ahead of realized scale is the entire point of the exercise, not something only possible in hindsight (D is wrong).
Q100
Cost monitoring & budgeting
Foundational
Why is it useful to track cost per unit of business value (e.g., cost per resolved support ticket) rather than only tracking raw API spend in isolation?
Not quite.
Raw spend alone can't distinguish 'healthy growth' from 'a real problem' — rising spend that tracks proportionally rising value delivered (more tickets resolved, more tasks completed) is a sign of healthy scaling, while the same raw number against flat or declining value is a real efficiency problem; only a per-unit-of-value framing makes that distinction visible (A is wrong to claim isolated spend is always more meaningful). This generalizes to any application with a definable unit of value, not just support tickets specifically (C is too narrow), and it's a standard, calculable metric (spend divided by value-units delivered) wherever a meaningful value unit exists (D is wrong).
Q101
Agent loop basics for developers
Foundational
At the implementation level, what is the basic loop structure a developer builds when giving Claude access to tools?
Not quite.
This is the concrete, implementation-level shape of the agent loop: the application code itself is responsible for detecting a tool-use response, executing the actual tool (that's application code, not something the API does on its own), feeding the result back as a new message, and repeating — the API does not internally manage multi-step tool execution for the application (A is wrong). This is standard practice for any custom application doing tool use via the API directly, not exclusive to Claude Code (D is wrong), and it's built on the same Messages API, not a separate one-shot-only endpoint (B is wrong).
Q102
Agent loop basics for developers
Applied
A developer builds an agent loop with no maximum iteration count, assuming the model will naturally stop calling tools once the task is done. What production risk does this create?
Not quite.
This is the iteration-budget principle made concrete at the implementation level: relying on the model to always naturally stop (A overstates reliability that doesn't hold under all conditions — a hard task, an unexpected or confusing tool result, or a genuine edge case can all extend a loop) leaves no ceiling on worst-case time and cost. The API itself doesn't impose this cap on the application's behalf (C is wrong) — enforcing a maximum iteration count is the calling application's responsibility. The risk exists for read-only tool loops too, not just ones with side effects (B is wrong) — even non-destructive loops have real time/cost cost if they run too long.
Q103
Agent loop basics for developers
Advanced
In a hand-rolled agent loop, after executing a tool and getting a result, what message role and structure should the tool's result be sent back as?
Not quite.
Tool results are sent back as a structured content block tied to the originating tool-use request's id, which is what lets the model correctly correlate which result answers which call — this matters a lot once a single turn can include multiple simultaneous tool calls, where ambiguity about which result goes with which request would be a real problem. Free-form, unstructured text (B) loses this correlation. Results absolutely need to go back to the model for it to continue reasoning with them (C is wrong — that defeats the entire point of tool use), and they belong in the same ongoing conversation, not a disconnected new one, since the model needs the full context to make sense of the result (D is wrong).
Q104
Agent loop basics for developers
Foundational
Why does the application, not the model itself, bear responsibility for actually executing a requested tool call (e.g., running the database query, calling the external API)?
Not quite.
This is a fundamental architectural boundary: the model's output is a structured description of an intended action, not the action itself — it has no direct access to execute code, query a database, or reach external services, so the application is necessarily the one that interprets the request and actually performs it, then reports the result back. This division applies to every kind of tool, not a special case for financial ones specifically (C is wrong), and it's a fixed architectural boundary, not something that varies arbitrarily per call (D is wrong) — the model genuinely cannot execute tools directly by design (A is wrong).
Q105
Agent loop basics for developers
Applied
A developer's agent loop executes a requested tool call synchronously and blocks on it before continuing. For a tool that takes a long time to complete (e.g., a slow external API), what architectural option exists to avoid blocking the entire application during that wait?
Not quite.
Synchronous blocking is the simplest implementation but not the only one — depending on the application's architecture, slow tool execution can be handled asynchronously (tracking the in-progress task and updating the user, or structuring the server to not let one slow tool call block its ability to serve other concurrent requests). This is a real architectural choice, not a forced constraint with no alternative (A is wrong), and removing a genuinely useful but slow tool (B) is a much more drastic response than necessary when async handling is available. This concern is relevant even at modest concurrency — a single slow synchronous call can degrade experience well before reaching 1,000 concurrent users (D is an arbitrary threshold).
Q106
Agent loop basics for developers
Advanced
A developer wants to let a user observe an agent loop's progress in real time (which tool is being called, intermediate results) rather than seeing only the final answer. What does this require architecturally, beyond the basic loop?
Not quite.
The basic loop (per question 1) just handles the request/execute/respond mechanics — surfacing that intermediate state to a user-facing UI in real time is additional application-level work: explicitly wiring each iteration's tool name, arguments, and results out to whatever renders the progress view, as it becomes available during the loop's execution. This doesn't happen automatically (A is wrong) but is a well-supported, common pattern (B is wrong to call it impossible) built on the same Messages API and streaming mechanics already discussed, not a specialized separate API (C is wrong).
Q107
Tool use implementation patterns
Foundational
When defining a tool for the API, what three things does the tool definition need to specify at minimum?
Not quite.
This is the concrete, minimum structure behind the tool-description-as-interface-contract principle: name, description, and input schema together form what the model uses to decide when and how to invoke the tool correctly — none of these are inferred automatically with no input (B is wrong), and the description genuinely matters for model behavior, not optional or inert (C is wrong, directly contradicting the interface-contract principle). Price and execution time (D) aren't part of the tool definition schema the model reads — those are operational/billing concerns handled elsewhere, not part of the model-facing contract.
Q108
Tool use implementation patterns
Applied
A tool's input schema marks a field as optional when the underlying function actually requires it to be present to work correctly. What problem does this schema/implementation mismatch create?
Not quite.
The model treats the schema as the actual contract — if it says a field is optional, the model may reasonably omit it sometimes, and if the underlying implementation actually needs that field to function correctly, this produces a real failure downstream at execution time. The schema needs to accurately reflect real requirements, not be more permissive than the implementation actually tolerates (contradicting A's claim that schemas are merely cosmetic). This mismatch risk applies to any parameter type, not just strings (C is wrong), and matters even for a tool with very few parameters — a single incorrectly-optional required field is enough to cause failures (D is wrong).
Q109
Tool use implementation patterns
Advanced
A developer is deciding whether to expose a complex multi-step business process (e.g., 'process a return') as one single tool, or as several smaller tools the model calls in sequence (e.g., 'look up order,' 'validate return eligibility,' 'issue refund'). What factors should drive this decision?
Not quite.
This is a genuine design tradeoff, not something that's inconsequential (A is wrong): breaking a process into smaller tools gives the model visibility and a chance to exercise judgment or react to an intermediate result (e.g., stopping if eligibility fails before attempting a refund), which matters when that judgment is actually valuable; consolidating into one tool is better when the process is rigid and always executes identically, since it reduces the model's job to a single correct call rather than a correctly-ordered sequence of several. Neither 'always one tool' (B) nor 'always maximally decomposed' (D) is a universal rule — the right granularity depends on how much intermediate judgment actually matters for that specific process.
Q110
Tool use implementation patterns
Foundational
What is the purpose of the `tool_choice` parameter (or equivalent) in the API, which can force, allow, or prevent tool use for a given call?
Not quite.
This parameter gives the application deliberate, per-call control over tool-use behavior rather than always deferring entirely to the model's judgment — forcing a specific tool is a common pattern for guaranteeing structured output, allowing free choice suits open-ended agentic behavior, and disabling tool use suits a call that should only produce text. This has a real, meaningful effect on behavior (B is wrong), can be set per individual call rather than being conversation-wide and fixed (C is wrong), and is part of the core API available across SDKs and raw HTTP alike, not Python-exclusive (D is wrong).
Q111
Tool use implementation patterns
Applied
A developer forces tool use (via `tool_choice`) to guarantee the model's response is a specific tool call matching a defined schema, effectively using tool calling as a structured-output mechanism. What should still be validated after this?
Not quite.
This directly reconnects to the schema-is-an-upper-bound-not-a-guarantee principle: forcing a tool call reliably produces output matching the schema's shape, but schema conformance and semantic/business correctness are different guarantees — a syntactically valid argument can still be wrong in content (e.g., a well-formed but logically invalid value). The same defense-in-depth validation discipline still applies on receipt, not something this technique exempts (A is wrong to call it an absolute guarantee of correctness). This is in fact a common, well-established way to get structured output (B is wrong), and the validation need isn't limited to numeric fields — any field can be schema-valid but semantically wrong (C is wrong).
Q112
Tool use implementation patterns
Advanced
A developer defines a tool whose input schema includes a deeply nested object with several levels of sub-objects and arrays. What practical implementation challenge does this raise, beyond what a flatter schema would?
Not quite.
This is a practical, implementation-level instance of the complexity-increases-error-surface principle: more nesting levels mean more places a generated call can go subtly wrong (a missing field three levels deep, a malformed nested array), and more complexity for a developer to validate and debug when it does — flattening the schema where reasonable, or decomposing into multiple simpler tool calls, are real mitigation strategies. Nested schemas are fully supported by the API, not rejected outright (B is wrong), and the practical difficulty shows up at far shallower nesting than an extreme 50-level threshold — even two or three levels can meaningfully increase error surface (D is wrong).
Q113
Workflow orchestration (sequential/parallel in code)
Foundational
At the code level, what determines whether two steps in a workflow should be implemented as sequential (one `await`ed after another) or parallel (both started concurrently)?
Not quite.
This is the dependency-driven parallel-vs-sequential principle made concrete at the implementation level: if step B genuinely needs step A's output as input, B must wait for A (sequential); if the two steps don't depend on each other's results, running them concurrently is both correct and faster in wall-clock terms. This isn't an arbitrary stylistic choice with no real effect (A is wrong), isn't a blanket safety requirement to always go sequential regardless of actual dependency (C is wrong), and parallel execution is achievable with standard language-level concurrency constructs (async/await, promises, threads) — no specialized SDK is required (D is wrong).
Q114
Workflow orchestration (sequential/parallel in code)
Applied
A developer runs three independent classification calls in parallel using `Promise.all` (or equivalent). One of the three calls fails while the other two succeed. What should the application's error handling account for?
Not quite.
Independent parallel calls genuinely can partially fail — a naive `Promise.all` would reject entirely on the first failure by default, potentially losing visibility into the two results that actually succeeded, so handling partial failure explicitly (e.g., using an all-settled pattern and deciding how to treat the lone failure) is a real implementation concern, not something that resolves itself automatically (B, C both wrong to assume atomic all-or-nothing behavior). This consideration applies to any independent parallel calls, whether they involve tool use or are plain generation calls (D is wrong) — the partial-failure risk is about concurrency, not call type.
Q115
Workflow orchestration (sequential/parallel in code)
Advanced
A workflow has five steps where steps 2 and 3 are independent of each other but both depend on step 1, and step 4 depends on both 2 and 3, with step 5 depending on step 4. How should this be orchestrated for the best latency, and what does this illustrate about dependency structure?
Not quite.
This illustrates that real workflows are often graphs, not simple chains — the correct orchestration follows the actual dependency structure: step 1 alone, then 2 and 3 concurrently (since each only depends on 1, not on each other), then step 4 waiting on both, then step 5. Treating the whole workflow as fully sequential just because some dependencies exist somewhere (A) wastes the real parallelization opportunity between 2 and 3. Being part of the 'same workflow' doesn't itself create a dependency between 2 and 3 (B is wrong) — their actual data dependencies (both depending only on step 1) are what matters. The presence of any dependency in the graph doesn't preclude parallelizing the independent parts (C is wrong).
Q116
Workflow orchestration (sequential/parallel in code)
Foundational
Why might a developer choose to implement workflow orchestration using an existing workflow/orchestration framework or library rather than hand-rolling the control flow from scratch?
Not quite.
This is a standard build-versus-reuse engineering tradeoff: an existing orchestration framework often bundles retry logic, state persistence across steps, observability, and dependency management that a hand-rolled implementation would need to build and maintain itself — worth considering deliberately rather than assuming hand-rolling is always better (A is wrong) or defaulting to a framework without reason. Such frameworks integrate with the Claude API the same way any other external call would be wired into their step model — there's no inherent incompatibility (B is wrong), and the tradeoff is worth considering at far smaller workflow sizes than 100 steps, even a handful of steps with real reliability needs can benefit (D is wrong).
Q117
Workflow orchestration (sequential/parallel in code)
Applied
A workflow step occasionally needs to branch — taking a different subsequent path depending on the result of a model call (e.g., if sentiment is negative, escalate; otherwise, proceed normally). How is this typically implemented, compared to a workflow with no branching?
Not quite.
Branching in an orchestrated workflow is just ordinary application-level conditional logic — the model produces output (ideally structured, so it's reliably inspectable) the application code reads and uses to decide which path to take next; the API itself doesn't execute branching logic directly, that control flow lives in the calling application, same as any other conditional in regular code. There's no dedicated 'branch' API parameter (C is wrong) — this is standard programming, not an API feature — and a workflow can branch into any number of paths, not capped at exactly two (D is wrong); branching workflows are a completely standard, supported pattern (A is wrong).
Q118
Workflow orchestration (sequential/parallel in code)
Advanced
A long-running, multi-step workflow needs to survive an application restart partway through (e.g., a server redeploy) without losing its progress. What architectural requirement does this impose, beyond the basic sequential/parallel orchestration logic?
Not quite.
In-memory state is lost on restart by definition — a workflow that needs to survive that event requires externally persisted progress (written to a database or equivalent durable store at each meaningful step boundary), so a freshly started process can read that state back and resume rather than starting over. This is precisely the scratchpad/resumability principle discussed for agentic reliability, applied concretely to workflow orchestration (contradicting B's claim that in-memory state survives automatically). Collapsing everything into one atomic step (C) isn't a real option for a genuinely multi-step process with real intermediate state — it would just remove the ability to track partial progress at all. This requirement is about the possibility of restart, not gated to very long-running workflows specifically — even a workflow taking a few minutes can be interrupted by a redeploy (D is wrong).
Q119
Error handling in agent workflows
Foundational
In an agent loop, a tool call fails with an error. What should the application do with that error, architecturally?
Not quite.
This connects the structured-tool-error-response principle to the hand-rolled loop's actual implementation: the application catches the failure and feeds a structured error result back as the tool's result message, giving the model the information to decide what to do next (retry, try differently, inform the user) rather than the loop crashing outright (A is wrong) or silently pretending success (B is wrong, which risks the model proceeding on a false premise). Tool call errors are a normal, expected possibility in any real system with external dependencies, not something a 'correct' implementation eliminates entirely (C is wrong) — robust error handling is part of what makes an implementation correct, not evidence that errors shouldn't occur.
Q120
Error handling in agent workflows
Applied
An agent loop's tool call fails, the error is fed back to the model, and the model tries again with slightly different arguments — but it keeps failing and retrying in a way that never converges on success. What should bound this behavior?
Not quite.
This is the retry-cap principle applied to a concrete, realistic failure mode: a model retrying with variations that still don't converge is exactly the non-converging scenario a bounded retry cap exists to catch, escalating to the application/user once the cap is hit rather than letting the loop run (and spend) indefinitely (A is wrong to suggest no bound is needed). Well-written error messages help the model retry more intelligently but don't guarantee eventual convergence for every possible failure, so this situation remains a real possibility even with good error design (C overstates what good messages can guarantee). Removing the tool entirely (C) is a much more drastic response than simply bounding and escalating the specific failing interaction.
Q121
Error handling in agent workflows
Advanced
A workflow step depends on an external API that is temporarily down. The agent loop keeps retrying that single step while the rest of the otherwise-independent workflow (steps that don't depend on this external API) sits idle waiting. How could this be architected better?
Not quite.
This connects dependency-driven orchestration directly to error handling: if other steps don't actually depend on the failing one's output, there's no architectural reason they need to wait — they can proceed concurrently while the failing step retries independently in the background, making better use of the time rather than leaving the whole workflow idle on a dependency it doesn't yet need (B is wrong to insist on strict full-sequential waiting regardless of real dependencies). Abandoning the whole workflow on any single failure (C) is a much blunter response than necessary when independent progress is possible, and this is an application-architecture decision entirely unrelated to model provider choice (D is a non-sequitur).
Q122
Error handling in agent workflows
Foundational
Why is it important for error handling logic in an agent workflow to distinguish between an error in the model's own response (e.g., an API error from Anthropic) versus an error from a tool execution the application performed?
Not quite.
These are genuinely different failure categories needing different handling: an API-level error (rate limiting, a 5xx) is about the call to Anthropic itself and calls for the backoff/retry handling discussed in the rate-limits subtopic, while a tool execution error is about something that went wrong in the application's own tool code and calls for feeding structured error context back to the model so it can adapt — treating both identically (A) risks applying the wrong recovery strategy to either. This distinction matters even for a single-tool workflow, since API-level errors can occur regardless of tool count (C is wrong), and the two are readily distinguishable in practice (by where in the code the error originates — the API call versus the tool execution code) rather than indistinguishable (D is wrong).
Q123
Error handling in agent workflows
Applied
An agent workflow fails partway through a multi-step task that had already made some real-world changes (e.g., it successfully updated one record before failing on a second, related update). What does this scenario highlight the need for?
Not quite.
A workflow with real-world side effects partway through isn't automatically all-or-nothing just because it's conceptually 'one task' — once step one's effect is real, the application needs visibility into what actually completed before deciding the right recovery: roll back, retry just the remainder, or flag the inconsistent state for human review. Ignoring this (A) risks leaving the system in a silently inconsistent state. This can happen in even a two-step workflow, not gated to longer ones (B is wrong), and always automatically rolling back everything (D) is only one of several valid responses — sometimes retrying the remainder or flagging for review is the more appropriate choice, depending on the situation.
Q124
Error handling in agent workflows
Advanced
A developer wants their agent workflow's error handling to be testable, so failure scenarios can be verified without waiting for a real external failure to occur naturally. What implementation approach supports this?
Not quite.
This is standard testing discipline (test doubles/fault injection) applied to agent workflow error paths specifically: designing the tool execution layer so a controlled, simulated failure can be injected lets retry logic, escalation behavior, and structured error feedback be verified deliberately, rather than depending on waiting for a real failure to happen to occur during testing (A is wrong — that's exactly the limitation deliberate fault injection solves). Error paths are just as important to verify as the happy path, often more so since they're less frequently exercised naturally and more likely to harbor undiscovered bugs (B is wrong to exempt them from testability concerns). Designing for testability/fault injection is a widely recommended practice, not something to avoid (C is wrong).
Q125
Testing agent behavior
Foundational
Why is testing an agentic feature (one involving model calls and tool use) generally harder than testing traditional deterministic code?
Not quite.
Two real properties make this genuinely harder: output non-determinism (the same prompt can yield somewhat different phrasing or even different reasonable decisions across runs) and a much larger, less fully enumerable space of possible inputs and behaviors than a typical deterministic function — exact-output assertions ('the output must be exactly this string') don't map cleanly onto either property, which is why agentic testing needs different strategies (e.g., evaluating against success criteria rather than exact string matches). This isn't untestable, just differently testable, not an excuse for only ad hoc manual checking (C is wrong), and the difficulty isn't gated by tool count — even a single-tool or tool-free generative feature has the same non-determinism and input-space challenges (D is wrong).
Q126
Testing agent behavior
Applied
A team writes a test that asserts a chatbot's response to a given input must exactly match one specific string. What problem does this testing approach run into for an agentic/generative feature?
Not quite.
This is the direct, practical consequence of output non-determinism for test design: an exact-string assertion is brittle against a feature where multiple different phrasings can all be equally valid and correct — the test fails not because the behavior is wrong, but because the wording happens to differ from the one string hardcoded in the test, producing noisy false failures that erode trust in the test suite. This connects to why evaluation-style testing (checking whether a response meets a success criterion, rather than matches one exact string) is more appropriate for generative features. The brittleness exists at any temperature setting where any output variation is possible, not gated to a specific threshold (C is wrong), and exact-string testing is a completely standard, implementable technique in general — it's just poorly suited to this particular kind of feature (D is wrong).
Q127
Testing agent behavior
Advanced
What is a more appropriate testing strategy than exact-string matching for verifying an agentic feature behaves correctly across a range of inputs?
Not quite.
Criterion/property-based evaluation is the standard, practical answer: define what a correct response actually needs to satisfy (contains the right extracted value, calls the expected tool, avoids a specific unsafe pattern) and check for that property across a representative test set, rather than demanding one single exact string — this accommodates valid output variation while still being automated and meaningful (directly contradicting A's claim that no viable alternative exists). This remains automatable, not requiring full manual review of every response (B is wrong), and pre-deployment testing and production monitoring are complementary, not substitutes for each other — relying solely on monitoring after the fact (D) forgoes catching problems before they reach real users, which pre-deployment evaluation is specifically meant to do.
Q128
Testing agent behavior
Foundational
Why is it valuable to maintain a fixed, curated set of test inputs (a regression test suite) for an agentic feature, re-run whenever the prompt or model changes?
Not quite.
A fixed, curated set provides a stable basis for comparison across changes — if a prompt edit or model upgrade causes a previously-passing case to regress, the fixed suite catches it directly, which is exactly the evaluation-drift concern from monitoring, applied proactively pre-ship rather than discovered reactively in production. Using entirely random inputs each run (A) would lose this comparability — there'd be no stable baseline to detect regression against. This value applies just as much to generative features as deterministic ones, arguably more given the non-determinism and drift risks discussed (B is wrong), and it's valuable regardless of a feature's prior bug history — preventing a regression doesn't require having already had a past incident to justify it (C is wrong).
Q129
Testing agent behavior
Applied
A team's regression test suite for an agentic feature only includes 'easy,' typical inputs and no genuinely hard or adversarial edge cases. What gap does this leave in their testing coverage?
Not quite.
This is the same benchmark-representativeness concern from model-tier evaluation, here applied to regression-suite design: a suite covering only easy, typical cases can look perfectly healthy (all green) while missing real regressions or failures that only surface on harder, less common, or deliberately adversarial inputs — passing the suite doesn't mean the feature is actually reliably correct everywhere it needs to be (A overstates what a narrow suite proves). This coverage gap matters for internal tools too, not only externally-facing paying-customer features — a wrong answer is still a wrong answer regardless of audience (C is wrong). Hard and adversarial cases are fully includable in automated suites — there's no technical barrier to writing a difficult or edge-case test (D is wrong).
Q130
Prompt structuring for applications
Foundational
Why do many production prompts use explicit structural markers (e.g., XML-style tags) to separate distinct pieces of content, such as instructions, reference material, and the actual input to process?
Not quite.
Explicit structure (like clearly delimited sections) helps the model reliably tell apart genuinely different roles within one prompt — an instruction to follow, reference material to draw on, and the specific input to act on — reducing ambiguity about what's actually being asked for. This has a real, practical effect on parsing reliability, not merely cosmetic (A is wrong). It's a recommended practice, not an API-enforced requirement — unstructured prompts are still accepted (B is wrong) — and the benefit applies to prompts of any length where multiple distinct content types are mixed together, not gated to very long prompts specifically (C is wrong).
Q131
Prompt structuring for applications
Applied
An application's prompt interleaves instructions and user-provided input without any clear separation (e.g., 'Summarize this: ' immediately followed by raw, unescaped user text that could itself contain phrases that look like instructions). What risk does this create?
Not quite.
This is a security-adjacent instance of the structural-separation principle: without clear delimiting, content from an untrusted source (user input) sitting right next to developer instructions creates real ambiguity about which parts are instructions and which are just data to process, a mechanism that is part of how prompt-injection attacks work — clearly wrapping/delimiting user input is a real, meaningful mitigation, not a cosmetic nicety (contradicting A's assumption the model reliably sorts this out unaided). This risk doesn't depend on relative length (B is wrong) and isn't only fixable by refusing all user input (D is far more drastic than necessary) — structural separation is a practical, usable mitigation that preserves the feature.
Q132
Prompt structuring for applications
Advanced
A team maintains prompts as raw strings concatenated directly in application code, making small wording tweaks directly in the codebase whenever they want to adjust behavior. What architectural downside does this create as the application and team grow?
Not quite.
As an application and team scale, prompts hardcoded inline in application logic inherit all the code-coupling costs discussed for hardcoded secrets earlier: every tweak requires a full code change and deploy, non-engineers with useful input on wording are locked out of easy iteration, and there's no clean way to version, test, or audit prompt changes separately from unrelated code changes — treating prompts as their own managed asset (separate files, a dedicated system) decouples these concerns. This isn't a scale-invariant non-issue (A is wrong) or gated to only very long prompts (C is wrong) — it's a workflow and team-structure cost that grows with how often prompts need iteration, and prompts absolutely remain changeable post-deployment, just coupled to the code deploy process unless decoupled (D is wrong).
Q133
Prompt structuring for applications
Foundational
Why might an application use prompt templates (with variable placeholders filled in at request time) rather than constructing each prompt as an entirely bespoke string per request?
Not quite.
Templating separates the stable, reusable structure (instructions, formatting conventions) from the genuinely variable, per-request content, so improvements to the stable part apply consistently everywhere it's used, duplication is reduced, and subtle drift between different call sites constructing similar-but-not-identical prompts ad hoc is avoided. This is a real, meaningful benefit over bespoke per-request strings (B is wrong), and templates are specifically valuable BECAUSE they have variable content to fill in — a prompt with zero variability wouldn't need a template at all, the opposite of C's claim. The benefit compounds with request volume and reuse, not limited to tiny volumes (D is wrong).
Q134
Prompt structuring for applications
Applied
A prompt template interpolates raw user input directly into a template string without any consideration for how special characters or template-syntax-like content in that input might be handled. What is the architectural risk?
Not quite.
This is a direct analogy to well-known templating risks in other domains (unescaped input corrupting HTML structure, SQL injection from unescaped query interpolation) — if a prompt-templating approach doesn't robustly handle arbitrary input content, user-supplied text containing patterns that collide with the template's own syntax or the structural markers discussed earlier could corrupt the resulting prompt in unintended ways. This isn't a purely theoretical, never-realized concern (B is wrong) — it's the same well-established category of bug that motivates escaping/sanitization practices broadly, and it's a conceptual risk independent of which specific programming language implements the templating (C is wrong) — any language's naive string interpolation is susceptible if not handled carefully.
Q135
Prompt structuring for applications
Advanced
A team wants to make a significant change to a core, widely-used prompt template (used across many different application features). What process consideration does this warrant, beyond simply editing the template and deploying?
Not quite.
A widely-shared prompt template is functionally a shared dependency, just like a shared library function — a change ripples across every feature that uses it, which is exactly the shape of change that warrants regression testing (per the testing-agent-behavior subtopic) and a cautious rollout, not a lower bar just because the change happens to be prompt text rather than traditional code (A is wrong to assume prompt changes are inherently lower-risk). Templates remain editable in place — there's no requirement to always fork into an entirely new template for every change (B is wrong) — and the blast-radius concern is about behavioral impact across features, not specifically gated to changes that happen to affect token count (D is wrong).
Q136
System prompts in production apps
Foundational
In a production application serving many different end users, what is a typical architectural pattern for constructing the system prompt per request?
Not quite.
The typical, practical pattern blends a stable, shared core (role, tone, standing rules — the same across requests, which also supports prompt caching discussed earlier) with a smaller amount of assembled, request-specific context (the current user's relevant state, the specific task at hand) — not a rigidly single static string with zero variation (A is wrong), and definitely not a fully bespoke, individually hand-authored prompt per end user, which wouldn't scale (C is wrong). System prompts are a core, widely-used production pattern, not limited to development/testing (D is wrong).
Q137
System prompts in production apps
Applied
An application's system prompt grants the model a broad persona ('You are a helpful assistant that can discuss anything') for a product that's actually meant to be scoped narrowly (e.g., only answering questions about the company's own product). What problem can this mismatch create?
Not quite.
A system prompt's framing genuinely shapes what the model treats as in-scope — granting a broad, open-ended persona for a product meant to be narrowly focused can reasonably invite the model to engage with off-topic requests the business never intended to support, simply because the prompt told it that's its role. The system prompt should reflect the product's real intended boundaries, which is conceptually the same least-privilege-style scoping principle applied to role/persona framing rather than tool access specifically — directly contradicting D's claim they're unrelated. This mismatch risk exists regardless of the product's pricing model (C is an irrelevant distinction).
Q138
System prompts in production apps
Advanced
A team wants their system prompt to enforce that the assistant never discusses a specific competitor by name, under any circumstances. Given the deterministic-vs-model-judgment principle, how reliable should they expect a system-prompt-only instruction to be for this requirement?
Not quite.
This is the deterministic-vs-model-judgment principle applied directly to a system-prompt scenario: an instruction meaningfully influences behavior (contradicting B's claim of no effect) and will likely work the large majority of the time, but 'most of the time' falls short of the 'under any circumstances' bar the requirement actually states — for that level of guarantee, the same logic that put the refund cap in deterministic code applies here: an application-level check (scanning output before display) provides the hard guarantee that prompt instruction alone cannot (contradicting A's claim of unconditional reliability). Fine-tuning is one possible lever but not the only way to add this kind of guarantee — a post-generation check is a simpler, more directly applicable fix (C is wrong).
Q139
System prompts in production apps
Foundational
Why might a system prompt include explicit instructions about what the assistant should do when it doesn't know an answer or lacks sufficient information, rather than leaving this unaddressed?
Not quite.
Default behavior under uncertainty is exactly the kind of dimension worth shaping explicitly rather than leaving to chance — different applications reasonably want different tradeoffs here (a creative brainstorming tool might want confident speculation; a factual lookup tool wants explicit 'I don't know' over a guess), and an explicit instruction is a concrete, direct way to communicate which behavior this particular application needs rather than assuming default handling happens to match (A is wrong to assume uniformity). This guidance lives naturally in the standing system prompt, applying across the whole conversation rather than needing restatement each turn (C is wrong), and the value of being explicit about this applies broadly, not only in regulated-industry contexts specifically (D is too narrow).
Q140
System prompts in production apps
Applied
An application injects dynamic, per-user data (e.g., the user's name, account tier, recent activity) into the system prompt on every call. What consideration from prompt caching does this interact with?
Not quite.
This is a direct, practical interaction with the caching-prefix-ordering principle discussed earlier: per-user variable content placed early in the system prompt breaks the cache match for the stable content that follows it, losing the caching benefit — the fix is the same structural one discussed for caching generally, placing stable shared content first and variable per-user data after the cacheable boundary, not treating the two concerns as unrelated (A is wrong). Dynamic per-user data is commonly and successfully included in system prompts (B is wrong) — the consideration is about ordering for caching efficiency, not about feasibility, and applies to any variable text data, not specifically image-type content like a profile picture (D is wrong and an odd specific case).
Q141
Context window management in long-running apps
Foundational
What is the first thing an application should do when designing for a conversation that could, in principle, grow indefinitely long?
Not quite.
Given that the context window is finite (established early in the API fundamentals), any application that could plausibly have long-running conversations needs an explicit, deliberate strategy for approaching that limit (truncation, summarization, a hard cap, or some mix) decided during design — not discovered reactively once a real conversation actually hits the wall in production (A is wrong to assume no practical limit needs planning). Long conversations are supportable with the right application-level strategy, just not infinitely so without one (B is wrong), and this planning matters for even a single long-running conversation, independent of total user count (C is wrong) — one user with one very long chat already needs this handled.
Q142
Context window management in long-running apps
Applied
An application simply truncates (drops) the oldest messages once a conversation exceeds a certain length, with no summarization. What is the tradeoff of this approach compared to summarization?
Not quite.
This is a genuine implementation-complexity-versus-fidelity tradeoff: truncation is simple (just drop old messages) but total — nothing about the dropped content survives at all, while summarization is more work to implement but preserves at least a condensed trace of what came before, which matters if older context has any residual relevance later. Whether that residual value justifies the added complexity is specific to the application (a simple support bot might be fine losing ancient history entirely; a long-term coaching assistant might not) — neither approach is unconditionally better (C is wrong), and the tradeoff is relevant well before 1,000 turns — even a few dozen turns raises the same question once the window starts to fill (D is an arbitrary threshold).
Q143
Context window management in long-running apps
Advanced
An application summarizes older conversation turns using a separate Claude API call (asking the model to condense prior history), adding this as an extra step in the application's flow. What operational cost does this summarization strategy itself introduce?
Not quite.
Summarization isn't free just because the output is smaller — generating that summary requires its own API call, consuming tokens (to process the content being summarized) and adding latency, so the strategy is really trading an ongoing per-turn context-budget savings against a periodic summarization cost, a genuine operational tradeoff to account for rather than assume away (A is wrong to call it free). This cost applies on the first summarization event, not only if it recurs multiple times per conversation (C is wrong) — even one summarization call has this cost. Summarization via a standard API call is a completely standard, implementable technique using the same Messages API already discussed (D is wrong).
Q144
Context window management in long-running apps
Foundational
Why might an application set a hard maximum on conversation length (e.g., after N turns, prompting the user to start a new conversation) rather than relying solely on truncation or summarization to handle unbounded growth?
Not quite.
A hard cap is the simplest, most predictable strategy of the three — it gives both the application and the user a clear, unambiguous reset point, which can be a reasonable choice on its own or combined with truncation/summarization as a backstop, rather than letting the conversation degrade indefinitely through ever-more-aggressive truncation or many compounded rounds of lossy summarization (contradicting A's claim it offers no benefit). This is purely an application-level design choice, not an API-enforced requirement (B is wrong), and applies to any long conversation regardless of whether tools are involved — plain long text conversations face the same context-growth problem (D is wrong).
Q145
Context window management in long-running apps
Applied
An application's context management strategy summarizes aggressively, keeping conversations very short in terms of token count. A user later complains the assistant seems to 'forget' things they mentioned earlier in the same session. How does this connect to a tradeoff discussed for context compaction generally?
Not quite.
This is the headroom-versus-fidelity tradeoff showing up as a concrete, user-facing symptom: choosing aggressive summarization to maximize context headroom real-costs some fidelity, and a user noticing the assistant 'forgot' something they mentioned is exactly what that fidelity loss looks like in practice, not a sign of an unrelated bug (B is wrong — this is the expected cost of the chosen tradeoff, working as designed, just landing at an uncomfortable point on the tradeoff curve) or something disconnected from the summarization choice (A is wrong). The fix is likely tuning how aggressively summarization compacts (preserving more, summarizing less often, or being more deliberate about what's preserved), not necessarily abandoning summarization for a hard cap altogether, which has its own different tradeoffs (D overstates the only available fix).
Q146
Context window management in long-running apps
Advanced
A team wants to preserve specific, important facts (e.g., a customer's stated preferences) across a summarization event, without relying on the summary to capture them reliably by chance. What architectural pattern addresses this?
Not quite.
This is the same structured-state pattern discussed for tracking a specific important fact (like 'current product') reliably across a drifting conversation, applied here to the summarization-fidelity problem specifically: rather than hoping a general-purpose summary reliably captures and retains a specific important detail among everything else being condensed, extracting and storing that fact as its own explicit structured state (separate from the lossy free-text summary) guarantees it survives regardless of how the summarization step handles the rest. This directly solves the problem rather than leaving it unavoidable (B is wrong), doesn't require abandoning summarization for any conversation containing such facts (C is an overly broad restriction), and has no dependency on vector database usage — it's just structured application state, unrelated to retrieval infrastructure (D is wrong).
Q147
Few-shot examples in production prompts
Foundational
In a production application, where do few-shot examples typically live, and why does this matter for maintainability?
Not quite.
Examples naturally live inside the stable, reusable part of a prompt template discussed earlier — written once, reused across every call using that template, which is exactly why their ongoing quality matters: a stale, suboptimal, or narrow example set (recall the narrow-examples risk discussed earlier) silently affects every subsequent call until someone updates it, making example maintenance a genuine ongoing concern rather than a one-time setup task to forget about (contradicting A's claim they must be retyped per request, and C's claim they're experimentation-only). This maintenance concern scales with usage volume, mattering more — not less — as request count grows, the opposite of D's claim it only matters at tiny volumes.
Q148
Few-shot examples in production prompts
Applied
An application's few-shot examples were written when the product launched and have never been revisited, even as the product's real usage patterns have evolved significantly. What risk does this create?
Not quite.
This connects the narrow-example risk to product evolution over time: examples that were representative at launch can become a stale, unrepresentative sample as real usage patterns shift, reintroducing exactly the bias-toward-an-outdated-narrow-case risk discussed earlier — periodic review of examples against current real traffic is the natural mitigation, paralleling the same evolve-with-the-product discipline recommended for prompts and regression suites generally. This isn't a 'set once, forget forever' asset (A is wrong), the risk doesn't require a complete domain pivot to materialize — even organic drift in usage patterns within the same domain is enough (B is wrong), and examples are static text the team must deliberately maintain — they don't self-update from traffic automatically (D is wrong).
Q149
Few-shot examples in production prompts
Advanced
A team considers dynamically selecting which few-shot examples to include per request (e.g., picking examples most similar to the current input from a larger example pool) rather than using one fixed set of examples for every call. What tradeoff does this introduce relative to a fixed example set?
Not quite.
This is a genuine, multi-sided tradeoff: dynamically choosing examples most relevant to the specific current input (a RAG-like selection process) can improve tailoring and relevance over one fixed set, but it adds real implementation complexity (building that selection logic) and, critically, connects back to the caching-prefix principle — varying examples per call breaks the stable prefix a fixed example set would otherwise let get cached, trading a caching benefit for improved relevance (contradicting A's claim of no downside). This is implementable with standard application logic plus the existing API (B is wrong), and the tradeoff is relevant with example pools of any meaningful size, not gated to an arbitrarily large pool (D is wrong) — even a modest pool of a dozen candidate examples raises the same dynamic-selection tradeoff.
Q150
Few-shot examples in production prompts
Foundational
Why might a team include a deliberately negative example (showing an input and an explicitly incorrect or undesired output, labeled as such) alongside positive few-shot examples?
Not quite.
A clearly labeled negative example is a legitimate, sometimes very effective technique — it directly illustrates a specific mistake or boundary to avoid, which can rule out a failure mode more precisely than positive examples alone might manage to convey implicitly. This isn't a technique to avoid altogether (A is wrong), isn't gated behind a specific ratio requirement (C is wrong — the technique can be used with any number of accompanying positive examples, including zero), and doesn't inherently degrade overall output quality when used thoughtfully and clearly labeled as negative (D overstates a risk that applies mainly to poorly-labeled or confusing negative examples, not the technique itself).
Q151
Few-shot examples in production prompts
Applied
A team is deciding how many few-shot examples to include for a moderately well-specified task. What should guide this number, beyond 'more examples are always better'?
Not quite.
This connects back to the example-density tradeoff: additional examples cost context budget (and caching efficiency, if dynamically selected) while their marginal benefit typically diminishes once the underlying pattern is already clearly conveyed — for a moderately well-specified task, a small, diverse, well-chosen set often captures most of the achievable benefit, which is a more principled guide than either an arbitrary choice (B is wrong) or a fixed universal number like exactly one (C is wrong) or a number tied to some unrelated property of the task's answer space (D is an arbitrary and unmotivated rule).
What schema format does the Claude API use for defining a tool's expected input parameters?
Not quite.
Tool input schemas use JSON Schema, a widely-adopted standard for describing JSON data shape, types, and constraints — this is a deliberate choice that makes tool definitions interoperable with existing JSON Schema tooling and lets developers lean on a well-documented, familiar standard instead of learning a bespoke, proprietary format (B is wrong). Parameters can have real types, required/optional status, and constraints like enums or value ranges, not just freeform strings (C is wrong), and defining a schema requires no separate paid service — it's just a JSON object in the tool definition (D is wrong).
A tool's schema defines a parameter as an enum with a fixed set of allowed string values (e.g., `status: ['open', 'closed', 'pending']`). What practical benefit does this provide over defining the parameter as an unconstrained free-text string?
Not quite.
An enum constraint narrows the model's valid output space for that field to exactly the known-good set, meaningfully reducing the chance of an invalid, misspelled, or near-but-not-quite-matching value compared to an unconstrained free-text field — which in turn reduces (though per defense-in-depth, doesn't fully eliminate the need for) application-side validation on receipt. Enums work for strings just as commonly as any other type — the example given is literally a string enum, directly contradicting C. This benefit is present even with just one enum parameter in a tool — it doesn't require multiple to matter (D is wrong).
A developer is writing a tool's JSON Schema and must decide whether a given parameter should be marked `required`. What should drive this decision?
Not quite.
This should be driven by the real behavior of the underlying implementation, mirroring the schema/implementation mismatch problem discussed earlier from the other direction: over-marking parameters required when the implementation actually has sensible defaults unnecessarily forces the model to always supply a value it might not have a good reason to pick, while under-marking a genuinely necessary parameter as optional risks exactly the failure mode already described (the model omitting something the implementation needs). This has a real, non-stylistic consequence (D is wrong) and obviously shouldn't be assigned randomly (B is wrong) — required status should track actual functional necessity, not be marked uniformly true regardless of real flexibility (A is wrong).
Beyond type information, what other kind of constraint can a JSON Schema parameter definition include that helps guide the model toward valid input (e.g., for a numeric 'quantity' field that must be positive)?
Not quite.
Standard JSON Schema supports constraints well beyond bare type — numeric bounds (minimum/maximum), string length limits, regex pattern matching, and more — all of which narrow the valid input space the same way an enum narrows a string to a known set, giving the model more precise guidance than type alone (contradicting A's claim type is the only available constraint). These constraints apply across applicable types, not string-only (B is wrong) — numeric bounds are a prime example applying to numbers specifically — and they're part of the standard JSON Schema specification itself, requiring no custom plugin (C is wrong).
A developer writes a tool schema with a parameter description that just restates the parameter's name (e.g., a field named `start_date` with description 'the start date'). What improvement would make this more useful to the model?
Not quite.
A description that just restates the field name adds zero information beyond what the name already conveys — a genuinely useful description adds what the name alone doesn't: expected format, what the date means in context, and any real constraints (must it be in the future, is it inclusive). This connects to the parameter-level interface-contract principle discussed for tool descriptions generally. An empty description (C) is strictly worse than even a restating one, let alone a genuinely informative one, since it provides even less signal — that's not a real improvement. This benefit applies to any parameter type where the name alone is ambiguous, not limited to strings (D is wrong) — a numeric field's expected range or unit (seconds? milliseconds?) benefits from the same kind of elaboration.
A tool accepts a parameter that is an array of objects, each with several required fields. What practical implementation consideration does this nested-array-of-objects shape raise for the developer, beyond a flat set of top-level parameters?
Not quite.
This is the nested-schema complexity concern made concrete: the schema must define not just 'this is an array' but the full shape of each item within it, and validation logic needs to check each individual array item's fields, not just confirm the array exists — more moving pieces than a flat parameter set, with correspondingly more surface area for something to be missing or malformed (directly connecting to the earlier nested-schema construction-error-surface discussion). Arrays of objects are fully supported by JSON Schema — no forced flattening is required (C is wrong) — and this consideration is relevant regardless of array length; even a two-item array with required nested fields raises the same validation need (D is wrong).
Q158
Using MCP servers as a client/developer
Foundational
From an application developer's perspective, what does connecting to an existing MCP server provide, compared to writing custom integration code for that same external system from scratch?
Not quite.
This is the practical, developer-facing payoff of MCP's standardization value: if an external system already exposes an MCP server, a developer connects via the standard protocol instead of writing and maintaining bespoke integration code for that specific system from scratch — real, meaningful savings versus building custom glue code (contradicting A's claim of no difference). MCP servers can be connected to by any compatible client, including a custom application built directly on the Claude API, not just Claude Code specifically (B is wrong), and connecting as a client doesn't require the connecting developer to host the server themselves — that's the server provider's responsibility (D is wrong).
Q159
Using MCP servers as a client/developer
Applied
A developer's application connects to multiple different MCP servers simultaneously (e.g., one for a CRM, one for an internal database, one for a ticketing system). How does the application's experience of tool availability change as a result?
Not quite.
Multiple simultaneous MCP server connections are fully supported, and their tools combine into the model's overall available toolset — which is exactly why this connects directly back to the tool-selection-degrades-with-count and dynamic-filtering/namespacing concerns discussed earlier: as more servers (and therefore more tools) are connected, those scaling considerations become more, not less, relevant (contradicting A's claim only one connection is possible). Tools from different servers don't inherently conflict just by being available together — they coexist in the toolset, with any actual naming or purpose overlap being a design concern to manage, not an automatic conflict (B is wrong). This is a standard capability, not gated behind a special tier (C is wrong).
Q160
Using MCP servers as a client/developer
Advanced
A developer connects their application to a third-party MCP server they don't control or operate themselves. What operational risk does this external dependency introduce, beyond the risk of using their own first-party tools?
Not quite.
A third-party MCP server is an external dependency like any other — its uptime, latency, and behavior sit outside the connecting developer's direct control, and the application inherits whatever risk that introduces (unlike a first-party tool the developer fully controls and can fix directly) — this connects to the exact same resilience patterns (backoff/retry, circuit breakers, graceful degradation) already discussed for external dependencies generally, not a fundamentally new category of risk (contradicting B's claim of equivalent control). Refusing all third-party MCP servers (C) is a far more drastic response than applying standard resilience patterns, and third-party servers are absolutely capable of going down or failing — that's precisely the risk being described (D is wrong).
Q161
Using MCP servers as a client/developer
Foundational
Before connecting an application to a third-party MCP server, what should a developer evaluate, similar to vetting any other external dependency or library?
Not quite.
Using the MCP protocol says nothing about a specific server's actual quality, reliability, or trustworthiness — the protocol standardizes the connection mechanism, not the server's behavior or operational quality (contradicting A's claim of inherent trustworthiness). The same due diligence applied to adopting any third-party library or service is appropriate here: reliability track record, what access/scope would be granted (directly tying to least-privilege), maintenance status, and whether the server's own documentation and behavior are trustworthy. A server's name alone (B) says nothing meaningful about these properties, and cost (free vs. paid) has no bearing on whether due diligence is warranted (D is wrong) — a free server can be just as risky or well-run as a paid one.
Q162
Using MCP servers as a client/developer
Applied
An application connects to an MCP server whose available tools can change over time (the server adds, removes, or modifies tools independently of the connecting application's own release cycle). What does this imply for the application's robustness?
Not quite.
A server's toolset isn't guaranteed to stay frozen just because the connection was established once — a server operator can evolve their offering on their own schedule, independent of the connecting application's release cycle, so robust applications should handle a changed or missing tool gracefully (an error-handling and resilience concern, not an assumption of eternal stability, contradicting A). This applies even for a server operated internally by the same company/team, not just third-party ones — internal services get updated independently too (C is wrong). Handling a toolset change doesn't require re-implementing all tool-calling logic from scratch — targeted error handling for the specific changed/missing tool is the proportionate response, not a full rebuild (D is wrong).
Q163
Using MCP servers as a client/developer
Advanced
A developer is deciding whether to connect their application directly to several third-party MCP servers, or to build a single internal MCP server that itself aggregates and re-exposes a curated subset of those external capabilities. What architectural benefit might the aggregation approach provide?
Not quite.
An internal aggregating/curating layer is a real architectural option with real benefits: centralized access control and monitoring in one place, and the ability to expose only a curated subset of each external server's full capability (rather than everything being directly available), at the cost of building and maintaining that extra layer — a genuine tradeoff, not an equivalent no-op compared to direct connections (contradicting B). Building an MCP server that itself acts as a client to other MCP servers (or wraps other APIs) is a well-supported architectural pattern (C is wrong), and the tradeoff is worth considering even with just two or three external servers, not gated to a large number like 20 (D is an arbitrary threshold).
Q164
Building a basic MCP server
Foundational
At a basic level, what two things must a developer implement to expose a capability as an MCP server?
Not quite.
At minimum, building an MCP server means defining the tools it exposes (the same name/description/schema structure discussed in the tool-definitions subtopic) and implementing the actual handler logic that runs when a client invokes one of those tools, wired together through the protocol's expected server interface so any compatible client can discover and call them. None of this is automatically generated from just a name (A is wrong). A graphical interface is not a requirement for an MCP server to function — many are purely programmatic backends (B is wrong), and MCP servers can be implemented in any language with a suitable SDK/library, not restricted to one specific language (C is wrong).
Q165
Building a basic MCP server
Applied
A developer builds an MCP server wrapping their company's internal API. The underlying internal API already has its own authentication requirements. How should the MCP server handle this?
Not quite.
The MCP server sits in the middle of two distinct authentication relationships: how its own callers (MCP clients) authenticate to it, and how it itself authenticates to the underlying internal API it wraps — these are separate concerns the server needs to handle appropriately, not something that can be skipped (A, C are both wrong) just because a wrapping layer now exists. The MCP server typically holds and manages its own credentials for calling the underlying API, parallel to the API-key-management principles discussed for application backends generally. There's no requirement that the underlying API and the MCP protocol share the same authentication method — the server's job is precisely to bridge between whatever each side actually requires (D is wrong).
Q166
Building a basic MCP server
Advanced
A developer building an MCP server is deciding how granular to make the exposed tools — one broad tool mirroring the internal API's single flexible endpoint, or several narrower, purpose-specific tools built on top of it. How does the earlier granularity discussion (single vs. several smaller tools) apply here?
Not quite.
This is a direct, practical application of the tool-granularity tradeoff already established, now specifically in the context of designing an MCP server: several narrow, purpose-specific tools support clearer model selection and tighter least-privilege scoping per tool (an agent only needing read access doesn't need to also have the broad endpoint's write capability exposed), while one broad tool is simpler to implement and maintain but reintroduces the selection-ambiguity and over-broad-access concerns already discussed (directly contradicting A's claim of no relationship). MCP servers can expose any number of tools, not capped at exactly one (C is wrong), and this is entirely the server-building developer's own design decision — not something requiring Anthropic's involvement (D is wrong).
Q167
Building a basic MCP server
Foundational
Why should error handling inside an MCP server's tool handler logic follow the structured-error-response principle discussed earlier, rather than letting an unhandled exception propagate or returning a bare failure signal?
Not quite.
This connects the tool-error-design principle directly to the server-implementation side: the server's handler code is the place with the most direct visibility into what actually failed (a downstream API error, a validation failure, a timeout), and translating that into a structured response gives the calling model and application real information to act on — an unhandled crash or a bare pass/fail flag loses exactly that diagnostic value (contradicting B's claim the protocol handles this automatically with no server code needed). This matters regardless of how many tools the server exposes — even a single-tool server benefits from good error design (C is wrong), and structured error responses are a design choice implementable in any MCP server regardless of which specific SDK is used to build it (D is wrong).
Q168
Building a basic MCP server
Applied
A developer wants to test their newly built MCP server's tools before connecting it to a production application. What is a sound approach?
Not quite.
An MCP server's tool handlers are, underneath the protocol, ordinary application code — they can and should be unit/integration tested directly like any other code, with separate verification that the full protocol-level interaction (discovery, invocation) also works correctly through a real client — applying the same testing and fault-injection discipline already discussed for agent workflows generally, not skipping testing until production (contradicting A). This applies to read-only tools too — a read tool returning wrong data is still a real bug worth catching before production (B is wrong), and automated tests are both possible and preferable to one-off manual invocation as the only verification (C is wrong).
Q169
Debugging tool-call failures
Foundational
A developer is debugging a case where the model called a tool with unexpected or incorrect arguments. What is the first thing to inspect?
Not quite.
This directly follows from the tool-description-as-interface-contract principle: since the model's call is shaped by what the tool definition told it and what context it had available, those are the first, most directly inspectable and fixable levers — an ambiguous description or missing context is a far more common and addressable cause than assuming something unfixable about training data (A is wrong to treat that as the only explanation and dead end). This is absolutely debuggable through ordinary inspection, not an unavoidable cost to simply accept (B is wrong), and a bare status code alone (D) doesn't reveal why the arguments were wrong — the actual tool definition and context are what need review.
Q170
Debugging tool-call failures
Applied
A tool call intermittently fails — it works correctly most of the time but occasionally produces malformed arguments for what appears to be the same type of request. What debugging approach fits this pattern better than assuming a single deterministic bug?
Not quite.
This connects directly to the non-determinism property discussed for testing agentic features: 'the same type of request' isn't the same as 'the literally identical input every time,' and model output variability means an intermittent failure might not have one single deterministic root cause the way a traditional code bug would — collecting multiple failure instances to look for a correlating pattern (a specific input trait, a particular phrasing) is a more productive debugging approach than hunting for one single deterministic cause (contradicting A's claim that identical requests always produce identical results). The failure could stem from genuine model output variability, not only network issues (C is too narrow), and intermittent issues are worth debugging via pattern-finding, not an immediate reason to remove the tool outright (D is a disproportionate response before even attempting to understand the pattern).
Q171
Debugging tool-call failures
Advanced
A developer suspects a tool-call failure is caused by the model running low on relevant context by the time it reaches that point in a long conversation (e.g., earlier context got compacted or pushed out). What would help confirm or rule out this hypothesis?
Not quite.
This connects directly to the intermediate-reasoning-and-context-logging principle from reliability: the way to actually test this specific hypothesis is to inspect what context genuinely survived to the point of the failing call (post-compaction, if applicable) and check whether it plausibly still contained what the tool call needed — a concrete, evidence-based way to confirm or rule out the theory, not an unfalsifiable guess (contradicting A). Context exhaustion from compaction or a long conversation is a real, valid cause worth investigating, not something to dismiss outright (C is wrong), and this investigation is relevant at whatever point compaction or context pressure actually occurs for a given application — it doesn't require reaching an extreme token count like 1 million specifically (D is wrong, and well beyond any realistic context window besides).
Q172
Debugging tool-call failures
Foundational
Why is comprehensive logging of tool calls (what was called, with what arguments, what result came back) valuable for debugging, beyond just helping with the specific failure currently being investigated?
Not quite.
This connects logging to the broader monitoring and evaluation-drift principles discussed for reliability: a historical record across many calls over time supports pattern and trend detection (a specific tool failing disproportionately, a particular input trait correlating with failures) that's invisible when only looking at one incident in isolation — the value compounds over time, not just for the single case currently being debugged (contradicting A). This benefit applies to read-only tools too — understanding why a read tool intermittently returns wrong or incomplete data is just as valuable to track as debugging a write tool (B is wrong), and the model's final response typically doesn't show the full intermediate tool-call detail (arguments sent, raw results received) that a dedicated log captures (D is wrong).
Q173
Input validation / prompt injection defenses
Foundational
What is prompt injection, in the context of an application that includes untrusted content (user input, retrieved documents, web content) in its prompts?
Not quite.
Prompt injection exploits exactly the structural-separation ambiguity discussed earlier — untrusted content (a document, a user message, scraped web content) crafted to resemble an instruction, attempting to hijack the application's intended behavior by blurring the line between 'data to process' and 'instruction to follow.' This is a real security concern, not a performance technique (A) or a mandatory developer step (C), and it's unrelated to whether a response happens to be streamed — the vulnerability lives in prompt construction, not delivery mechanism (D is wrong).
Q174
Input validation / prompt injection defenses
Applied
An application summarizes web pages fetched based on a URL the user provides. What is the realistic prompt-injection risk here, and what does it imply for how much trust the application should place in the fetched content?
Not quite.
Web content is about as untrusted as input gets — literally anyone can put arbitrary text on a page the application might fetch — so it should be treated purely as data to summarize, never as a source of instructions, with clear structural separation from real developer instructions (contradicting B's claim it's inherently safe). Beyond prompt-level separation, defense in depth also means scoping what the application/agent can actually do downstream, so that even a successful injection attempt has limited real-world blast radius — connecting to least-privilege. This risk doesn't scale with page length specifically (C is wrong) and refusing to fetch any external content (D) is a far more drastic response than the realistic, usable mitigations available.
Q175
Input validation / prompt injection defenses
Advanced
A developer considers prompt injection 'solved' once they've clearly delimited user/retrieved content from instructions using structural markers (e.g., XML tags). Is this a complete mitigation?
Not quite.
This is the deterministic-vs-probabilistic distinction applied directly to security: structural delimiting is a real, meaningful risk reduction (contradicting B's claim of zero effect) but it's still a prompt-level, probabilistic mitigation — a sufficiently crafted injection could still potentially succeed, so it isn't an absolute guarantee (contradicting A). For consequential actions, the same defense-in-depth logic from tool-boundary enforcement applies: scoped access, output validation, and human review for high-stakes cases provide guarantees prompt-level mitigation alone cannot. Prompt injection is an active, researched area with real (if imperfect) mitigations, not an unanswerable or unmitigable problem (C is wrong).
Q176
Input validation / prompt injection defenses
Foundational
Why does an agentic application with tool access raise the stakes of prompt injection compared to a simple text-generation-only application?
Not quite.
This connects prompt injection directly to the tool-side-effect-communication and tool-boundary-enforcement principles discussed earlier: a successful injection against a text-only system produces unwanted text with no further consequence, while the same success against a tool-using agent could potentially induce a real action with a genuine side effect — meaningfully raising the stakes (contradicting A's claim of equal stakes). This risk exists with even a single consequential tool available, not gated to a high tool count (B is wrong), and tool-using agents are not immune — if anything the opposite is true, which is the entire point of this question (D is backwards).
Q177
Input validation / prompt injection defenses
Applied
A developer wants to add a mitigation specifically for the scenario where an agent might be induced (via injection) to call a destructive tool unexpectedly. Beyond prompt-level delimiting, what architectural mitigation directly addresses this, drawing on a principle from tool design?
Not quite.
This directly combines two already-established principles into a concrete injection mitigation: tool-boundary enforcement (a deterministic confirmation gate a prompt injection can't talk its way past) and least-privilege scoping (simply not granting destructive capability to an agent whose job involves processing untrusted content) both limit what the agent *can* do structurally, independent of whether an injection attempt is successfully detected or resisted at the prompt level — a stronger posture than hoping manipulation never succeeds (contradicting A). Refusing to process any untrusted content at all (C) would eliminate the agent's actual usefulness for a far more drastic tradeoff than necessary, and this risk has no dependency on the content's language (D is an irrelevant and incorrect constraint).
Q178
Secrets & credential handling in agent code
Foundational
An agentic application's tool handler code needs credentials to call an internal API. Where should those credentials come from, consistent with the API-key-management principles discussed for application backends generally?
Not quite.
A tool handler is still application backend code, and credentials it needs should follow the same secrets-management discipline already established — environment variables or a secrets manager, decoupled from the code — rather than being hardcoded inline, which reintroduces the exact rotation, access-control, and deploy-coupling problems already discussed for hardcoded secrets generally (B is wrong). The end user has no business holding or supplying internal-system credentials for backend-to-backend calls they're not party to (C is wrong and a real security problem). Tool handler code isn't a special exemption from standard practice — if anything it deserves the same or greater care, since a tool handler often has real-world side effects (D is wrong).
Q179
Secrets & credential handling in agent code
Applied
An agent's conversation logs (kept for debugging) occasionally capture tool call arguments that include sensitive values (e.g., an internal auth token passed as a parameter for a specific call). What risk does this create, and what should the application do about it?
Not quite.
This connects the earlier PII-in-logs concern directly to credentials specifically: a log capturing a sensitive value (an auth token, a secret) becomes another place that value now lives and must be protected, expanding the attack surface the team has to secure rather than being automatically safe just because it's 'only for debugging' (A is wrong). The practical fix is redacting known-sensitive fields before they're ever written to logs, not an all-or-nothing choice between full logging and no logging at all (C is a false dichotomy) — selective redaction preserves debugging value for everything else. The risk exists from the moment of capture, regardless of retention duration — a short-lived log still represents real exposure while it exists (C is wrong).
Q180
Secrets & credential handling in agent code
Advanced
An agent is given a tool that requires a highly privileged credential (e.g., full administrative database access) because 'it might need that level of access for some edge case someday.' How does this connect to the least-privilege and tool-availability-defines-scope principles discussed earlier?
Not quite.
The credential backing a tool is effectively part of that tool's real scope, so provisioning maximum privilege 'just in case' is the identical anti-pattern already called out for tool access generally — and because availability itself invites use (the tool-availability-defines-scope principle), an overly privileged credential is a live risk the moment it exists, not only in the narrow anticipated edge case someone had in mind (directly contradicting A's claim these are unrelated concerns, and B's framing of this as a safe default). This is a real risk regardless of how many tables a database happens to have — the issue is privilege scope, not table count (D is wrong).
Q181
Secrets & credential handling in agent code
Foundational
Why is credential rotation (periodically replacing API keys/secrets with new ones) a recommended practice for agentic application infrastructure, same as for any backend system?
Not quite.
Rotation addresses a specific residual risk: even with careful handling, an undetected leak is always a possibility (a log slip, a misconfigured access grant, a compromised dependency), and periodic rotation bounds how long such a leak stays exploitable rather than letting a compromised-but-undetected credential remain valid forever — a real security benefit, not merely a compliance formality (A is wrong). This benefit applies to any credential, including ones backing read-only access — unauthorized read access to sensitive data is still a real harm (C is wrong), and rotation provides meaningful benefit at reasonable intervals well short of daily — the specific cadence is a risk-based tuning choice, not a fixed daily requirement to have any value at all (D is wrong).
Q182
Output handling safety (sanitization before executing/displaying)
Foundational
An application takes code generated by the model and executes it directly with no review or sandboxing. What is the architectural risk?
Not quite.
Generated code is still just model output — it carries the same not-guaranteed-correct property as any other output, and direct, unreviewed execution risks running something buggy, with unintended side effects, or (especially if the generation process was influenced by untrusted input somewhere upstream) something genuinely harmful. Sandboxing, human review for consequential changes, or restricted/limited execution environments are the standard, usable mitigations — not an all-or-nothing ban on code generation features (D is far more drastic than necessary). This risk exists regardless of code length — a short but harmful snippet is just as dangerous as a long one (B is wrong).
Q183
Output handling safety (sanitization before executing/displaying)
Applied
An application displays model-generated Markdown directly to users by rendering it as HTML, without restricting which Markdown/HTML features are allowed. What specific risk does this raise, beyond the general innerHTML/XSS concern discussed for streamed output?
Not quite.
This is a specific, concrete instance of the earlier XSS concern, routed through Markdown's own feature set: many Markdown renderers allow embedded raw HTML or permit arbitrary URLs in links/images, which — if the renderer isn't configured to restrict or sanitize these — reopens the same unsanitized-output-rendering risk via a Markdown-specific path, not a fundamentally new category of risk (so A undersells the connection, but the specific mechanism is worth calling out distinctly). Markdown is not inherently safe just because it's 'not HTML' (D is wrong) — its common extensibility into raw HTML is exactly the gap. This isn't limited to tables — any Markdown feature permitting embedded markup or arbitrary URLs carries the same risk (C is wrong).
Q184
Output handling safety (sanitization before executing/displaying)
Advanced
An application constructs a SQL query by directly interpolating a value the model generated (e.g., as part of a text-to-SQL feature) into a query string, rather than using parameterized queries. What risk does this introduce?
Not quite.
Model output is not an inherently trusted, pre-verified-safe category just because a model (rather than a user directly) produced the string — especially since the model's generation could itself have been shaped by untrusted upstream content (a prompt-injected instruction, unsanitized input earlier in the pipeline) — so directly interpolating it into a SQL query carries the exact same injection risk as interpolating any other untrusted string (contradicting A's assumption of inherent safety). Parameterized queries are the standard mitigation regardless of where the value originated. This isn't SQL-specific — equivalent injection-style risks exist in NoSQL query construction too if values are unsafely interpolated rather than properly parameterized (C is wrong), and HTTPS protects data in transit, entirely unrelated to this query-construction vulnerability (D is wrong).
Q185
Output handling safety (sanitization before executing/displaying)
Foundational
What is the general principle connecting all of these output-handling risks (XSS from rendered output, unsafe code execution, SQL injection from interpolated model output)?
Not quite.
This is the unifying thread tying all three scenarios together (directly contradicting A's claim of no connection): wherever model output flows into something with its own execution or rendering semantics — a browser interpreting HTML, a shell executing code, a database interpreting a query — it needs to be treated as untrusted data at that boundary, the same discipline already applied to any other untrusted input, rather than getting a free pass just because it originated from the model instead of directly from a user. This applies to internal tools just as much as external ones — an internal tool with a SQL injection risk is still a real vulnerability (B is wrong), and the principle holds even with exactly one downstream consumer — multiplicity of consumers isn't what creates the risk (D is wrong).
Q186
Least-privilege for API keys/tool scopes (dev-level)
Foundational
At implementation time, what is a concrete way a developer applies least-privilege to a specific agent's tool configuration?
Not quite.
This is the concrete implementation mechanics of least-privilege at the code level: explicitly constructing and passing a scoped toolset parameter per agent instance (just the tools that specific agent's job requires) rather than defaulting every agent to a shared, maximal set and only trimming reactively after a problem surfaces (A is backwards — that's the 'just in case' anti-pattern already identified as risky, applied reactively instead of proactively). This is squarely the application developer's own configuration responsibility, not something that requires provider-side enforcement (B is wrong), and it's worth doing with any number of tools, not gated to a small total count — if anything, the benefit grows as the overall tool count grows (C is wrong).
Q187
Least-privilege for API keys/tool scopes (dev-level)
Applied
A single backend service handles requests for multiple different agent features (one customer-support agent, one internal-reporting agent) using one shared API key and one shared, combined toolset passed to every call regardless of which feature is making the request. What is the architectural improvement here?
Not quite.
Even with shared underlying infrastructure (one service, one API key), least-privilege can and should still be applied at the call level — scoping the toolset passed to each individual call to match what that specific feature actually needs, so the support agent's calls never carry reporting tools and vice versa, regardless of whether they technically share a backend (contradicting B's claim shared-everything is always optimal). This is a lighter-weight, more proportionate fix than fully separating into independent services (C overstates what's needed — call-level toolset scoping achieves the real goal without the added operational overhead of full separation), and the risk/benefit of proper scoping applies regardless of team structure — even a single team maintaining both features benefits from this discipline (D is wrong).
Q188
Least-privilege for API keys/tool scopes (dev-level)
Advanced
A developer scopes an agent's toolset tightly for its primary task, but one of the included tools (a general-purpose 'run_database_query' tool) is far broader than the agent's actual need (it only ever needs to read from one specific table). What does true least-privilege require here, beyond just limiting which tools are included?
Not quite.
Least-privilege operates at two levels that both matter: which tools are included at all, and how broad each included tool's own capability is — a short toolset that happens to include one wildly over-scoped tool (arbitrary SQL access, when only single-table reads are needed) hasn't actually achieved least-privilege just because the tool count is small (contradicting A). This connects directly back to the earlier tool-scoping principle from agentic architecture: the fix is narrowing or replacing that specific tool with a purpose-built, appropriately restricted one, not necessarily removing database access altogether — a narrower read-only, single-table tool still lets the agent do its real job (C overstates what's needed). This consideration applies to any data the table holds, not gated specifically to financial data — overly broad access is a problem on principle regardless of content sensitivity (D is wrong).
Q189
Using Claude Code CLI & SDK as a developer
Foundational
From a developer's perspective, what distinguishes using Claude Code interactively in a terminal from building a custom application on the raw Claude API?
Not quite.
Claude Code is a complete, opinionated product built for a specific use case (agentic coding work), with its own established tool set, configuration system (CLAUDE.md), permission model, and interfaces (CLI, IDE extensions) ready to use directly — versus the raw API, which is a general-purpose building block a developer uses to construct their own custom application, prompts, and tool integrations for whatever use case they have, coding-related or not. This is a meaningful, practical distinction, not a non-difference (A is wrong). Claude Code is available for use by external developers generally, not restricted to Anthropic's own engineers (B is wrong), and the raw API is fully capable of supporting coding-related applications too — a developer could build a custom coding assistant on the API directly, it just wouldn't be Claude Code specifically (D is wrong).
Q190
Using Claude Code CLI & SDK as a developer
Applied
A developer wants to build a custom application that embeds agentic coding capability (similar to what Claude Code does) rather than using Claude Code's own CLI/IDE interface directly. What does the Claude Agent SDK provide for this purpose?
Not quite.
The Claude Agent SDK is specifically meant to give developers the underlying agentic infrastructure (the loop, tool-use plumbing, permission handling) that powers products like Claude Code, packaged so it can be embedded into a developer's own custom application rather than being locked into Claude Code's specific CLI/IDE surface — a direct, relevant capability, not something unrelated (A is wrong). It's general-purpose infrastructure for building agents, not restricted to coding-specific use cases only (B is wrong), and it's built on top of (not a bypass of) the standard API primitives already discussed throughout this domain (C is wrong).
Q191
Using Claude Code CLI & SDK as a developer
Advanced
A developer wants their custom agentic application to support something like Claude Code's CLAUDE.md pattern — persistent, project-scoped configuration automatically loaded into context. How does this relate to the system-prompt-construction principles discussed for applications generally?
Not quite.
This connects Claude Code's own well-known configuration pattern back to general system-prompt-construction principles already covered: CLAUDE.md is, functionally, a stable body of context (project conventions, standing instructions) automatically assembled into the model's context on each session — the same underlying technique (stable + request-specific context combined) a developer building any custom application's system prompt already uses, just applied specifically to 'project configuration' as the stable content (contradicting B's claim of no relationship). A developer can absolutely build an analogous pattern into their own application using standard system-prompt-assembly techniques — this requires no special access to Anthropic's own infrastructure (C is wrong), and the underlying pattern is valuable at any configuration size, not gated to extremely long files (D is wrong).
Q192
Automating workflows with Claude Code (CI/headless)
Foundational
What does running Claude Code in a 'headless' mode (e.g., as part of a CI/CD pipeline) mean, compared to its typical interactive terminal use?
Not quite.
This directly reconnects to the headless/CI execution-mode concept already established for agentic architecture generally: running non-interactively means no human is present in real time to approve each step, so permissions and guardrails must be decided and configured in advance rather than confirmed moment-to-moment — this is a difference in interaction model and upfront configuration need, not a reduction in underlying capability (A is wrong). Headless Claude Code can perform real actions, not just read-only ones, provided it's configured with appropriate permissions to do so (C is wrong) — the configuration need is exactly what makes real actions safe to automate. This is the same product used differently, not a separately licensed one (D is wrong).
Q193
Automating workflows with Claude Code (CI/headless)
Applied
A team wants to use Claude Code in a CI pipeline to automatically review pull requests, but is deciding how much autonomy to grant it (e.g., can it merge PRs itself, or only leave review comments). What principle from earlier in this domain should drive this decision?
Not quite.
This is a direct, concrete application of least-privilege and tool-boundary enforcement to a realistic CI scenario: granting only what the actual task needs (commenting) rather than broader capability 'just in case' (merge access) is exactly the right instinct, and it matters even more in a headless context specifically because no human is present in the moment to catch and stop a mistake before a consequential action (a bad merge) takes effect — directly contradicting A's claim the choice has no real consequence. There's no requirement to grant full merge autonomy — a narrower, comment-only configuration is a completely valid and often preferable choice (B is wrong), and the stakes of this decision don't scale specifically with contributor count — even a small repository can be meaningfully harmed by an inappropriate automated merge (D is wrong).
Q194
Automating workflows with Claude Code (CI/headless)
Advanced
A headless Claude Code automation in a CI pipeline occasionally produces an unexpected or incorrect result with no human watching in real time to catch it immediately. What mitigation approach connects most directly to the monitoring and evaluation principles discussed for agentic reliability generally?
Not quite.
Absence of real-time human oversight doesn't mean absence of any oversight — this is exactly where the comprehensive-logging and ongoing-monitoring principles already discussed for agentic reliability apply: logging actions and outcomes lets the team review after the fact, catch problems that did occur, and build an evidence base to inform whether the automation's permission scope should be adjusted going forward (contradicting A's claim headless automation is simply unmonitorable). This doesn't require restricting headless automation only to trivial tasks (C is an overcorrection) — well-logged, reviewed automation can reasonably handle more consequential tasks over time as trust is established through evidence. Automated tests catch a different category of problem (code correctness) and don't substitute for monitoring the automation's own actions and decisions specifically (C is wrong).
Q195
Writing evals for Claude-powered features
Foundational
What is the basic structure of an eval (evaluation) for a Claude-powered feature?
Not quite.
This is the concrete structure behind the criterion-based evaluation approach already discussed for testing agent behavior: a representative input set plus a defined way to judge each output (success criteria, a rubric, a comparison method) run systematically, which is what makes it an eval rather than just an informal impression from a handful of manual tries (A is wrong to equate the two). A meaningful eval needs more than a single data point to be representative of real variation (B is wrong), and evals are just as relevant for tool-using agentic features as for plain generation — arguably more so, given the added complexity tool use introduces (C is wrong).
Q196
Writing evals for Claude-powered features
Applied
A team builds an eval set but populates it entirely with inputs the developers personally thought of while writing the code, with no real production data or deliberately adversarial cases included. What limitation does this introduce, connecting to a principle from testing agent behavior?
Not quite.
This is a direct instance of the representativeness concern already established for regression suites and model-tier benchmarking: cases a developer thinks to write tend to reflect their own expectations and mental model of the feature, which systematically differs from the genuine variety (including weird, unexpected, or adversarial inputs) real usage actually surfaces — supplementing developer-authored cases with real or realistic production-like data and deliberately hard cases is the established mitigation (contradicting A's claim of equal representativeness). This gap exists from day one, before any production history accumulates — it doesn't require a full year of operation to matter (B is wrong), and developer-authored cases aren't inherently invalid — they're a reasonable starting point, just insufficient alone without broader supplementation (D overstates the limitation into total invalidity).
Q197
Writing evals for Claude-powered features
Advanced
A team wants to use Claude itself to grade the outputs of another Claude call as part of an automated eval (an 'LLM-as-judge' pattern), rather than writing exact-match or rule-based checks. What should they be aware of with this approach?
Not quite.
An LLM grader is a real, useful, and increasingly common technique — but it's still a model making a probabilistic judgment call, not an infallible, perfectly objective oracle (contradicting A), so the grading prompt/rubric deserves the same careful design attention as any other prompt, and periodically spot-checking the grader's judgments against human review is how the team validates that the automated grading is actually trustworthy, rather than assuming so by default. This technique is particularly valuable for subjective or open-ended outputs that are hard to grade with simple rule-based checks — exactly the case where it shines, not where it's inapplicable (C is backwards). LLM-as-judge is a well-established, technically feasible pattern, not something models are incapable of (D is wrong).
Q198
Debugging unexpected model output
Foundational
A developer sees unexpected output from a model call and wants to debug it. What is the most direct, first piece of information to inspect?
Not quite.
Since output is a direct function of input, the most direct, first debugging step is reconstructing exactly what the model actually received for that specific call — the full system prompt, message history, and tool definitions as they existed at that moment (not as currently written, if anything has since changed) — rather than starting from abstract training methodology (B, which offers little actionable, call-specific insight) or skipping the input entirely (D is backwards — the input is precisely what's most directly inspectable and actionable). This is a systematic, well-supported debugging approach, not something to simply accept as unknowable (C is wrong).
Q199
Debugging unexpected model output
Applied
A developer reconstructs the exact request sent for a problematic call and finds nothing obviously wrong with the prompt itself. What should they check next, connecting to the non-determinism property discussed for testing agent behavior?
Not quite.
This connects directly to the non-determinism property from testing agent behavior: once the prompt itself looks sound, the next useful diagnostic step is determining whether the unexpected output is a one-off instance of normal variability or a reproducible pattern — re-running the same request and/or checking historical logs for similar cases is how that distinction gets made, which meaningfully changes the right response (accept as normal variation vs. treat as a real, fixable issue). This is far from unexplainable or requiring acceptance with no further investigation (A is wrong), assuming the API is simply broken (B) skips a much more likely and directly testable explanation, and there are clearly other diagnostic options available before reaching for a model version change as the first lever (C is wrong).
Q200
Debugging unexpected model output
Advanced
A developer identifies that a specific, reproducible prompt reliably produces an undesired output. Having confirmed the issue is real and not just one-off variability, what should guide the fix, connecting to prompt engineering principles discussed elsewhere?
Not quite.
Once an issue is confirmed reproducible (not just incidental variability), the fix should be driven by diagnosing the actual likely cause — tracing back to the specific prompt-engineering concepts already covered (an ambiguous instruction, missing context, a misleading example, insufficient structural separation between instructions and data) — and addressing that specific root cause, rather than reaching for one fixed response regardless of actual cause, whether that's always switching models (A) or always adding more examples (B, which could even make things worse if a bad existing example is the actual problem, per the narrow-examples risk already discussed). Prompt-level fixes are the first and most direct lever for behavior shaped by prompt content — retraining a custom model is a much heavier-weight, rarely-necessary response for most such issues (D overstates what's required).