GSTIN / Company Registration Number: 20AOBPD8112K1ZI
    All certifications
    CCAR-P

    Claude Architect – Professional (CCAR-P) Practice Questions

    The senior tier: governance, stakeholder lifecycle and operational enablement on top of Foundations.

    63 items on the real exam 120 min $175 Pass: 720/1000
    Take the timed CCAR-P mock exam

    Claude CCAR-P exam guide

    The Claude Architect – Professional (CCAR-P) exam is 63 items, closed-book, proctored by Pearson VUE, with a 120-minute time limit. It costs $175 and uses a scaled score out of 1000, with 720 needed to pass. Senior partners responsible for Claude solutions after they ship. This page's question bank and flashcards are free and domain-weighted to match the breakdown below; the timed mock exam reproduces the full exam length and scoring.

    Exam domains & weighting

    DomainWeightYour progress
    Integration19%
    0/76
    Solution Design & Architecture17%
    0/68
    Evaluation, Testing & Optimization16%
    0/64
    Governance, Safety & Risk14%
    0/56
    Stakeholder Communication & Lifecycle14%
    0/56
    Claude Models, Prompting & Context Engineering13%
    0/52
    Developer Productivity & Operational Enablement7%
    0/28

    About these questions

    Every question below is written from scratch by GetDev, grounded in Anthropic's published exam domains and weightings plus general Claude, Claude Code and MCP product knowledge. None are reproduced from any other prep site. Each is tagged with its domain, subtopic and difficulty so you can target weak spots — use the filters in the Question bank tab.

    400 of 400 questions · 0 answered · 0 correct so far

    Q1
    Enterprise integration architecture
    Foundational

    A senior partner is designing how a Claude-based solution will integrate across an enterprise with dozens of existing systems. What architectural principle should take priority over integrating with every available system immediately?

    Q2
    Enterprise integration architecture
    Applied

    A Claude-based solution needs to integrate with both a modern, well-documented API-first system and a decades-old legacy system with no API, only a database the solution can query directly. What architectural approach should a senior partner recommend for the legacy integration?

    Q3
    Enterprise integration architecture
    Advanced

    A senior partner is designing an integration architecture that will need to support multiple business units with different (and evolving) requirements over the solution's lifetime. What architectural pattern best supports this kind of sustained flexibility?

    Q4
    Enterprise integration architecture
    Foundational

    Why does a senior partner need to consider an enterprise's existing identity and access management (IAM) infrastructure when designing how a Claude-based solution's users authenticate?

    Q5
    Enterprise integration architecture
    Applied

    A senior partner is evaluating whether a Claude-based solution's integration architecture should route all external system calls through a single internal gateway/aggregation layer, or let each component of the solution call external systems directly. What tradeoff should inform this decision?

    Q6
    Enterprise integration architecture
    Advanced

    A senior partner discovers mid-engagement that an enterprise's data governance policy requires certain integrated systems' data to remain within specific geographic boundaries, a requirement not accounted for in the original integration architecture. How should this be handled?

    Q7
    Enterprise integration architecture
    Foundational

    Why should a senior partner document integration architecture decisions and their rationale for an enterprise client, beyond the technical implementation itself?

    Q8
    Enterprise integration architecture
    Applied

    A senior partner is designing integration architecture for a solution that will need to scale from an initial pilot group to the entire enterprise. What architectural consideration, beyond the pilot's immediate functional needs, should inform the design from the start?

    Q9
    Enterprise integration architecture
    Advanced

    A senior partner is deciding how to handle version compatibility as an enterprise's various integrated systems are independently upgraded over time by their own separate owning teams. What architectural approach manages this risk?

    Q10
    Enterprise integration architecture
    Foundational

    Why might a senior partner recommend periodically reviewing an enterprise solution's overall integration architecture, not just individual integration points, as the enterprise and solution evolve?

    Q11
    API lifecycle and versioning strategy
    Foundational

    A senior partner is advising an enterprise on a versioning strategy for pinning to specific Claude model versions across a large, multi-team organization. What governance consideration goes beyond the individual model-pinning decision already covered in foundational material?

    Q12
    API lifecycle and versioning strategy
    Applied

    An enterprise has dozens of Claude-based applications built by different teams, each pinned to different model versions at different times. A senior partner is asked to propose an organization-wide upgrade governance process. What should this process address?

    Q13
    API lifecycle and versioning strategy
    Advanced

    A senior partner is deciding between a 'pin and manually upgrade' strategy versus an 'always use latest' strategy for a critical, high-visibility enterprise application, weighing the tradeoff already discussed in foundational material. What additional enterprise-level factor should weigh into this specific decision beyond the individual application's own tradeoff?

    Q14
    API lifecycle and versioning strategy
    Foundational

    Why should an enterprise maintain a clear inventory of which Claude-based applications are pinned to which model versions, connecting to a documentation principle established in foundational material?

    Q15
    API lifecycle and versioning strategy
    Applied

    An enterprise's version-pinning inventory reveals that several business-critical applications are pinned to a model version approaching the end of its support lifecycle. What should a senior partner prioritize in response?

    Q16
    API lifecycle and versioning strategy
    Advanced

    A senior partner is advising on whether an enterprise should adopt a formal API deprecation and migration policy for their own internal Claude-based services (services other internal teams build on top of), not just for how they consume Anthropic's own API. What principle applies here by extension?

    Q17
    API lifecycle and versioning strategy
    Foundational

    Why might a senior partner recommend an enterprise maintain a staging or pre-production environment specifically for testing model version upgrades before they reach production applications?

    Q18
    API lifecycle and versioning strategy
    Applied

    A senior partner discovers that an enterprise's internal teams have been inconsistently interpreting what 'tested and validated' means before upgrading a production application's model version — some teams do thorough testing, others barely any. What should this prompt?

    Q19
    API lifecycle and versioning strategy
    Advanced

    A senior partner is advising on how an enterprise should handle a scenario where a model version upgrade is expected to improve most applications' performance but has a known, specific behavioral change that could affect a small number of applications with unusual, highly specific prompt patterns. What governance approach addresses this?

    Q20
    API lifecycle and versioning strategy
    Foundational

    Reflecting on this subtopic as a whole, how does API lifecycle and versioning strategy at the enterprise level relate to the model-tier and model-version principles established in foundational material?

    Q21
    Cross-functional stakeholder integration (data/security/legal)
    Foundational

    A senior partner is architecting a Claude-based solution's integration with enterprise data systems. Why should the security team be engaged as a design stakeholder from early in the architecture process, rather than only reviewing the finished design before launch?

    Q22
    Cross-functional stakeholder integration (data/security/legal)
    Applied

    A senior partner is coordinating between a client's legal, security, and business stakeholders who have genuinely different priorities for a Claude-based solution (legal wants maximum caution, business wants speed to market, security wants tight access control). How should the senior partner approach reconciling these?

    Q23
    Cross-functional stakeholder integration (data/security/legal)
    Advanced

    A senior partner is managing a situation where legal's requested data-handling restriction, if implemented exactly as requested, would make the solution's core use case technically infeasible. What should guide the response?

    Q24
    Cross-functional stakeholder integration (data/security/legal)
    Foundational

    Why should a senior partner establish clear, ongoing communication channels with cross-functional stakeholders throughout a solution's lifecycle, not just during initial design?

    Q25
    Cross-functional stakeholder integration (data/security/legal)
    Applied

    A senior partner is onboarding a new security stakeholder mid-engagement, replacing someone who departed. What should the senior partner prioritize to avoid losing the benefit of prior security alignment work?

    Q26
    Cross-functional stakeholder integration (data/security/legal)
    Advanced

    A senior partner notices that legal and security stakeholders at a client organization rarely communicate directly with each other, with the senior partner serving as the only bridge between them. What risk does this illustrate, and what should the senior partner consider?

    Q27
    Cross-functional stakeholder integration (data/security/legal)
    Foundational

    Why is it valuable for a senior partner to translate technical architecture decisions into terms that legal and business stakeholders (who may not have deep technical backgrounds) can genuinely understand and evaluate?

    Q28
    Cross-functional stakeholder integration (data/security/legal)
    Applied

    A senior partner is deciding how to document a cross-functional architectural decision that involved input from legal, security, and business stakeholders, each with different concerns. What should this documentation capture, beyond the final decision itself?

    Q29
    Cross-functional stakeholder integration (data/security/legal)
    Advanced

    A senior partner observes that cross-functional review (legal, security, business) for new features has become a significant bottleneck, with each new feature facing a lengthy, uncoordinated sequential review by each function independently. What architectural/process response addresses this?

    Q30
    Cross-functional stakeholder integration (data/security/legal)
    Foundational

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to cross-functional stakeholder integration?

    Q31
    Integration testing and validation at scale
    Foundational

    A senior partner is designing a testing strategy for a Claude-based solution integrated with a dozen enterprise systems. Why is testing each integration point in isolation not sufficient on its own?

    Q32
    Integration testing and validation at scale
    Applied

    A senior partner is deciding how to prioritize testing effort across a dozen integration points with limited testing time before launch. What should guide this prioritization?

    Q33
    Integration testing and validation at scale
    Advanced

    A senior partner is designing integration tests and must decide how to handle a dependency on an external system that is itself occasionally unreliable (intermittent outages, inconsistent test data). What approach addresses this testing challenge?

    Q34
    Integration testing and validation at scale
    Foundational

    Why should integration tests be run automatically and regularly (not just once before initial launch), connecting to a principle established for regression testing generally?

    Q35
    Integration testing and validation at scale
    Applied

    A senior partner's integration test suite passes consistently, but production incidents still occasionally occur from integration issues the test suite didn't catch. What should this prompt?

    Q36
    Integration testing and validation at scale
    Advanced

    A senior partner is deciding whether integration testing should include deliberately adversarial or malformed inputs from integrated systems (not just well-formed, expected data), connecting to the red-teaming principle established in foundational material. Why does this matter for enterprise integrations specifically?

    Q37
    Integration testing and validation at scale
    Foundational

    Why might a senior partner recommend a formal sign-off process for integration testing before a Claude-based solution moves to production, connecting to a principle discussed for evaluation generally?

    Q38
    Integration testing and validation at scale
    Applied

    A senior partner is testing a Claude-based solution's integration with a system that has different behavior in its test/staging environment versus its actual production environment. What risk does this create, and what should the consultant check?

    Q39
    Integration testing and validation at scale
    Advanced

    A senior partner is deciding how to test an integration point whose real-world behavior depends on data volume and concurrency patterns that are hard to fully replicate in a pre-production environment. What approach addresses this gap?

    Q40
    Integration testing and validation at scale
    Foundational

    Reflecting on this subtopic, how does integration testing at enterprise scale relate to the evidence-based-validation discipline established throughout foundational material?

    Q41
    Third-party and partner ecosystem integration
    Foundational

    An enterprise wants a Claude-based solution to integrate with several third-party SaaS partners in their ecosystem. What governance question should a senior partner raise before any specific integration is designed?

    Q42
    Third-party and partner ecosystem integration
    Applied

    A partner integration the enterprise depends on changes its API with limited notice, breaking part of the Claude-based solution. What architectural practice, already established for internal systems, reduces this risk?

    Q43
    Third-party and partner ecosystem integration
    Advanced

    A senior partner is evaluating whether to build a direct integration with each individual partner or route all partner integrations through a single internal partner-integration layer. What should inform this choice, connecting to the gateway-tradeoff principle discussed earlier?

    Q44
    Third-party and partner ecosystem integration
    Foundational

    Why should contractual terms with ecosystem partners be reviewed with an eye toward data handling and liability, not purely technical integration feasibility, before a senior partner recommends a partner integration?

    Q45
    Third-party and partner ecosystem integration
    Applied

    An enterprise relies on a single critical partner integration with no readily available alternative. What architectural and governance consideration should a senior partner raise about this concentration?

    Q46
    Third-party and partner ecosystem integration
    Advanced

    A senior partner is advising on an ecosystem where the enterprise's own Claude-based solution will itself be a partner integration point for other organizations. What additional responsibility does this reverse relationship create?

    Q47
    Third-party and partner ecosystem integration
    Foundational

    Why should a senior partner maintain visibility into the overall health and status of each partner integration in an ecosystem, not just whether each one is technically functioning at a given moment?

    Q48
    Third-party and partner ecosystem integration
    Applied

    A senior partner is deciding how much custom, partner-specific logic to build into the core solution versus keeping partner-specific handling cleanly separated. What principle, already established for business-unit configuration, applies by analogy?

    Q49
    Third-party and partner ecosystem integration
    Advanced

    A senior partner is advising on offboarding a partner integration that's being discontinued. What should this process prioritize, connecting to principles already established for workspace and access offboarding?

    Q50
    Third-party and partner ecosystem integration
    Foundational

    Reflecting on this subtopic, how does third-party partner ecosystem integration relate to the vendor due-diligence and build-vs-buy principles established in foundational material?

    Q51
    Integration monitoring and observability
    Foundational

    A senior partner is designing observability for a Claude-based solution's enterprise integrations. Why should monitoring cover each integration point individually, not just the solution's overall health?

    Q52
    Integration monitoring and observability
    Applied

    A senior partner is deciding what signals to track for each integration point's health. Beyond simple up/down status, what should be included?

    Q53
    Integration monitoring and observability
    Advanced

    A senior partner notices that integration monitoring alerts are frequently triggered but rarely represent a genuine actionable problem, leading the team to start ignoring alerts. What established principle does this connect to, and what should be done?

    Q54
    Integration monitoring and observability
    Foundational

    Why should integration monitoring data be accessible to both technical and business stakeholders, in appropriately tailored views, rather than existing only as raw technical logs for engineers?

    Q55
    Integration monitoring and observability
    Applied

    A senior partner is deciding how long to retain detailed integration monitoring/log data. What tradeoff should inform this decision, connecting to the data-retention principle established in foundational material?

    Q56
    Integration monitoring and observability
    Advanced

    A senior partner is designing monitoring for a solution with many integration points, each individually monitored. What additional, higher-level observability consideration goes beyond per-point monitoring?

    Q57
    Integration monitoring and observability
    Foundational

    Why should a senior partner ensure monitoring and observability requirements are considered during integration architecture design, rather than added only after an integration is already built?

    Q58
    Integration monitoring and observability
    Applied

    An integration's monitoring shows a gradual, sustained increase in latency over several months, with no single dramatic failure. What should this pattern prompt, connecting to the evaluation-drift principle established in foundational material?

    Q59
    Integration monitoring and observability
    Advanced

    A senior partner is deciding whether integration monitoring should be owned centrally by a platform/infrastructure team or distributed to each individual application team. What should inform this choice?

    Q60
    Integration monitoring and observability
    Foundational

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to integration monitoring and observability?

    Q61
    Integration failure recovery and business continuity
    Foundational

    A senior partner is designing business continuity planning for a Claude-based solution that's become business-critical. What should this planning address beyond the technical retry/backoff patterns already established?

    Q62
    Integration failure recovery and business continuity
    Applied

    A senior partner is designing recovery procedures for a scenario where a critical integration fails for an extended period (hours, not minutes). What should this recovery plan specify, beyond 'retry until it works'?

    Q63
    Integration failure recovery and business continuity
    Advanced

    A senior partner is planning recovery from an outage and considers how to handle data that was supposed to flow through a failed integration during the downtime. What architectural pattern addresses this, connecting to the idempotency principle established in foundational material?

    Q64
    Integration failure recovery and business continuity
    Foundational

    Why should a senior partner ensure recovery procedures are actually tested (not just documented) before relying on them during a real outage?

    Q65
    Integration failure recovery and business continuity
    Applied

    A senior partner is deciding how often business continuity and recovery procedures should be re-tested as the solution evolves. What principle should guide this cadence?

    Q66
    Integration failure recovery and business continuity
    Advanced

    A senior partner is designing continuity planning for a scenario where the Claude API itself (not a specific integration) experiences a broader outage. What should this planning specifically address, distinct from planning for a single integration's failure?

    Q67
    Integration failure recovery and business continuity
    Foundational

    Why should recovery procedures include clear communication plans for affected internal and external stakeholders during an outage, not just the technical recovery steps themselves?

    Q68
    Integration failure recovery and business continuity
    Applied

    After a significant outage is resolved, what should a senior partner ensure happens, connecting to the knowledge-preservation and continuous-improvement principles established throughout this certification?

    Q69
    Integration failure recovery and business continuity
    Advanced

    A senior partner is deciding how much redundancy (e.g., a secondary integration path, a backup data source) to build into a business-critical solution's architecture. What should guide this investment, connecting to the risk-proportionality principle established in foundational material?

    Q70
    Integration failure recovery and business continuity
    Foundational

    Why should business continuity planning for a Claude-based solution be reviewed alongside the enterprise's broader, existing disaster-recovery and business-continuity framework, rather than developed as an entirely standalone plan?

    Q71
    Integration failure recovery and business continuity
    Applied

    A senior partner is advising on recovery time objectives (RTO) and recovery point objectives (RPO) for a Claude-based solution. How should these specific targets be determined?

    Q72
    Integration failure recovery and business continuity
    Advanced

    A senior partner discovers that a solution's actual recovery time during a real incident significantly exceeded its documented RTO target. What should this prompt, beyond simply noting the target was missed?

    Q73
    Integration failure recovery and business continuity
    Foundational

    Why might a senior partner recommend a tiered continuity approach (different recovery expectations for different parts of a solution), rather than a single uniform recovery standard across the entire solution?

    Q74
    Integration failure recovery and business continuity
    Applied

    A senior partner is advising a client whose business continuity plan assumes a single named individual will always be available to execute manual recovery steps during an outage. What risk does this illustrate, connecting to a principle established earlier in this domain?

    Q75
    Integration failure recovery and business continuity
    Advanced

    Reflecting on this subtopic, how does integration failure recovery and business continuity relate to the technical reliability patterns (idempotency, retry caps, circuit breakers) established in foundational material?

    Q76
    Integration failure recovery and business continuity
    Foundational

    Reflecting on the entire Integration domain (enterprise architecture, API lifecycle, cross-functional stakeholders, testing at scale, partner ecosystem, monitoring, and business continuity) as a whole, what single theme ties all seven subtopics together?

    Q77
    Solution design patterns for enterprise AI
    Foundational

    A senior partner is designing a reusable solution pattern for 'AI-assisted document review' that will be applied across several different enterprise clients. Why design this as a pattern rather than a one-off solution each time?

    Q78
    Solution design patterns for enterprise AI
    Applied

    A senior partner applies a proven 'document review' solution pattern to a new client, but the new client's documents have a structural characteristic (e.g., heavy use of embedded tables) the original pattern wasn't designed around. What should guide the senior partner's response?

    Q79
    Solution design patterns for enterprise AI
    Advanced

    A senior partner is deciding how to document a solution pattern so it remains useful as it's applied and refined across many engagements over time. What should this documentation emphasize beyond the pattern's current, latest form?

    Q80
    Solution design patterns for enterprise AI
    Foundational

    Why should a senior partner be cautious about over-generalizing a solution pattern based on success with just one or two client engagements?

    Q81
    Solution design patterns for enterprise AI
    Applied

    A senior partner is choosing between a pattern that favors a single, monolithic AI-assisted workflow versus one that favors a modular, composable set of smaller AI-assisted components a client can mix and match. What should inform this choice for a specific client?

    Q82
    Solution design patterns for enterprise AI
    Advanced

    A senior partner notices that a widely-reused solution pattern within their practice has an embedded assumption (e.g., about typical document length) that no longer holds as client use cases have diversified over time. What should this prompt?

    Q83
    Solution design patterns for enterprise AI
    Foundational

    Why might a senior partner intentionally design a solution pattern with explicit 'extension points' (places where client-specific customization is expected and architecturally supported) rather than designing for one fixed, uniform implementation?

    Q84
    Solution design patterns for enterprise AI
    Applied

    A senior partner is deciding whether a new, genuinely novel client requirement warrants creating an entirely new solution pattern or can be handled as a one-off customization within an existing pattern. What should guide this decision?

    Q85
    Solution design patterns for enterprise AI
    Advanced

    A senior partner's practice has accumulated many solution patterns over time, some now rarely used and possibly outdated. What organizational practice addresses this accumulation, connecting to a principle established for other accumulated organizational assets?

    Q86
    Solution design patterns for enterprise AI
    Foundational

    Why should a senior partner validate a solution pattern against a new client's actual data and context before fully committing to it, even for a pattern with a strong track record elsewhere?

    Q87
    Solution design patterns for enterprise AI
    Applied

    A senior partner is deciding how prescriptive a documented solution pattern should be — a detailed, step-by-step implementation guide versus a higher-level set of principles and a reference example. What tradeoff should inform this choice?

    Q88
    Solution design patterns for enterprise AI
    Advanced

    Reflecting on this subtopic, how does solution pattern design at the senior-partner level relate to the prompt-template and few-shot-example principles established in foundational material?

    Q89
    Reference architecture and design standards
    Foundational

    A senior partner is establishing design standards for how Claude-based solutions should be architected across an organization. What is the primary benefit of a documented reference architecture, beyond individual solution patterns already discussed?

    Q90
    Reference architecture and design standards
    Applied

    A senior partner proposes a reference architecture standard requiring all Claude-based solutions to implement a specific logging format. A project team pushes back, arguing their specific use case doesn't need it. How should the senior partner approach this tension?

    Q91
    Reference architecture and design standards
    Advanced

    A senior partner is deciding how to handle a reference architecture standard that was sound when established but has become outdated as the underlying technology and best practices have evolved. What governance process addresses this?

    Q92
    Reference architecture and design standards
    Foundational

    Why should a reference architecture document the rationale behind each standard, not just the standard itself (e.g., not just 'use this logging format' but why)?

    Q93
    Reference architecture and design standards
    Applied

    A senior partner is introducing a new reference architecture standard to an organization that already has many existing Claude-based solutions predating the standard. How should the rollout be approached, connecting to a change-management principle established earlier?

    Q94
    Reference architecture and design standards
    Advanced

    A senior partner is deciding how prescriptive a reference architecture should be regarding specific technology choices (e.g., mandating a specific logging library) versus specifying only the required behavior (e.g., 'logs must be structured and queryable,' leaving the specific tool open). What tradeoff should inform this?

    Q95
    Reference architecture and design standards
    Foundational

    Why should a senior partner solicit input from the teams who will actually implement against a reference architecture, before finalizing its standards, rather than designing it in isolation?

    Q96
    Reference architecture and design standards
    Applied

    A senior partner discovers that a reference architecture standard, though technically correct, is consistently misunderstood and misapplied by implementing teams. What does this suggest, connecting to a principle established for other kinds of guidance in this certification?

    Q97
    Reference architecture and design standards
    Advanced

    A senior partner is deciding whether conformance to the reference architecture should be enforced through automated tooling (linting, automated checks) or through manual review alone. What should inform this choice, connecting to the deterministic-vs-probabilistic principle established throughout this certification?

    Q98
    Reference architecture and design standards
    Foundational

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to reference architecture and design standards?

    Q99
    Technical debt and architectural tradeoffs
    Foundational

    A senior partner deliberately chooses a simpler, faster-to-ship architecture for an initial launch, knowingly deferring a more robust approach to a later phase. How should this decision be framed and tracked?

    Q100
    Technical debt and architectural tradeoffs
    Applied

    A senior partner is deciding whether to accept a specific piece of technical debt for a new solution. What should guide this decision, connecting to the risk-assessment framework established in foundational material?

    Q101
    Technical debt and architectural tradeoffs
    Advanced

    A senior partner maintains a technical debt register across several client solutions. Several items have remained unaddressed for a long time with no apparent consequence. What should this prompt the senior partner to consider, rather than simply concluding the debt was never actually a real risk?

    Q102
    Technical debt and architectural tradeoffs
    Foundational

    Why should a senior partner communicate accepted technical debt clearly to a client, rather than treating it as an internal engineering detail the client doesn't need to know about?

    Q103
    Technical debt and architectural tradeoffs
    Applied

    A senior partner is weighing whether to refactor an existing, working but architecturally suboptimal integration versus leaving it as-is and focusing effort elsewhere. What should guide this prioritization, connecting to the measure-before-optimizing principle established in foundational material?

    Q104
    Technical debt and architectural tradeoffs
    Advanced

    A senior partner notices that technical debt decisions across a portfolio of client solutions tend to be made ad hoc by whoever is under the most immediate delivery pressure, with no consistent evaluation framework. What governance response addresses this?

    Q105
    Technical debt and architectural tradeoffs
    Foundational

    Why might a senior partner distinguish between 'deliberate, informed' technical debt (a conscious tradeoff with known consequences) and 'inadvertent' technical debt (suboptimal architecture that emerged without anyone deliberately deciding to accept it)?

    Q106
    Technical debt and architectural tradeoffs
    Applied

    A senior partner is deciding how to weigh a client's short-term budget constraints against a more robust, higher-upfront-cost architectural approach. What synthesis, drawing on the total-cost-of-ownership principle established in foundational material, should inform this recommendation?

    Q107
    Technical debt and architectural tradeoffs
    Advanced

    A senior partner is deciding whether to pay down a specific piece of technical debt now versus bundling it with a larger, already-planned future architectural change. What should inform this sequencing decision?

    Q108
    Technical debt and architectural tradeoffs
    Foundational

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to technical debt and architectural tradeoffs?

    Q109
    Scalability and performance architecture
    Foundational

    A senior partner is designing a Claude-based solution expected to scale from hundreds to tens of thousands of users. What architectural principle, already established for pilot scaling, applies here at the design stage?

    Q110
    Scalability and performance architecture
    Applied

    A senior partner is deciding whether a Claude-based solution's architecture should scale primarily by adding more parallel processing capacity (horizontal scaling) or by making each individual request more efficient (vertical/per-request optimization). What should inform this choice?

    Q111
    Scalability and performance architecture
    Advanced

    A senior partner is designing a solution where some components scale easily (stateless API calls) and others are harder to scale (a shared, stateful resource several requests must coordinate through). What architectural approach addresses this asymmetry?

    Q112
    Scalability and performance architecture
    Foundational

    Why should a senior partner establish performance targets (acceptable latency, throughput) before designing a solution's scalability architecture, rather than designing first and measuring performance only after the fact?

    Q113
    Scalability and performance architecture
    Applied

    A senior partner is deciding how to validate that a solution's architecture will actually meet its performance targets at the expected future scale, before that scale is actually reached. What approach addresses this, connecting to the evidence-based-validation principle established throughout this certification?

    Q114
    Scalability and performance architecture
    Advanced

    A senior partner discovers that a solution's performance degrades non-linearly as scale increases (performance is fine at moderate scale but degrades sharply past a certain threshold). What does this suggest for the architecture, connecting to the chunking/granularity principles established elsewhere?

    Q115
    Scalability and performance architecture
    Foundational

    Why should a senior partner consider cost scaling alongside performance scaling when designing an architecture expected to grow significantly over time?

    Q116
    Scalability and performance architecture
    Applied

    A senior partner is deciding whether to invest in scalability architecture for a solution whose future growth is uncertain, versus building a simpler architecture now and investing in scalability only once growth actually materializes. What tradeoff, connecting to the pilot-value and premature-optimization principles, should inform this?

    Q117
    Scalability and performance architecture
    Advanced

    A senior partner is designing a solution where performance at the 99th percentile (the slowest 1% of requests) matters significantly for user experience, even though average performance is already good. What architectural consideration does this introduce, connecting to the tail-risk principle established in foundational material?

    Q118
    Scalability and performance architecture
    Foundational

    Why should a senior partner document the specific assumptions underlying a scalability architecture's design (e.g., 'assumes requests are evenly distributed across time, not concentrated in bursts')?

    Q119
    Scalability and performance architecture
    Applied

    A senior partner is deciding how to handle a scenario where a solution's actual usage pattern has shifted meaningfully from the pattern originally assumed during scalability design (e.g., usage has become much burstier than originally assumed). What should this prompt?

    Q120
    Scalability and performance architecture
    Advanced

    Reflecting on this subtopic, how does scalability and performance architecture relate to the risk-proportionality and evidence-based-validation principles established throughout this certification?

    Q121
    Architecture decision records and governance
    Foundational

    A senior partner recommends adopting a formal architecture decision record (ADR) practice for a client's Claude-based solutions. What is the core purpose of an ADR, connecting to a principle established throughout this certification?

    Q122
    Architecture decision records and governance
    Applied

    A senior partner is deciding which architectural decisions warrant a formal ADR versus which can be made and implemented without this level of documentation. What should guide this threshold?

    Q123
    Architecture decision records and governance
    Advanced

    A senior partner discovers that an existing ADR documents a decision that made sense at the time but has since become outdated as circumstances changed, yet the ADR is still being cited as current guidance. What should this prompt, connecting to the evolve-over-time principle?

    Q124
    Architecture decision records and governance
    Foundational

    Why should an ADR document the alternatives that were considered and rejected, not just the option that was ultimately chosen?

    Q125
    Architecture decision records and governance
    Applied

    A senior partner is establishing an architecture governance process for a client organization with multiple project teams. How should the governance process balance consistency across teams against each team's need for some autonomy in their own technical decisions?

    Q126
    Architecture decision records and governance
    Advanced

    A senior partner notices that architecture governance review has become a significant bottleneck, similar to the cross-functional review bottleneck discussed earlier in this domain. What established principle applies here by direct analogy?

    Q127
    Architecture decision records and governance
    Foundational

    Why should architecture governance include a clear, documented process for how a team can request an exception to an established standard, rather than treating every standard as absolute with no exception path?

    Q128
    Architecture decision records and governance
    Applied

    A senior partner is deciding how to make an organization's accumulated ADRs genuinely discoverable and useful to the teams who need them, rather than existing as a growing but effectively unsearchable archive. What practice addresses this?

    Q129
    Architecture decision records and governance
    Advanced

    A senior partner is deciding whether architecture governance for Claude-based solutions should be a distinct function from the organization's broader, pre-existing software architecture governance, or integrated into it. What should inform this choice?

    Q130
    Architecture decision records and governance
    Foundational

    Why should a senior partner periodically review whether an organization's architecture governance process itself (not just the individual standards it enforces) remains fit for purpose as the organization grows?

    Q131
    Architecture decision records and governance
    Applied

    Reflecting on this subtopic, how do architecture decision records and governance relate to the documentation and knowledge-preservation principles established throughout this certification as a whole?

    Q132
    Legacy modernization strategy
    Foundational

    A senior partner is advising a client on using a Claude-based solution as part of modernizing a legacy system, rather than a full rip-and-replace. What architectural role can the AI-assisted layer play in this strategy?

    Q133
    Legacy modernization strategy
    Applied

    A senior partner is deciding which functions of a legacy system to prioritize for AI-assisted modernization first. What should guide this prioritization, connecting to the bottleneck-focused scoping principle established earlier?

    Q134
    Legacy modernization strategy
    Advanced

    A senior partner is designing an AI-assisted modernization layer that will eventually make the underlying legacy system's specific implementation irrelevant from the user's perspective. What architectural discipline does this 'eventual legacy independence' goal require from the start?

    Q135
    Legacy modernization strategy
    Foundational

    Why should a senior partner document the legacy system's actual current behavior (including undocumented quirks) before designing an AI-assisted layer meant to eventually supersede specific legacy functions?

    Q136
    Legacy modernization strategy
    Applied

    A senior partner is deciding how to validate that an AI-assisted modernization layer correctly replicates a legacy function's real behavior, including its undocumented quirks. What approach addresses this, connecting to the evidence-based-validation principle established throughout this certification?

    Q137
    Legacy modernization strategy
    Advanced

    A senior partner is advising on a modernization effort where stakeholders disagree about whether a specific undocumented legacy quirk should be preserved (because some users depend on it) or fixed (because it's clearly a bug). What should guide this decision?

    Q138
    Legacy modernization strategy
    Foundational

    Why should a senior partner plan for a transition period where both the legacy function and its AI-assisted replacement run in parallel, rather than switching over all at once?

    Q139
    Legacy modernization strategy
    Applied

    A senior partner is deciding when it's appropriate to finally retire a legacy function after its AI-assisted replacement has been running in parallel. What criteria should guide this decision, connecting to the evidence-based-decision principle established throughout this certification?

    Q140
    Legacy modernization strategy
    Advanced

    A senior partner is advising on a modernization effort where the legacy system itself is the source of significant institutional knowledge embedded in its business logic (accumulated rules, exceptions, and edge-case handling built up over years). What risk does modernization pose here, beyond the technical replication challenge already discussed?

    Q141
    Legacy modernization strategy
    Foundational

    Why might a senior partner recommend using the modernization effort itself as an opportunity to interview long-tenured staff who understand the legacy system's institutional knowledge, before that knowledge is lost to attrition?

    Q142
    Legacy modernization strategy
    Applied

    A senior partner is deciding how to communicate a legacy modernization roadmap to executive stakeholders who are primarily interested in business outcomes, not technical architecture details. What approach addresses this, connecting to the stakeholder-translation principle established earlier in this domain?

    Q143
    Legacy modernization strategy
    Advanced

    Reflecting on this subtopic, how does legacy modernization strategy synthesize the principles established across this entire Solution Design & Architecture domain so far (patterns, reference architecture, technical debt, scalability, governance)?

    Q144
    Legacy modernization strategy
    Foundational

    Reflecting on the entire Solution Design & Architecture domain (solution patterns, reference architecture, technical debt, scalability, governance, legacy modernization) as a whole, what single theme ties all six subtopics together?

    Q145
    Evaluation framework design at enterprise scale
    Foundational

    A senior partner is designing a single evaluation framework meant to serve many different Claude-based solutions across an enterprise. What tension does this scale introduce, connecting to the rubric-design principle established in foundational material?

    Q146
    Evaluation framework design at enterprise scale
    Applied

    A senior partner is deciding what common baseline evaluation criteria should apply across every Claude-based solution in an enterprise, regardless of each solution's specific task. What kinds of criteria belong in this common baseline?

    Q147
    Evaluation framework design at enterprise scale
    Advanced

    A senior partner is deciding how to aggregate evaluation results across many solutions into an enterprise-wide quality dashboard for executive visibility. What risk should inform this design, connecting to the multi-signal-monitoring principle established in foundational material?

    Q148
    Evaluation framework design at enterprise scale
    Foundational

    Why should an enterprise evaluation framework be designed with input from the individual solution teams who will actually use it, rather than imposed entirely top-down by a central governance function?

    Q149
    Evaluation framework design at enterprise scale
    Applied

    A senior partner is deciding how often the enterprise evaluation framework itself should be reviewed and updated, connecting to the evolve-over-time principle established throughout this certification. What should guide this cadence?

    Q150
    Evaluation framework design at enterprise scale
    Advanced

    A senior partner notices that different solution teams across the enterprise interpret 'passing' the common baseline evaluation criteria inconsistently. What does this connect to, and what's the fix?

    Q151
    Evaluation framework design at enterprise scale
    Foundational

    Why should an enterprise evaluation framework explicitly distinguish evaluation criteria that are mandatory (every solution must pass) from those that are recommended best-practice (encouraged but not strictly required)?

    Q152
    Evaluation framework design at enterprise scale
    Applied

    A senior partner is deciding how to handle a new solution category (e.g., the first agentic, tool-using solution in a portfolio that previously only had simple text-generation solutions) that doesn't fit the existing evaluation framework's assumptions well. What should guide this situation?

    Q153
    Evaluation framework design at enterprise scale
    Advanced

    A senior partner is deciding how the enterprise evaluation framework should account for solutions built by external partners or vendors on the Claude platform, not just internally-built solutions. What principle applies by extension?

    Q154
    Evaluation framework design at enterprise scale
    Foundational

    Why should the enterprise evaluation framework itself be documented with the reasoning behind its structure (not just the criteria themselves), connecting to the documentation-of-reasoning principle established throughout this certification?

    Q155
    Evaluation framework design at enterprise scale
    Applied

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to enterprise evaluation framework design?

    Q156
    A/B and experimentation governance
    Foundational

    A senior partner is establishing governance for how experiments (A/B tests comparing prompt versions, model tiers, or workflow designs) are run across an enterprise's Claude-based solutions. Why does this warrant explicit governance rather than leaving each team to experiment however they see fit?

    Q157
    A/B and experimentation governance
    Applied

    A senior partner is reviewing an experiment design where the success metric was decided after seeing preliminary results, rather than before the experiment began. What problem does this illustrate, connecting to a principle established in foundational material?

    Q158
    A/B and experimentation governance
    Advanced

    A senior partner is deciding what proportion of production traffic should be allocated to a new experimental variant of a Claude-based solution, versus the current proven version. What should inform this allocation, connecting to the risk-tiered and pilot-value principles established throughout this certification?

    Q159
    A/B and experimentation governance
    Foundational

    Why should an experiment have a pre-defined stopping condition (when to conclude the experiment and act on the result) rather than running indefinitely until a desired result appears?

    Q160
    A/B and experimentation governance
    Applied

    A senior partner discovers that several experiments run across different teams within the enterprise have reached contradictory conclusions about a similar underlying question (e.g., one team found a lighter model tier performs just as well for a task, another found it performs notably worse for a similar task). What should this prompt?

    Q161
    A/B and experimentation governance
    Advanced

    A senior partner is deciding whether experiments involving a solution with real consequence for users (not a purely internal or low-stakes tool) should receive additional governance scrutiny beyond a standard experiment review. What should guide this?

    Q162
    A/B and experimentation governance
    Foundational

    Why should the results of experiments (both successful and unsuccessful) be documented and shared across the organization, not just acted upon by the team that ran them?

    Q163
    A/B and experimentation governance
    Applied

    A senior partner is deciding how to handle an experiment that shows a statistically significant improvement, but the actual magnitude of the improvement is small enough that the real business value may not justify the cost of implementing the change broadly. What principle should inform this decision?

    Q164
    A/B and experimentation governance
    Advanced

    A senior partner is deciding how experimentation governance should handle a scenario where an experimental variant shows a clear negative effect partway through the planned experiment duration. What should guide the response, connecting to the bounded-risk and graceful-degradation principles established throughout this certification?

    Q165
    A/B and experimentation governance
    Foundational

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to A/B testing and experimentation governance?

    Q166
    Production monitoring and quality assurance
    Foundational

    A senior partner is designing production quality monitoring for a Claude-based solution already in use by real users. Why does quality monitoring remain necessary after a solution has already passed pre-launch evaluation?

    Q167
    Production monitoring and quality assurance
    Applied

    A senior partner is deciding what proportion of production outputs should be sampled for human quality review, versus relying entirely on automated signals. What should guide this decision, connecting to the LLM-as-judge caveat and risk-tiered review principles established throughout this certification?

    Q168
    Production monitoring and quality assurance
    Advanced

    A senior partner notices that a solution's production quality metrics have remained stable, but user-reported complaints have quietly increased over the same period. What does this discrepancy suggest, connecting to the multi-signal-monitoring principle established in foundational material?

    Q169
    Production monitoring and quality assurance
    Foundational

    Why should production quality monitoring be designed to detect quality issues specific to particular user segments or input types, not just an overall average quality score?

    Q170
    Production monitoring and quality assurance
    Applied

    A senior partner is deciding how quickly a detected production quality issue should trigger an actual response (investigation, rollback, escalation), connecting to the risk-proportionality principle established throughout this certification. What should inform this response-time expectation?

    Q171
    Production monitoring and quality assurance
    Advanced

    A senior partner is deciding how production quality assurance should account for a solution's output being used as input to a downstream automated process (not reviewed by a human before being acted upon). What additional consideration applies here, connecting to the downstream-consequence-tracing principle established earlier in this domain?

    Q172
    Production monitoring and quality assurance
    Foundational

    Why should a senior partner ensure production quality assurance findings feed back into the solution's development and evaluation process, not remain siloed as a purely operational concern?

    Q173
    Production monitoring and quality assurance
    Applied

    A senior partner is deciding how to handle a situation where production quality assurance reveals a real, confirmed issue, but fixing it would require a change to a widely-shared, foundational component used by several other solutions. What should guide the response, connecting to the shared-dependency principle established elsewhere in this domain?

    Q174
    Production monitoring and quality assurance
    Advanced

    A senior partner is deciding how production quality assurance practices should differ (if at all) between a solution in its first month of production versus one that's been stable in production for two years. What should inform this?

    Q175
    Production monitoring and quality assurance
    Foundational

    Why should a senior partner ensure production quality assurance practices are documented and transferable, not dependent on one individual's personal knowledge of the solution's quirks?

    Q176
    Production monitoring and quality assurance
    Applied

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to production monitoring and quality assurance?

    Q177
    Performance benchmarking and optimization strategy
    Foundational

    A senior partner is establishing a benchmarking practice to compare Claude-based solution performance over time and across model tiers. Why should benchmarks be run against a fixed, versioned test set rather than an ad hoc, freshly-chosen set each time?

    Q178
    Performance benchmarking and optimization strategy
    Applied

    A senior partner is deciding what to include in a benchmark suite meant to compare different optimization approaches for a solution. Beyond the task's core success criteria, what else should the benchmark capture, connecting to the cost/latency-attribution principles established elsewhere?

    Q179
    Performance benchmarking and optimization strategy
    Advanced

    A senior partner is evaluating two optimization approaches that perform similarly on the benchmark's average-case metrics but differ in how they perform on the benchmark's harder, edge-case test items. How should this inform the final decision, connecting to the tail-risk principle established throughout this certification?

    Q180
    Performance benchmarking and optimization strategy
    Foundational

    Why should a senior partner establish a baseline benchmark measurement before implementing any optimization effort, rather than only measuring after the optimization is complete?

    Q181
    Performance benchmarking and optimization strategy
    Applied

    A senior partner is deciding how to prioritize which of several possible optimization efforts to pursue first for a solution with limited available engineering time. What should guide this prioritization, connecting to the proportionate-effort-allocation principle established throughout this certification?

    Q182
    Performance benchmarking and optimization strategy
    Advanced

    A senior partner is deciding how to benchmark an optimization that's expected to help primarily under high-concurrency production conditions, which the standard single-request benchmark doesn't capture. What should this prompt, connecting to the representative-testing principle established throughout this certification?

    Q183
    Performance benchmarking and optimization strategy
    Foundational

    Why should benchmark results and the specific optimization decisions they informed be documented, connecting to the documentation-of-reasoning principle established throughout this certification?

    Q184
    Performance benchmarking and optimization strategy
    Applied

    A senior partner is deciding how to re-validate a previously-chosen optimization configuration as the underlying model and usage patterns evolve over time, connecting to the evolve-over-time principle established throughout this certification. What should trigger this re-validation?

    Q185
    Performance benchmarking and optimization strategy
    Advanced

    A senior partner is deciding whether benchmarking and optimization efforts across different solutions in an enterprise portfolio should share common tooling and methodology, connecting to the pattern-reuse principle established in the Solution Design domain. What should inform this?

    Q186
    Performance benchmarking and optimization strategy
    Foundational

    Why should a senior partner be cautious about optimizing a solution specifically for its current benchmark, if that optimization doesn't correspond to genuine, real-world improvement?

    Q187
    Performance benchmarking and optimization strategy
    Applied

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to performance benchmarking and optimization strategy?

    Q188
    Continuous evaluation and model lifecycle testing
    Foundational

    A senior partner is designing a continuous evaluation practice (re-running evaluation regularly, not just at launch) for a portfolio of Claude-based solutions. What is the core rationale for this, connecting to the evaluation-drift principle established throughout this certification?

    Q189
    Continuous evaluation and model lifecycle testing
    Applied

    A senior partner is deciding how to structure continuous evaluation so that a detected regression can be traced back to its likely cause (a model update, a configuration change, a genuine usage-pattern shift). What practice supports this, connecting to the change-tracking principle established in the Integration domain?

    Q190
    Continuous evaluation and model lifecycle testing
    Advanced

    A senior partner is deciding how continuous evaluation should handle a scenario where the underlying model provider releases an update that's a clear net improvement generally, but the continuous evaluation reveals a specific regression for one narrow use case within a broader solution. What should guide the response, connecting to the risk-tiered rollout principle established in the Integration domain?

    Q191
    Continuous evaluation and model lifecycle testing
    Foundational

    Why should continuous evaluation include periodic re-testing against genuinely new, previously-unseen test cases, not only re-running the same fixed benchmark repeatedly?

    Q192
    Continuous evaluation and model lifecycle testing
    Applied

    A senior partner is deciding how continuous evaluation results should factor into the organization's model-version upgrade governance process established in the Integration domain. How do these two practices connect?

    Q193
    Continuous evaluation and model lifecycle testing
    Advanced

    A senior partner is deciding how to evaluate a Claude-based solution's behavior specifically around a model's deprecated-version sunset date, connecting to the API lifecycle principles established in the Integration domain. What should continuous evaluation specifically verify as this date approaches?

    Q194
    Continuous evaluation and model lifecycle testing
    Foundational

    Why should continuous evaluation infrastructure itself be tested and validated (not just assumed to be working correctly), connecting to the verify-after-fixing principle established throughout this certification?

    Q195
    Continuous evaluation and model lifecycle testing
    Applied

    A senior partner is deciding how to communicate continuous evaluation findings to business stakeholders who aren't closely tracking the solution's day-to-day technical operation. What approach addresses this, connecting to the stakeholder-translation principle established earlier in this domain?

    Q196
    Continuous evaluation and model lifecycle testing
    Advanced

    A senior partner is deciding how to balance the real cost of running continuous evaluation (compute, engineering time to maintain it) against its real risk-mitigation value. What should guide this, connecting to the risk-proportionality principle established throughout this certification?

    Q197
    Continuous evaluation and model lifecycle testing
    Foundational

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to continuous evaluation and model lifecycle testing?

    Q198
    Cost-quality optimization tradeoffs
    Foundational

    A senior partner is advising on an enterprise-wide cost-optimization initiative across a portfolio of Claude-based solutions. What foundational principle should guide this effort, connecting to the model-tier-matching principle established throughout this certification?

    Q199
    Cost-quality optimization tradeoffs
    Applied

    A senior partner is deciding how to validate a proposed cost-saving change (e.g., a lighter model tier, more aggressive caching) before rolling it out broadly. What approach addresses this, connecting to the evidence-based-validation and regression-testing principles established throughout this certification?

    Q200
    Cost-quality optimization tradeoffs
    Advanced

    A senior partner is deciding how to present a cost-quality tradeoff decision to a client where a cheaper configuration performs nearly as well on average but has a meaningfully worse tail-risk profile. How should this be communicated, connecting to the tail-risk and honest-communication principles established throughout this certification?

    Q201
    Cost-quality optimization tradeoffs
    Foundational

    Why should a senior partner consider the total cost of a quality regression (including remediation cost, reputational cost, lost trust) when evaluating a cost-saving change, not just the direct cost savings themselves?

    Q202
    Cost-quality optimization tradeoffs
    Applied

    A senior partner is deciding how to sequence a portfolio-wide cost-optimization effort across many solutions with limited engineering time. What should guide this sequencing, connecting to the proportionate-effort-allocation principle established throughout this certification?

    Q203
    Cost-quality optimization tradeoffs
    Advanced

    A senior partner is deciding how to handle a cost-optimization proposal that would save money by reducing a solution's human-review sampling rate, relying more heavily on automated quality signals. What should inform this specific decision, connecting to the LLM-as-judge caveat and risk-tiered review principles established throughout this certification?

    Q204
    Cost-quality optimization tradeoffs
    Foundational

    Why should a senior partner track cost-quality tradeoff decisions over time as a portfolio, not just as isolated individual decisions, connecting to the knowledge-preservation principle established throughout this certification?

    Q205
    Cost-quality optimization tradeoffs
    Applied

    A senior partner discovers that cost pressure has led several teams across a portfolio to make cost-saving tradeoffs without the same validation rigor established for other optimization decisions earlier in this domain. What does this suggest, connecting to the organizational-governance-scaling principle established earlier?

    Q206
    Cost-quality optimization tradeoffs
    Advanced

    A senior partner is deciding how to frame cost-quality optimization work to a client who views it purely as 'cutting corners.' How should this framing be addressed, connecting to the honest-communication and value-framing principles established throughout this certification?

    Q207
    Cost-quality optimization tradeoffs
    Foundational

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to cost-quality optimization tradeoffs?

    Q208
    Cost-quality optimization tradeoffs
    Applied

    A senior partner is deciding whether a cost-quality tradeoff decision should be revisited after the underlying model's pricing changes (e.g., a tier that was previously expensive becomes meaningfully cheaper). What should guide this?

    Q209
    Enterprise AI governance program design
    Foundational

    A senior partner is designing an enterprise-wide AI governance program for a client with many Claude-based solutions. What should this program's scope cover, beyond the per-solution risk assessment already established in foundational material?

    Q210
    Enterprise AI governance program design
    Applied

    A senior partner is deciding how much centralized control an AI governance program should exert over individual solution teams' decisions, connecting to the governance-balance principle established in the Solution Design domain. What should guide this?

    Q211
    Enterprise AI governance program design
    Advanced

    A senior partner is designing how an AI governance program should handle a solution that was built before the governance program existed (a 'brownfield' case) versus a new solution built after. What approach addresses this, connecting to the staged-rollout principle established for standards adoption earlier in this domain?

    Q212
    Enterprise AI governance program design
    Foundational

    Why should an AI governance program include a clear process for how new or emerging risks (not anticipated when the program was first designed) get incorporated into governance scope?

    Q213
    Enterprise AI governance program design
    Applied

    A senior partner is deciding how to structure governance program metrics so leadership can assess whether the program is actually working, not just that it formally exists. What should these metrics capture, connecting to the multi-signal-monitoring principle established throughout this certification?

    Q214
    Enterprise AI governance program design
    Advanced

    A senior partner notices that an AI governance program has become primarily a documentation exercise (teams fill out forms) with little actual risk-reducing effect. What does this illustrate, connecting to the automation-complacency and substance-over-form principles established throughout this certification?

    Q215
    Enterprise AI governance program design
    Foundational

    Why should an AI governance program designate clear escalation paths for when a solution team disagrees with a governance decision (e.g., a risk classification they believe is too strict or too lenient)?

    Q216
    Enterprise AI governance program design
    Applied

    A senior partner is deciding how an AI governance program should account for solutions that span multiple business units with different risk appetites. What approach addresses this, connecting to the modular-configuration principle established in the Integration domain?

    Q217
    Enterprise AI governance program design
    Advanced

    A senior partner is deciding how to measure whether an AI governance program has achieved appropriate organizational maturity, connecting to the organizational-scaling-transition-point principle established earlier in this certification. What should this assessment consider?

    Q218
    Enterprise AI governance program design
    Foundational

    Why should a senior partner document the reasoning behind the AI governance program's own structural design (not just the resulting policies), connecting to the documentation-of-reasoning principle established throughout this certification?

    Q219
    Enterprise AI governance program design
    Applied

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to enterprise AI governance program design?

    Q220
    Safety program leadership and incident response
    Foundational

    A senior partner is establishing an incident response process specifically for AI safety incidents (a harmful output, an inappropriate autonomous action) across a solution portfolio. Why should this process be distinct from (while connected to) the general technical incident response (outages, bugs) already established?

    Q221
    Safety program leadership and incident response
    Applied

    A senior partner is deciding how quickly a suspected (not yet confirmed) safety incident should trigger an initial response, connecting to the bounded-risk principle established throughout this certification. What should guide this?

    Q222
    Safety program leadership and incident response
    Advanced

    A senior partner is deciding how to balance transparency about a confirmed safety incident (informing affected users and stakeholders) against the real risk of causing undue alarm before the full scope and remediation are understood. What should guide this?

    Q223
    Safety program leadership and incident response
    Foundational

    Why should a safety incident response process include a structured post-incident review, connecting to the post-incident-review principle established in the Integration domain for technical outages?

    Q224
    Safety program leadership and incident response
    Applied

    A senior partner is deciding how to ensure lessons from one solution's safety incident inform other solutions across the portfolio that might share a similar vulnerability. What approach addresses this, connecting to the cross-team pattern-recognition principle established earlier in this domain?

    Q225
    Safety program leadership and incident response
    Advanced

    A senior partner is deciding how to calibrate the severity classification system used for safety incidents (minor, moderate, severe, critical), connecting to the rubric-clarity principle established throughout this certification. Why does this calibration matter?

    Q226
    Safety program leadership and incident response
    Foundational

    Why should a senior partner ensure the safety incident response process itself is tested (through tabletop exercises or simulations), not just documented, connecting to the recovery-procedure-testing principle established in the Integration domain?

    Q227
    Safety program leadership and incident response
    Applied

    A senior partner is deciding how to document safety incidents in a way that preserves organizational learning while being mindful of legal sensitivity around how incident documentation might be used later. What should guide this?

    Q228
    Safety program leadership and incident response
    Advanced

    A senior partner is deciding how to handle a near-miss (a situation that could have become a safety incident but was caught before causing real harm). Why should near-misses receive genuine attention, not just confirmed incidents?

    Q229
    Safety program leadership and incident response
    Foundational

    Why should a senior partner ensure the safety incident response process has clear ownership that doesn't depend on any single individual's availability, connecting to the bus-factor principle established throughout this certification?

    Q230
    Safety program leadership and incident response
    Applied

    Reflecting on this subtopic, what single theme best captures the senior-partner-level approach to safety program leadership and incident response?

    Q231
    Risk management framework and organizational accountability
    Foundational

    A client's steering committee asks why a Claude-based claims-triage assistant needs its own risk register instead of being tracked inside the general IT risk list. What is the strongest justification?

    Q232
    Risk management framework and organizational accountability
    Applied

    Your team scores a risk 'Claude produces an incorrect policy citation in a customer reply' as low likelihood, high impact. The client wants to close it because 'it rarely happens'. What is the most defensible response?

    Q233
    Risk management framework and organizational accountability
    Applied

    During a design review the product owner, the security lead and the data-protection officer each assume one of the others owns the risk of customer data appearing in prompts sent to Claude. What should you do?

    Q234
    Risk management framework and organizational accountability
    Advanced

    A client wants one composite 'AI risk score' of 1–100 for each deployed Claude workflow so executives can rank them at a glance. What is the main caution you should raise?

    Q235
    Risk management framework and organizational accountability
    Applied

    Which situation most clearly calls for a formal risk acceptance, as opposed to mitigation, transfer or avoidance?

    Q236
    Risk management framework and organizational accountability
    Advanced

    Six months after launch a vendor model update changes how a Claude workflow phrases refusals, and complaint volume rises. The original risk assessment never mentioned model updates. What is the best framework-level correction?

    Q237
    Risk management framework and organizational accountability
    Foundational

    What is the main purpose of a RACI-style matrix in an AI governance program?

    Q238
    Risk management framework and organizational accountability
    Applied

    A bank's internal audit team asks how they can verify that the AI risk controls you described are actually operating. Which deliverable best supports them?

    Q239
    Risk management framework and organizational accountability
    Advanced

    A client's risk committee wants to approve every prompt edit individually. The team ships prompt tweaks weekly, and the committee meets monthly. How do you reconcile oversight with delivery speed?

    Q240
    Risk management framework and organizational accountability
    Applied

    Why should the risk register record the control's owner and last-tested date, not just the control's description?

    Q241
    Risk management framework and organizational accountability
    Advanced

    Two business units want to deploy the same Claude-based document summarizer. One is a marketing team, the other a legal-discovery team. Should they share a single risk assessment?

    Q242
    Risk management framework and organizational accountability
    Applied

    Which statement best sums up the senior-partner approach to risk frameworks and accountability for Claude solutions?

    Q243
    Regulatory and compliance program management
    Foundational

    A client in a regulated industry asks you to 'make the Claude solution compliant'. What should you establish first?

    Q244
    Regulatory and compliance program management
    Applied

    Your solution sends customer messages containing personal data to Claude through the API. Legal asks where the data goes and how long it is kept. What is the right way to answer?

    Q245
    Regulatory and compliance program management
    Applied

    A new AI-specific regulation will take effect in nine months and may classify your client's hiring-screen assistant as high risk. What is the most useful step now?

    Q246
    Regulatory and compliance program management
    Advanced

    A multinational client operates in three jurisdictions with different rules on automated decisions affecting individuals. How should the Claude-based decision-support workflow be designed?

    Q247
    Regulatory and compliance program management
    Applied

    An auditor asks for the documentation trail for a Claude-powered eligibility recommendation made last quarter. Which artifacts would most directly let you reconstruct what happened?

    Q248
    Regulatory and compliance program management
    Foundational

    Why is a data protection impact assessment often worth completing before a Claude workflow processes personal data at scale?

    Q249
    Regulatory and compliance program management
    Applied

    The client's compliance officer insists that every customer-facing Claude answer includes a legal disclaimer, and the product team says users ignore them. What is the balanced position?

    Q250
    Regulatory and compliance program management
    Advanced

    A regulator publishes guidance that appears to conflict with a contractual commitment your client made to a major customer about automated processing. What is the most appropriate course?

    Q251
    Regulatory and compliance program management
    Applied

    Which practice best demonstrates ongoing compliance for a Claude workflow, as opposed to point-in-time compliance?

    Q252
    Regulatory and compliance program management
    Advanced

    A client wants to use Claude to draft responses to regulatory inquiries. What control design is most appropriate?

    Q253
    Regulatory and compliance program management
    Applied

    What is the best summary of the partner's role regarding legal interpretation in compliance work?

    Q254
    Responsible AI culture and organizational change
    Foundational

    A client has a thorough responsible-AI policy, yet staff routinely paste confidential material into unapproved tools. What does this most likely indicate?

    Q255
    Responsible AI culture and organizational change
    Applied

    Employees in the claims department fear the new Claude assistant will replace them and are quietly refusing to use it. Which approach is most likely to build real adoption?

    Q256
    Responsible AI culture and organizational change
    Applied

    A frontline analyst notices Claude confidently misstating a figure and hesitates to report it, worried about being seen as slowing the project. What organizational mechanism most directly helps?

    Q257
    Responsible AI culture and organizational change
    Advanced

    Executives at a client announce that AI will 'cut review time by 70%', and the review team is now measured on that figure. You suspect quality will suffer. What is the most constructive action?

    Q258
    Responsible AI culture and organizational change
    Applied

    What is the best way to build durable AI literacy in a client organization after delivery?

    Q259
    Responsible AI culture and organizational change
    Foundational

    Which leadership behavior most strongly signals that responsible AI is a real priority and not a slogan?

    Q260
    Responsible AI culture and organizational change
    Applied

    A business unit wants to move quickly with its own Claude pilots while central governance wants everything reviewed first. How do you propose resolving the tension?

    Q261
    Responsible AI culture and organizational change
    Advanced

    After a minor incident, a client's leadership wants to identify and discipline the engineer who wrote the prompt. What is the better organizational response?

    Q262
    Responsible AI culture and organizational change
    Applied

    Which indicator best suggests a healthy responsible-AI culture in a deployed program?

    Q263
    Responsible AI culture and organizational change
    Advanced

    A client's board asks how to make sure responsible AI practices survive staff turnover and shifting priorities. What would you recommend?

    Q264
    Responsible AI culture and organizational change
    Applied

    Which statement best summarizes the senior-partner approach to responsible AI culture and organizational change?

    Q265
    Executive and board communication
    Foundational

    You have five minutes with a CFO to update them on a Claude-based invoice-matching pilot. Which opening serves them best?

    Q266
    Executive and board communication
    Applied

    A board member asks, 'Can we be certain the AI will not make a mistake?' What is the most credible answer?

    Q267
    Executive and board communication
    Applied

    Your monthly executive dashboard for a Claude support assistant currently shows token counts, average latency and prompt length. Which change makes it more useful to executives?

    Q268
    Executive and board communication
    Advanced

    A pilot's accuracy dropped from 94% to 89% after a data source changed. The steering committee meets tomorrow. How should you present it?

    Q269
    Executive and board communication
    Applied

    An executive keeps describing the system as 'basically a smart employee'. Why might you gently adjust this framing?

    Q270
    Executive and board communication
    Applied

    You are asked to present a business case for expanding a successful Claude workflow to three more departments. Which structure is most persuasive and honest?

    Q271
    Executive and board communication
    Advanced

    A board wants a one-page statement on 'AI risk exposure' across eleven Claude workflows. What should the page prioritize?

    Q272
    Executive and board communication
    Foundational

    What is the main reason to state the baseline when reporting AI-driven improvement to executives?

    Q273
    Executive and board communication
    Applied

    An executive sponsor wants to announce a Claude assistant to customers as 'always accurate'. How do you respond?

    Q274
    Executive and board communication
    Advanced

    Two executives disagree sharply: one wants to accelerate rollout, the other wants to pause pending more testing. You are asked for a recommendation. What is the strongest approach?

    Q275
    Executive and board communication
    Applied

    How should you handle a technical term such as 'hallucination' in an executive briefing?

    Q276
    Executive and board communication
    Advanced

    Which summary best expresses the senior-partner approach to executive and board communication?

    Q277
    Expectation setting and scope negotiation
    Foundational

    A client says, 'We want Claude to handle all our customer emails.' What is the best first step?

    Q278
    Expectation setting and scope negotiation
    Applied

    The client expects the assistant to reach '100% accuracy' before launch. How do you reset the expectation?

    Q279
    Expectation setting and scope negotiation
    Applied

    Mid-project a stakeholder requests three new features that were not in the agreed scope. What approach maintains the relationship and delivery?

    Q280
    Expectation setting and scope negotiation
    Advanced

    A client's success definition is 'users like it'. How do you make this usable as a project objective?

    Q281
    Expectation setting and scope negotiation
    Applied

    What belongs in an explicit 'out of scope' section for a Claude-based solution?

    Q282
    Expectation setting and scope negotiation
    Applied

    A sponsor wants a delivery date before discovery has produced any data on document quality. What is the most responsible way to give a timeline?

    Q283
    Expectation setting and scope negotiation
    Foundational

    Why demonstrate a prototype with real, messy client data early, rather than polished sample data?

    Q284
    Expectation setting and scope negotiation
    Advanced

    A client's operations lead assumes Claude can learn permanently from corrections made by staff each day. How do you address this misunderstanding?

    Q285
    Expectation setting and scope negotiation
    Applied

    How should acceptance criteria be written for a summarization feature so both sides can agree when it is done?

    Q286
    Expectation setting and scope negotiation
    Advanced

    Procurement insists on a fixed-price, fixed-scope contract for a Claude solution whose feasibility is unproven. What structure reduces risk for both sides?

    Q287
    Expectation setting and scope negotiation
    Applied

    A stakeholder is disappointed that the assistant declines certain requests. What explanation best sets expectations?

    Q288
    Expectation setting and scope negotiation
    Advanced

    Which statement best captures the senior-partner approach to expectation setting and scope negotiation?

    Q289
    Delivery communication and status reporting
    Foundational

    A weekly status report for a Claude integration project reads 'work continues, no issues'. What is missing that stakeholders most need?

    Q290
    Delivery communication and status reporting
    Applied

    Evaluation results show the assistant is below target on one document type, but overall accuracy meets the goal. How should the weekly report handle this?

    Q291
    Delivery communication and status reporting
    Applied

    What makes a red/amber/green status indicator trustworthy in AI delivery reporting?

    Q292
    Delivery communication and status reporting
    Applied

    An unexpected vendor model deprecation notice arrives mid-sprint. Who should hear about it, and how fast?

    Q293
    Delivery communication and status reporting
    Advanced

    A demo went poorly in front of the client because the model gave a wrong answer to a live question. How should you follow up?

    Q294
    Delivery communication and status reporting
    Foundational

    Why should a status report distinguish 'blocked' from 'at risk'?

    Q295
    Delivery communication and status reporting
    Applied

    Which format best communicates the output of an evaluation run to a mixed audience of product, legal and engineering readers?

    Q296
    Delivery communication and status reporting
    Advanced

    Two stakeholders receive different numbers for 'accuracy' for the same assistant from different reports. How do you prevent this recurring?

    Q297
    Delivery communication and status reporting
    Applied

    How should the team communicate a planned prompt and model change to the support staff who use the assistant daily?

    Q298
    Delivery communication and status reporting
    Advanced

    A stakeholder pushes you to omit a known limitation from a public case study about the solution. What is the right approach?

    Q299
    Delivery communication and status reporting
    Applied

    Which summary best fits the senior-partner approach to delivery communication and status reporting?

    Q300
    Solution lifecycle, versioning and deprecation
    Foundational

    Why should a production Claude workflow record the exact model identifier it uses rather than a floating alias?

    Q301
    Solution lifecycle, versioning and deprecation
    Applied

    A provider announces that the model your workflow uses will be retired in six months. What should the lifecycle plan include?

    Q302
    Solution lifecycle, versioning and deprecation
    Applied

    Where should prompts for a production workflow live, and why?

    Q303
    Solution lifecycle, versioning and deprecation
    Advanced

    Usage of a workflow has fallen to a few requests per week, but it still consumes a maintained integration. How should you approach retirement?

    Q304
    Solution lifecycle, versioning and deprecation
    Applied

    How often should the regression evaluation set for a live Claude workflow be refreshed?

    Q305
    Solution lifecycle, versioning and deprecation
    Advanced

    The client's knowledge base feeding a retrieval-based Claude assistant is edited weekly by several teams. What lifecycle practice protects answer quality?

    Q306
    Solution lifecycle, versioning and deprecation
    Foundational

    What is the purpose of a rollback plan for a prompt or model change?

    Q307
    Solution lifecycle, versioning and deprecation
    Applied

    Which signals should trigger a lifecycle review of a mature Claude workflow, beyond the calendar schedule?

    Q308
    Solution lifecycle, versioning and deprecation
    Advanced

    A newer model scores higher on public benchmarks, and the client wants an immediate upgrade. What is the appropriate lifecycle decision process?

    Q309
    Solution lifecycle, versioning and deprecation
    Applied

    What is the main benefit of maintaining a change log that links each production change to its evaluation results and approver?

    Q310
    Solution lifecycle, versioning and deprecation
    Advanced

    Which statement best sums up the senior-partner approach to solution lifecycle management?

    Q311
    Handover, support and knowledge transfer
    Foundational

    What should a handover package for a Claude-based workflow contain at minimum?

    Q312
    Handover, support and knowledge transfer
    Applied

    The client's support team will own the assistant after launch. What handover approach gives them the best chance of success?

    Q313
    Handover, support and knowledge transfer
    Applied

    Which runbook entry is most useful for first-line support of a Claude assistant?

    Q314
    Handover, support and knowledge transfer
    Advanced

    The only engineer who understands the prompt design is leaving the client's team next month. What should you do?

    Q315
    Handover, support and knowledge transfer
    Applied

    What defines a useful hypercare period after launch?

    Q316
    Handover, support and knowledge transfer
    Applied

    How should you design the ongoing support model for an AI workflow with respect to incident severity?

    Q317
    Handover, support and knowledge transfer
    Advanced

    After handover, the client team makes prompt edits that break an important behavior, and they ask the partner to fix it. What lifecycle lesson should be built in?

    Q318
    Handover, support and knowledge transfer
    Foundational

    Why include decision records, not just final designs, in the handover?

    Q319
    Handover, support and knowledge transfer
    Applied

    What is a sound way to confirm a handover is complete?

    Q320
    Handover, support and knowledge transfer
    Advanced

    Which summary best captures the senior-partner approach to handover, support and knowledge transfer?

    Q321
    Model selection and capability tiering
    Foundational

    A workflow classifies 2 million short support tickets a day into twelve categories, and the labels are simple and well defined. Which model-selection starting point is most sensible?

    Q322
    Model selection and capability tiering
    Applied

    An architecture uses a large model for every step of a pipeline: routing, extraction, drafting and final review. Costs are high. What redesign principle applies?

    Q323
    Model selection and capability tiering
    Applied

    Your team compared two models on 40 hand-picked examples and found no difference. The client concludes the cheaper model is safe for production. What is the main concern?

    Q324
    Model selection and capability tiering
    Advanced

    A product needs both sub-second responses for chat and deep analysis for a nightly report. How should model choice be approached?

    Q325
    Model selection and capability tiering
    Applied

    When might it be justified to use extended reasoning or additional thinking budget for a task?

    Q326
    Model selection and capability tiering
    Foundational

    Which task is best handled through the asynchronous batch interface rather than real-time calls, assuming the provider offers one with a lower price in exchange for delayed results?

    Q327
    Model selection and capability tiering
    Advanced

    A regulated client requires that behavior not change without review, yet wants to benefit from model improvements. How should model versions be handled?

    Q328
    Model selection and capability tiering
    Applied

    A workflow's quality is acceptable but too slow. Which options should be considered before simply switching to a smaller model?

    Q329
    Model selection and capability tiering
    Advanced

    A model comparison shows Model X scoring higher on accuracy but Model Y is cheaper and nearly as accurate. How should the recommendation be framed to the client?

    Q330
    Model selection and capability tiering
    Applied

    Which situation most clearly calls for a multimodal-capable model?

    Q331
    Model selection and capability tiering
    Advanced

    Which statement best sums up the senior-partner approach to model selection and tiering?

    Q332
    Prompt design for production reliability
    Foundational

    A prompt for contract review says only 'Review this contract and tell me what's risky.' Results vary widely. What is the most effective first improvement?

    Q333
    Prompt design for production reliability
    Applied

    Why is it useful to separate instructions from the data to be processed, for example by delimiting the document with clear tags?

    Q334
    Prompt design for production reliability
    Applied

    A prompt contains twenty rules, some contradictory, accumulated through patches over months. Outputs are inconsistent. What is the best remediation?

    Q335
    Prompt design for production reliability
    Advanced

    You add three few-shot examples to a classification prompt and accuracy improves on the test set but drops in production. What is the likely issue to investigate?

    Q336
    Prompt design for production reliability
    Applied

    What is the benefit of telling the model explicitly what to do when the answer is not in the provided material?

    Q337
    Prompt design for production reliability
    Foundational

    Which prompt-writing habit most improves maintainability across a team?

    Q338
    Prompt design for production reliability
    Applied

    An output must always be in the customer's language, which varies per request. What is a robust way to achieve this?

    Q339
    Prompt design for production reliability
    Advanced

    A prompt instructs 'never mention competitors', yet a user's question about market alternatives leads to a mention. What is the sound design response?

    Q340
    Prompt design for production reliability
    Applied

    Why test prompts with adversarial and unusual inputs before release?

    Q341
    Prompt design for production reliability
    Advanced

    When the same prompt gives different tone across runs, what are the main levers?

    Q342
    Prompt design for production reliability
    Applied

    Which summary best expresses the senior-partner approach to prompt design for production reliability?

    Q343
    Structured output and tool-use prompting
    Foundational

    A downstream system parses Claude's reply as JSON, and about 2% of replies include a friendly sentence before the JSON, breaking the parser. What is the most robust fix?

    Q344
    Structured output and tool-use prompting
    Applied

    A tool definition named 'process' has a one-word description and accepts a free-text 'data' field. Claude frequently calls it with inappropriate arguments. What is the best improvement?

    Q345
    Structured output and tool-use prompting
    Applied

    Why should free-text fields in an extraction schema often be paired with enumerated values or constrained formats where possible?

    Q346
    Structured output and tool-use prompting
    Advanced

    An agent can call a tool that sends customer emails. During testing it sometimes sends before the draft was approved. Which design is most appropriate?

    Q347
    Structured output and tool-use prompting
    Applied

    When a tool returns a very large result, such as 50,000 rows, what is the sound pattern?

    Q348
    Structured output and tool-use prompting
    Applied

    What should an error returned from a tool to Claude contain to support recovery?

    Q349
    Structured output and tool-use prompting
    Advanced

    Two tools — search_orders and lookup_customer — overlap in what they can return, and Claude picks inconsistently between them. How should you resolve it?

    Q350
    Structured output and tool-use prompting
    Applied

    Why validate model-produced structured output even when a schema-constrained mode is used?

    Q351
    Structured output and tool-use prompting
    Advanced

    A multi-step agent needs to call five independent lookups before answering. How can the design reduce latency without hurting reliability?

    Q352
    Structured output and tool-use prompting
    Applied

    Which summary best captures the senior-partner approach to structured output and tool-use prompting?

    Q353
    Context window management and caching
    Foundational

    A support assistant resends a 12,000-token policy document with every request. Costs and latency are high. Which technique is the most direct improvement?

    Q354
    Context window management and caching
    Applied

    Cache hit rates are low even though the system prompt is large and stable. What should you check first?

    Q355
    Context window management and caching
    Applied

    A long conversation approaches the context limit, and early details matter later. Which strategy best preserves quality?

    Q356
    Context window management and caching
    Advanced

    Retrieval returns 40 passages for each query, and answer quality is worse than with the top 5. What is the likely explanation and fix?

    Q357
    Context window management and caching
    Applied

    Where should the most important instruction be placed in a very long prompt that includes large documents?

    Q358
    Context window management and caching
    Advanced

    The team wants to put an entire 800-page policy library into each prompt because the context window is large enough. What should they weigh?

    Q359
    Context window management and caching
    Applied

    Which information is worth keeping outside the prompt, in a store retrieved on demand, instead of embedded in every call?

    Q360
    Context window management and caching
    Advanced

    A multi-tenant application uses shared cached prefixes. What data-handling concern should the design address?

    Q361
    Context window management and caching
    Applied

    How can you detect that context handling is degrading answer quality in production?

    Q362
    Context window management and caching
    Advanced

    Which statement best sums up the senior-partner approach to context management and caching?

    Q363
    Prompt evaluation and iteration at scale
    Foundational

    A team edits a prompt and judges the improvement by trying five examples they like. What is the better practice?

    Q364
    Prompt evaluation and iteration at scale
    Applied

    For open-ended outputs like summaries, exact-match scoring is not usable. What is a sound alternative evaluation design?

    Q365
    Prompt evaluation and iteration at scale
    Advanced

    A model-based grader gives higher scores to longer answers regardless of correctness. How should you respond?

    Q366
    Prompt evaluation and iteration at scale
    Applied

    Production logs reveal a failure pattern not covered by the evaluation set. What should happen next?

    Q367
    Prompt evaluation and iteration at scale
    Advanced

    A prompt change improves average quality by 1.5 points but reduces accuracy on the highest-value customer segment. How do you decide?

    Q368
    Prompt evaluation and iteration at scale
    Applied

    Why run an evaluation multiple times when outputs are nondeterministic?

    Q369
    Prompt evaluation and iteration at scale
    Advanced

    The team wants to use production conversations to build the evaluation set. What must be addressed first?

    Q370
    Prompt evaluation and iteration at scale
    Applied

    How should prompt experiments be rolled out when the evaluation set cannot capture all real behavior?

    Q371
    Prompt evaluation and iteration at scale
    Advanced

    Evaluation scores keep rising while customer complaints are flat. What should you suspect?

    Q372
    Prompt evaluation and iteration at scale
    Applied

    Which summary best captures the senior-partner approach to prompt evaluation and iteration at scale?

    Q373
    Engineering workflows with AI coding assistants
    Foundational

    A 60-engineer team adopts Claude Code. Each developer gets different results because they describe the project's conventions differently. What is the most effective standardization step?

    Q374
    Engineering workflows with AI coding assistants
    Applied

    Leads worry that AI-generated pull requests are being approved with only a glance. Which control best addresses the risk?

    Q375
    Engineering workflows with AI coding assistants
    Applied

    A team wants an AI assistant to run shell commands and edit files autonomously in their repository. Which permission design is most defensible?

    Q376
    Engineering workflows with AI coding assistants
    Advanced

    Developers use the assistant to generate large refactors across dozens of files. Defects are rising. What process change best reduces them?

    Q377
    Engineering workflows with AI coding assistants
    Applied

    How should a team decide which tasks to hand to an AI coding assistant first?

    Q378
    Engineering workflows with AI coding assistants
    Applied

    A security team asks how to prevent secrets or customer data from leaking through developer prompts to an AI tool. Which combination is strongest?

    Q379
    Engineering workflows with AI coding assistants
    Advanced

    Teams increasingly encode repeated workflows — release checklists, migration steps — as reusable commands or automations for the assistant. What governance applies?

    Q380
    Engineering workflows with AI coding assistants
    Applied

    Junior developers rely heavily on the assistant and struggle to explain their code in reviews. What is the best response?

    Q381
    Engineering workflows with AI coding assistants
    Advanced

    Licensing counsel asks what the engineering process does to manage the provenance and license risk of AI-assisted code. What should the partner help put in place?

    Q382
    Engineering workflows with AI coding assistants
    Advanced

    Which summary best captures the senior-partner approach to engineering workflows with AI coding assistants?

    Q383
    Operational enablement and runbooks
    Foundational

    At 2 a.m. an on-call engineer sees complaints that the assistant is returning empty answers. Which asset would help most?

    Q384
    Operational enablement and runbooks
    Applied

    Which monitoring signals should an operations team watch for a production Claude workflow?

    Q385
    Operational enablement and runbooks
    Applied

    The provider's API starts returning rate-limit errors during peak hours. What operational design handles this best?

    Q386
    Operational enablement and runbooks
    Advanced

    Monthly Claude spend doubled with no increase in users. What is the best investigative approach?

    Q387
    Operational enablement and runbooks
    Applied

    What belongs in a post-incident review for an AI workflow incident?

    Q388
    Operational enablement and runbooks
    Advanced

    A critical workflow depends on a single provider API. What resilience design is proportionate?

    Q389
    Operational enablement and runbooks
    Applied

    How should an operations team handle user feedback that an AI answer was wrong?

    Q390
    Operational enablement and runbooks
    Advanced

    Logging prompts and responses helps debugging but may capture personal data. How should this be handled?

    Q391
    Operational enablement and runbooks
    Applied

    Which summary best captures the senior-partner approach to operational enablement and runbooks?

    Q392
    Measuring productivity and adoption
    Foundational

    A manager proposes measuring AI coding assistant success by the number of lines of code generated. What is the main problem?

    Q393
    Measuring productivity and adoption
    Applied

    Before rolling out an assistant to a department, what should be captured to evaluate the benefit later?

    Q394
    Measuring productivity and adoption
    Applied

    Usage dashboards show that half the licensed users never open the tool. What is the most productive response?

    Q395
    Measuring productivity and adoption
    Advanced

    Time saved on drafting tasks is high, but reviewers report that the review burden has grown. How should the productivity case be assessed?

    Q396
    Measuring productivity and adoption
    Applied

    How should self-reported time savings from surveys be used?

    Q397
    Measuring productivity and adoption
    Advanced

    An executive asks for the 'ROI of AI' on a single slide. What should the partner provide?

    Q398
    Measuring productivity and adoption
    Applied

    What makes an internal champions network effective for spreading good AI practice?

    Q399
    Measuring productivity and adoption
    Advanced

    Two years in, adoption has plateaued. Which approach best finds new value?

    Q400
    Measuring productivity and adoption
    Advanced

    Which summary best fits the senior-partner approach to measuring productivity and adoption?

    0/63 known

    Questions about the CCAR-P exam