Sample Questions & Distractor Analysis

A) Implement a hook (PostToolUse) that blocks lookup_order and process_refund until get_customer returns a verified customer ID B) Expand the descriptions of both tools to clarify their different purposes and expected inputs C) Add a few-shot example in the system prompt showing the agent calling get_customer first D) Replace the agentic loop with a fixed orchestration that always calls get_customer first Correct Answer: A Why A is correct: Hooks provide deterministic guarantees that cannot be bypassed by prompt compliance issues. A PostToolUse hook can programmatically enforce that any lookup_order or process_refund call is blocked if get_customer hasn't been called yet. This is a prerequisite enforcement pattern that eliminates the 12% failure rate through code-level guarantees rather than probabilistic prompt compliance. Why other options are wrong: - B: Tool descriptions help with selection, but they address intent uncertainty, not enforcement. The agent might still skip get_customer even with clearer descriptions—this happened despite prompt improvements. - C: Few-shot examples are probabilistic. They improve likelihood of correct ordering but can't guarantee the agent won't skip steps, especially under edge cases. - D: Fixed orchestration removes agent flexibility for cases where get_customer calls might fail and require recovery logic.

A) Rewrite both tool descriptions to be more distinct and emphasize their differences B) Expand each tool's description with input formats, example queries, edge cases, and clear boundaries C) Add examples in the system prompt showing when to use each tool D) Remove one of the tools and have the other do both operations Correct Answer: B Why B is correct: Tool descriptions are the primary mechanism that the model uses to select between tools. Expanding descriptions with input formats (what parameters each accepts), example queries (typical use cases), edge cases (when NOT to use it), and clear boundaries makes the selection decision unambiguous. For example: "lookup_order: Accepts order_id or order_number. Use when you already know the order ID. Does NOT return customer name or account balance. Edge case: Fails if provided a customer name instead of order ID." Why other options are wrong: - A: "More distinct" isn't specific enough. Distinct descriptions still need concrete examples and boundaries to guide selection. - C: System prompt examples improve behavior but are less reliable than clear tool descriptions, which the model references directly during tool selection. - D: Tool consolidation reduces modularity and makes recovery from failures harder.

A) Add explicit escalation criteria with few-shot examples showing when to escalate vs. when to handle autonomously B) Remove escalation capability entirely and force the agent to handle all cases C) Implement a confidence scoring system that escalates low-confidence responses D) Add a hook that tracks escalation patterns and automatically handles similar cases after first escalation Correct Answer: A Why A is correct: The root cause is unclear decision boundaries between "standard" and "complex" cases. Few-shot examples in the system prompt (or explicit tool descriptions for an escalate tool) can show exact scenarios: "Standard damage replacements (visible physical damage, no dispute) → handle with process_refund. Policy exception (past return window, customer reports defect without proof) → escalate." This directly addresses the decision boundary problem. Why other options are wrong: - B: Escalation is valuable for genuinely complex cases. Removing it removes an important safety mechanism. - C: Confidence scoring doesn't directly address unclear decision boundaries. The agent may be escalating for reasons other than low confidence. - D: Hooks track patterns but don't clarify the decision criteria; the agent would still need guidance on when to escalate initially.

A) In .claude/commands/ in the project repository B) In each developer's ~/.claude.json personal configuration C) In a central team configuration file committed to the repo D) In the Claude Code CLI installation directory Correct Answer: A Why A is correct: Commands in .claude/commands/ are stored in the project repository and are automatically available to all developers who work on that project. They're version-controlled, team-scoped, and don't require individual setup. This is the standard pattern for team-wide tooling. Why other options are wrong: - B: ~/.claude.json is for personal, user-specific configuration. It wouldn't be available to all developers unless each configured it individually. - C: While you can commit custom files, .claude/commands/ is the designated location with built-in support. - D: CLI installation directories are system-wide and not project-scoped.

A) Use plan mode to explore the codebase, understand dependencies, and design the refactoring approach B) Use direct execution to start making changes immediately while iterating C) Use batch API to process all files in parallel for speed D) Start with a small subset of files to test the approach Correct Answer: A Why A is correct: Plan mode is designed for complex, multi-file architectural decisions. It explores the codebase, understands dependencies, and develops a comprehensive approach before making changes. This prevents costly mistakes in large refactors and helps you design the optimal path forward. Why other options are wrong: - B: Direct execution without understanding dependencies can cause cascading failures. - C: Batch API is for processing independent items in parallel (like processing documents), not for orchestrating interdependent code changes. - D: Testing on a subset first delays understanding the full scope and may not reveal cross-module issues.

A) Create .claude/rules/ with YAML frontmatter glob patterns that match file paths and apply area-specific conventions B) Create a single CLAUDE.md with all conventions and rely on Claude to choose the appropriate one C) Create separate projects for each area and manage them independently D) Add explicit comments to each file documenting its convention Correct Answer: A Why A is correct: .claude/rules/ with YAML frontmatter and glob patterns (e.g., path: "frontend/**" vs. path: "backend/**") applies rules automatically based on file location. Rules are evaluated regardless of directory depth, so /frontend/components/Button.js and /frontend/hooks/useForm.js both match the frontend pattern. This is the designated mechanism for path-specific conventions. Why other options are wrong: - B: A single CLAUDE.md doesn't automatically apply conventions; Claude would need to choose them manually each time. - C: Separate projects increase operational overhead and prevent sharing common infrastructure. - D: Comments require reading and manual interpretation; automated rules are more reliable.

A) Subagents failed to execute their tasks correctly B) Coordinator's task decomposition was too narrow; it should have included more creative domains C) Subagents ran in series instead of parallel, causing the last subagent to rush D) The initial query was too vague for the coordinator to understand Correct Answer: B Why B is correct: The subagents worked correctly within their assignments. The problem is what they were assigned. The coordinator decomposed "AI in creative industries" into only visual arts, music, and writing when the broader domain includes film, gaming, design, and performance art. Better decomposition would include: visual arts, music/audio, writing, film/video, gaming, and design/interactive media. Why other options are wrong: - A: Subagents executed well. The problem isn't execution; it's task definition. - C: Serial vs. parallel execution doesn't explain why music and writing subagents underdelivered (they all had full execution time). - D: The query "AI in creative industries" is clear enough; the issue is coordinator decomposition, not query understanding.

A) Return structured error context including failure type, attempted query, partial results (if any), and suggested alternatives B) Re-raise the error up the chain so the coordinator knows to retry C) Implement exponential backoff and retry automatically before returning error D) Switch to a different search API and try again Correct Answer: A Why A is correct: Structured error context enables the coordinator to make intelligent recovery decisions. For example: {errorType: "timeout", attemptedQuery: "AI in music production", partialResults: [3 documents], suggestedAlternatives: ["narrow query to genres", "search academic papers only"]}. This lets the coordinator decide to retry with narrower parameters, proceed with partial results, or escalate. Why other options are wrong: - B: Re-raising doesn't provide context for recovery; the coordinator still doesn't know what failed or why. - C: Automatic retry may exacerbate the timeout issue if the service is overloaded; the coordinator should decide. - D: Switching APIs requires code changes and doesn't address the root timeout problem.

A) Give the synthesis agent a scoped verify_fact tool for simple lookups; keep complex verifications routing through the coordinator B) Remove fact verification entirely and trust the subagent results C) Have the coordinator pre-verify all facts before assigning research to the synthesis agent D) Implement a caching layer so repeated fact-checks don't round-trip Correct Answer: A Why A is correct: This follows the principle of least privilege for the common case. 85% of fact-checks are simple (factual lookups with objective answers), which the synthesis agent can do directly with a scoped tool. 15% of complex verifications (policy interpretations, conflicting sources) still route through the coordinator's more sophisticated reasoning. This reduces latency while maintaining accuracy for edge cases. Why other options are wrong: - B: Removing verification removes a safety mechanism; unchecked facts propagate errors. - C: Pre-verification requires the coordinator to know all facts needed before synthesis begins, which is impractical. - D: Caching helps but doesn't address the fundamental latency of round-tripping to the coordinator for each check.

A) Add the -p or --print flag to run in non-interactive mode B) Pipe the PR content into stdin instead of using the prompt C) Use the batch API instead of direct execution D) Run the command with a timeout and fail if it hangs Correct Answer: A Why A is correct: The -p or --print flag is the documented way to run Claude Code in non-interactive mode, suitable for CI/CD. It produces output to stdout and exits without waiting for user input, which is exactly what a pipeline needs. Why other options are wrong: - B: Piping stdin doesn't prevent interactive prompts; the tool might still request clarification. - C: Batch API is for cost-optimized asynchronous processing, not for interactive execution concerns. It doesn't address the hanging issue. - D: Using a timeout masks the problem rather than fixing it. The command will fail on timeout even with -p flag missing.

A) Batch API for overnight reports only; keep pre-merge checks synchronous B) Use batch API for both; accept the latency trade-off C) Use neither; optimize code instead D) Use batch for pre-merge checks only Correct Answer: A Why A is correct: Batch API has no latency guarantee and processes requests within a 24-hour window. It's perfect for non-time-sensitive workloads like overnight reports. However, pre-merge checks are blocking workflows where developers wait for results, so latency matters. Synchronous API is appropriate there. This maximizes cost savings without sacrificing developer experience. Why other options are wrong: - B: Accepting latency for blocking pre-merge checks frustrates developers and defeats the purpose of fast feedback. - C: Optimization is good but doesn't address the batch vs. sync decision. - D: Batch API is unsuitable for blocking workflows regardless of which one it is.

A) Split into per-file focused passes followed by a cross-file integration pass B) Increase max_tokens to give Claude more space for detailed analysis C) Use few-shot examples to enforce consistent analysis style D) Review one file at a time in serial to avoid confusion Correct Answer: A Why A is correct: Reviewing 14 files in a single pass causes attention dilution. The model spreads its reasoning across too many contexts, resulting in some files getting deep analysis and others getting shallow treatment (attention drop-off). It also misses cross-file patterns. The solution is to split into focused passes: Pass 1 - detailed per-file review (faster, more attention per file), Pass 2 - cross-file integration review (catches consistency issues and architectural concerns). Why other options are wrong: - B: More tokens might help slightly but doesn't address the core issue of attention dilution across too many files. - C: Few-shot examples improve consistency style but don't address the depth-of-analysis problem caused by too many files. - D: Serial review is slower and doesn't improve analytical depth; it's the same problem spread over time.