Ussama Assad Start a project

Self-check · 10 questions

The 10-question check

Two questions per domain, one at a time, about five minutes. After each answer you see why every option is right or wrong. At the end you get a result per domain and the notes to read first.

Without JavaScript this page shows a fixed set of 10 with the answers folded under each question. With it, you get one question at a time and a new set on each retake.

Q1 · Domain 1 · Agentic Architecture & Orchestration

A trip-planning assistant runs its own loop on the Messages API. The loop ends as soon as the latest response contains a text block. Users report the assistant saying "Let me check flights for you" and then going quiet. What is the bug, and what is the fix?

  1. A. The model needs a firmer instruction to finish its work before it replies, so add one to the system prompt.
  2. B. The loop needs a hard cap of 20 iterations so that it cannot stop before the work is done.
  3. C. Drive the loop from stop_reason: run the tools and continue on "tool_use", stop on "end_turn".
  4. D. Set tool_choice to "any" on every turn so that the model never answers in plain text at all.
Answer and explanation

Answer: C. stop_reason is the field the API sets to say why generation stopped. A text block can come before a tool_use block in the same response, so the text check exits while a tool call is still waiting. A cap is a safety net, not a completion signal. "any" on every turn would stop the assistant from ever giving a final answer. A prompt line is advice the model can still miss.

  • A. A prompt line is advice the model can still miss, and the loop would still exit on the first text block.
  • B. A cap is a safety net against a runaway loop. It is not a completion signal, and the loop would still exit on the first text block.
  • C. Right. stop_reason is the field the API sets to say why generation stopped. A text block can come before a tool_use block in the same response, so the text check exits while a tool call is still waiting.
  • D. Forcing a tool call on every turn means the assistant can never give its final answer in text.

Q2 · Domain 1 · Agentic Architecture & Orchestration

Your agent gets a response with stop_reason "max_tokens". The last content block is a tool_use block for create_invoice, and its input is cut off halfway through the line items. What should the loop do?

  1. A. Don't run it. Retry the request with a higher max_tokens so the whole tool call comes back.
  2. B. Run create_invoice with the line items that did arrive, then ask the model for the rest.
  3. C. Treat the turn as finished and show the user whatever text came before the tool call.
  4. D. Change tool_choice to "auto" and then resend exactly the same request a second time.
Answer and explanation

Answer: A. max_tokens means the output limit you set cut the response off. Anthropic's stop-reason docs say a truncated response with an incomplete tool_use block needs a retry with a higher max_tokens. Running a half-formed call writes a wrong invoice. Treating it as finished ignores pending work, and changing tool_choice does nothing about the limit that caused the cut.

  • A. Right. max_tokens means the output limit you set cut the response off. Anthropic's stop-reason docs say an incomplete tool_use block needs a retry with a higher max_tokens.
  • B. Running a half-formed call writes a wrong invoice. A truncated tool call should never run.
  • C. The response stopped because of the limit, not because the work was done. Treating it as finished ignores the pending tool call.
  • D. tool_choice does nothing about the output limit that caused the cut, so the same request gets cut off again.

Q5 · Domain 2 · Tool Design & MCP Integration

An intake service has three tools: extract_invoice, extract_receipt and extract_contract. You don't know which kind of document will arrive, but every response must be one of those tool calls, never prose. Your model supports forced tool use. Which tool_choice fits?

  1. A. {"type": "auto"}
  2. B. {"type": "any"}
  3. C. {"type": "tool", "name": "extract_invoice"}
  4. D. {"type": "none"}
Answer and explanation

Answer: B. "any" means the model must call one of the tools and picks which. "auto" lets it answer in text. "tool" forces one named tool, which is wrong for two of the three document types. "none" turns tools off. Check model support before relying on forcing: Anthropic's docs list newer models where "any" and "tool" return a 400, and point to "auto" with strict tool use or structured outputs instead.

  • A. auto lets the model answer in plain text, which the scenario rules out.
  • B. Right. any means the model must call one of the tools and picks which. Check model support before relying on it: the docs list newer models where any and tool return a 400.
  • C. tool forces one named tool, which is wrong for two of the three document types.
  • D. none turns tools off, so every response would be prose.

Q6 · Domain 2 · Tool Design & MCP Integration

A get_shipments MCP tool returns an empty list both when a customer has no shipments and when the carrier API times out. During an outage the agent tells customers they have nothing on the way. What should change?

  1. A. Add a prompt line: "If the shipment list comes back empty, tell the customer to try again later."
  2. B. Retry inside the tool up to five times, and return an empty list if the call still fails.
  3. C. On failure, return isError: true, a category, a retryable flag and a message. [] means none.
  4. D. Remove the tool entirely and have the agent ask each customer for their tracking numbers instead.
Answer and explanation

Answer: C. The agent can't tell "nothing found" from "couldn't look" because the tool gives the same answer for both. A structured error lets it say the right thing and decide whether to retry. The prompt line wrongly treats every real empty result as an outage, silent retries still hide the failure, and removing the tool throws away a working lookup.

  • A. This treats every real empty result as an outage, so customers who have nothing on the way get told to try again.
  • B. After the retries it still returns an empty list, so the failure is still hidden.
  • C. Right. The agent can't tell "nothing found" from "couldn't look" while both come back as []. A structured error lets it say the right thing and decide whether to retry.
  • D. Removing the tool throws away a lookup that works whenever the carrier API is up.

Q9 · Domain 3 · Claude Code Configuration & Workflows

SQL migration conventions should apply to db/migrations/**/*.sql and services/*/migrations/*.sql and nowhere else. The root CLAUDE.md is already long. What keeps these conventions out of unrelated sessions?

  1. A. Move the conventions into migrations.md and pull it into CLAUDE.md with an @migrations.md import.
  2. B. Create .claude/rules/migrations.md with a paths: list in its frontmatter holding both globs.
  3. C. Put a copy of the conventions in a CLAUDE.md file inside every migrations folder in the repo.
  4. D. Move the conventions out of the project and into your personal ~/.claude/CLAUDE.md file.
Answer and explanation

Answer: B. A rule with paths: frontmatter loads when Claude reads, writes or edits a matching file. Imported files load together with the CLAUDE.md that imports them, so an import tidies the file but saves no context. Per-folder copies duplicate the rule and drift. The personal file loads in every project and isn't shared with the team.

  • A. An imported file loads together with the CLAUDE.md that imports it. The file gets tidier but no context is saved.
  • B. Right. A rule with paths: frontmatter loads when Claude reads, writes or edits a matching file, and both globs fit in one list.
  • C. Per-folder copies duplicate the rule, and copies drift apart.
  • D. The personal file loads in every project and isn't shared with the team.

Q10 · Domain 3 · Claude Code Configuration & Workflows

Claude Code must never run git push --force against main in your repository. Which mechanism guarantees that?

  1. A. A line in the project CLAUDE.md stating that force pushes to main are forbidden.
  2. B. A PostToolUse hook on Bash that detects a force push and then reverts it.
  3. C. A PreToolUse hook on Bash that denies any command matching a force push to main.
  4. D. An instruction asking Claude to check with you before it runs any git command.
Answer and explanation

Answer: C. A PreToolUse hook runs before the tool call and can deny it. PostToolUse runs after the command has already happened, and a force-pushed history can't be reliably taken back. The CLAUDE.md line and the check-with-me instruction both depend on the model remembering.

  • A. A CLAUDE.md line depends on the model remembering it, so it can't guarantee anything.
  • B. PostToolUse runs after the command has already happened, and a force-pushed history can't be reliably taken back.
  • C. Right. A PreToolUse hook runs before the tool call and can deny it, so the push never runs.
  • D. An instruction to check with you still depends on the model remembering it.

Q13 · Domain 4 · Prompt Engineering & Structured Output

You extract the delivery date from scanned delivery notes. The instruction "Return the delivery date as YYYY-MM-DD" works on clean notes and fails on unusual layouts, where the date sits in a stamp or a handwritten box. What do you try next?

  1. A. Fine-tune a model on a few thousand labelled delivery notes before going any further.
  2. B. Ask for step-by-step written reasoning on every single note before it gives its final answer.
  3. C. Add a few examples of awkward layouts with the correct output, including one with no date (null).
  4. D. Raise max_tokens so that the model has more room to search each note carefully.
Answer and explanation

Answer: C. The desired behaviour is easier to show than to describe, which is what few-shot examples are for, and including a no-date case stops the model inventing one. Fine-tuning is a last resort. Step-by-step reasoning helps when the reasoning path matters; spotting a date in a stamp is recognition. max_tokens caps output length and has nothing to do with this.

  • A. Fine-tuning is a last resort. Showing the model a few examples comes first.
  • B. Step-by-step reasoning helps when the reasoning path matters. Spotting a date in a stamp is recognition.
  • C. Right. The behaviour is easier to show than to describe, which is what few-shot examples are for. The no-date example stops the model inventing a date.
  • D. max_tokens caps output length. It has nothing to do with where the date sits on the page.

Q14 · Domain 4 · Prompt Engineering & Structured Output

Timesheet extraction returns daily_hours (a list of numbers) and week_total. Every output passes schema validation, yet some week_total values don't equal the sum of daily_hours. What is the fix?

  1. A. Add minimum and maximum bounds to week_total in the schema so that out-of-range totals get rejected.
  2. B. Add a model-level check that sum(daily_hours) equals week_total, and retry with that exact error.
  3. C. Set the temperature to zero so that the arithmetic comes out identical on every single run.
  4. D. Append the sentence "Always double-check your totals before answering" to the prompt.
Answer and explanation

Answer: B. Schema validation checks each field's type on its own. A rule that relates two fields needs a model-level validator (in Pydantic, a model_validator). Feeding the failed output and the specific error into the retry gives the model something to correct. Bounds check one field in isolation, temperature zero can be consistently wrong, and a reminder in the prompt is advice.

  • A. Bounds check one field on its own. A total can be in range and still not match the days.
  • B. Right. Schema validation checks each field on its own. A rule that relates two fields needs a model-level validator, and sending the specific error with the retry gives the model something to correct.
  • C. Temperature zero makes the output consistent, not correct. It can be consistently wrong.
  • D. A reminder in the prompt is advice. Nothing checks that it worked.

Q17 · Domain 5 · Context Management & Reliability

A billing support agent can hand a conversation to a human. Which two are sound triggers for that handoff? (Select 2)

  1. A. The customer explicitly asks to speak to a person.
  2. B. A sentiment score on the customer's last message drops below a set threshold.
  3. C. The request falls outside documented policy, or the agent hasn't resolved it within its allowed attempts.
  4. D. The model rates its own confidence in its answer below 70%.
  5. E. The customer's message runs longer than 150 words.
Answer and explanation

Answer: A and C. Escalate on conditions you can state and check: the customer asked, the case isn't covered, or the agent is stuck. The public exam guide calls sentiment and self-reported confidence unreliable proxies for how complex a case is. Message length tells you nothing about whether a human is needed.

  • A. Right. An explicit request for a person is a condition you can state and check.
  • B. The public exam guide calls sentiment an unreliable proxy for how complex a case is.
  • C. Right. Outside documented policy, or stuck after the allowed attempts, are both conditions you can state and check.
  • D. Self-reported confidence is the other proxy the exam guide calls unreliable.
  • E. Message length tells you nothing about whether a human is needed.

Q18 · Domain 5 · Context Management & Reliability

Two sources disagree on when a supplier was founded. A company-registry record dated 2024 says 2011; a 2019 news article says 2009. Your research agent is writing the supplier profile. What should it do?

  1. A. Use 2010, the midpoint of the two, and mark the year as approximate in the profile.
  2. B. Report both years with their source and date, and flag the conflict for the reader.
  3. C. Keep the registry's year and drop the news article from the profile without comment.
  4. D. Leave the founding year out of the profile entirely, since the sources can't agree.
Answer and explanation

Answer: B. Keep each claim tied to its source and date, and show conflicts. Averaging invents a year neither source states. Dropping one silently hides evidence the reader may need. Leaving the field out loses both. Saying the registry is probably more reliable is fine, as long as both values stay visible.

  • A. Averaging invents a year neither source states.
  • B. Right. Each claim stays tied to its source and date, and the reader sees the conflict. Saying the registry is probably more reliable is fine, as long as both values stay visible.
  • C. Dropping one source silently hides evidence the reader may need.
  • D. Leaving the field out loses both pieces of evidence.