1. Every answer is tied to a primary source
We start from the documents the exam itself is built on: federal regulations in the Electronic Code of Federal Regulations, government handbooks and advisory circulars, and the exam owner’s published outline of what the test covers. We do not write questions from other question banks, from memory or from leaked exam content.
For every question we store, alongside the question itself:
- the publisher and title of the document (with its edition or revision where that matters),
- the exact location — the section and paragraph of a regulation, or the chapter and page of a handbook,
- a word-for-word quote of the passage that proves the answer,
- the date the source was opened and checked.
We keep a saved copy of each source document. Before a question can be published, software checks that the stored quote really appears in that saved copy. If the quote cannot be found — even because of a single changed word — the question is blocked.
Sources are graded. Primary sources are the law, the regulator or the exam owner. Secondary sources (for example a university textbook) are used only when no primary document covers a point. Every published question has at least one primary or secondary source that supports its answer, and the sources are shown under each answer so you can check them yourself.
2. A separate review of the whole question
Questions are drafted with the help of AI language models working from the saved source text. A separate review pass — run independently of the drafting pass, with its own instructions — then re-reads the whole question as it will be published: the question, every option, the marked answer, the explanation and the sources. It records, for each of the points below, whether the question passed and which document it was checked against:
| Check | What the reviewer confirms |
|---|---|
| Correct answer | The marked answer is right under every reasonable reading, and each wrong option is wrong in every case — not just under one interpretation. |
| Sources | The cited passage actually supports the answer and the facts in the explanation. |
| Explanation | It says why the answer is right, why the others are wrong, and defines key terms. |
| Giveaways | You cannot guess the answer from the wording alone — for example because the right option is the longest, or the only one without “always” or “never”. |
| Scope | The question tests something in the official exam outline. |
| Duplicates | The same fact is not tested over and over. |
| Language | The wording is clear and has only one reading. |
If any part of a question changes later, the checks that depend on that part are automatically reset and must be done again before the new version can go live. This stops a “fix” from being checked only where it was edited.
We are open about this: the drafting and review passes are carried out by AI models under our supervision, with automated checks on top. We do not claim that a human subject-matter expert has signed off every question. That is precisely why every answer shows its source — so you never have to take our word for it.
3. Automated checks
- Quote matching against the saved source document (described above).
- Giveaway detection: the longest-option pattern, absolute words appearing only in wrong options, words from the question echoed only in the answer, evenly spaced numbers with the answer in the middle, and true/false sets that lean too far one way.
- Near-duplicate detection across the whole exam.
- Format checks: no “all of the above” (we shuffle options), explanations never refer to an option by its letter, and figures carry their own source.
- Dates: every change to a question records the date it changed. The “Last updated” dates on this site, and in our sitemap, come from those records — not from when the site was last deployed.
4. Corrections are public
Every question has a Report button. Each report is checked against the primary source. If the report is right, we correct the question, look for the same kind of error across the rest of the site, and log the change — with the date and what was wrong — on the corrections page. If the question was right after all, we keep a note of why.
5. What we cannot promise
Rules and exam outlines change. We record when each source was checked and re-check sources on a schedule, but the official documents always win. Exam Kata is independent and is not affiliated with the FAA, the EPA, CompTIA, Amazon Web Services or any other exam owner. Use it to practise and to find the official text quickly — and confirm anything important with the source.