Quiz review that asks better questions, and keeps the answers.

CoQui is meant to support the collaboration between authors and reviewers: authoring, review, revision, and approval. Review is the first part built, and the vague-question problem above is only its most visible symptom.

A four-option item isn’t one judgment. It’s closer to two dozen, and they don’t all belong to the same person. Asking a domain expert to spot a cueing flaw, or a designer to verify a clinical fact, is how review surfaces go wrong. And when comments carry no resolution record, revision churns: the author can’t tell which concerns are still open, and the reviewer can’t tell what changed since their last pass, so every pass starts over.

Claims, not comments. Each part of an item carries one stated claim, and the reviewer affirms it or declines with a reason. The key: “Incontrovertibly correct — not merely the strongest of the options.” Each distractor: “Wrong — not merely weaker than the key.” A four-option item has twelve content-review claims, one of which the blind answer settles. The item’s shape fixes how many there are, so “done” is something you can count.

An open channel for everything else. The claims cover only what a reviewer can be asked to affirm. Everything else goes in a note or a suggested rewrite, attached to one claim or to the whole item. That includes the objection expert reviewers most often have and current tools most reliably lose: something is off, and I can’t say what.

Answer first, then see the key. The reviewer answers the item as a learner would and marks their confidence (sure because anyone could get it, sure because they know the material, or unsure), then commits. Only then does the key appear. If the two disagree, CoQui asks who was wrong rather than assuming it was the item: the reviewer, the key, both answers are defensible, or it needs checking against a source. That turns the hardest judgment in item review, is this ambiguous?, into a behavioral signal that costs one click. The key isn’t just hidden on screen: the server doesn’t send it to the browser until the answer is committed.

Roles matched to expertise. Author, content reviewer, craft reviewer, stakeholder: the same part carries a different claim depending on who is looking. People assign the roles; the software doesn’t enforce them. Review and approval are separate acts, often by the same person wearing two hats.

A record, not a thread. Every judgment is an entry in an append-only log. Nothing is edited or deleted; a correction is a new entry that says what it replaces. Once versioning is built, an edit will reset only the claims that rest on the changed text, so a late fix needs targeted re-review, not a fresh pass.

Instructional-design tools capture what was built and how learners performed. They rarely capture why the design decisions were made. Armature is design data infrastructure for learning engineering. It’s an open schema and API that models a course as a graph of objectives, assessments, items, and evidence, and keeps the rationale in the relationships between them. Think of it as version control underneath the authoring tools, making the history of the design visible.

Review is where much of that rationale gets spoken, and then lost. CoQui is designed to send Armature the outcomes: a change to an item and the reason for it, its approval, and findings against an item or an objective. The eighty attestations and fifteen threads that produced them stay in CoQui. The test for what crosses: would a researcher studying design process, or a designer inheriting this course in three years, need it?

CoQui is also Armature’s first real test. It’s designed from the review problem outward, not from the schema inward, because a tool shaped by the schema will always appear to fit it. Where a good design doesn’t map onto Armature, that’s a finding, and Armature is what changes. A framework designed without an application is a hypothesis, not infrastructure.

This demo runs without an Armature connection; its items come from CoQui’s own store.

Because every judgment is recorded, CoQui can run controlled comparisons of review methods, not just support them. The first compares answering blind before seeing the key with an otherwise identical review where everything is shown at once. It asks whether answering blind helps experts catch items that are miskeyed, ambiguous, or have more than one defensible answer. The demo isn’t that study; it’s the same workspace with the study switched off.

In the demo: the blind answer and the mismatch branch, content-review claims, notes and suggested rewrites, and the Record; and the craft review, the same items under a designer’s twelve claims, with the author’s declared purposes and level beside the claims that name them. Open the demo as either reviewer; each hat is a review of its own, and changing hats starts over.

Review the contentReview the craft

Not yet:

• Comparing items within a question bank.
• Comparing versions of an item, with claims marked stale by an edit.
• Seeing an item exactly as a learner would.
• Saved work, sign-in, and the Armature connection.

Folded away so the demo stays a fair test. Open it once you have reviewed the ten.