Items, not questions
The terminology correction arrived with theatrical severity: "anda semua telah pun didedahkan dengan terminologi yang salah" — you have all been exposed to the wrong terminology. The thing you sat in school is not a kertas soalan, a question paper; it is a kertas ujian, a test paper. Because not everything in it is a question. A question ends with a question mark — di manakah, bagaimanakah, kenapakah. But "calculate the value" and "justify your answer" end with full stops and assess just as hard. The correct word for everything inside a test paper is item. Items then divide along two axes. Selection items put the answer on the paper — multiple choice, matching, true/false — and the participant selects it. Supply items make the participant produce the answer — short answers, fill-in-the-blank ("the academic word is short answer"), and on up through structured essays to extended essays. Selection items are heavily objective: right is right, a machine could mark them. And she pointed out where the weight falls: "objektif — preparation berat untuk siapa? Yang buat." Objective items are cheap to mark and expensive to write; essays are cheap to write and expensive to mark. The trainer pays either way — the only choice is when.
Anatomy of a multiple-choice item
A multiple-choice item, properly dissected: the stem, the key answer — the correct option — and the rest. What are A, C and D, if B is the key? The room offered "options." Her word was better: "these are called noise." Distractors, in the textbook language — and their job description is precise. A well-written distractor is chosen by exactly one kind of participant: "bukan sebab dia tak tahu, bukan sebab dia tak faham, bukan sebab dia malas — sebab dia tak achieve learning outcome." If a participant who did achieve the outcome can be tempted by a distractor, the item is broken. If a participant who didn't can spot the key by elimination, the noise is too weak. The whole diagnostic power of an MCQ lives in the quality of its wrong answers. Two footnotes gave the format sharper edges. Malaysia's public exams once ran five options and later reduced to four — moving the guessing floor from 20% to 25%, a deliberate trade of difficulty against simplicity. And her old roommate, a medical student, lived under the harshest MCQ regime of all: correct +1, blank 0, wrong −1. Negative marking, "because in medicine there's no guessing." A doctor who guesses confidently is worse than one who knows they don't know — so the scoring system priced guessing accordingly. Scoring is not arithmetic; it is policy about what behaviour you want.
The matching test that punished one mistake twice
Then a story about fairness hiding inside item design. Her children's school test: ten marks, five items, five options, match one-to-one. A child gets one match wrong. But with one-to-one matching, one wrong forces a second wrong — the misused option was somebody else's answer. "If you get one wrong, you get two wrong. If you get two wrong, confirm you get four wrong." A single error costs double, and a child who knew four of five scores three of five — 60% knowledge reported as 60%... except with two errors, 80% knowledge collapses to 20%. The scoring no longer measures the learning outcome; it measures the cascade. The fix is one design move: more options than items, so the mapping stops being one-to-one and each mistake costs exactly one mark. She raised it with the school — "I told them. So they don't invite me anymore." The price of item-writing literacy is that you can never unsee a broken test, and the reward is that your own participants never sit one. Fairness, it turns out, is not only invigilation and equal time; it can be broken at the level of a five-line matching exercise.
Journals, logbooks, portfolios — and a crossword
The written test is one tool in a much wider kit. Journals: personal reflections on the learning — one PTC candidate she remembered hand-wrote hers, drew in it, pasted photographs, "and it was more of their own learning" than any typed submission. Logbooks: the diary of the doing — what did you do on day one, morning and afternoon — the backbone of competency-based training records. Reports: submit a report of your five days of learning, and the analysis in it is the evidence. Portfolios got the longest run, because the format has quietly conquered the world. Give me a digital portfolio, she asked. Website — "boleh, but not so accurate." The real answer was already in everyone's pocket: Instagram. Cake sellers run their entire evidence-of-competence operation on it — every cake a work sample, every post dated and authentic. TikTok, "even worse" — better still, because video. Models and photographers were the original portfolio professionals, and now "there is no ground for you to say I cannot have a portfolio." Her two rules: start one, and keep work separate from personal — the travel photos dilute the evidence. Then the humbler tools: the 3-2-1 card — three things I didn't know before, two things I'll improve, one thing to avoid — which is a measurement of changed behaviour disguised as a feedback ritual, "so you can already measure that there is learning." And crossword puzzles for terminology-heavy content, built free on Puzzle Maker, with one hard-won operational tip: the generated puzzle vanishes if you regenerate, so "you generate, you suka, you snapshot." Assessment tools do not have to be solemn to be valid — they have to map to the outcome.
Case Study
rebuilding the DJC AI course quiz
My DJC AI course ends with ten multiple-choice questions I wrote in one sitting, and today's module graded my item-writing without mercy. My distractors are not noise; they are filler — options so obviously wrong that a participant who slept through the course can eliminate three and collect the mark. By the anatomy lesson, my MCQs measure test-taking, not learning outcomes. The rewrite rule: every distractor must be a real misconception I have actually heard in class. "The AI's answer is always factually correct" is a distractor with a job, because believing it is precisely what the course exists to fix — anyone who picks it has told me exactly which outcome they missed. Second fix: the quiz stops being the only instrument, because my course's learning outcomes are mostly performance, not recall. The portfolio moves to the centre — every participant leaves with the prompts and outputs they built during class, kept in one folder per person, which is Instagram-portfolio logic applied to knowledge work: dated, authentic, work-sample evidence. The 3-2-1 card closes every session, and I keep the cards — three things I didn't know before is before-and-after evidence in the participant's own handwriting, the cheapest measurement of changed behaviour I will ever collect. And for the jargon layer — tokens, context window, hallucination, RAG — a Puzzle Maker crossword as the day-two warm-up, snapshot taken before I regenerate. The deeper correction is the preparation-weight lesson. I had put my effort where marking is easy (MCQs) and skimped where writing is easy (open tasks). The module says a competent assessor pays the cost deliberately: well-built selection items for the knowledge floor, well-briefed performance tasks for everything that matters above it.
Reflections
Item is doing the same work in my head that outcome verbs did on Day 2 — a small vocabulary change that forces a design change. A question is something you ask; an item is something you engineer, with a stem, a key and calibrated noise, and it can be broken in ways that have names. Once the components have names, quality stops being a matter of taste. And the matching-test story earns a permanent place in my design conscience. The school test was not written by a villain; it was written by someone who never checked the failure cascade. Every scoring scheme is a claim about what a mistake costs, and the cost structure — one mark, two marks, minus one in medicine — is a design decision that either serves the learning outcome or quietly betrays it. The medicine faculty and the school test sit at opposite ends of the same lesson: decide what behaviour your scoring pays for.