Valid means sahih
First principle: validity. Are you measuring what you are supposed to measure — and what you are supposed to measure is always the learning outcome, nothing else. Her Malay gloss made it memorable: valid means sahih — real. If your question is not measuring a learning outcome, it is not a valid item, whatever else it accomplishes. The example of invalid assessment she chose was uncomfortably recognisable: the trainer who uses a hard question as a weapon. A participant talks too much, so the trainer thinks "kita nak tunjuk dia siapa berkuasa" — let's show them who holds the power — and fires a question designed to embarrass. "Salah." Wrong twice over: it is not measuring any learning outcome, and — the sharper point — "you bagi ikut learning outcome pun belum tentu dia boleh jawab," a fair question would have done the job anyway. An assessment item deployed for ego is not an assessment at all. Validity is a discipline about purpose.
Reliable — the gemstone and the backwards marker
Reliability is consistency: the same performance, assessed by two competent assessors with the same criteria, should produce roughly the same judgment. Her illustration came from gemstones — a certified stone is trusted not because the seller says so but because the testing process behind the certificate produces the same verdict every time, and "you must go to the right person to test for consistency." The certificate is trustworthy exactly as far as the process is. Same reason an HRD Corp certificate means something: "I trust the process that gives me that certificate." Then the war story every assessor should hear before their first marking session. A panel is scoring participants; everyone gives fives, one assessor keeps giving ones. Investigation reveals: "rupanya dia ingat satu tu lima" — she thought one was the top of the scale. The exact same backwards-scale accident as Martin's one-one-one-one rating story yesterday, except this time the assessor was the one holding the pen. The fix is never to hope for better people; it is procedure — clear criteria, a structured marking scheme, calibration before anyone scores anything. "If we have a clear process, we have a clear procedure — that is where consistency comes from."
Fair, and flexible without lowering the bar
Fairness is equal and reasonable opportunity to demonstrate competence: everyone gets the same time, the same clarity about what will be assessed, how, and against what standard. It is also why tomorrow's assessment order will be decided by drawing lots — "remember, assessment principle kena fair" — no one gets to choose the comfortable afternoon slot, and no one gets ambushed with the 9 a.m. one. Flexibility was the principle with the most texture, because it sounds like a loophole and is actually a design tool. The exam is set for thirty minutes but after four days you can see the room needs more? Extend to forty-five — the standard is the competence, not the stopwatch. A calculation can be done three different ways, all correct? The marking scheme must accept all three, "because there is no one way to arrive there." You are teaching table etiquette but cannot assess in a hotel? Bring the hotel to the training room — napkins, forks, glasses on the table. Teaching senses in manufacturing? Bring the sensors to the class and have participants differentiate them by hand. Flexibility means varying the method while holding the standard absolutely still. The objective is never to make assessment easier — it is to give a fair route to the same evidence.
From occupational standards to training standards
Underneath the principles sits the architecture that makes them matter. An occupational standard describes the competence a job actually requires; a training standard translates it into learning outcomes, content and assessment; and the assessment standard is the bridge that lets evidence flow back to the workplace. The diagram — adapted from Bob Mansfield and Hermann Schmidt's 1999 work on standards — draws it as two worlds, work and learning, with translation in one direction and transfer back in the other. The principle is alignment, and it is the same triangle Day 3's constructive-alignment post drew: what the workplace requires, what the trainer teaches, and what the assessor measures must connect. Her standards-writing background sharpened the practical edge: every word in an occupational standard has been argued over, "because they will question you — is that correct?" A trainer who takes "handle customer complaints professionally" and teaches only complaint-logging has broken the chain at translation; an assessor who tests it with a multiple-choice quiz has broken it at measurement. The role-play or simulation is not a nice-to-have — it is what fidelity to the occupational standard demands.
Rules of evidence — VSAC
Principles govern the assessment; the rules of evidence govern what the assessment collects. Valid: the evidence must relate to the competency being assessed. Sufficient: enough of it to judge with confidence — her example was a cooking portfolio: if you claim you can make pizza, the portfolio needs the sauce, the dough, the topping and the bake, "kena ada contoh" for each, because one lucky Margherita is not competence. Authentic: "ini menunjukkan yang buat tu — you yang buat tu, bukan orang lain" — the work must be the participant's own, which is the fake engineer from this morning restated as a rule, and which the age of AI-assisted assignments has made the hardest rule on the list. Current: evidence must show present competence — "dulu masa you kerja" is history; are you still practicing it now? A certificate from a decade ago is a fact about the past. Four principles for the assessment, four rules for the evidence — the module is quietly building a chain of custody from learning outcome to judgment, and every link has a name.
Case Study
auditing Mana Mana's SOP test against the principles
We have one formal assessment at Mana Mana: the SOP quiz new front-desk staff sit after onboarding. Today's four principles took it apart in about ninety seconds. Validity: the quiz asks what the deposit amounts are, what the check-in hours are, what the WiFi password policy is. The learning outcome we actually care about is operate the check-in flow and handle the deposit process correctly. We are measuring recall of facts printed on the wall behind the desk, not the competence — a knowledge test standing in for a performance assessment, which is precisely the mismatch the validity principle forbids. Reliability: pass judgment currently belongs to whichever duty manager is on shift, with no marking guide — one DM's "okay, boleh" is another's "not yet." That is the backwards-marker problem in slow motion. The fix is Monday's competency checklist plus a two-line marking guide per item, and a ten-minute calibration where both DMs assess the same mock check-in and compare scores. Fairness: night-shift hires currently get assessed at hour nine of a shift, day-shift hires fresh at ten a.m. Same test, wildly different opportunity to demonstrate competence — schedule every assessment at shift start, full stop. Flexibility: our strongest housekeeping supervisor reads Bahasa far better than English; assessing her orally in Malay against the same checklist is not lowering the standard, it is removing an obstacle that was never the competence. And the rules of evidence fix our records: the file for each staff member should hold the scored checklist (valid), across three separate shifts (sufficient), signed by the assessing DM who watched them personally (authentic), re-verified annually (current). What we keep now is a signed attendance sheet — which, per this morning's story, is exactly the kind of evidence a man with his brother's certificate would pass.
Reflections
The four principles look like compliance vocabulary until you watch them fail one at a time: an invalid assessment measures the wrong thing confidently; an unreliable one measures the right thing randomly; an unfair one measures the participant's circumstances instead of their competence; an inflexible one measures their resemblance to the assessor's habits. Every bad test I have ever sat — or set — fails on one of the four. What I want to keep is the standard of the gemstone: trust flows from process, not from authority. Nobody trusts the stone because the seller is confident; they trust the certificate because the testing behind it is consistent. The moment my own assessments have clear criteria, calibrated markers and documented evidence, my certificate — a Mana Mana sign-off, a DJC course completion — starts meaning something. Until then it is an opinion with a logo.