A mystery shop is a controlled observation. A trained person visits your location as an ordinary customer, follows a scenario agreed in advance, and records what happened against criteria written before the visit. That last clause is the whole method. Criteria written afterwards describe the visit; criteria written beforehand measure it.
This is why mystery shopping and customer satisfaction research are not substitutes. A satisfaction survey asks a customer how they felt. A mystery shop records what occurred, timed and specified, by someone briefed to notice it. When the two disagree — and they frequently do — the disagreement is itself the finding.
What a scorecard can measure
A criterion earns its place if it is observable and unambiguous. "Was the shopper made to feel welcome" fails both tests: two shoppers will score it differently and neither score can be coached against. "Was the shopper greeted within thirty seconds of entering" passes both.
“Was the shopper made to feel welcome” is not a criterion. “Was the shopper greeted within thirty seconds of entering” is.On writing a scorecard
The criteria that survive that test are more numerous than most people expect:
- Durations — approach time, time to seat, time from order to delivery, wait against the wait that was quoted
- Sequences — whether a need was established before a product was recommended, whether a fee was disclosed before a signature was sought
- Presence or absence — was the test drive offered, was the check-back made, was the exclusion mentioned
- Physical condition — stock availability against a planogram, restroom condition against an itemised standard, equipment fitted and checked
- Verifiable claims — a specification statement that can be checked against published manufacturer data after fieldwork closes
What a scorecard cannot measure
It cannot measure clinical quality, repair quality, or anything else requiring expertise the shopper does not have. A mystery shopper can record whether a service adviser explained the estimate before work began; they cannot judge whether the repair was correct. Programmes that pretend otherwise produce reports that the department being measured can dismiss in one sentence, and they are right to.
It cannot measure attitude in the abstract. It can measure behaviours that people describe as attitude — being acknowledged, being interrupted, being handed a receipt rather than given one — and those behaviours are trainable in a way that attitude is not.
It also cannot measure anything from a single visit. One visit is one shift, on one day, with one staff member. A location's score becomes meaningful somewhere around the third visit in a wave, and a network's trend becomes meaningful at the second wave. Anyone selling you a single visit as a diagnosis is selling you an anecdote.
How the score gets built
Criteria are grouped into sections and each section carries a weight. The weighting is a commercial decision rather than a research one: it states what you consider important and it has to be defensible to the branch that scores badly because of it. A restaurant group that weights the check-back at twenty per cent is saying the check-back matters, and it should be prepared to say that out loud.
The written narrative matters as much as the score and is more often ignored. A branch scoring 71 tells you very little. A branch scoring 71 whose narrative reads "two staff were restocking and neither looked up during the four minutes I browsed" tells you exactly what to change. The number is for the trend; the narrative is for the action.
The test of a programme
A mystery shopping programme is working if a manager can read one report and know what to do differently on Monday. If the output is a percentage that goes into a slide, the programme is measuring something but it is not changing anything, and the two are commonly confused.