Most of the value in a mystery shopping programme is decided in the first two weeks, before a single visit happens. What follows is how our programmes are built and run, in the order it actually occurs.
Scoping
We start from the decision rather than the method: what will you do differently depending on what the data says? If the answer is nothing, we say so. That is a cheaper conversation in week one than in month six.
Scoping settles the reporting units — location, region, format, market — because they determine sample size and sample size determines cost. A programme sized to produce a national number cannot support a branch league table. Discovering that after fieldwork has closed is the most expensive mistake in this industry and it is entirely avoidable.
It also settles what evidence is collectible. Photography, audio recording and anything touching a staff member's personal data are agreed here with your legal or compliance function. The scorecard is then written to ask only for evidence a shopper can lawfully obtain, rather than being written first and negotiated afterwards.
Scorecard design
Criteria are written against the behaviours your own training asks for. A criterion that does not correspond to something a staff member was instructed to do is unfair to score and impossible to act on.
The instrument is then piloted on real visits before the wave opens. Every pilot we have run has changed something. A criterion that reads clearly at a desk turns out to be unanswerable in a live environment more often than not — usually because it asks the shopper to observe two things at once, or to know something only a staff member would know.
Recruitment and briefing
Shoppers are selected against the customer profile agreed at scoping: age band, gender, language, and where it matters income bracket or family composition. A shopper who is not credible in the environment receives a different experience from the one your actual customer receives, and that data is worse than no data at all.
New shoppers complete a written assessment before their first paid assignment, because the written narrative is the part that cannot be faked and it is the part most panels never check.
No shopper is told what score the client is hoping for, and no shopper is briefed on a location's previous result. Both are straightforward ways to contaminate a programme and both happen in this industry.
Fieldwork
Visits are scheduled across the dayparts and days of week agreed at scoping, so the sample represents the operation rather than whichever slot suited the shopper. Completion is monitored daily against quota, and shortfalls are re-issued while the wave is still open. A gap found in week two can be filled; a gap found at the close cannot.
Reports are submitted within twenty-four hours. Recall degrades quickly, and a narrative written three days later is a reconstruction rather than an observation.
Quality assurance
Every report is reviewed for internal consistency before it reaches you: does the score match the narrative, does the narrative match the timings, does the evidence support both. Reports that fail go back to the shopper with specific questions rather than being quietly corrected by an editor.
When a client challenges a report, it goes back through the same review against the same evidence. If the narrative and the score disagree, the report is corrected or withdrawn. A programme that cannot survive a challenge from the client is not worth running, and saying so in advance is what makes the rest of the data credible.
A programme that cannot survive a challenge from the client is not worth running.Quality assurance
Reporting
You receive the scored data, the written narratives and the evidence through the client portal as the wave progresses, rather than as a single delivery at the end. Where the sample cannot support a cut you asked for, the report says so rather than publishing a number that looks authoritative and is not.
Total elapsed time from a signed brief to a first report is typically seven to eight weeks in Qatar. Scorecard design is the part that should not be compressed.