How we work

We start from the decision, not the method. What will you do differently depending on what the data says? If the answer is nothing, the programme should not run, and we would rather establish that in week one than in month six.

Scoping settles the reporting units — location, region, format, market — because those determine sample size, and sample size determines cost. A programme sized for a national number cannot support a branch league table, and finding that out after fieldwork is the most common and most expensive mistake in this industry.

It also settles what evidence is collectible. Photography, audio and any handling of staff information are agreed here with your legal or compliance function, and the scorecard is written to ask only for evidence a shopper can lawfully obtain.

Criteria are written against the behaviours your training actually asks for. A criterion that does not correspond to something a staff member was told to do is unfair to score and useless to act on.

Every criterion has to be observable and unambiguous. 'Was the shopper made to feel welcome' is not a criterion; 'was the shopper greeted within thirty seconds of entering' is. Where a judgement is genuinely required, the scoring guidance defines what each point on the scale looks like, with examples.

Weighting is agreed with you, section by section, and it is a commercial decision rather than a research one. It says what you consider important, and it should be defensible to the branch that scores badly because of it.

The instrument is piloted on real visits before the wave opens. Every pilot to date has changed something — a criterion that reads clearly at a desk turns out to be unanswerable in a live environment more often than not.

Shoppers are selected against the customer profile agreed at scoping — age band, gender, language, and where relevant income bracket or family composition. A shopper who is not credible in the environment receives a different experience from your actual customer, which makes the data worse than no data.

New shoppers complete a written assessment before their first paid assignment. The written narrative is the part that cannot be faked, and it is the part most panels never check.

No shopper is told what score the client hopes for, and no shopper is briefed on the previous wave's result for the location they are visiting. Both are straightforward ways to contaminate a programme and both happen in this industry.

Shoppers are rotated. A shopper recognised on a return visit is spent for that location, and the panel is managed to avoid it.

Visits are scheduled across the dayparts and days of week agreed at scoping, so that the sample represents the operation rather than whichever slot was convenient for the shopper.

Completion is monitored daily against the quota. Where a location or a daypart is falling short, the assignment is re-issued while the wave is still open. A shortfall discovered at the end of fieldwork cannot be fixed; a shortfall discovered in week two can.

Reports are submitted within 24 hours of the visit, because recall degrades fast and a narrative written three days later is a reconstruction.

Each submitted report is reviewed for internal consistency: does the score match the narrative, does the narrative match the timings, does the evidence support both. Reports that fail go back to the shopper with specific questions rather than being silently corrected.

A proportion of visits are back-checked. Where a report cannot be substantiated it is withdrawn, and the shopper's subsequent work is reviewed.

Where a client challenges a report, it goes back through the same review against the same evidence. If the narrative and the score disagree, the report is corrected or withdrawn. A programme that cannot survive a challenge from the client is not worth running.

You receive the scored data, the full written narratives, and the evidence, through the client portal as the wave progresses rather than as a single delivery at the end.

The report names the specific behaviour to change rather than the general area. 'Improve customer service' is not a recommendation; 'the check-back is not happening at six of twelve branches and the four best-performing branches all do it within two minutes' is.

Where the sample cannot support a cut you asked for, the report says so instead of publishing a number that looks authoritative and is not. This is the single most useful discipline in the whole process and it is the one most often skipped.

Shoppers are identified to clients by reference only. Names, contact details and demographic records stay with us and are not disclosed as part of any report.

Where a narrative identifies a staff member, it does so by role, position and time rather than by name unless the client's own policy and the applicable law require otherwise. Reports are a management tool, not a disciplinary file, and they are written that way.

Evidence containing images of individuals who are not part of the scenario is not collected. Where it is captured incidentally it is removed before the report is published.

Retention periods for evidence and personal data are agreed at scoping and applied. The shopper data protection statement sets out what shoppers are told about their own data.