By Secret Customer Australia

A sound mystery shopping methodology connects a clear business objective with realistic scenarios, observable assessment criteria, suitable shoppers and evidence-based report validation. Quality assurance checks whether the submission supports its answers and ratings before findings are used. Results should be interpreted within the sample and scenario, with clear scoring rules, fair handling of uncertainty and practical actions for improvement.

This guide explains the principles in Secret Customer’s documented methodology. The agreed client brief and assessment rules determine the specific scenario, evidence and reporting requirements for each program.

Begin with an observable standard

Define what the employee or process should do before deciding how to score it. A standard might require an employee to clarify a customer’s intended use, explain a relevant option or confirm the next action.

Separate the observation from the interpretation. “The employee asked about the required size and budget” records behaviour. “The employee understood the customer” is a judgement that needs supporting evidence.

Design scenarios that allow the behaviour to occur

A shopper needs a realistic customer purpose, relevant background and clear instructions. The scenario should create an ordinary opportunity for the behaviour being assessed, without coaching the employee or provoking a failure.

Specify permitted prompts, evidence, spending boundaries and what happens when the expected interaction is unavailable. Different channels may need different criteria: a follow-up email has different opportunities and constraints from a busy counter transaction.

Brief shoppers and identify conflicts

Shopper suitability includes the ability to complete the scenario naturally, observe relevant details and describe them accurately. Briefing should explain the purpose, required conduct, evidence, submission requirements and exception handling.

Close relationships, recent employment or other conflicts can affect suitability. Recognition or an unexpected change in the scenario should be reported so its effect on the assessment can be considered.

Require evidence that explains the answer

Evidence may include factual notes, relevant timings, receipts or other material specified in the brief. The evidence should be necessary for the assessment and handled appropriately.

Weak statement Stronger evidence statement
The employee had a bad attitude. The employee continued a private conversation while I waited and did not acknowledge me before I asked for assistance.
The explanation was excellent. The employee compared the two options against the needs I had described and checked whether the difference was clear.
There was no follow-up. No follow-up was received through the specified channel by the agreed assessment deadline.

These examples are fictional. Accurate wording matters: a remembered summary should not be presented as an exact quotation.

Validate the report before relying on it

Secret Customer’s documented validation approach distinguishes several checks. The exact requirements depend on the assignment and the evidence it is designed to collect.

Check Question it answers
Assignment compliance Was the correct location, channel, time window and scenario assessed?
Completeness Are required responses and evidence present, or are exceptions explained?
Internal consistency Do closed answers, timings and narrative describe the same interaction?
Evidence support Does the account support the rating without unsupported assumptions?
Scoring Were the agreed weights, not-applicable rules and critical exceptions applied correctly?
Clarity and release Is the report understandable, appropriately presented and ready for its intended audience?

A query should ask the shopper to clarify what occurred. It must not suggest an answer that improves a score or manufactures missing evidence. Where the observation cannot be supported, the agreed exception rules should determine whether it is corrected, excluded or held.

Explain traffic light ratings

Red, amber and green can make findings easier to scan when each colour has a defined meaning and evidence behind it.

Rating Meaning within the agreed criterion
Green The required behaviour was clearly and effectively demonstrated to the expected standard.
Amber The behaviour was attempted or partly demonstrated, with a defined gap in completeness or effectiveness.
Red The required behaviour was absent or materially ineffective against the defined standard.

Amber describes a performance state; it should not be a substitute for reviewer uncertainty. Unclear evidence requires clarification. Numerical mappings and any aggregate calculation must be defined before colours are combined into a total score.

A fictional scored example

Criterion: confirm the customer’s need before recommending an option.

Observed interaction Illustrative rating Reason
The employee asks about the intended use, clarifies an important constraint and confirms the need before recommending. Green The defined discovery and confirmation behaviours are supported by the account.
The employee asks about intended use but recommends before clarifying the relevant constraint. Amber Discovery was attempted, with a defined part missing.
The employee recommends immediately without asking about the customer’s need. Red The required discovery behaviour was not demonstrated.

The example illustrates behavioural anchors. A live questionnaire must define its own criterion and opportunities; these ratings are not universal thresholds for every customer interaction.

Handle not-applicable answers and serious exceptions separately

“Not applicable” means the criterion genuinely did not arise under the agreed rules. It is different from failure and from insufficient evidence. Its treatment in the score denominator must be clear.

Some programs identify critical failures that need separate attention. A high average should not hide a serious exception, but neither should an unverified observation be presented as an established finding.

Calibrate interpretation and preserve comparability

Calibration means comparing interpretations of the same evidence against the agreed standard. It helps identify vague wording, borderline cases and differences between reviewers.

Comparisons also depend on scenario difficulty, channel, scoring weights, timing and sample size. A change from one questionnaire to another may change the total without showing an equivalent change in service. Document these differences before interpreting trends or comparing organisations.

There is no universal percentage that proves a business is delivering excellent service in every context. A benchmark needs a defined comparison group and compatible measures.

Use results fairly and protect information

Mystery shopping is most useful when it helps managers investigate patterns, recognise effective behaviour and improve systems or coaching. One encounter is not a complete measure of an employee’s performance or a customer population’s experience.

Limit personal information to what the purpose requires and control how evidence is shared. The OAIC’s Australian Privacy Principles guidance explains the relevant framework for organisations covered by those principles. Recording or other sensitive evidence requires the requirements of the actual activity to be checked before fieldwork.

Frequently asked questions

Does validation guarantee that every report is error-free?

No. Clear design, briefing and review reduce avoidable errors, but observation and sampling have limitations. Material disputed findings should be examined against the available evidence.

Is a traffic light score a national benchmark?

No. It is a defined interpretation of assessed evidence. A national benchmark would require an appropriate dataset, compatible measures and transparent sampling and publication rules.

What should happen when an answer and comment disagree?

The inconsistency should be queried and resolved through the evidence and agreed rules. The reviewer should not simply select whichever version produces the preferred result.

Related guides

Discuss assessment design and reporting

Explore Secret Customer’s mystery shopping services or contact our team to discuss your standards, channels and reporting needs. Our overview of mystery shopping explains where the method fits within customer experience measurement.

Sources and scope

This article summarises Secret Customer Academy guidance on ethical mystery shopping, scoring, evidence and quality assurance. It is a public explanation of methodology, not the full operational manual, a certification claim or a report of national industry averages.

Mystery Shopping

Please feel free to call us anytime

1300 88 33 16

Alternatively if you would like us to to give you a call,
simply fill out your details below