Few moments in a first Type II audit cause more anxiety than the evidence request list. Suddenly the auditor wants “a sample of 25 new hires” or “all terminated employees for the period,” and nobody on the team knows how those numbers were chosen. Understanding how sampling works makes the process predictable, and it shows why clean, complete records matter more than polished documents.
This article explains what populations and samples mean in SOC 2, how auditors typically think about sample selection, why “completeness of population” is a common trap, and how to prepare so sampling does not become a source of exceptions.
Why auditors sample instead of testing everything
A Type II report gives an opinion on whether controls were suitably designed and operated effectively over a period of time, often six to twelve months. A company might onboard hundreds of employees, process thousands of changes and generate millions of log lines in that time. Testing all of it is impractical. Attestation standards allow the practitioner to use sampling and other procedures to obtain sufficient appropriate evidence.
Sampling is therefore a means to an opinion about the whole period. If a sampled item fails, the auditor evaluates whether it points to an isolated lapse or a broader control problem. If you want a refresher on how the period fits into the engagement, read our SOC 2 observation period explainer.
Key terms: population, sample, exception
| Term | Meaning | Example |
|---|---|---|
| Population | Every instance of the control activity during the period | All production code changes merged between January and June |
| Sample | The subset the auditor selects to test | A selection of those changes |
| Attribute | What the auditor checks on each item | Peer review recorded, approval before deploy |
| Exception | A sampled item that does not meet the attribute | A change deployed without approval |
| Deviation rate | How often exceptions occur relative to the sample | One exception out of the sample tested |
How auditors think about sample size
Each audit firm has its own methodology, and the AICPA does not publish one mandatory table for SOC 2 samples. What is consistent across firms is the underlying logic.
- Frequency of the control. A control that runs once a year (such as an annual policy review) has a population of one, so that single instance is tested. A control that runs continuously or hundreds of times, such as access provisioning, calls for a larger selection.
- Nature of the control. Automated controls, such as a system-enforced branch protection rule, are often tested by inspecting the configuration plus a small number of confirming instances. Manual controls need more samples because human execution varies.
- Risk. Higher inherent risk or more important controls tend to draw larger samples.
- Length of the period. A twelve-month period typically involves more testing across time than a three-month window, to confirm the control operated consistently.
- Prior results. Previous exceptions or weak areas may lead to expanded testing.
Because firms differ, we deliberately do not quote fixed numbers here. If a guide tells you that “the auditor will always sample 25,” treat it with caution and ask your own auditor for their sampling approach early. A good firm will explain it during planning.
The step most teams miss: proving the population is complete
Before an auditor selects a sample, they need confidence that the list you gave them is complete and accurate. This is often called information produced by the entity (IPE) or completeness and accuracy. If you hand over a spreadsheet of “all new hires” that quietly omits contractors, the sample is meaningless, and the auditor may need to redo the work.
To support completeness:
- Generate populations directly from the source system, such as HR, the ticketing tool, the CI/CD platform or the identity provider, rather than assembling them manually.
- Record the report parameters: date range, filters, the person who ran it and the date it was run.
- Avoid editing exports. If you must filter, document why.
- Cross-check totals against another source (for example, headcount from payroll versus the identity provider).
- Keep the query or screenshot of the report settings as evidence.
Typical populations auditors request
| Control area | Population usually requested | What they test |
|---|---|---|
| Onboarding | All new hires and contractors in the period | Background check, policy acceptance, access approval |
| Offboarding | All terminations in the period | Access removed within the stated timeframe |
| Access reviews | All review cycles | Review completed, reviewer identified, changes actioned. See SOC 2 user access review evidence |
| Change management | All production changes or pull requests | Review, testing and approval before release |
| Incident response | All security incidents or tickets | Triage, severity, resolution. See the incident response guide |
| Vulnerability management | Scan results and remediation tickets | Findings addressed within policy timelines |
| Vendor management | List of critical vendors | Due diligence and review of vendor assurance |
| Backups and recovery | Backup job logs, restore tests | Jobs ran, failures followed up. See backup and DR testing |
| Training | All employees required to complete training | Completion within the required window |
What happens when a sample item fails
An exception does not automatically mean a qualified opinion. The auditor considers the nature of the failure, whether it is an isolated human error or evidence the control is not operating, whether compensating controls exist and whether to extend testing with additional items. Possible outcomes include:
- The exception is noted in the report in the tests of controls section, with management’s response, and the opinion stays unqualified.
- The auditor expands the sample to determine how widespread the issue is.
- If the control did not operate effectively enough to meet the related criteria, the opinion may be qualified.
For detail on that last scenario, see our article on qualified SOC 2 opinions, exceptions and remediation. And remember that SOC 2 has no pass or fail score. The outcome is the auditor’s opinion plus the described test results.
How to prepare so sampling goes smoothly
- Ask for the sampling approach early. During planning, request the auditor’s expected populations and sample method.
- Self-test before the auditor does. Pull your own sample for each control mid-period and check it. This is exactly what a gap analysis or readiness review is for.
- Use consistent evidence locations. Keep tickets, approvals and screenshots in a predictable structure so requests can be answered quickly.
- Fix process problems as they arise. If you find a missed access removal in month three, correct it and document the remediation. Do not wait until the final month.
- Preserve timestamps. Screenshots without dates and exports without parameters are weak evidence.
- Do not backdate. Creating approvals after the fact to cover a missing one can turn a minor exception into an integrity concern.
- Respond to evidence requests promptly. Delays lengthen fieldwork and can extend the engagement.
Automation and sampling
Compliance platforms can export populations and collect evidence continuously, which helps with completeness. But they also expose gaps more visibly, since every unapproved change or late offboarding is logged. Tools do not decide your auditor’s sample, and they do not replace judgment about whether a population is complete. We discuss that balance in why GRC tools do not replace expert SOC 2 talent.
What this means for your business
Sampling rewards consistency. A team that follows its process every time, and can pull a clean record for any date in the period, rarely worries about sample size. A team that relies on heroics and memory will see exceptions no matter how small the sample is. Build habits now: ticket every change, record every approval, and keep the proof where an auditor can find it.
Our SOC 2 readiness service runs sample-style tests before the real audit so surprises surface early, and our Type I and Type II audit support helps coordinate evidence requests with your auditor. The final examination and opinion are issued by an independent CPA firm.
Frequently asked questions
How many samples does a SOC 2 auditor test?
It varies by audit firm, control frequency, risk and period length. There is no universal number in SOC 2, so ask your auditor for their methodology rather than relying on a figure from another company’s experience.
Does the auditor choose the sample or do we?
The auditor selects the sample from the population you provide. Management should not pre-select items, because that undermines the independence of the test.
What if our population is zero?
If no instances occurred, for example no terminations or no incidents, the auditor documents that and may use alternative procedures such as inquiry and inspection of the process design. Be ready to prove that the population is genuinely empty.
Is one exception enough to fail?
No. SOC 2 does not use pass or fail. One exception is evaluated in context and may simply be reported with management’s response. Multiple or systemic exceptions raise the risk of a qualified opinion.
Do automated controls get sampled too?
Often less heavily. Auditors may inspect configuration once and test that it was in place throughout the period, sometimes with a few supporting instances, because automated controls behave consistently unless changed.
Can we reduce the number of items tested?
Not by negotiating the sample. You can reduce risk by automating controls, shortening the number of manual steps and keeping populations clean and well documented.
Conclusion
Understanding SOC 2 audit sampling removes much of the mystery from a Type II examination. Auditors test a selection from complete populations, using methods that vary by firm, and they evaluate every exception in context. Your best preparation is not guessing a number but running a consistent process, producing trustworthy populations from source systems and self-testing throughout the period.
