Dana Okafor had survived two SOC 2 Type II cycles at Brightloom Systems, a 40,000-seat HR technology platform, and she still felt her stomach drop when the fieldwork kickoff email landed. Kestrel & Boyce LLP was starting testing on Monday, and the engagement lead, Senior Audit Manager Owen Petrakis, had attached a request list with 61 line items. Somewhere in that list was the sentence that made her palms sweat every year: "Please provide the full population of user access changes for the period, from which we will select a sample for testing." Brightloom's renewal with its largest customer — a $3.1 million annual contract with an option to expand into two new product lines — was explicitly contingent on a clean SOC 2 Type II report with no unresolved exceptions. Dana knew the controls existed. She had watched her team run quarterly access reviews, approve every production deployment through a ticketing workflow, and rotate encryption keys on schedule. What she had not fully internalized, in her first two cycles, was that having a control was never the point. The point was whether Owen's team could independently verify, using one of four specific test procedures, that the control operated exactly as designed, every time it was supposed to, across the entire audit period — and whether Dana's evidence could survive that scrutiny without a single unexplained gap.
That distinction — between a control existing and a control being tested — is the one most compliance teams underestimate walking into their first SOC 2 examination. Auditors do not take your word for it. They do not read your policy document and move on. They apply a defined methodology, rooted in professional auditing standards, that determines exactly how much evidence they need, how they will gather it, and what threshold of failure turns a minor hiccup into a reportable exception. Understanding that methodology before fieldwork starts is the single highest-leverage thing a compliance owner can do to make an audit go smoothly.
Who This Is For and What You'll Walk Away With
This article is for compliance leads, engineering managers, and control owners preparing for their first (or fifth) SOC 2 Type I or Type II fieldwork engagement, and for auditors-in-training who want a practitioner's view of how testing decisions actually get made. You'll walk away understanding the four core test procedures and when auditors escalate between them, how sample sizes are actually calculated by control frequency, the mechanical difference between testing control design and testing operating effectiveness, and the path a single failed test item travels before it becomes a documented exception in your final report.
What "Testing" Actually Means in a SOC 2 Engagement
A SOC 2 examination is governed by SSAE 18, the AICPA's attestation standard, and it is fundamentally different from a checklist audit. The auditor is not confirming that documents exist — they are forming an independent opinion, backed by evidence they gathered and evaluated themselves, about whether your controls were suitably designed and, for a Type II report, operating effectively across the review period. Every control in your system description gets mapped to one or more of the Trust Services Criteria, and every mapped control gets a test plan. That test plan specifies which of four procedures the auditor will use, how large a sample they will pull from the population of instances where that control operated, and what "pass" looks like for that specific test.
This matters because the rigor of the test procedure is not arbitrary — it is a direct function of how much audit evidence the auditor believes they need to reach a supportable conclusion, which in turn depends on the risk the control is mitigating, how often it operates, and how strong the surrounding control environment is. A control protecting cardholder-adjacent data gets tested harder than a control governing the office recycling policy. Understanding that logic lets you anticipate, rather than react to, what your auditor is going to ask for.
The Four Test Procedures, In Increasing Order of Rigor
Every SOC 2 test, without exception, is built from one or more of four procedures: inquiry, observation, inspection, and re-performance. They are listed in professional auditing guidance in that order deliberately, because each one provides progressively stronger evidence than the one before it. An auditor's methodology decisions — which procedure to use, and whether to combine several — flow directly from this hierarchy.
Test Procedure | What the Auditor Does | Evidence Strength | Typical Standalone Use |
|---|---|---|---|
Inquiry | Asks control owners and staff how a control works | Weakest | Rarely used alone; supports other tests |
Observation | Watches the control being performed in real time | Moderate | Physical security, onboarding/offboarding walkthroughs |
Inspection | Examines documentation, logs, tickets, or system configuration as evidence the control operated | Strong | Change tickets, access review records, policy sign-offs |
Re-performance | Independently redoes the control to confirm the same result | Strongest | Reconciliations, access provisioning logic, encryption configuration |
"Inquiry tells me what someone believes happens. It never tells me what actually happened. If inquiry is the only evidence in my workpapers for a control, I haven't tested anything yet — I've had a conversation." — Owen Petrakis, Senior Audit Manager, Kestrel & Boyce LLP
Inquiry: Necessary, Never Sufficient
Inquiry is the starting point of nearly every test, and it is indispensable for understanding intent, but under professional auditing standards it can never stand alone as the sole basis for a conclusion. When Owen's team sits down with a Brightloom engineer and asks, "Walk me through what happens when someone requests elevated production access," that conversation tells the auditor what the control is supposed to do. It does not tell them whether the last 12 months of access requests actually followed that process. Auditors use inquiry to build context, identify who the real control owner is (which is not always the person listed in the policy), and surface informal workarounds that never made it into documentation. Every inquiry response gets corroborated with a stronger test procedure before it appears as support for a conclusion in the report.
Observation: Watching the Control Happen
Observation moves the auditor from being told about a control to watching it execute. During fieldwork, this often looks like a live screen-share where a control owner performs a task — approving a deployment, badging into a data center, running a vulnerability scan — while the auditor watches and takes notes. Observation is powerful for controls that leave a thin evidentiary trail, like physical access control at a facility, but it has an inherent limitation: the auditor is only seeing one instance, at one moment, and people tend to perform more carefully when they know they're being watched. Auditors treat observation as strong evidence for design and for a single point-in-time instance, but they still need inspection or re-performance to conclude the control operated consistently across the full period.
Inspection: Examining the Evidence Trail
Inspection is the workhorse of most SOC 2 fieldwork. The auditor examines a document, a system log, a signed-off ticket, or a configuration screen and forms a conclusion based on what that record shows. If Brightloom's change management process requires a peer-reviewed pull request and a documented approval before production deployment, the auditor inspects a sample of change tickets and confirms each one carries evidence of review, approval, and testing prior to the deploy timestamp. Inspection is popular because it produces durable, reviewable audit evidence that supports the auditor's own workpapers, and because — unlike observation — it can be applied retrospectively across an entire audit period rather than a single moment in time.
Re-performance: Independently Redoing the Control
Re-performance is the highest-rigor test available and is reserved for controls where the auditor needs to independently confirm the outcome, not just confirm that a process was followed. If Brightloom asserts that terminated employees lose system access within 24 hours, an inquiry-only test would ask HR to describe the process. An inspection test would review the offboarding ticket. A re-performance test goes further: the auditor pulls the actual list of terminations for the period, independently cross-references it against active directory or the identity provider's access logs, and calculates the actual elapsed time themselves rather than trusting a report Brightloom generated. Re-performance is common for reconciliations, encryption key configuration checks, firewall rule reviews, and any control whose output can be independently recalculated from raw system data.
Why Auditors Blend Methods
In practice, very few controls get tested with a single procedure in isolation. Auditors design "combined" tests that layer methods to build a defensible conclusion efficiently — inquiry to understand the process, inspection of a documentation sample to confirm it happened, and observation or re-performance for the highest-risk elements.
Control Example | Inquiry | Observation | Inspection | Re-performance | Rationale |
|---|---|---|---|---|---|
Quarterly access reviews | Yes | No | Yes (review sign-off) | Sometimes (recalculate reviewer's decision) | Confirms process and documented completion |
New hire background checks | Yes | No | Yes (vendor report on file) | No | Third-party attestation is sufficient corroboration |
Production deployment approval | Yes | Sometimes (live demo) | Yes (ticket trail) | Sometimes (verify no bypass path exists) | High-risk control gets layered evidence |
Data center badge access | Yes | Yes (site visit or camera footage review) | Yes (badge logs) | No | Physical controls favor direct observation |
Encryption key rotation | Yes | No | Yes (rotation logs) | Yes (verify current key age against policy) | Objective, recalculable outcome |
"The clients who go smoothest through fieldwork are the ones who assume we'll ask for the strongest evidence available, not the easiest evidence to produce. If a screenshot is your only artifact for a control that runs 200 times a year, that's a conversation we need to have before testing, not during it." — Renata Ilic, Partner, Ashgrove Assurance Partners
Type I vs Type II: Testing Design vs Testing Operating Effectiveness
The single biggest driver of how much testing you'll experience is whether your engagement is a Type I report or a Type II report. This distinction changes not just the report's conclusion but the entire testing methodology behind it, and it's worth understanding in detail alongside SOC 2 Operating Effectiveness, which covers the demonstration side of this same question in depth.
Dimension | Type I Testing | Type II Testing |
|---|---|---|
What's being tested | Suitability of design at a single point in time | Operating effectiveness across the full audit period (commonly 3–12 months) |
Primary test procedures used | Inquiry, observation, inspection of a single instance | All four procedures, applied to samples pulled across the period |
Sampling approach | Test of one — confirm the control exists and is designed correctly today | Statistically or risk-informed samples reflecting control frequency over months |
Evidence dated | As of the report date only | Spread across the entire period under review |
What a "pass" demonstrates | The control, if operated, would meet the criterion | The control actually did operate as designed, consistently, over time |
Buyer perception | Lower assurance; often a stepping stone | Higher assurance; what most enterprise buyers require |
Type I testing is comparatively quick because the auditor is confirming that a control is well-designed and exists as of one date — a snapshot. Type II testing is where most of the rigor described in this article applies, because the auditor must gather evidence spanning the entire review window and conclude the control didn't just exist on day one, but ran correctly on every applicable occasion in between.
The Walkthrough: Where Testing Begins
Before any sample gets pulled, auditors conduct a walkthrough of each significant process — access provisioning, change management, incident response, vendor onboarding, backup and recovery. A walkthrough is not itself the test; it's the reconnaissance that determines what the test plan will look like.
Walkthrough Component | Purpose |
|---|---|
Process narrative from the control owner | Establish who does what, in what order, and why |
Identification of control points | Pinpoint exactly where a preventive or detective action occurs |
System and tool identification | Confirm which platforms generate the evidence auditors will later request |
Population source confirmation | Determine where the complete list of control instances lives (ticketing system, IAM platform, HRIS) |
Exception-handling discussion | Understand what happens when the control doesn't work as designed |
Segregation of duties check | Confirm no single person can both execute and approve a sensitive action |
A well-run walkthrough tells the auditor exactly which of the four test procedures will be efficient for each control, and it's where auditors first flag segregation of duties gaps — for example, discovering that the same engineer who deploys code can also approve their own pull request, which changes the entire testing strategy for that control.
Control Frequency and Why It Drives Everything
How often a control operates determines how many instances exist in the population for the audit period, and population size is the single biggest input into sample size. A control that runs continuously (like a firewall rule set or an encryption configuration) is tested differently from a control that runs annually (like a disaster recovery tabletop exercise).
Frequency Category | Typical Examples | Approximate Population Size (12-month period) |
|---|---|---|
Annual | Risk assessment, DR test, penetration test, policy review | 1 |
Quarterly | Access reviews, vendor risk reassessment | 4 |
Monthly | Vulnerability scan review, patch compliance report | 12 |
Weekly | Backup verification, log review meeting | 52 |
Daily | Automated backup jobs, malware scan completions | 260–365 |
Continuous / on-demand | Access provisioning, deployments, terminations | Varies (often 50–500+) |
Sample Size Guidance by Frequency
Because a 100% test of every instance would make audits prohibitively expensive and slow, auditors rely on audit sampling — testing a representative subset of the population and extrapolating a conclusion to the whole. AICPA-aligned firms generally follow frequency-based sample size conventions that scale with how often a control runs, since a higher-frequency control offers more opportunities for something to go wrong and therefore needs a larger sample to support a conclusion with reasonable assurance.
Control Frequency | Population Size | Typical Sample Size | Notes |
|---|---|---|---|
Annual | 1 | 1 (test of one) | Only one instance exists in the period |
Quarterly | 4 | 2 | Roughly half the population |
Monthly | 12 | 2–4 | Scales with auditor risk judgment |
Weekly | 52 | 5–8 | Covers different times of year |
Daily | 260–365 | 25–45 | Large enough to detect systemic issues |
Continuous/on-demand (high volume) | 100+ | 25–60 | Often risk-stratified rather than purely random |
These figures are illustrative conventions, not a rigid formula — a firm's methodology can flex the sample size upward if the control environment is weak, if prior-period testing surfaced deviations, or if the control protects especially sensitive data. Conversely, a strong control environment and clean testing history rarely justifies a smaller sample than the convention; auditors are far more likely to expand a sample than shrink one below guidance.
"New clients always ask if they can negotiate the sample size down. You can't negotiate math. What you can influence is how clean and complete your population list is, because a messy population is the single biggest reason a sample balloons mid-engagement." — Tomas Reyes, IT Audit Senior, Kestrel & Boyce LLP
How Auditors Select Samples
Once the sample size is set, the auditor still has to decide which items from the population to pull. Three approaches dominate SOC 2 practice, and auditors often blend them within a single test.
Sampling Method | How It Works | When Auditors Use It |
|---|---|---|
Random sampling | Every item in the population has an equal chance of selection, often via a random-number generator against a numbered list | Default for most medium-to-large populations |
Haphazard sampling | Auditor selects items without conscious bias, spreading picks across the period (e.g., one from each month) | Useful for smaller populations or when true randomization tools aren't practical |
Judgmental (risk-based) sampling | Auditor deliberately selects higher-risk items — a privileged account, a change deployed on a weekend, a large financial transaction | Applied on top of random samples for high-risk controls |
Auditors generally avoid letting the client pick which items get tested, since a self-selected sample undermines independence — instead, they request the complete, timestamped population and make the selection themselves, often before the walkthrough concludes.
Population Completeness: Testing the List Before Testing the Sample
One of the most consequential and least understood steps in control testing happens before a single sample item is examined: the auditor has to satisfy themselves that the population they're sampling from is complete and accurate. If Brightloom hands over a spreadsheet of "all access changes for the period" that was manually maintained and missing entries, any sample pulled from it is meaningless — the auditor would be testing a subset of a subset, with no assurance the excluded items weren't the problematic ones.
Auditors typically corroborate a population's completeness by reconciling it against an independent system-generated source: comparing a manually tracked change log against the ticketing system's own export, or cross-referencing an HR-provided termination list against the identity provider's deprovisioning audit trail. This is why system-generated populations — pulled directly from Jira, ServiceNow, an IAM platform, or a CI/CD pipeline — are dramatically preferred over hand-maintained spreadsheets, and why organizations that automate their evidence pipelines tend to sail through this step while manual-tracking shops lose days reconciling gaps.
"I've seen engagements where the control itself was flawless and the entire delay came from proving the population was complete. Give me a system export with a timestamp and a row count I can reconcile, not a spreadsheet someone updated by hand." — Renata Ilic, Partner, Ashgrove Assurance Partners
Testing Procedures by Control Type
The four test procedures apply differently depending on what kind of control is being examined. Below is how testing typically plays out across the control domains most SOC 2 engagements cover.
Access Control Testing
Access-related controls draw some of the heaviest sampling in any SOC 2 engagement because they touch least privilege, role-based access control, and provisioning/deprovisioning — all high-risk areas tied directly to the Common Criteria.
Control Tested | Primary Method(s) | Evidence Auditor Requests |
|---|---|---|
New user provisioning | Inspection | Access request ticket, manager approval, provisioning timestamp |
Access termination | Inspection, re-performance | HR termination date vs. deprovisioning timestamp, system logs |
Quarterly access recertification | Inspection | Reviewer sign-off, evidence of access removed for flagged accounts |
Privileged/admin account inventory | Inspection, re-performance | Current admin list cross-checked against approved role list |
Multi-factor authentication enforcement | Inspection, observation | System configuration screenshot, policy enforcement report |
Change Management Testing
Change management controls (CC8 in the Common Criteria) get tested against the full software development lifecycle — from code review through deployment approval.
Control Tested | Primary Method(s) | Evidence Auditor Requests |
|---|---|---|
Peer code review before merge | Inspection | Pull request approval record with reviewer identity distinct from author |
Change approval before deployment | Inspection | Change ticket with documented sign-off predating deploy timestamp |
Testing performed prior to release | Inspection | Test execution logs or QA sign-off attached to the change record |
Emergency change process | Inquiry, inspection | Sample of emergency changes, retroactive approval evidence |
Segregation between dev and production access | Re-performance | Independent verification that deploy permissions exclude authors who lack approval rights |
Logical and Physical Security Testing
Control Tested | Primary Method(s) | Evidence Auditor Requests |
|---|---|---|
Firewall/network rule configuration | Inspection, re-performance | Current rule set export, comparison against approved baseline |
Data center physical access | Observation, inspection | Badge access logs, visitor logs, site walkthrough or camera review |
Endpoint encryption | Inspection | MDM console report showing device-level encryption status |
Vulnerability scanning cadence | Inspection | Scan schedule configuration, scan completion reports for sampled periods |
Availability, Backup, and Continuity Testing
Control Tested | Primary Method(s) | Evidence Auditor Requests |
|---|---|---|
Backup job completion | Inspection | Backup job success/failure logs for sampled dates |
Backup restoration testing | Inspection, observation | Restoration test results, evidence of a defined test cadence |
Uptime/availability monitoring | Inspection | Monitoring dashboard exports, incident tickets tied to downtime |
Disaster recovery exercise | Inquiry, inspection | Tabletop exercise documentation, after-action report |
Risk Assessment and Monitoring Testing (CC3/CC4)
Control Tested | Primary Method(s) | Evidence Auditor Requests |
|---|---|---|
Annual risk assessment | Inspection | Completed risk register, evidence of management review and sign-off |
Ongoing risk monitoring | Inquiry, inspection | Meeting minutes, updated risk scores, remediation tracking |
Fraud risk consideration | Inquiry, inspection | Documentation showing fraud risk was explicitly considered in the assessment |
Vendor and Third-Party Risk Testing
Control Tested | Primary Method(s) | Evidence Auditor Requests |
|---|---|---|
Vendor security review before onboarding | Inspection | Vendor questionnaire, SOC 2 report review, contract security clauses |
Annual vendor reassessment | Inspection | Sample of reassessment records for critical vendors |
Subservice organization monitoring | Inquiry, inspection | Evidence the organization reviewed the subservice organization's own SOC report |
Evidence Auditors Expect
The test procedure chosen determines the shape of evidence an auditor will accept — this is worth reading alongside SOC 2 Evidence Collection: Documentation and Testing Requirements, which covers the full evidence lifecycle. In short, the stronger the procedure, the more objective and system-generated the underlying evidence needs to be.
Test Procedure | Acceptable Evidence Examples | Weak / Unacceptable Evidence |
|---|---|---|
Inquiry | Meeting notes corroborated by another test | Uncorroborated verbal claims alone |
Observation | Screen-share recording, live demo notes, timestamped photos | Auditor's own memory without contemporaneous notes |
Inspection | System-exported logs, signed tickets, configuration screenshots with timestamps | Manually re-typed summaries of what a log "would have shown" |
Re-performance | Independently recalculated result matching system output | Client-provided calculation the auditor didn't verify independently |
Every artifact submitted needs a clear audit trail: who performed the action, when, and what system generated the record. Items on the PBC list — "provided by client" — that arrive without timestamps or without a clear source system are the single most common cause of extended fieldwork timelines.
From Deviation to Exception: How a Single Miss Becomes a Finding
When a sample item doesn't match what the control was supposed to do, auditors call that a deviation, not automatically an exception — the distinction matters enormously and is explored in full in SOC 2 Exception Management: Handling Control Deficiencies. Here's the mechanical path a deviation follows.
Step | What Happens |
|---|---|
1. Deviation identified | A sampled item fails the test (e.g., a change ticket missing approval) |
2. Root cause requested | Auditor asks the control owner to explain why the deviation occurred |
3. Sample expansion (sometimes) | Auditor may pull additional items to determine if the issue is isolated or systemic |
4. Deviation rate calculated | Number of failed items divided by sample size |
5. Materiality judgment | Auditor weighs deviation rate, root cause, and compensating controls |
6. Exception documented (if material) | Deviation is written into the report as an audit exception affecting the opinion |
A single deviation in a sample of two (a 50% deviation rate on a quarterly control) is treated very differently from one deviation in a sample of 45 (roughly 2%) — but rate alone doesn't decide the outcome. A compensating control that caught the failure downstream, or a root cause showing the deviation was a one-time documented anomaly rather than a process breakdown, can keep a deviation from becoming a reportable exception. Auditors document this reasoning explicitly; "we found it and decided it didn't matter" is never acceptable without a written rationale.
Deviation Rate | Common Auditor Response |
|---|---|
Isolated, well-explained, sample of 1 (annual control) | Likely still an exception — no room to average it out |
1 of 25 (4%), clear root cause, no compensating gap | Often noted, sometimes elevated to exception depending on control criticality |
3+ of 25 (12%+) | Near-certain exception; may trigger population-wide review |
Any deviation on a high-risk security control (e.g., a former employee retaining access for weeks) | Almost always an exception regardless of rate |
Case Study: Brightloom Systems — Turning a Messy Population Into a Clean Type II
Back at Brightloom, Dana's team pulled its access change population directly from the identity provider rather than the manually maintained spreadsheet it had used the previous cycle. Owen's team selected a sample of eight items from a population of 61 access changes over the nine-month review period. Inspection of the sample turned up one deviation: a contractor's access had been granted correctly but the manager-approval ticket was logged four days after the access was already active. Dana's team produced the root cause immediately — a ticketing automation bug that had since been fixed — and showed the compensating control (a quarterly access review) had already flagged and remediated the same account before the auditor even found it. The item was documented as a deviation but did not rise to a reportable exception, because the compensating control demonstrated the broader control environment caught what the primary control missed. Brightloom's Type II report came back unqualified, the $3.1 million renewal closed on schedule, and the fixed automation bug eliminated the entire category of deviation before the next audit cycle.
Case Study: Fernwood Analytics — When a Deviation Rate Becomes an Exception
Fernwood Analytics, a data analytics SaaS provider, wasn't as fortunate on its change management testing. Auditors selected 25 deployment tickets from a population of roughly 340 production changes over the 12-month period. Three tickets — a 12% deviation rate — showed deployments that occurred before the documented peer-review approval was logged, all tied to a single engineer who had been bypassing the pull-request gate during an incident-response weekend. VP of Engineering Marcus Webb's team could not produce a compensating control that would have caught unauthorized deployments in real time. The auditor documented a reportable exception for the change management criterion. Fernwood's report shipped with a qualified opinion language for that specific control, the engagement expanded by roughly three weeks as the auditor tested an additional 15 items to determine whether the issue was isolated, and the extra testing and remediation review added close to $45,000 in incremental audit fees. Fernwood closed the gap in its next cycle by adding a branch-protection rule that made bypassing peer review technically impossible rather than just procedurally discouraged — a control design fix, not just a training fix.
"The fix that actually works isn't 'remind engineers to follow the process.' It's removing the technical ability to skip it. Auditors trust a control they can't route around far more than a control that depends on someone remembering a rule." — Marcus Webb, VP of Engineering, Fernwood Analytics
Case Study: Lumenwell Health — Fixing Access Review Evidence Gaps Before They Cost a Deal
Lumenwell Health, a virtual care platform, had a real quarterly access review process but a chronically incomplete evidence trail — reviewers approved access in a spreadsheet, but roughly 30% of quarterly cycles were missing a documented reviewer sign-off for at least one system. Director of Security Priya Nandakumar's team knew this would draw scrutiny before their next Type II engagement began, since access recertification evidence is inspected on nearly every SOC 2 engagement. Ahead of fieldwork, Lumenwell moved access certification into its identity governance platform, which timestamps every reviewer decision automatically and cannot be closed out with a missing item. Across the subsequent nine-month observation period, the auditor sampled six of Lumenwell's four quarterly reviews and one interim ad hoc review, found zero deviations, and the report came back clean. Priya credits the fix with directly preserving a healthcare enterprise deal worth roughly $500,000 in first-year contract value that had a signed security addendum requiring an unqualified SOC 2 Type II.
"We didn't need a better policy. We needed a system that made it physically impossible to finish a review with a blank field. That one change is the reason our access testing went from our worst section to our fastest." — Priya Nandakumar, Director of Security, Lumenwell Health
Visualizing the Testing Workflow
flowchart TD
A[Planning: Map controls to TSC] --> B[Walkthrough: understand process & control points]
B --> C[Request complete population from client]
C --> D[Reconcile population for completeness]
D --> E[Select sample: random / haphazard / judgmental]
E --> F[Apply test procedure: inquiry, observation, inspection, re-performance]
F --> G{Deviation found?}
G -- No --> H[Document test as passed]
G -- Yes --> I[Request root cause & compensating controls]
I --> J{Material to opinion?}
J -- No --> H
J -- Yes --> K[Document as reportable exception]
K --> L[Draft description of tests and results]
H --> L
L --> M[Report finalized: unqualified / qualified opinion]This is the loop that repeats for every control in scope — often dozens of times in parallel across an engagement — which is why fieldwork can feel chaotic from the client side even though each individual test follows this same disciplined sequence.
Common Mistakes Companies Make During Fieldwork
Mistake | Why It Backfires | Better Approach |
|---|---|---|
Hand-picking "good" examples for the auditor | Undermines sample independence; auditors select from the full population, not curated highlights | Provide the complete, system-generated population and let the auditor sample |
Treating inquiry answers as sufficient | Auditor will always corroborate with inspection or re-performance | Have documentary evidence ready before the conversation happens |
Maintaining populations in spreadsheets | Manual lists are hard to reconcile and often incomplete | Pull populations directly from source systems (ticketing, IAM, CI/CD) |
Explaining a deviation after the fact instead of documenting root cause in real time | Retroactive explanations read as less credible | Log root cause and remediation the moment an anomaly is caught internally |
Assuming a low deviation rate is automatically fine | Materiality depends on control criticality, not just rate | Understand which controls carry zero tolerance regardless of sample size |
Waiting until fieldwork to gather evidence | Creates evidence scrambles and increases deviation risk | Build continuous evidence collection into BAU operations |
How to Prepare for Control Testing
Preparation Step | Owner | Timing |
|---|---|---|
Confirm each control has a system-generated population source | Control owner + compliance | 60–90 days before fieldwork |
Run an internal control self-assessment against likely sample items | Compliance team | 30–60 days before fieldwork |
Reconcile manual lists against source systems | Control owner | 30 days before fieldwork |
Pre-stage evidence for high-frequency controls (access, change, backups) | Control owner | Ongoing, continuous |
Brief control owners on inquiry technique (concise, factual, evidence-first answers) | Compliance lead | 1–2 weeks before fieldwork |
Identify compensating controls for known weak spots | Compliance + control owner | Before walkthroughs |
Designate a single point of contact for PBC list coordination | Compliance lead | Start of engagement |
Remote and Screen-Share Testing Considerations
Most SOC 2 fieldwork today happens remotely, which changes how observation and inspection get executed in practice. Screen-shares replace in-person walkthroughs for most controls, and auditors increasingly request recorded sessions or timestamped screenshots as a durable substitute for a live observation they can't personally witness again later. This shifts more weight toward inspection-friendly evidence — system exports, signed records, configuration screenshots with visible timestamps — because a remote observation, unlike an in-person one, produces no independent physical memory the auditor can rely on later if the recording is lost or unclear. Organizations that build screen-recording and structured evidence capture into their remote fieldwork process consistently move faster than those improvising live demos on request.
"Remote testing is efficient until the recording fails or the screen-share cuts a corner off the log timestamp. I ask every client now: assume I'll need to review this evidence six months from now without you in the room. Does the artifact stand on its own?" — Tomas Reyes, IT Audit Senior, Kestrel & Boyce LLP
What "Good" Looks Like: Auditor Expectations at a Glance
Expectation | What It Looks Like in Practice |
|---|---|
Complete, source-system populations | No manually curated lists; exports come directly from the system of record |
Fast evidence turnaround | PBC requests fulfilled within 1–3 business days |
Consistent control owners | The same person who described the process in the walkthrough can produce evidence for it |
Root cause fluency | Control owners can explain why a deviation happened, not just that it happened |
Documented compensating controls | Weak spots have a second line of defense that's already evidenced, not improvised |
No sample gaming | Client never attempts to influence which items get tested |
Proactive disclosure | Known issues are raised by the client before the auditor finds them independently |
Control Testing as a Business Opportunity
It's tempting to treat control testing purely as a compliance cost center — something to survive rather than something to leverage. That framing undersells what strong testing outcomes actually buy you. A Brightloom-style clean Type II with zero material exceptions isn't just a report; it's a sales asset that shortens security review cycles with every prospect who asks for it, a credibility signal that reduces the friction of custom security questionnaires, and — increasingly — a prerequisite that gates access to entire categories of enterprise and regulated-industry customers. Organizations that treat testing readiness as a year-round engineering discipline rather than a pre-audit scramble consistently report shorter fieldwork windows, lower audit fees (fewer expanded samples, fewer remediation cycles), and fewer surprises in the deviation-to-exception pipeline described above. The companies that struggle are almost always the ones that built their evidence trail retroactively, days before the auditor asked for it, rather than as a byproduct of how they already operate. Investing in system-generated populations, automated evidence capture, and clearly owned controls pays for itself many times over across a company's audit lifetime — not just in audit fees saved, but in deals that close on schedule instead of stalling on a security review.
Get Ahead of Your Next Testing Cycle
If you're heading into fieldwork and want to know exactly where your evidence trail has gaps before your auditor finds them, PentesterWorld's SOC 2 Control Matrix / RACI Template maps every Common Criteria control to a named owner and expected evidence type, and the SOC 2 Readiness Checklist walks through the exact preparation steps outlined above. If you're newer to the framework, our SOC 2 Report Reader's Guide breaks down how to read a description of tests and results section so you know what "good" testing outcomes look like on paper, and the SOC 2 Glossary keeps every term in this article — deviation rate, re-performance, population — one click away while you prepare your team.