Security

Real offensive depth

Testing and defence led by a published security researcher with five CVEs, including a CVSS 9.1 Mirai botnet kill-switch.

All security →
Compliance

Audit-ready, fixed scope

SOC 2, ISO, and the Canadian privacy stack, run end to end with an independent auditor.

All frameworks →
Resources

Learn the space

Original research, free tools, and plain-language guides on security and compliance, from a published security researcher.

Read the blog →
Compliance

What Your SOC 2 Auditor Actually Checks During Fieldwork

Most SOC 2 explanations stop at the framework. None of that helps on the Tuesday morning when a senior associate from the licensed CPA firm joins a call and asks your infrastructure lead to walk through how a production change gets approved.

Fieldwork is where the report is decided. Everything before it is preparation and everything after it is formatting. This is what happens in that window, from the client side of the table.

What happens between kickoff and report

Kickoff. Usually one hour. The audit team confirms scope, system description boundaries, the criteria in play, the window dates, and who the control owners are. Leave with names against controls or the rest of the engagement is spent chasing.

System description review. Management writes it; the auditor comes back on boundaries, meaning which subservice organizations are carved out, which are inclusive, and what the complementary user entity controls are. This is where quiet scope creep happens: a sentence saying the platform "monitors customer environments continuously" creates a control the auditor will now test.

Evidence request. The list arrives. More on this below.

Walkthroughs, then sample selection and testing. The auditor watches controls operate live, then picks items from populations you provide and tests each against the control description.

Findings discussion. Anything that did not pass gets raised, usually informally first. This is the most important conversation in the engagement and most companies treat it as a status update.

Draft, then final report. Management reviews, writes any responses, confirms the system description, and the report is signed.

On a well-prepared Type II, fieldwork is typically three to six weeks of calendar time and far less actual effort. On an unprepared one it runs months, because every request loops.

The evidence request list, and how it actually arrives

The evidence request list, sometimes called the PBC list (prepared by client), is a spreadsheet delivered as an attachment or inside the audit firm's portal. Each row is one numbered request with a description, the control it maps to, a due date, and a status column the auditor updates.

The Ontario medtech engagement we published, a SOC 2 Type I for a company putting an AI clinical assistant in front of practitioners, had 84 items on that list, which is normal for a Type I at a small company. A Type II runs longer because population requests multiply.

The requests are not written for you. They are written in audit language against the criteria. "Provide evidence of the operating effectiveness of the logical access provisioning control for the period" is a real sentence you will read, and somebody on your side translates it into "export the service desk ticket list for the window and pull the approval field."

Follow-ups are where the real work is. Each item carries a status: open, received, under review, follow-up, closed. Items sitting in follow-up become findings, so watch that column more closely than the open one. First submissions are rarely the last. The auditor opens your screenshot, sees no timestamp, and asks for one with the system clock visible. Budget two rounds on roughly a third of the list.

Screenshots are weak evidence and system exports are strong. A screenshot proves a state at an unknown moment unless it carries a date and a system identifier. An export from the source system, with the query or filter visible, proves a population. Name every file with its request number.

Build the index before the list arrives. On the medtech Type I, the largest single time saving was building the evidence index before the list arrived, so that when request 41 asked for the access review, the answer was a file path rather than a Slack thread.

How sampling works, and why you must never pick the sample

Step one: the auditor asks for the population, not the evidence. For change management the request is not "send us ten change tickets." It is "send the complete list of production changes deployed in the window, with ticket ID, date, requester, and approver." For access provisioning it is every account created in the window; for terminations, every person who left.

Step two: the auditor tests the population for completeness, tying your list back to an independent source. Terminations get compared against payroll or the HR system, production changes against deployment logs or repository history. If the population is missing rows, nothing after it matters, because a sample drawn from an incomplete population tests nothing.

Step three: the auditor selects the sample, from their side, using their own method, and tells you which items they picked.

Step four: you produce evidence for exactly those items, not similar ones and not better ones.

Sample sizes follow frequency of operation, and working conventions land in a similar place: daily controls in the range of twenty-five to forty items, weekly five to fifteen, monthly two to five, quarterly two, annual one. The auditor's own methodology sets the final number, so ask for theirs at kickoff rather than assuming.

You must not select the sample, suggest items, or filter the population before sending it.

The auditor's ability to conclude anything rests on independent selection from a complete population. Hand over a curated list, or say "those three were unusual, use these instead," and you have removed the basis for the conclusion. A careful team notices, expands testing, and may treat the population as unreliable. One that does not issues a report that does not mean what it says, which is worse for you, because that report carries your name to your customers.

When you know a sampled item will fail, send the evidence anyway and flag it in the same message: "Item 14's approval was verbal on an incident call and the ticket was updated after the deploy. Here is the incident record." Auditors form a view of management integrity during fieldwork, and that view shapes how much extra testing they feel they need.

Walkthroughs and control owner interviews

A walkthrough is a live session where the auditor watches the control operate rather than reading about it. Thirty to sixty minutes per control area, screen sharing, control owner driving. The auditor is establishing that the control as described matches the control as it exists, that the person operating it understands it, and that the evidence it produces is reliable.

The structure is usually describe the process, show me the system, take one real item end to end, show me what happens when it fails. That last one is where people are unprepared. "What happens if someone deploys without approval?" is not a trick; the auditor is testing whether the control has teeth or is only a policy sentence.

On the zero-exceptions Type II we published, the client ran a multi-layer change approval flow across 76 controls with a team of fifteen. Walkthroughs went quickly there, not because the controls were elaborate, but because the control owner could open the system and show the approval gate blocking a merge in real time. Demonstration beats description every time.

What a bad answer sounds like

Bad answers are rarely lies. They are vagueness, and vagueness triggers expanded testing.

"We review access quarterly." That is a policy statement, not a control description. The auditor needs who reviews, against what source list, what the review produces, what happens to a flagged account, and how you know it happened in March rather than June.

"That's handled automatically." By what? Configured by whom? How would you know if it stopped? A control nobody monitors is one the auditor cannot rely on.

"Let me get back to you" is fine and far better than guessing. Guessing produces a statement in the workpapers that your evidence later contradicts, and contradictions generate more testing, not less.

Good answers are specific and tied to both a system and an artifact. "I run it the first Monday of the quarter. I pull the user list from the identity provider, compare it against the HR roster export, and log the review as a ticket. Here is April's, and the two accounts it caught." Thirty seconds, and the auditor has what they need.

Where exceptions actually originate

Across engagements, exceptions cluster in a few places and are almost never exotic.

Access reviews not performed inside the window. The control says quarterly and the window is twelve months. You performed three, or four with one done the week before fieldwork covering a period already elapsed. A quarterly control needs four occurrences spread across the year, evidenced with dates.

Offboarding evidence missing a date. The account is disabled and nobody disputes that. The evidence has to show it was disabled within the period your policy commits to, which means the HR termination date and the system deactivation timestamp side by side for each sampled leaver. A screenshot of a disabled account today says nothing about when it happened.

Change tickets without approval. The deploy happened, the ticket exists, the approval field is empty or the approver is the requester. Self-approval is the most common single finding in change management, usually from emergency changes that never got retroactive sign-off.

Backup restore never tested. Everybody has backup jobs. The control says restoration is tested at a stated frequency, and the test is the part that does not happen. There is no remediating this once the window closes.

Vendor reviews not done. Your policy commits to annual review of critical vendors. The auditor asks for the vendor list, picks four, and asks for review evidence. Two have a SOC 2 report on file that nobody read and no record of assessment.

Notice the pattern. Almost none of these are failures of security. They are failures of evidence and cadence: the control worked, but nothing recorded it, or it ran at the wrong time.

Deviation, exception, qualified opinion: three different words

These get used interchangeably and are not the same thing.

A deviation is a single sampled item that did not operate as described: one change without approval, one leaver disabled on day nine against a five-day policy.

An exception is the auditor's conclusion, after evaluating the deviations for a control, that the control did not operate effectively throughout the period. One deviation in a sample of forty may be judged isolated, with testing expanded to confirm. Four in a sample of five is an exception with nothing to discuss.

A qualified opinion is what the licensed CPA firm expresses on the report as a whole when exceptions are significant enough that they cannot say the controls operated effectively to achieve the applicable criteria.

The distinction that matters commercially: a report can describe an exception and still carry an unqualified opinion, depending on the auditor's evaluation of severity and compensating controls. Customers see the testing results section either way, and what they react to hardest is a qualification in the opinion paragraph. The zero-exceptions Type II we published means no control in the tested set produced an exception in the results section, which is why we describe it that way rather than saying the audit was passed.

Management response: what it is and when you write one

When the report contains an exception, management may include a response. The auditor's description stays as written; your response sits beside it, labelled as management's.

Write one when there is something true and useful to say: what caused it, what was done, when the fix took effect. "The Q2 access review was completed in July rather than June following a change in control ownership. Ownership was reassigned and Q3 and Q4 were completed on schedule."

Do not write one that argues with the finding, minimizes it, or reads as legal positioning. Every customer's security reviewer reads the responses, and a defensive one draws more attention to the exception than the exception does alone. Where there is nothing useful to add, no response is a legitimate choice.

The observation window is the whole point

A Type I is a point in time. A Type II covers a period, commonly three, six or twelve months, and the auditor tests operation throughout it.

Throughout is the operative word. Evidence has to exist across the whole window, created when the control operated, not assembled afterwards. If your window ran January to December and all four access reviews were performed in the second week of the following January, zero of them fall inside the window. If your quarterly scans all carry timestamps from the same three days, the auditor sees that too. Evidence carries its own dates, and the dates are the finding.

The fix is unglamorous. Pick the window start date deliberately instead of defaulting to the contract signature, and start it only once controls are genuinely operating, because a window that opens too early turns your learning period into tested months. Put every periodic control on a calendar with a named owner and a deadline inside the period. File evidence when it is produced. On a Type II, the audit is decided by what your team did in month two, not the last fortnight.

Questions worth asking before fieldwork starts

Ask the licensed CPA firm during selection rather than after the engagement letter is signed: what sample sizes does your methodology use by control frequency; how do you test population completeness, and against what source; when are findings communicated; who signs the report, and can we speak to a reference.

Ask any readiness partner: have you completed engagements in Canada you can show; is remediation priced before or after the gap assessment; which entity signs the contract and under which province's law; where does our engagement data live. Remediation priced before anybody has looked at your environment is a number invented before the gaps are known. Our own work runs in two phases for that reason: a gap assessment that sets scope and produces a findings register, then remediation priced from those findings.

What to do next

If you are heading into fieldwork, the useful preparation is not more policy writing. It is three things: name a control owner for every control, build the evidence index before the request list arrives, and check every periodic control for whether it operated inside the window with a dated artifact to show for it.

Two published engagements cover this in sequence. The SOC 2 Type I for an Ontario medtech company running an AI clinical assistant walks the gap analysis and the 84-item evidence request: /blog/soc-2-type-i-medtech-ai-case-study. The venture-backed startup that reached a SOC 2 Type II with zero exceptions across 76 controls covers the control set and change approval flow: /blog/zero-to-soc2-type-ii-case-study.

For a read on where your evidence would fail before an audit team finds it, that is what a gap assessment is for. TrazTech is the readiness partner; the audit is done by an independent licensed CPA firm. Start at traztech.ca/contact, or getsoc2.ca if you are still deciding which framework comes first.

Before you go

Want the rest of this by email?

If this was useful, I send a few short notes on SOC 2 and compliance. Unsubscribe in one click, and replies reach me directly.

From Jacob Masse, principal of traztech. No spam, unsubscribe in one click.

Want a second opinion on where you stand?

We run SOC 2, ISO 27001 and the rest of the compliance stack for startups and SMEs, and the security testing that sits behind it. The first call is free, and we will tell you if you are not ready to start yet.

Book a free call

Track record

Who is actually doing the work

5
Published CVEs, including a CVSS 9.1
76
Controls taken from nothing to a passed SOC 2 Type II
Zero
Exceptions on that Type II report
20+
Penetration testing engagements delivered

Published vulnerability research

Five published CVEs. CVE-2024-45163 (CVSS 9.1) is a flaw in the Mirai botnet itself, which gave defenders a way to shut down attacker infrastructure. CVE-2026-42626 takes HP ENVY 5000 printers offline from any unauthenticated device on the same network.

A SOC 2 Type II built from nothing

At Humera, a venture-backed US security company, Jacob built the compliance programme in-house from nothing: no report, no policies, no documented controls. It ended in a Type II attestation across 76 controls with zero exceptions, on a team of 15.