Security

Real offensive depth

Testing and defence led by a published security researcher with five CVEs, including a CVSS 9.1 Mirai botnet kill-switch.

All security →
Compliance

Audit-ready, fixed scope

SOC 2, ISO, and the Canadian privacy stack, run end to end with an independent auditor.

All frameworks →
Resources

Learn the space

Original research, free tools, and plain-language guides on security and compliance, from a published security researcher.

Read the blog →
Security

Penetration Test vs Vulnerability Scan: What Is the Difference?

If you have ever filled out a security questionnaire or scoped work for a SOC 2 report, you have run into this question: do we need a penetration test, or is a vulnerability scan enough? The two terms get used interchangeably in RFPs and sales calls, but they are not the same service, they do not cost the same, and they do not tell you the same thing about your risk. Buying the wrong one wastes budget. Skipping the right one leaves a real gap in your security posture.

Vulnerability scanning: automated, broad, continuous

A vulnerability scan is an automated process. Tooling sweeps your network, applications, or cloud environment, compares what it finds against a database of known vulnerabilities (CVEs, misconfigurations, outdated software versions), and produces a report ranked by severity. It is fast, it is repeatable, and it is cheap relative to the alternative.

Scanning is good at what it is built for: finding unpatched software, expired certificates, open ports that should be closed, weak TLS configurations, and known CVEs across a large surface area. Because it is automated, you can run it weekly, monthly, or continuously, which matters because your environment changes constantly. A new server gets spun up, a dependency gets bumped, a firewall rule gets loosened for a deploy and never gets tightened back. A scan catches that drift.

What a scan cannot do is think. It does not understand your business logic, it cannot chain three low-severity findings into one critical exploit path, and it will flag things that are not actually exploitable in your specific environment right alongside things that are. That noise is the tradeoff for speed and cost.

Penetration testing: human-led, targeted, adversarial

A penetration test is a person, or a small team, actively trying to break into your systems the way a real attacker would. A pen tester uses scanning tools as a starting point, but the value is in what happens after the scan finishes: manual exploitation, privilege escalation, lateral movement, and testing business logic that no automated tool understands. Can a standard user manipulate an API call to see another customer's data? Can a low-privilege account be chained through three separate misconfigurations to reach admin access? A scanner will not find that. A human tester, working through your application the way an attacker would, might.

Penetration tests are also the only way to validate that a vulnerability is actually exploitable, not just theoretically present. That distinction matters a lot when you are trying to prioritize remediation with a small engineering team and a finite amount of time. It also matters to auditors and enterprise security teams reviewing your questionnaire responses. "We ran a scan" and "we had a licensed tester confirm exploitability and provide a report with proof of concept" read very differently to a buyer's security team.

Need the testing done? Penetration testing and vulnerability management, with the retest that proves a finding is actually closed. Penetration testing

When you need each one

Use a vulnerability scan when you need broad, frequent, low-cost visibility into your attack surface. This is table stakes for ongoing hygiene: new assets, new dependencies, and configuration drift all need to be caught continuously, not once a year. Scanning is also usually a baseline requirement embedded inside broader compliance frameworks, even when the framework's language talks about penetration testing separately.

Use a penetration test when you need a point-in-time, defensible assessment of whether your defenses actually hold up against a skilled attacker, and when a third party (a customer, an auditor, an investor, a cyber insurance underwriter) needs evidence of that. SOC 2 audits, enterprise vendor security reviews, and cyber insurance renewals frequently require a pen test report specifically, not a scan output. Substituting one for the other in that context is a common and costly mistake we see teams make when they are trying to move fast and assume the terms are interchangeable.

Why most teams need both

The honest answer for most growing companies is that vulnerability scanning and penetration testing are not competing options, they are complementary layers. Scanning gives you continuous, low-friction coverage of the known and the obvious. Penetration testing gives you a periodic, deep validation of what a determined attacker could actually do, including the things scanners structurally cannot see. Relying on scanning alone means you are blind to exploit chains and business logic flaws until someone finds them for you, and that someone is not always on your payroll. Relying on penetration testing alone, run once a year, means the six months of configuration drift between engagements go unmonitored.

This is the logic behind treating vulnerability management as an ongoing program rather than a one-off purchase. Our vulnerability management service is built around that continuous layer, scanning your environment on a regular cadence, triaging findings by real-world exploitability rather than raw CVSS score, and tracking remediation so nothing sits open for months. Penetration testing then sits on top of that as the periodic, human-led check that validates whether the gaps that matter are actually closed.

How to decide what to buy first

If you have neither today, start with a vulnerability scan. It is faster to stand up, gives you an immediate baseline, and surfaces the low-hanging fruit that would otherwise pad out a penetration test report with findings a scanner could have caught for a fraction of the cost. Once that baseline is under control, layer in a penetration test, particularly if you are heading into a SOC 2 audit, responding to a vendor security questionnaire, or preparing for a cyber insurance renewal that specifically asks for one. If you are further along and building out a broader program, it is worth looking at how vulnerability management fits into your overall compliance posture rather than treating each control as an isolated purchase.

The one thing to avoid is treating either service as a checkbox. A scan report nobody reads or a pen test finding nobody remediates does not reduce risk, it just produces a PDF. The value is in what happens after the report lands: triage, prioritization, and fixing what actually matters.

Get a straight answer for your environment

If you are not sure whether your next step is a vulnerability scan, a penetration test, or both, we can walk through your environment and your compliance timeline and tell you plainly what you need and what you do not. Contact traztech to talk it through.

Scanning is not one thing, and the differences matter

People say "we run scans" as if it describes a single control. Buyers and auditors ask which kind, because the coverage varies enormously.

Unauthenticated network scanning looks at your systems the way an anonymous internet user would: open ports, service banners, TLS configuration, known CVEs inferred from version strings. It is the cheapest form and the least accurate, because banner-based detection produces both false positives and false negatives at a rate that surprises people the first time they check.

Authenticated or agent-based scanning logs in, or runs on the host, and reads the actual installed package versions. Coverage and accuracy jump, and so does the finding count, which is why teams who switch from unauthenticated to authenticated scanning often think their posture got worse. It did not; the measurement got honest.

Software composition analysis reads your dependency manifests and tells you which libraries carry known CVEs. This is where most modern findings actually originate, and it is the one class of scanning that belongs in the build pipeline rather than on a weekly schedule.

Cloud posture scanning reads your AWS, Azure, or GCP configuration and flags public buckets, over-permissive IAM roles, unencrypted volumes, and missing logging. Nothing else on this list will find those, and for a cloud-native SaaS company this class of finding is frequently the highest actual risk.

ASV scanning is a specific, PCI DSS-mandated external scan run by an Approved Scanning Vendor, quarterly, with a passing scan required. It is a compliance artefact with a defined format and cannot be substituted with your own tooling output. If you are in scope for PCI DSS, budget for it separately; see PCI DSS for SaaS for how scope reduction changes what you owe here.

A team that runs authenticated host scanning, dependency analysis, and cloud posture checks has a genuinely useful programme. A team running only unauthenticated external scans has a thin one, and any competent auditor will spot the difference in about two questions.

What a penetration test report has to contain to be usable

Most disputes about pen test quality come down to the report, because the report is the only part anyone outside the engagement ever sees. A report that will survive an auditor, an enterprise security reviewer, and your own engineering team's scepticism needs several things that budget reports routinely omit.

It needs the scope stated precisely: which hosts, which applications, which API endpoints, which user roles, and what was explicitly excluded. Reviewers read the exclusions carefully, because a test that excluded authentication or excluded the payment path is not the test they were asking for. It needs the methodology and the standard it followed, along with the testing window dates, because a report with no dates is worthless for a compliance period. It needs each finding with reproduction steps, evidence, affected components, a severity with reasoning, and a concrete remediation recommendation that names the fix rather than saying "implement input validation". And it needs a statement of what was not achievable, because "we attempted to escalate from the standard user role and could not" is a positive result that belongs on the record.

One more detail that matters commercially: you will want a version you can share with customers. Full reports contain exploitation detail you do not want circulating. Ask up front whether a redacted summary or an attestation letter is included, because prospects will ask for something and you do not want to be negotiating that after the fact.

Scoping: what actually drives the price

Penetration testing starts from $1,000 for a tightly scoped engagement, and the spread above that floor is driven by a handful of variables rather than by vendor whim.

The first is surface count. A single web application with two user roles is a different quantum of work from a platform with a public API, an admin console, a mobile client, and a partner integration. Testers price in days, and each distinct surface consumes them. The second is role depth. Testing as an anonymous user is cheap. Testing every privilege boundary in a multi-tenant application, which is where the findings that actually matter tend to live, requires provisioned accounts in each role and enough tenants to prove cross-tenant isolation holds. The third is knowledge level. A black-box test spends a meaningful fraction of the engagement on reconnaissance you could have handed over. A grey-box test with credentials, an architecture diagram, and API documentation puts that time into finding things instead. For anything other than an adversary simulation exercise, grey-box gives you more findings per dollar, and we recommend it in almost every case.

The fourth is environment. Testing in production is more realistic and more constrained. Testing in staging is safer but only meaningful if staging is a faithful copy, which it usually is not: different IAM, seeded data, a WAF that is disabled, feature flags in different states. If you test staging, document the deltas, because an auditor who notices that the tested environment differs from the production one will discount the report.

Rules of engagement, and how tests go wrong

Before a test starts, write down the testing window, the source IP addresses the tester will use, the emergency contact on both sides, what is out of bounds, and how you want denial of service handled. Then tell your cloud provider if their terms require notice, and tell your own on-call team, because a pen test that pages an engineer at 2am and triggers an incident response that nobody knew was a drill burns goodwill you will want later.

The failure modes are consistent across the twenty-plus penetration tests we have run. Someone allowlists the tester's IP addresses at the WAF to "make testing easier", which means the test measures your application without the control that protects it in reality. Someone provisions a single test account and the tester cannot prove tenant isolation. The environment gets redeployed mid-window and half the findings evaporate before they can be reproduced. Or the report lands, three findings get fixed, and nobody ever confirms the fixes worked.

That last one is the most expensive. A finding is not closed because a ticket was closed. Our five published CVEs, including one rated CVSS 9.1 that appeared in Mirai botnet activity, all came from the same discipline that makes retesting worth paying for: verify what the code actually does after the change, not what the change was supposed to do.

Triaging findings when CVSS lies to you

A scan will hand you 400 findings sorted by CVSS, and CVSS is a severity score, not a priority order. It knows nothing about whether the vulnerable code path is reachable in your deployment, whether the component is internet-facing, or whether anyone is actually exploiting the flaw in the wild.

A more useful order starts with the CISA Known Exploited Vulnerabilities catalogue, which tells you what is confirmed to be under active exploitation, then applies exploit prediction scoring to the rest, then filters by reachability and exposure in your own environment. A CVSS 9.8 in a library function your application never calls, on a host with no inbound network path, is a lower priority than a CVSS 6.5 authentication weakness on your public login endpoint. Engineering teams stop trusting security backlogs when the ordering does not survive that kind of scrutiny, and once trust goes, remediation rates fall off regardless of how good the tooling is.

Set remediation targets by severity band, measure time to remediate rather than finding count, and accept that finding count going up is often a sign of better coverage rather than worse security.

When you should not buy a penetration test from us

There are several situations where a penetration test is the wrong purchase, and we will say so on the call.

If you already know about serious unpatched issues, fix them first. Paying a tester to rediscover an unauthenticated admin panel or an out-of-date framework with public exploits is buying an expensive confirmation of something you could have written down yourself. Spend the money on the fix, then test.

If nobody is available to remediate for the next quarter, defer. A report with unremediated criticals sitting in it is worse than no report during a due diligence review, because it demonstrates you knew and did nothing. Test when there is capacity to act on the results.

If your requirement is genuinely just an annual compliance artefact for a low-risk internal tool, a scoped, lightweight test is appropriate and you should resist being upsold into a red team exercise. Red teaming answers whether your detection and response work, and it is wasted on organizations that have not yet built detection and response to test.

And if you have a mature internal security team already running authenticated scanning and dependency analysis with good remediation discipline, what you probably need from an outside firm is targeted testing of one new high-risk surface, not a broad annual assessment that mostly reconfirms your own results. Say that when you request a quote and any honest vendor will scope smaller.

Making the two work together

The practical arrangement that works for most growing companies looks like this. Dependency and cloud posture scanning run continuously in the pipeline with build-breaking thresholds for critical, exploitable findings. Authenticated infrastructure scanning runs weekly with findings triaged into the same backlog engineering already uses, not a separate security tracker nobody opens. A penetration test runs annually and after any significant architectural change, scoped to the surfaces that changed. Every test is followed by a retest of the findings, and the retest letter is what you send to customers.

Where that programme needs an owner rather than a vendor, a retainer covers the triage and the follow-through between tests, and our security work lays out how testing, vulnerability management, and incident readiness fit together. If you only need the test itself with a fixed scope and a fixed number, the SKU pricing is the simpler path.

Need the testing done? Penetration testing and vulnerability management, with the retest that proves a finding is actually closed.

Penetration testingOr talk about a retainer

Before you go

Want the rest of this by email?

If this was useful, I send a few short notes on vulnerability management. Unsubscribe in one click, and replies reach me directly.

From Jacob Masse, principal of traztech. No spam, unsubscribe in one click.

Want a second opinion on where you stand?

We run SOC 2, ISO 27001 and the rest of the compliance stack for startups and SMEs, and the security testing that sits behind it. The first call is free, and we will tell you if you are not ready to start yet.

Book a free call

Track record

Who is actually doing the work

5
Published CVEs, including a CVSS 9.1
76
Controls taken from nothing to a passed SOC 2 Type II
Zero
Exceptions on that Type II report
20+
Penetration testing engagements delivered

Published vulnerability research

Five published CVEs. CVE-2024-45163 (CVSS 9.1) is a flaw in the Mirai botnet itself, which gave defenders a way to shut down attacker infrastructure. CVE-2026-42626 takes HP ENVY 5000 printers offline from any unauthenticated device on the same network.

A SOC 2 Type II built from nothing

At Humera, a venture-backed US security company, Jacob built the compliance programme in-house from nothing: no report, no policies, no documented controls. It ended in a Type II attestation across 76 controls with zero exceptions, on a team of 15.