Application Security — Penetration Testing, Code Review and SCA

  • Home
  • Application Security — Penetration Testing, Code Review and SCA
Application Security — Penetration Testing, Code Review and SCA
Application Security — Penetration Testing, Code Review and SCA
Application Security — Penetration Testing, Code Review and SCA
Application Security — Penetration Testing, Code Review and SCA

A buyer’s guide · Operating since 2002

Application securityWhat each testing method actually catches, where it sits in the lifecycle, and the flaws none of them find

Most application security budgets buy tools. Tools find the defects that look like other defects — the injection pattern, the outdated library, the missing header. They do not find the workflow that lets a customer apply the same discount twice, because nothing about that code looks wrong. It is doing exactly what it was written to do.

This page sets out the methods, what each one genuinely covers, and the honest limitation of each — the column most vendor comparisons leave out. If you are looking for a scoped testing engagement rather than a programme, that is VAPT.

A note on the term. “Application security” here means securing software you build or run. It is not Application Security Groups, which are a network filtering feature in Azure, and it is not the same as a vulnerability scan with a report cover on it.
3K+
Projects delivered
1,000+
Enterprises protected
50+
In-house NOC & SOC engineers
24+
Years, since 2002
ISO/IEC 27001:2022
Certified — operations in scope

The comparison

Six methods, and what each one misses

These are complementary rather than competing, and no serious programme uses only one. The useful question is not which is best but which class of defect each leaves behind.

Scroll the table sideways →

Method What it is What it catches The honest limitation
Manual penetration testingthe engagement A tester attacking a running application the way an attacker would. Business logic, chained flaws, authorisation gaps, anything requiring intent. Point in time. The build you tested is not the build you shipped last Tuesday.
Secure code review A human reading the source, usually targeted at the risky parts. Design errors, dangerous patterns, missing authorisation checks, crypto misuse. Slow and expensive per line, so it must be aimed rather than sprayed.
SASTstatic analysis Automated inspection of source code without running it. Injection patterns, unsafe functions, hardcoded secrets, tainted data flows. Notorious false-positive rates. Sees no runtime context, so it cannot tell exploitable from theoretical.
DASTdynamic analysis Automated probing of the running application from outside. Reflected inputs, misconfiguration, exposed endpoints, missing headers. Only finds what it can reach. Authenticated and multi-step flows are routinely missed unless configured with care.
SCAdependency scanning Inventory of third-party libraries checked against known vulnerabilities. Known CVEs in dependencies, licence problems, unmaintained packages. Reports the vulnerable version, not whether your code ever calls the vulnerable function. Most findings are not reachable.
Threat modelling Structured reasoning about what could go wrong, before code exists. Whole categories of design flaw, at the only point they are cheap to fix. Produces no findings list, which makes it the first thing cut when a deadline slips.

Read the limitation column first. Every one of these methods is sold on what it finds; the difference between a programme that works and one that produces reports is whether somebody planned for what each method does not.

Coverage

Where each method sits, and the band nothing covers

Methods are usually argued about as though they compete. Laid against the lifecycle they mostly do not overlap at all — and one band has no automated coverage whatsoever.

DesignCodeBuildTestDeployRuntimeThreat modellingSecure code reviewSASTSCA / dependenciesDASTManual pen testBusiness-logic flawsno tool spans this — found by people who understand the businessA scanner-only programme is not an application security programme.

The band with no tool in it

Business-logic flaws span the entire lifecycle and are invisible to every automated method, because the code contains no defect. Applying a discount twice, skipping a step in a payment flow, changing an identifier to read another customer’s record — each is the application working as written.

They are found by people who understand what the application is for, and they are consistently among the most damaging, because they need no exploit and often leave no anomaly in the logs.

Why shifting left is not enough on its own

Design-stage work is cheap and catches whole categories early, which is why threat modelling earns its place. But it reasons about the system you intended to build.

Runtime testing examines the system you actually deployed, including the configuration nobody documented and the endpoint somebody added in a hurry. Programmes that lean entirely on either end consistently miss what the other would have caught.

Straight answers

What buyers get wrong about application security

Fact

No tool finds business-logic flaws

Every automated method looks for patterns it was taught. A workflow that lets a user apply a discount twice, or skip a payment step, or read another customer’s record by changing an identifier, breaks no pattern — the code is doing exactly what it was written to do. Those flaws are found by people who understand what the application is for, and they are consistently among the most damaging.

Ask

“We run scans” is not a programme

Scanners are useful and cheap and should absolutely run. But a programme is a set of decisions about which risks matter, which methods cover them, who fixes what by when, and how you know it stayed fixed. A tool with nobody reading its output is a subscription, not a control.

Fact

Most dependency findings are not reachable

SCA reports that a vulnerable version is present. It rarely establishes whether your code ever calls the vulnerable function. Without reachability analysis, teams burn weeks upgrading libraries to fix vulnerabilities they were never exposed to, while genuinely reachable ones sit in the same undifferentiated list.

Ask

“Annual pen test” means eleven untested months

If you deploy weekly and test yearly, fifty-one of your fifty-two builds went to production untested. That may be an acceptable risk, but it should be a decision rather than an accident of the audit calendar.

Fact

A CVSS score is not a priority

CVSS describes a vulnerability in the abstract. Whether it matters depends on whether the affected code is reachable, whether the data behind it is sensitive, and whether compensating controls exist. A 9.8 on an internal admin tool behind SSO may matter less than a 6.1 on your checkout.

Ask

“Do you test the API separately?”

Modern applications are mostly API. Testing the web front end and calling the job done leaves the actual attack surface unexamined — particularly authorisation, which is enforced per endpoint and is where most real breaches begin.

Before you sign

Ten questions for any application security provider

Use these on us and on everyone else you shortlist. Question three separates a test from a scan, and question ten separates a current provider from one still testing last year’s architecture.

Scope

Is the test authenticated, and against how many roles?

An unauthenticated test of an authenticated application checks the front door of a building nobody uses. Ask which roles were tested and whether cross-role authorisation was exercised.

Depth

Is this a scanner run with a report cover, or did a person attack it?

Ask how many hours a human spent. A tool-only engagement is a legitimate product, but it should be priced and named as one.

Logic

How do you test business logic?

This is the class of flaw no tool finds and the class that costs the most: negative quantities, skipped payment steps, tampering with another customer’s identifier. If the answer is vague, the test is a scan.

Evidence

Can we see a redacted sample report?

Ask for one before signing. A findings list with CVSS scores and no reproduction steps is not something a developer can act on.

Retest

Is a retest included, and for how long after the fix?

Findings you cannot prove are closed are findings you still have. Retest should be in the price, not a change order.

Triage

Who separates the exploitable from the theoretical?

A raw scanner export with four hundred findings transfers the work to your developers rather than doing it. Ask what the deliverable actually contains.

Dependencies

Does the SCA output tell us what is reachable?

Most dependency findings are in code paths your application never executes. Reachability analysis is the difference between a prioritised list and a wall of noise.

Cadence

How does this run continuously rather than annually?

An application tested once a year is untested for eleven months. Ask how findings enter the development workflow rather than a PDF.

Secrets

Do you check for exposed credentials in source and build history?

Deleted secrets remain in git history. This is one of the cheapest checks to run and one of the most commonly skipped.

AI

How do you handle AI-assisted or AI-embedded features?

Prompt injection, insecure output handling and model-supply-chain risks are not covered by a conventional web application test. If a provider has no answer here, they are testing last year’s architecture.

Working with us

How we run this

We have tested and defended enterprise applications since 2002, and we run the operations centre that watches them afterwards. That combination shapes how we report: a finding is only useful if somebody can act on it.

People, not just tooling

Automated analysis runs because it is cheap and catches the obvious. The engagement is priced on the hours a person spends attacking the application, and we will tell you what that number is.

Findings developers can use

Reproduction steps, evidence, real exploitability in your context rather than a bare CVSS score, and remediation specific enough to act on. Into your issue tracker if you would rather not receive a PDF.

Retest included

A finding you cannot demonstrate is closed is a finding you still have. Retest is part of the engagement, not a change order.

Feeds the operations centre

What testing learns about your application becomes detection content for the team watching it — see managed SOC services and threat hunting.

Certified operations

ISO/IEC 27001:2022 certified, with our operations centre inside the certified scope and the scope statement available on request.

US presence

P J Networks LLC is our US entity, based in Dallas, Texas — the company a US client contracts with. It is a subsidiary of P J Networks Pvt Ltd, which holds the ISO/IEC 27001:2022 certification referenced on this page.

3K+
Projects delivered
1,000+
Enterprises protected
50+
In-house NOC & SOC engineers
24+
Years, since 2002
ISO/IEC 27001:2022
Certified — operations in scope

Questions we get asked

Application security, answered

What is application security?

Application security is the practice of finding and fixing flaws in software before an attacker uses them, and of building so that fewer flaws exist. In practice it combines threat modelling at design, static and dependency analysis during development, dynamic testing and manual penetration testing against running builds, and secure code review aimed at the parts that matter most. The distinguishing feature of a programme, as opposed to a purchase, is that someone decides which methods cover which risks and what happens to a finding after it is raised.

What is the difference between SAST and DAST?

SAST reads your source code without running it and finds patterns that look dangerous — unsafe functions, tainted data reaching a sink, hardcoded secrets. DAST attacks the running application from outside and finds what is actually exposed — misconfiguration, reflected inputs, missing controls. They fail in opposite directions: SAST sees code it cannot execute and produces false positives, while DAST sees behaviour it cannot attribute to a line of code and misses anything it cannot reach. Used together they overlap usefully; used alone either leaves obvious gaps.

Is a vulnerability scan the same as a penetration test?

No, and the difference is the most common source of disappointment in this market. A scan is automated pattern matching, delivered as a list. A penetration test is a person attempting to achieve an objective in your application — chaining small weaknesses, abusing logic, escalating between roles. Both are legitimate; they cost different amounts and find different things, and a scan sold as a test is the thing to watch for.

What is secure code review?

A human reading source code specifically looking for security defects, usually aimed at authentication, authorisation, cryptography, input handling and anything touching money or personal data. It is slower and more expensive per line than automated analysis, which is exactly why it should be targeted rather than applied uniformly — the value comes from putting expert attention on the code where a mistake would matter most.

What is SCA and why do most findings not matter?

Software composition analysis inventories your third-party dependencies and matches them against known vulnerabilities. It tells you that a vulnerable version is present. It usually does not tell you whether your application ever calls the vulnerable function, and in most codebases the majority of flagged vulnerabilities sit in code paths that never execute. Reachability analysis is what turns that output from a wall of noise into a short, actionable list.

How often should applications be tested?

Tie it to change rather than to the calendar. Automated analysis belongs in the pipeline on every commit or merge. Manual testing is worth scheduling around significant releases, architectural changes, new authentication flows, and anything that alters how data moves. An annual test on a weekly release cycle leaves fifty-one builds unexamined, which may be acceptable but should be a decision rather than a habit.

Do you test APIs and mobile applications?

Yes. Modern applications are predominantly API, and testing only a web front end leaves the real attack surface unexamined — particularly authorisation, which is enforced endpoint by endpoint and is where a large share of real breaches begin. Mobile testing covers the application binary, its local storage and the APIs behind it, since the client is fully under an attacker’s control.

What about AI features and LLM-based applications?

They need testing their own architecture calls for, not last year’s. Prompt injection, insecure handling of model output, excessive agency granted to a model, and the model supply chain are real categories that a conventional web application test does not cover. If an application takes untrusted input into a prompt, or lets a model take actions, that path needs examining specifically.

What does the OWASP Top 10 actually give us?

A shared vocabulary and a sensible starting point, not a certification and not a test plan. It describes the categories of risk most commonly seen across web applications, which makes it useful for prioritising and for talking to developers. Treating it as a checklist to be completed is how organisations end up compliant with a list while remaining vulnerable to the flaw specific to their own business.

How is this different from your VAPT service?

VAPT is the testing engagement — a scoped exercise with a start, an end and a report, covering network, application and cloud. This page describes the ongoing programme that testing sits inside: which methods run continuously, how findings reach developers, what gets retested and how you know coverage is improving. Many organisations buy the engagement first and build the programme afterwards, which is a perfectly reasonable order.

Do we get a retest after fixing?

Yes, and you should insist on it from any provider. A finding you cannot demonstrate is closed is a finding you still have, and fixes routinely address the symptom rather than the cause. Retest should be included in the engagement rather than sold as a change order.

What does the report contain?

Findings with reproduction steps a developer can follow, evidence, an assessment of real exploitability in your context rather than a bare CVSS score, and remediation guidance specific enough to act on. A prioritised summary for people who must decide what to fund, and the technical detail for people who must fix it. Sample reports are available before you commit.

Can you work with our developers rather than around them?

That is usually where the value is. Findings that arrive as an adversarial PDF get argued with; findings that arrive in the workflow developers already use, explained by someone who can discuss the fix, get fixed. We are happy to work through your issue tracker and to talk directly to the people writing the code.

What do you need from us to scope this?

The application type and rough size, whether testing is authenticated and how many roles exist, whether there is an API and whether it is documented, your release cadence, and any areas you already suspect. If a previous report exists, that is the single most useful document you can share.

Next step

Send us the application, not a requirements document

Tell us the application type, whether testing is authenticated and how many roles exist, whether there is an API, and your release cadence. We will scope it, tell you how many human hours it warrants, and say plainly where automated analysis would serve you better than an engagement.

sanjay@pjnetworks.com