advanced35 minby DevFox

Security Testing for Engineers

The testable part of security: authorisation matrices, injection at every boundary, dependency and secret scanning, and the difference between a scan and a pentest.

Testing
Security

Security testing has a reputation for requiring a specialist, and part of it does. The larger part is ordinary engineering: a set of checks with clear right answers, automatable, and routinely missing from test suites.

This lesson is about that larger part. By the end of it you will have an authorisation matrix you can generate tests from, know where injection actually lives, have dependency and secret scanning in the pipeline, and understand the honest boundary between what you can do and what needs a penetration test.

Authorisation is the biggest one

Broken access control is consistently the top entry in the OWASP Top Ten, and it is almost entirely testable. It is also the defect class most likely to be absent from an otherwise good suite, because happy-path tests cannot see it.

The technique is a matrix: every protected resource against every kind of actor, with the expected outcome in each cell.

text

resource / action        owner  same-acct  other-acct  admin  none
------------------------------------------------------------------
GET    /contacts          200      200        200(f)    200    401
GET    /contacts/{id}     200      200        404       200    401
PATCH  /contacts/{id}     200      200        404       200    401
DELETE /contacts/{id}     204      403        404       204    401
POST   /contacts/import   201      403        404       201    401
GET    /accounts/{id}     200      200        404       200    401
PATCH  /accounts/{id}     200      403        404       200    401
GET    /admin/users       403      403        403       200    401

Two conventions in that table are worth adopting deliberately.

200(f) — filtered

Succeeds, and returns only their rows.

The listing case, where authorisation is enforced by what comes back rather than by refusal.

A test that only checks the status code passes on a listing that leaks every account's data.

404 rather than 403

For another account's resource.

"Forbidden" confirms the record exists, which is enough to enumerate ids and learn who your customers are.

"Not found" says the same thing to a legitimate typo and to a probe.

Both are about what a response tells an attacker who is guessing

Then generate the tests from the matrix rather than writing them by hand:

python

CASES = [
    # (method, path, actor, expected)
    ("GET",    "/contacts/{id}", "owner",      200),
    ("GET",    "/contacts/{id}", "same_acct",  200),
    ("GET",    "/contacts/{id}", "other_acct", 404),
    ("GET",    "/contacts/{id}", "none",       401),
    ("DELETE", "/contacts/{id}", "same_acct",  403),
    ("DELETE", "/contacts/{id}", "other_acct", 404),
    # ...
]


@pytest.mark.parametrize("method,path,actor,expected", CASES)
def test_authorisation_matrix(client, actors, contact, method, path,
                              actor, expected):
    response = client.request(
        method,
        path.format(id=contact.id),
        headers=actors[actor].headers,
    )
    assert response.status_code == expected

Two additions make this genuinely strong.

Assert on the data for writes, not only the status. A 403 returned after the write has happened is a 403 and a data breach.

python

def test_a_rejected_update_changes_nothing(client, actors, contact):
    original = contact.name
    client.patch(f"/contacts/{contact.id}",
                 headers=actors["other_acct"].headers,
                 json={"name": "Changed"})
    assert reload(contact).name == original

Fail the build on an unlisted endpoint. Enumerate the application's routes and assert every one appears in the matrix, so a new endpoint cannot ship without an authorisation decision.

python

def test_every_route_has_authorisation_cases(app):
    covered = {(method, path) for method, path, _, _ in CASES}
    for route in app.routes:
        for method in route.methods - {"HEAD", "OPTIONS"}:
            assert (method, route.path) in covered, (
                f"{method} {route.path} has no authorisation cases"
            )

Injection, where it actually lives

"Injection" means untrusted input being interpreted as instructions rather than as data, and it exists at more boundaries than people check.

Tests are cheap and specific:

The last payload in that list is doing something different and important. O'Brien must be accepted and stored correctly — a system that rejects apostrophes has usually "fixed" injection by blocking valid data, which is a different bug wearing a security badge. The correct fix is parameterisation, which handles both.

Fuzzing, from the earlier lesson, is the systematic version of this list: point it at any parser or input handler and it will generate hostile input more imaginatively than you can.

The scanners, in the pipeline

Four automated checks belong in CI, and all four are close to free.

Dependency scanning. Known vulnerabilities in what you depend on — which is where a large share of real incidents originate.

Secret scanning. Credentials committed by accident. Run it on history as well as on the diff, and run it as a pre-commit hook so the commit never happens.

Static analysis for security. Patterns a linter can recognise: unsafe functions, shell=True, unparameterised queries, weak cryptography.

Container and image scanning, if you ship images.

Scan results treated as noise

Bad — the scan runs, reports, and nothing happens:

Good — blocking, with a triaged and shrinking exception list:

The bad version is the usual state of dependency scanning, and it is worse than not scanning: it produces a green build, a report nobody reads, and the appearance of a control. Forty-seven findings means nobody can tell which one matters, so the one that does is invisible.

The good version blocks on high severity and requires each exception to carry a reason, an owner and an expiry. Then the list is a backlog with a shape, and ignoreUntil forces the decision to be revisited rather than becoming permanent by default.

The rest of the testable surface

Six more checks that need no specialist and find real problems.

Several of those are assertable in the API tests you already have, which is the cheapest place to put them.

Where the boundary is

Being honest about the limit is what makes the rest credible.

What you can do: everything above. Authorisation matrices, injection cases, scanners, header and cookie checks, upload validation, rate limiting, and fuzzing your parsers. This finds a large share of real vulnerabilities, and it finds them continuously rather than annually.

What needs a specialist: chained exploits across several small weaknesses, business-logic abuse that no rule describes, cryptographic review, authentication protocol flaws in OAuth or SAML implementations, and infrastructure and cloud configuration review. A penetration test is a time-boxed engagement by people who do this full time, and it finds a different category from any scan.

The relationship between them is worth stating: automated checks handle the known classes so a pentest is not spent reporting a missing security header. Fix the automatable half first, and the specialist time is spent on the part only a specialist can do.

Cheat sheet

Where to go next

The next lesson is about the infrastructure all of this runs on: ephemeral environments created per branch and destroyed after, and why a shared staging box becomes a queue that slows every team using it.

Before that, build the authorisation matrix for one resource in your own API — five actors, four methods, twenty cells. It is an afternoon, and the hit rate on a matrix that has never been built is high enough that it is rarely a quiet afternoon.