Scoring and hardening: the 19 Scorecard checks

The four previous guides built a pipeline that already ticks twelve guaranteed checks of OpenSSF Scorecard at 10, plus two (CI-Tests, Vulnerabilities) that hold 10 depending on the current state of the repository. This guide closes the loop: it takes each check apart from its source code, explains how its score is computed, shows the exact code satisfying it in our repository, and separates the actionable levers from the structural ceilings no configuration unlocks. The audience is an advanced maintainer who wants the best score without cheating. By the end, you will read a Scorecard report line by line and act on what genuinely depends on you.

What you will learn

Run Scorecard in CI and locally, then read its report
Understand the weighted average and the weight of each check
Reproduce the code satisfying Signed-Releases, Fuzzing and Pinned-Dependencies
Tell apart the acquired checks, the levers to activate, and the Maintained ceiling

Prerequisites

The full pipeline is in place: hardened CI, verifiable build and branch protection. The repository is public (Scorecard only evaluates public repositories fully) and the gh CLI is authenticated. All the code in this guide comes from the reference repository, public and browsable: github.com/stephrobert/secure-python-pipeline.

How Scorecard computes a score

The overall score is a weighted average of the checks, brought back to 10. Each check carries a risk level that fixes its weight in the average:

Risk level	Weight	Rationale
Critical	10	A directly exploitable flaw
High	7.5	An essential hardening of the chain
Medium	5	A robustness good practice
Low	2.5	A quality signal

That weighting has a practical consequence: a point gained on a High check is worth three points gained on a Low one. When optimising, you look first at the heavy checks still below 10.

Two calculation subtleties come up everywhere. An inconclusive check (the token could not read the needed data) is not counted in the average: making it readable but badly configured can therefore lower your score instead of raising it. And each check is either binary (0 or 10, the presence of a file) or proportional (a fraction, often with an integer division that punishes the slightest gap hard).

Running Scorecard

There are two ways to run it: a workflow publishing the score continuously, and a local command to iterate without waiting.

The workflow below runs on pushes to main, weekly, and on every branch rule change. It publishes the result (badge and OpenSSF API) and uploads a SARIF visible in the Security tab.

name: Scorecard

on:
  branch_protection_rule:
  push:
    branches: [main]
  schedule:
    - cron: "0 7 * * 1"

permissions: {}

jobs:
  analysis:
    name: OpenSSF Scorecard
    runs-on: ubuntu-24.04
    timeout-minutes: 20
    permissions:
      security-events: write # uploading the SARIF
      id-token: write        # signed publication of the results
      contents: read
      actions: read
    steps:
      - name: Harden runner
        uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
        with:
          egress-policy: audit
      - name: Checkout
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - name: Scorecard analysis
        uses: ossf/scorecard-action@2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4
        with:
          results_file: results.sarif
          results_format: sarif
          publish_results: true
      - name: Publish the SARIF
        uses: github/codeql-action/upload-sarif@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3
        with:
          sarif_file: results.sarif

To iterate without waiting for the next run, you launch Scorecard locally through its official image, with a read-only token:

export GITHUB_AUTH_TOKEN=$(gh auth token)

docker run -e GITHUB_AUTH_TOKEN \
  gcr.io/openssf/scorecard:stable \
  --repo=github.com/stephrobert/secure-python-pipeline

The output lists each check with its score and the aggregate. That is the view we read line by line in the rest of this guide:

Aggregate score: 8.0 / 10

Check scores:
| SCORE    | NAME                    |
|----------|-------------------------|
| 10 / 10  | Token-Permissions       |
| 10 / 10  | Pinned-Dependencies     |
| 10 / 10  | Signed-Releases         |
| 10 / 10  | SAST                    |
|  0 / 10  | Maintained              |
|  ? / 10  | Branch-Protection       |

The dashboard of the nineteen checks

Here is the overview. The Our score column reflects the state of the secure-python-pipeline repository built in this series. The Nature column separates what the pipeline acquires, what remains a lever to activate, and the structural ceiling.

Check	Risk	What it measures	Our score	Nature
Token-Permissions	High	Least privilege of the `GITHUB_TOKEN`	10	Acquired
Dangerous-Workflow	Critical	No dangerous workflow pattern	10	Acquired
Pinned-Dependencies	Medium	Dependencies frozen by SHA, digest or hash	10	Acquired
SAST	Medium	Static analysis on every PR	10	Acquired
Binary-Artifacts	High	No executable binary committed	10	Acquired
Dependency-Update-Tool	High	Dependabot or Renovate present	10	Acquired
License	Low	A `LICENSE` file recognised as SPDX	10	Acquired
Packaging	Medium	Artifacts published by a workflow	10	Acquired
Webhooks	Critical	Webhooks with a secret (or none at all)	10	Acquired
Security-Policy	Medium	A complete `SECURITY.md`	10	Acquired (lever)
Signed-Releases	High	Provenance or signature on the releases	10	Acquired (lever)
Fuzzing	Medium	A fuzzing integration	10	Acquired (lever)
CI-Tests	Low	Green CI tests on recent PRs	10*	Acquired
Vulnerabilities	High	No open OSV CVE	10*	Acquired
Code-Review	High	Changes approved by a third party	variable	Lever
Contributors	Low	Distinct entities (Company)	variable	Lever
Branch-Protection	High	Branch protection rules	tiered	Lever
CII-Best-Practices	Low	The bestpractices.dev badge	0	Lever
Maintained	High	Recent activity and repository age	0	Ceiling

The scores marked * depend on a state that can change without a commit: one untested PR breaks the CI-Tests 10, a new CVE drops Vulnerabilities.

The checks the pipeline already ticks

Most of the checks are satisfied simply by having followed the previous guides. Understanding why saves you from believing more work is needed.

The "free" gains of a hardened pipeline

These checks require no work dedicated to the score: a clean pipeline ticks them by construction.

Token-Permissions (High) measures the least privilege of the GITHUB_TOKEN. Our workflows declare permissions: {} globally and minimal permissions per job. The check starts at 10 and subtracts points for each global write: with none, you stay at 10.
Dangerous-Workflow (Critical) is binary: 10 or 0. It looks for two patterns, the untrusted checkout in a privileged context (pull_request_target followed by a checkout of the PR) and script injection. None of our seven workflows shows either, so 10.
Binary-Artifacts (High) checks that no executable binary is committed. Our code is Python source, the image is built from the Dockerfile: the probe finds no artifact, so 10.
License (Low) is additive: the LICENSE file at the root is worth +9, and an SPDX identifier recognised by the FSF or the OSI (MIT here) earns the final +1.
Packaging (Medium) looks for a recognised publishing workflow with at least one successful run. Our release.yml pushes an image to GHCR through docker/build-push-action, and the v1.0.0 release proves a successful run.
Webhooks (Critical) requires every webhook to carry a secret. The repository has none: the check is NotApplicable and returns 10 by default. Do not confuse that 10 "by absence" with an earned one.
CI-Tests (Low) measures the share of recently merged PRs carrying a successful test check. Our ci.yml runs pytest on every pull_request. The 10 requires 100 % of the last thirty PRs to be covered.
Vulnerabilities (High) queries OSV.dev and removes one point per open CVE. You stay at 10 as long as no known vulnerability touches your dependencies, hence the active watch through pip-audit, Trivy and Dependabot.

Dependency-Update-Tool (High) is also binary: the mere presence of .github/dependabot.yml at the right path is enough, the check does not judge the configuration. The repository takes the opportunity to add an anti-supply-chain quarantine (the cooldown block, forbidding the adoption of a version on its release day): it earns no point but hardens for real. Its full configuration is detailed in the bootstrap.

Pinned-Dependencies: freezing by hash

Pinned-Dependencies (Medium) requires every ecosystem to be frozen. A pip install fastapi==0.139.1 fixes the version but not the hash: Scorecard counts it as unpinned. The key point: == is not enough, you need a lockfile installed with --require-hashes, and every ecosystem must be frozen, actions by commit SHA, the base image by @sha256 digest and Python by hash. The full mechanics of the .in compiled with --generate-hashes are described in the bootstrap.

SAST: CodeQL on pull requests

SAST (Medium) wants static analysis on the PRs. Our codeql.yml runs github/codeql-action/analyze on pull_request: every merged PR carries the github-code-scanning check. An important calculation detail: CodeQL triggered on cron alone would cap at 7; it is the PR trigger that reaches 10.

Security-Policy: the tiered SECURITY.md

Security-Policy (Medium) is additive in tiers. The presence of a SECURITY.md unlocks the check, then the score adds up: +6 for a link or an email, +3 for real text (not an empty template), +1 for at least two disclosure markers (the "vuln" and "disclos" roots, plus stated deadlines). Our file carries the security@ email, the advisory URL, genuine prose and the deadlines (48 h, 90 days): 10.

Signed-Releases: the provenance as an asset

Signed-Releases (High) looks, over the last five releases, for a provenance or signature asset. A key point found in the code: the check looks at a *.intoto.jsonl file attached to the release, not the OCI attestation of the image, two distinct artifacts. One provenance is worth 10 points for that release, and the score is the floored average over the releases: with a single release carrying provenance, floor(10/1) = 10. The release.yml step copying the bundle to provenance.intoto.jsonl and attaching it to the release is detailed in Verifiable build.

Fuzzing: the Atheris harness

Fuzzing (Medium) is binary and disarmingly simple: for a Python project, the probe turns True as soon as a file contains import atheris. To stay honest rather than dropping an empty file, our harness sends real random inputs to the API and fails if the server returns a 5xx:

import sys
import atheris

# We do NOT instrument the imports: pydantic-core is a compiled extension
# (Rust) whose loader segfaults atheris.instrument_imports().
from fastapi.testclient import TestClient  # noqa: E402
from app.main import app  # noqa: E402

client = TestClient(app)

def test_one_input(data: bytes) -> None:
    fdp = atheris.FuzzedDataProvider(data)
    path = fdp.ConsumeUnicodeNoSurrogates(64)
    try:
        response = client.get("/" + path)
    except Exception:  # noqa: BLE001
        return
    # A 5xx on a malformed input signals a bug to fix.
    assert response.status_code < 500

def main() -> None:
    atheris.Setup(sys.argv, test_one_input)
    atheris.Fuzz()

if __name__ == "__main__":
    main()

The fuzz.yml workflow runs it with a bounded budget, which keeps the check honest without blocking the CI for too long:

# .github/workflows/fuzz.yml (the execution step)
      - name: Run fuzzing (limited budget)
        run: python fuzz/fuzz_api.py -runs=20000 -max_total_time=120

The score would be the same with a dummy file; the difference is ethical, not mechanical.

The levers left to activate

After the previous guides, four checks still depend on a targeted effort.

Branch-Protection: climbing the tiers

Branch-Protection (High) evaluates the default branch through cumulative tiers: each tier must be full to unlock the next, from the foundation (deletion and non_fast_forward) up to zero admin bypass. The check reads the rules through the token, so an active and readable ruleset is a prerequisite of the scoring. The detail of the five tiers, the ruleset.json descriptor and the gh api command that applies it are covered in Protection and governance.

The high tiers require a team: two approvals assume two reviewers. The last tier (zero admin bypass) stays debatable on a solo repository, where the author needs to be able to unblock a situation. You climb as high as the organisation allows, without lying to yourself about what is sustainable.

Code-Review: the discipline of the PR

Code-Review (High) computes the share of the last thirty changesets approved by an account different from the author, bot changesets being excluded from the denominator. Concretely, every change must go through a PR approved by a third party and direct pushes to main must be forbidden. Our two collaborators make that cross approval possible; the score climbs with time and discipline, not with a single setting.

Contributors: several organisations

Contributors (Low) counts the number of distinct entities among the contributors with at least five contributions, an entity being a public organisation or the Company field of the profile. Three distinct entities are enough for 10 (the denominator is capped at 3, with integer division). So three accounts need to fill in a different Company and contribute for real. It is a social lever, not a technical one.

CII-Best-Practices: the badge

CII-Best-Practices (Low) queries the bestpractices.dev API for the repository URL. The tiers give points: In Progress 2, Passing 5, Silver 7, Gold 10. It is the best remaining effort-to-gain ratio.

Register on bestpractices.dev, sign in with GitHub, add the project by its URL. Registration alone already gives In Progress (2 points).
Fill in the Passing questionnaire (about 66 criteria). A repository following this series already meets most of it: licence, README, HTTPS, versioning, SECURITY.md, tests in CI, static analysis, signed delivery.
Answer honestly: each criterion is marked Met, Unmet or N/A with a justification. The cryptography criteria are N/A when the project implements no cryptography of its own.

The structural ceiling: Maintained

Maintained (High) is the only check no configuration unlocks in the short term. The Scorecard code short-circuits the calculation for a recent repository: the lookBackDays = 90 constant and a recentlyCreated test force the minimum score while the repository is less than 90 days old, whatever the activity. That is time, not configuration. Past that delay, the check measures commits and issues over the sliding window; regular activity is then needed to hold the score.

Two other checks carry, on top of their lever, an incompressible share of time: Code-Review and Contributors climb with the history of reviewed PRs and the accumulation of contributions. You activate them, but their real ceiling only appears over months.

Do not game the score

The score is a means, not an end. Adding a tool solely to tick a check is gaming: if a check is already satisfied by the right tool (CodeQL for static analysis), you do not stack a second one to "lock in" the score. In the same way, you do not mark as Met a bestpractices.dev criterion you do not genuinely meet, and you do not drop an import atheris into an empty file to simulate fuzzing. A score earned honestly reflects a real security posture; an inflated score fools nobody but yourself, and collapses at the first serious audit.

Key points

The score is a weighted average: Critical counts 10, High 7.5, Medium 5, Low 2.5. A point on a High weighs three times a point on a Low.
An inconclusive check made readable but badly configured lowers the score: only make readable what is correctly configured.
Twelve checks are guaranteed at 10 by construction, plus two (CI-Tests, Vulnerabilities) depending on the current state; understanding their calculation saves you from overdoing it.
Remaining levers: Branch-Protection (tiers), Code-Review (cross-reviewed PRs), Contributors (three entities), CII-Best-Practices (the Passing badge).
Structural ceiling: Maintained (the 90-day constant, proven by the code), plus the time share of Code-Review and Contributors.
The score is a means: you do not game it, you harden for real.

Next steps

Debugging workflows: taking back control when a hardened check fails a job with no readable explanation.
Securing the runners: extending the hardening to the infrastructure that actually runs your jobs.
GitHub CLI (gh): driving the repository settings and replaying the scoring from the terminal.