needs and job outputs: modelling a pipeline as a graph

Read this page in French

By default, the jobs of a workflow run in parallel and share nothing. A real CI is not a list of independent jobs: it is a graph where some steps wait for others and consume their results. This page covers needs, job outputs, and above all what happens when one branch of the graph fails.

What you will learn

• Chain jobs with needs, in parallel and in sequence
• Pass a value from one job to another with outputs
• Carry a structure through JSON and fromJSON
• Master failure: skipped jobs, always(), failure(), cancelled()
• Combine needs and matrices without trapping yourself

Parallelism by default, and what needs changes

Without needs, every job starts at the same time, each on a fresh machine. That is fast, and it is wrong as soon as one step depends on another: publishing before testing makes no sense.

needs declares a dependency and, as a consequence, an order:

name: CI

on:
  pull_request:
    branches: [main]

permissions: {}

jobs:
  lint:
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - run: npm ci && npm run lint

  test:
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - run: npm ci && npm test

  build:
    needs: [lint, test]
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - run: npm ci && npm run build

lint and test start together, build waits for both. The shape of the graph is read in the needs, not in the order the jobs are written: moving build to the top of the file changes nothing.

lint ───┐
        ├──> build ──> scan ──> publish
test ───┘

This structure, several analyses in parallel then a convergence, is the most common shape of a serious CI. It gives the fastest possible feedback on the most frequent errors, while guaranteeing that nothing is built on code that does not pass.

Passing a value between jobs

Every job runs on a different machine. Nothing is passed automatically: neither files nor variables. For files, that is what artifacts are for; for values, it is outputs.

The mechanism is declared at two levels. The step writes to $GITHUB_OUTPUT, the job promotes that value to a job output:

jobs:
  version:
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    outputs:
      number: ${{ steps.compute.outputs.number }}
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - name: Compute the version number
        id: compute
        run: echo "number=1.4.$GITHUB_RUN_NUMBER" >> "$GITHUB_OUTPUT"

  build:
    needs: version
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    env:
      VERSION: ${{ needs.version.outputs.number }}
    steps:
      - name: Build with the computed version
        run: echo "Building version $VERSION"

Three rules avoid most of the mistakes:

• The step id is required for it to be referenced. Without id:, the value is unreachable.
• The consuming job must declare needs. The needs.<job>.outputs context only exists for jobs you depend on, even when the producing job has already finished.
• Every output is a string. There is no integer, no boolean, no array.

Never route a secret through an output

A job output travels through the orchestrator to reach the next job. It is not the channel designed for a sensitive value, and relying on it produces an empty value at best, an exposure at worst. A job that needs a secret reads it directly from secrets, or from the environment it is attached to.

Carrying a structure through JSON

Since an output is a string, carrying a list or an object goes through serialised JSON, decoded on arrival with fromJSON.

jobs:
  targets:
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    outputs:
      list: ${{ steps.detect.outputs.list }}
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - name: Detect the changed services
        id: detect
        run: echo 'list=["api","worker","frontend"]' >> "$GITHUB_OUTPUT"

  deploy:
    needs: targets
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    strategy:
      matrix:
        service: ${{ fromJSON(needs.targets.outputs.list) }}
    steps:
      - name: Deploy one service
        env:
          SERVICE: ${{ matrix.service }}
        run: echo "Deploying $SERVICE"

This is the dynamic matrix pattern: a first job computes the list of targets, a second derives that many parallel runs from it. It avoids maintaining by hand a matrix that repeats what the repository already holds.

One limitation to know: the JSON string must be valid and on a single line. JSON produced by a tool can contain newlines that break the write to $GITHUB_OUTPUT. Compacting it solves the problem:

echo "list=$(jq -c . targets.json)" >> "$GITHUB_OUTPUT"

What happens when a job fails

This is the part that surprises people, and it fits in one sentence from the GitHub documentation:

If a job fails or is skipped, all jobs that need it are skipped unless the jobs use a conditional expression that causes the job to continue.

Two practical consequences. A dependent job is not marked failed, it is skipped: the workflow shows a result that is not red everywhere, which misleads a quick reading. And a skipped job propagates: its whole descent is skipped in turn.

Forcing execution despite a failure

The documentation is explicit about the means:

If you want a job to run even when a job it depends on did not succeed, use the always() conditional expression.

  report:
    needs: [lint, test, build]
    if: always()
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    steps:
      - name: Publish the report whatever the result
        env:
          TEST_RESULT: ${{ needs.test.result }}
        run: echo "Test result: $TEST_RESULT"

The needs.<job>.result context carries the real outcome of the job you depend on: success, failure, cancelled or skipped. That is what lets a report job say what failed.

Four functions cover the common cases:

Condition	The job runs
success()	If every job in needs succeeded, the implicit default
failure()	If at least one job in needs failed
cancelled()	If the workflow was cancelled
always()	Systematically, including after a cancellation

Glissez pour voir

always() on a publishing job is an anti-pattern

always() really means always, including when the tests failed and including when somebody cancelled the workflow. On a report or cleanup job, that is what you want. On a job that publishes, deploys or signs, it is an open door: you are shipping an artifact your own CI has just rejected.

For a cleanup that must ignore failure but respect cancellation, prefer if: !cancelled().

The case of continue-on-error

A job marked continue-on-error: true is considered successful from the point of view of its dependants, even when it failed. That is useful for an informational check, and dangerous for a security check: a scanner running under continue-on-error no longer blocks anything, it decorates.

If you want to keep the information without blocking, keep the job blocking and let the next job decide based on needs.<job>.result.

Combining needs and matrices

A job depending on a matrix job waits for every combination, not the first one. That is the desirable behaviour: you do not build an image because the tests pass on a single Python version.

jobs:
  test:
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    strategy:
      fail-fast: false
      matrix:
        python: ["3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
        with:
          python-version: ${{ matrix.python }}
      - run: pytest -q

  build:
    needs: test
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    steps:
      - run: echo "Every version passed the tests"

Two points of vigilance. fail-fast: false lets every branch run to the end: without it, the first failure cancels the others and you lose the most useful information, namely whether the problem hits one version or all of them. And the outputs of a matrix job are ambiguous: every combination writes to the same output, and the last one to finish wins. Do not rely on them, use artifacts named per combination.

Rereading your graph

Once the pipeline is written, three questions are enough to spot the common structural defects.

1. Which jobs could run in parallel but do not? A needs added for comfort lengthens the pipeline without guaranteeing anything.

2. Which job publishes or deploys, and what does it really depend on? If its chain of needs does not reach back to the tests and the security scanners, it can ship code that was rejected elsewhere.

3. Which job carries an always() or a continue-on-error? Each one is an exception to justify, and that is where controls evaporate most quietly.

Key points

• Jobs are parallel by default; needs declares a dependency and draws the graph, independently of the writing order.
• The useful shape is fan-out then convergence: analyses in parallel, then build, then publication.
• A value travels through outputs: the step writes to $GITHUB_OUTPUT, the job promotes it, the consumer declares needs.
• Every output is a string: structures go through compacted JSON and fromJSON, the dynamic matrix pattern.
• Do not route a secret through an output: the job that needs it reads it directly.
• A job whose dependency fails is skipped, not failed, and it propagates that status to its descent.
• always() runs even after a cancellation: acceptable for a report, never for a job that publishes or deploys; prefer !cancelled().
• continue-on-error makes a job look successful to its dependants: to be banned on a security check.
• A job depending on a matrix waits for every combination; its outputs are ambiguous, prefer artifacts.

Next steps

• Reusable workflows: factoring out a graph that repeats from one repository to the next, with its own inputs and outputs.
• Composite actions: grouping a recurring sequence of steps, where a reusable workflow would be oversized.
• Securing GitHub Actions: what the dependency graph changes for security, starting with the jobs that publish.

×

📖 Voir la documentation →