Release & Deployment Pipeline¶

One place to see how a change becomes a published release. Everything from a merged PR to multi-arch Docker images, an enriched GitHub Release, an SBOM, and the live docs site — as a single directed flow instead of six workflow files you have to read side-by-side.

• Release orchestration: release-please.yml
• Deployment (images): release.yml
• Deployment (docs): docs.yml

For the PR-check DAG (the 8 required checks that gate a merge) see the companion CI/CD Pipeline page — this page picks up after a PR is green and merged.

The whole flow, in one DAG¶

flowchart TD
    %% ---------- 1 · change sources ----------
    subgraph S1["1 · Change sources"]
      DB["Dependabot
npm daily · actions + docker weekly
safe updates auto-merge"]
      HUM["Human PR"]
    end

    %% ---------- 2 · PR gate ----------
    PR{{"pr.yml · PR gate
8 required checks:
Build · Test · Docs · CodeQL
Trivy · Licenses · Sonar · E2E"}}
    DB --> PR
    HUM --> PR
    PR -->|squash merge| MAIN[("main")]

    %% ---------- 3 · on push to main ----------
    MAIN --> RP["release-please.yml
on push: main"]
    MAIN --> DOCS1["docs.yml
on push: main"]
    RP --> RPJOB["job: release-please
create / update Release PR
bump version + CHANGELOG"]
    RP --> BADGE["job: update-badge
unit + e2e counts → gist badges"]

    %% ---------- 4 · cut the release ----------
    RPJOB --> RELPR["Release PR
chore(main): release X.Y.Z"]
    RELPR -->|merge| TAG["release-please creates
tag vX.Y.Z + GitHub Release"]

    %% ---------- 5 · tag-triggered deploy ----------
    TAG --> REL["release.yml
on push: tag v*"]
    TAG --> DOCS2["docs.yml
on push: tag v* @ released SHA"]

    subgraph S5["release.yml — prepare → build per architecture → publish"]
      direction TB
      B0["prepare
release mode · tag = manifest · CHANGELOG entry"]
      B0 --> B1A["build amd64 · ubuntu-24.04
push untagged by digest"]
      B0 --> B1R["build arm64 · ubuntu-24.04-arm
push untagged by digest"]
      B1A --> B2A["smoke test the amd64 candidate
image contents E2E"]
      B1R --> B2R["smoke test the arm64 candidate
image contents E2E"]
      B2A --> B3["publish: tag the multi-arch
index on GHCR · REQUIRED"]
      B2R --> B3
      B3 --> B4["regctl promote → Docker Hub
best-effort"]
      B3 --> B5["SBOM (anchore)"]
      B3 --> B6["enrich GitHub Release notes"]
      B4 --> B7["update Docker Hub description
best-effort"]
    end
    REL --> B0

    %% ---------- 6 · outputs ----------
    B3 --> O1[["GHCR image
ghcr.io/sergienko4/israeli-bank-importer:vX.Y.Z"]]
    B4 --> O2[["Docker Hub image
sergienko4/israeli-bank-importer:vX.Y.Z"]]
    B5 --> O3[["SBOM (sbom.spdx.json)"]]
    B6 --> O4[["GitHub Release (enriched notes)"]]
    DOCS1 --> O5[["Docs site · GitHub Pages"]]
    DOCS2 --> O5

    classDef req fill:#1f7a1f,stroke:#0d3d0d,color:#fff;
    classDef best fill:#7a5c00,stroke:#3d2e00,color:#fff;
    classDef out fill:#0b3d91,stroke:#052247,color:#fff;
    class B3,O1 req;
    class B4,B7,O2 best;
    class O3,O4,O5 out;

Legend: green = required (release fails if it fails) · amber = best-effort (logged as a warning, never fails the release) · blue = published output.

At a glance¶

release.yml jobs¶

Three jobs. Nothing is tagged until both architectures passed their smoke test, so a broken image never reaches a user.

1. prepare resolves the release mode (below). For a tag it checks out the tag, verifies it matches config/release-please/manifest.json, and verifies the CHANGELOG entry.
2. build, one leg per architecture, each on its own native runner (ubuntu-24.04 for amd64, ubuntu-24.04-arm for arm64). A failed leg cancels the other (fail-fast). Each leg:
3. builds via docker/build-image and pushes the image to GHCR untagged, by digest;
4. pulls that exact candidate and runs the image contents E2E (tests/e2e/image-contents.e2e.test.ts) against it. Among other things, that test launches Camoufox and checks the scraper API, the font set and the Camoufox layer. Tags built before that test existed (up to v1.43.8) get the older createScraper().scrape check instead;
5. logs the candidate's compressed layer sizes and the runner's free disk;
6. uploads the candidate digest as an artifact.
7. publish runs only when both legs passed. It checks that exactly two digests arrived and joins them into one multi-arch index with docker buildx imagetools create, which writes the GHCR tags. This is the required publish target. Then:
8. promote GHCR → Docker Hub with regctl image copy (best-effort). regctl is used instead of a buildx push because Docker Hub's edge proxy rejects BuildKit's concurrent multi-arch uploads;
9. SBOM (anchore, scanned from the GHCR image) → enriched release notes → attach SBOM + notes to the GitHub Release → update Docker Hub description (best-effort).

GHCR is primary and required; Docker Hub is a best-effort mirror. If Docker Hub auth or push fails, the release still succeeds and the notes advertise only the GHCR pull command.

Each job gets only the token permissions it needs: build can write packages, and only publish can write release contents.

Release modes¶

Rehearsing a release¶

Run the full build, smoke test and index publish from a branch without releasing anything:

gh workflow run release.yml --ref <branch> -f rehearsal=true

The run tags only ghcr.io/sergienko4/israeli-bank-importer:rehearsal. Run one rehearsal at a time: the workflow has no concurrency group, and two runs would race for that tag.

Why the build is fast¶

• Native runners. arm64 builds on an arm64 runner instead of under QEMU emulation on amd64, and both legs run in parallel.
• No layer cache. Exporting the GHA layer cache cost about 170 s per release build, more than the layers it saved, so no release build reads or writes one.
• A reproducible Camoufox layer. The Dockerfile installs Camoufox into a staging root with fixed owners, modes and timestamps, so a release that keeps the same Camoufox pin and base image produces the same layer digest. GHCR and Docker Hub then skip that layer on push, and users skip it on pull: it is about 600 MB of the roughly 830 MB compressed image per architecture.
• Only the Windows font groups. The image ships only the font groups that the default Windows fingerprint reads; see CAMOUFOX_OS.

Reused building blocks (composite actions)¶

Every build path funnels through the same composites, so a fix lands once and propagates everywhere.

flowchart LR
    subgraph BLD["docker/build-image"]
      CAM["docker/camoufox-cache"]
      BX["docker/setup-buildx"]
      BP["docker/build-push-action
SKIP_BROWSER_FETCH, no layer cache"]
    end

    SETUP["ci/setup
node + npm ci"]
    TRIVY["security/trivy-scan"]

    RELW["release.yml"] --> BLD
    E2EW["_e2e-suite.yml"] --> BLD
    E2EW --> SETUP
    PRW["pr.yml"] --> BLD
    PRW --> TRIVY
    PRW --> SETUP
    RPB["release-please.yml
update-badge"] --> SETUP
    RPB --> CAM
    DOCSW["docs.yml"] --> SETUP

_e2e-suite.yml is the only reusable workflow (called by pr.yml and e2e-schedule.yml); it builds the :e2e image, seeds a budget, runs the scraper smoke + import runs, then the full E2E suite. update-badge needs docker/camoufox-cache because portal-ui.e2e.test.ts launches the product's Camoufox browser and never skips.

Secrets by workflow¶

No secrets: inherit anywhere — every reusable-workflow call maps secrets explicitly.

Publish targets¶

Both image tags are byte-identical multi-arch manifest lists (linux/amd64, linux/arm64).

Dependency bumps always cut a release¶

A dependency that ships inside the published image changes what users actually run, so it must reach them as a tagged release — never as a silent merge.

Two mechanisms keep this true:

1. .github/dependabot.yml sets prefix/prefix-development so runtime bumps arrive already titled fix(deps). It deliberately omits include: scope, which used to append a second scope (chore(deps)(deps):) that the conventional-commit parser cannot read — so release-please dropped those commits entirely.
2. The Release signal guard step in pr.yml → Build & Audit fails any PR that changes dependencies, overrides or the Dockerfile FROM base image under a non-releasing title. Configuration alone is not a guarantee; the guard is the enforcement half. The Dockerfile is covered because Dependabot's docker ecosystem never touches package.json, so a manifest comparison alone would let a base-image bump merge silently. Policy and messages live in scripts/release-signal-logic.mjs, exercised end to end by tests/e2e/release-signal.e2e.test.ts.

Before this, runtime bumps merged as chore(deps) and produced no version bump, no tag and no image — scraper updates had to be retitled by hand.

typescript majors are pinned back by an ignore entry: TypeScript 7 is rejected by the typedoc and typescript-eslint peer ranges, so npm ci fails with ERESOLVE before any job runs. Remove the entry once both peers publish stable TS 7 support.

Cutting a release (operator steps)¶

1. Merge feature PRs to main with conventional-commit titles (feat: minor, fix:/refactor: patch). release-please keeps a Release PR up to date automatically.
2. When ready to ship, merge the Release PR (chore(main): release X.Y.Z).
3. release-please tags vX.Y.Z and opens a GitHub Release; the tag push then fires release.yml (images) and docs.yml (docs) in parallel.
4. Confirm the Docker Build & Publish run is green and pull ghcr.io/sergienko4/israeli-bank-importer:vX.Y.Z.

If the deploy fails¶

• A smoke test or build leg failed — nothing was tagged. The candidate stays untagged on GHCR, users see no change. Fix and re-run, or re-run the failed jobs if the failure was transient.
• Docker Hub push failed — non-fatal by design; the image is on GHCR. Pull from GHCR or re-run to retry the mirror.
• Tag vX.Y.Z does not match manifest — the tag was pushed out of band; it must equal config/release-please/manifest.json.