1 Automation Pipeline
Matt Cupp edited this page 2026-05-29 17:35:45 -04:00

Automation Pipeline (Renovate → Komodo)

Image updates for all managed Docker stacks are handled automatically: Renovate detects newer tags and opens a pull request; you review and merge it; Forgejo fires a webhook that tells Komodo to pull the updated compose file and redeploy.

You never manually pull images or restart containers for managed services.


Architecture

flowchart TD
    A["🕐 Cron on Nexus<br/>top of every hour"]
    B["Renovate container<br/><i>docker compose run --rm renovate</i>"]
    C["Scan all compose files<br/>for newer image tags"]
    D{New tag found?}
    E["Open PR in Forgejo<br/>192.168.1.240:3000"]
    F["You review & merge PR<br/>in Forgejo UI"]
    G["Forgejo push webhook →<br/>Komodo Core<br/>192.168.1.226:9120"]
    H["Stage 1 — Pull Repo<br/><i>git pull</i>"]
    I["Stage 2 — Deploy Stacks (parallel)"]
    J1["dashy"]
    J2["bookstack"]
    J3["tandoor"]
    J4["homeassistant"]
    J5["pinchflat"]
    K["<i>docker compose up -d</i><br/>only restarts containers<br/>whose config changed"]

    A --> B --> C --> D
    D -->|yes| E --> F --> G --> H --> I
    D -->|no| L["No action — try again next hour"]
    I --> J1 & J2 & J3 & J4 & J5
    J1 & J2 & J3 & J4 & J5 --> K

The Four Moving Parts

Component Where Role
Renovate Nexus (Docker container, cron-driven) Scans compose files for newer image tags; opens PRs in Forgejo
Forgejo PVE LXC, 192.168.1.240:3000 Git hosting; fires a push webhook to Komodo when a PR merges to main
Komodo Core Nexus (Docker container), 192.168.1.226:9120 Receives the webhook; orchestrates the deploy procedure
Komodo Periphery Nexus (Docker container, same host) Executes commands on the host: runs git pull and docker compose up -d

For deep configuration details, see the dedicated pages: Renovate · Forgejo · Komodo · SOPS & Secrets


End-to-End Walkthrough

Step 1 — Renovate scans for updates (hourly, automatic)

A cron job on Nexus runs Renovate as a one-shot container at the top of every hour:

cd /home/matt/repos/homelab-docker/renovate
docker compose run --rm renovate

Renovate reads its config from /mnt/server/containers/renovate/config.js (server-side only; not in the repo). It queries Docker Hub and GHCR for all images referenced in the compose files, comparing pinned tags against what's available.

Log file: /mnt/server/containers/renovate/renovate.log

Step 2 — Renovate opens a PR (if an update exists)

If a newer tag exists, Renovate opens a PR in Forgejo. Example:

chore(deps): update ghcr.io/kieraneglin/pinchflat docker tag to v2025.7.1

PR behavior:

  • All PRs require manual approval — automerge is disabled
  • Major version bumps get a needs-manual-review label
  • Max 5 open PRs at once

Step 3 — You review and merge

In Forgejo at http://192.168.1.240:3000/matt/homelab-docker/pulls, review:

  • Check the image changelog (linked in the PR body)
  • Be cautious with needs-manual-review PRs — these are major version bumps
  • Back up before merging Bookstack, Tandoor, or Home Assistant (stateful data; potential breaking changes)
  • Postgres/MariaDB major versions require manual migration steps

Click Merge when satisfied.

Step 4 — Forgejo fires the webhook

A push event on main triggers the webhook configured in the repo settings. Forgejo sends a POST to http://192.168.1.226:9120 (Komodo Core).

To inspect webhook delivery: Forgejo → repo Settings → Webhooks → recent deliveries. A 200 or 204 response means Komodo accepted it.

Step 5 — Komodo Stage 1: Pull Repo

Komodo Periphery runs:

git -C /home/matt/repos/homelab-docker pull

The "Latest Commit" output in the Komodo run history should show the HEAD SHA that just merged on Forgejo. If it shows an older SHA, the pull didn't reach Forgejo — see Troubleshooting.

Step 6 — Komodo Stage 2: Deploy Stacks (parallel)

Five Deploy Stack actions run in parallel, each executing:

docker compose up -d

in its respective service directory under /home/matt/repos/homelab-docker/.

docker compose up -d only restarts a container if its configuration changed — if only pinchflat's image tag was bumped, only the Pinchflat container restarts. Dashy, Bookstack, etc. are untouched.

Step 7 — Verify

Open http://192.168.1.226:9120Procedures → the deploy procedure → run history.

  • Procedure should complete in a few seconds (not ~1ms — see Troubleshooting)
  • Stage 1 shows the expected commit SHA
  • Stage 2 shows 5 green Deploy Stack actions

What Doesn't Auto-Deploy and Why

Service Managed by Komodo? Reason
dashy, bookstack, tandoor, homeassistant, pinchflat Yes Main 5 stacks in the deploy procedure
dispatcharr Yes (via LXC 113 Periphery) Runs on PVE LXC 113, not Nexus; deployed through a separate Periphery agent
renovate No Including it would trigger an unwanted scan on every push to main
komodo Never Would kill itself mid-deploy. Update via SSH only.
frigate, wyze-bridge, mosquitto Not in auto-deploy Camera/IoT stack; managed separately. Frigate has no semver tags on latest.

Renovate note for Dispatcharr: ghcr.io/dispatcharr/dispatcharr only publishes latest — no semver tags. Renovate will never open a PR for it. Check for updates manually.


Images Do Not Auto-Pull

Komodo does not pull new Docker images on its own. A container only gets a new image when:

  1. The image tag in its compose file changes (via a Renovate PR or manual edit), and
  2. A deployment runs (webhook or manual trigger)

docker compose up -d will pull the new tag at that point. This keeps all versions pinned and auditable through git history.


Troubleshooting

Procedure completes in ~1ms and nothing deployed

Stage 2 is misconfigured. It must have 5 explicit Deploy Stack actions. If it uses "Batch Deploy Stack If Changed," that action always skips — stacks have no linked repo hash to compare against.

Fix: Komodo UI → Procedures → edit → Stage 2 → replace with 5 individual Deploy Stack actions: dashy, bookstack, tandoor, homeassistant, pinchflat.

Pull Repo fails — "fatal: not a git repository"

Git is rejecting the path due to ownership mismatch. Periphery runs as root but the repo is owned by matt. Check komodo/.gitconfig (mounted as /root/.gitconfig in the container):

[safe]
    directory = /home/matt/repos/homelab-docker
    directory = *

The directory = * wildcard is recommended. After editing the file, the change is live immediately (no container restart needed). Confirm via Komodo UI → Repos → homelab-docker → Pull.

Pull succeeds but stacks didn't update

Expand Stage 1 in the run history and check the "Latest Commit" output. If the SHA doesn't match Forgejo's current HEAD, git pulled but couldn't reach the remote. Verify network reachability from Nexus to Forgejo (192.168.1.240:3000).

Also confirm each Stack's run_directory points to /home/matt/repos/homelab-docker/<service>.

Webhook didn't fire after merge

  1. Forgejo → repo Settings → Webhooks → recent deliveries — look for failed deliveries
  2. Confirm Komodo is reachable from PVE LXC: curl http://192.168.1.226:9120
  3. Manual deploy fallback:
    ssh matt@192.168.1.226 'cd /home/matt/repos/homelab-docker/<service> && docker compose up -d'
    

Service broken after update — rollback

# Restore the old tag in the compose file on Nexus, then redeploy
ssh matt@192.168.1.226 'nano /home/matt/repos/homelab-docker/<service>/docker-compose.yml'
ssh matt@192.168.1.226 'cd /home/matt/repos/homelab-docker/<service> && docker compose up -d'
# Then commit the rollback in git so Komodo and the repo stay in sync

Manually trigger the deploy procedure (API)

# Full deploy (both stages)
curl -s -X POST "http://192.168.1.226:9120/execute/RunProcedure" \
  -H "x-api-key: <key>" \
  -H "x-api-secret: <secret>" \
  -H "Content-Type: application/json" \
  -d '{"procedure":"<procedure-id>"}'

API keys: Komodo UI → user profile → API Keys. Each key generates a key (K_…) and a secret (S_…) — save both immediately.