2 Forgejo
Matt Cupp edited this page 2026-07-27 14:53:24 -04:00

Forgejo

Self-hosted git forge. Stores all Docker Compose stacks, Renovate config, and personal docs. The webhook from homelab-docker drives Komodo auto-deploys on every push to main.

Where it runs

  • Host: PVE LXC 100 (alpine-forgejo, 192.168.1.240) — Alpine Linux, so service management is rc-service, not systemctl
  • UI: http://192.168.1.240:3000
  • Created via: ProxmoxVE community script
  • Binary: /usr/bin/forgejo — installed manually (official release binary); the Alpine apk package is stale and unused
  • Version: 16.0.1 (upgraded 2026-07-27 from 15.0.2, issue #204)
  • Config file: /etc/forgejo/app.ini (inside the LXC)
  • Database: SQLite at /var/lib/forgejo/db/forgejo.db (inside the LXC)

Repositories

Repo Purpose
matt/homelab-docker Docker Compose stacks; webhook triggers Komodo deploy on push to main
matt/renovate Renovate global config (the server-side config.js is on Nexus at /mnt/server/containers/renovate/config.js)
matt/docs Personal documentation

Backup

The Forgejo LXC is included in a PBS backup job that runs nightly. This is the most important backup in the homelab — Forgejo holds all commit history and automation config. Verify it is in the backup job via PVE → Datacenter → Backup.

Wiki

The wiki for homelab-docker is a git repo. Clone it locally to edit:

git clone http://192.168.1.240:3000/matt/homelab-docker.wiki.git

Edit markdown files, commit, and push to publish. Changes appear immediately.

Webhook (homelab-docker → Komodo)

Configured on matt/homelab-docker repo → Settings → Webhooks.

Field Value
Target URL http://192.168.1.226:9120/listener/github/procedure/<procedure-id>/main
Content type application/json
Secret Must match KOMODO_WEBHOOK_SECRET in Komodo's compose.env on Nexus
Events Push only

The /main suffix is a branch filter — Komodo only acts on pushes to main. The procedure ID is the MongoDB ObjectId; find it in Komodo UI → Procedures → the procedure → Config tab → Webhook section.

Note: New Komodo procedures default to webhook_enabled: false. After creating the procedure, open its Config tab and enable the "Webhook Enabled" toggle, then save. Without this, Komodo authenticates the request but silently does nothing.

Check delivery log: repo → Settings → Webhooks → click the webhook → Recent Deliveries.

Useful commands

Run from PVE shell or via SSH to PVE (root@192.168.1.227):

# Check Forgejo service status (Alpine — rc-service, not systemctl)
pct exec 100 -- rc-service forgejo status

# Enter the LXC interactively
pct enter 100

# List Forgejo users (without entering the LXC)
ssh root@192.168.1.227 "pct exec 100 -- forgejo admin user list"

# Restart Forgejo
ssh root@192.168.1.227 "pct exec 100 -- rc-service forgejo restart"

Upgrading

Forgejo is a single static binary; upgrades are a binary swap. Downgrades are NOT supported once the DB migrates, so the PBS backup is the rollback path.

# 1. Fresh PBS backup (fast — incremental on top of nightly)
ssh root@192.168.1.227 'vzdump 100 --storage pbs-backup --mode snapshot'

# 2. Download + verify new release inside the LXC (substitute version)
ssh root@192.168.1.227 'pct exec 100 -- sh -c "cd /root &&
  wget -q https://codeberg.org/forgejo/forgejo/releases/download/vX.Y.Z/forgejo-X.Y.Z-linux-amd64{,.sha256} &&
  sha256sum -c forgejo-X.Y.Z-linux-amd64.sha256"'

# 3. Swap binary and restart (SQLite migration runs on first start)
ssh root@192.168.1.227 'pct exec 100 -- sh -c "rc-service forgejo stop &&
  cp -a /usr/bin/forgejo /root/forgejo-OLD.bak &&
  install -o root -g www-data -m 755 /root/forgejo-X.Y.Z-linux-amd64 /usr/bin/forgejo &&
  rc-service forgejo start && sleep 8 && forgejo --version"'

# 4. Verify, then delete the downloaded artifact (1G rootfs — keep it lean)
curl -s http://192.168.1.240:3000/api/v1/version

Check the release notes for breaking changes first. After upgrading, verify the deploy pipeline end-to-end: push a trivial commit to main (bump the # test: comment in dashy/docker-compose.yml) and confirm Komodo runs Deploy Stacks within seconds.

Why we left 15.0.2: its merge endpoint intermittently half-completed — merge commit pushed but PR left unmerged and no push webhook fired, so Komodo silently skipped deploys (PRs #189, #200; issue #204). Safety nets if it recurs: Renovate patch automerge pushes directly to main (ignoreTests), and the "Deploy Stacks" procedure also runs daily at 7:30am as reconciliation.

Troubleshooting

Forgejo is unreachable

SSH to PVE and restart the service:

ssh root@192.168.1.227 "pct exec <lxc-id> -- systemctl restart forgejo"

If the LXC itself is down: pct status <lxc-id> and pct start <lxc-id> from the PVE shell.

Webhook not firing

Check the delivery log: repo → Settings → Webhooks → Recent Deliveries. The error detail is shown per delivery. Common causes:

  • Komodo is down on Nexus (docker ps | grep komodo on Nexus)
  • Webhook secret mismatch between Forgejo and Komodo's compose.env
  • Procedure has webhook_enabled: false (enable in Komodo UI → Procedures → Config tab)