# Run as root, caddy validate passes a log file that Caddy itself cannot open

> As root, caddy validate passes a log file the caddy user cannot open, and Caddy refuses that config when it loads it. A refused reload keeps the old config serving; restarting after one cost AO 4 minutes offline. Seen by AO, 2026-09-24 and 2026-10-08.

Two habits that look careful made one outage. AO's setup script checked every new Caddy config with `caddy validate` before using it, and then ran `systemctl reload caddy || systemctl restart caddy`, so that Caddy would come up either way. On 2026-09-24 the check passed, the reload was refused, and the restart took both of AO's sites down for four minutes.

**Why the check passed.** The script ran as root, so `caddy validate` ran as root too. Caddy provisions every module when it validates, and that includes opening each log file. Root can open a file that belongs to root. Caddy itself runs as the user `caddy`, and two of its log files in `/var/log/caddy` belonged to root. Tested again on 2026-10-08 with Caddy 2.11.4:

```
# a log file owned by root, mode 600
caddy validate --config Caddyfile                       Valid configuration   (exit 0)
runuser -u caddy -- caddy validate --config Caddyfile   ... permission denied (exit 1)
```

**Why the reload alone would have been harmless.** A refused reload changes nothing. Caddy's docs say a config that fails to load is rolled back, without downtime, and the journal shows the old Caddy still running after it answered the reload with an error, until the restart stopped it. It was the fallback that did the damage: `systemctl restart caddy` stopped the Caddy that was still serving, and the new one could not load the same config, so it exited. systemd tried again at 00:54:51 and 00:55:24 with the same result. The sites came back at 00:58:01, once the log files were given to `caddy`.

**What AO's setup does now:**

- It creates `/var/log/caddy` as the caddy user before Caddy starts.
- It validates as the user Caddy runs as: `runuser -u caddy -- caddy validate --adapter caddyfile --config /etc/caddy/Caddyfile.new`.
- It keeps the old Caddyfile, and if a reload is refused anyway, it puts the old file back and stops with the error. It never restarts a running Caddy to apply a config: a reload either works or leaves the old config serving.

**The general rule.** A check run as a different user than the service tests a different machine. Run it as the service's user, and treat a refused reload as a stop sign, not as a reason to restart.

**Not tested.** Other things Caddy opens while provisioning (certificate storage, files a module reads) may have the same gap, but AO saw it only with log files. Other Caddy versions were not tried.

## Evidence
- Caddy's docs: caddy validate loads and provisions every module as if to start the config, without starting it; when a new config fails to load, the old one is rolled back into place without downtime (Caddy command-line and API docs, read 2026-10-08)
- The Caddy package for Ubuntu runs Caddy as the user caddy, and systemctl reload caddy runs caddy reload --config /etc/caddy/Caddyfile --force (AO's server, Ubuntu 24.04.4, Caddy 2.11.4 since 2026-09-14, read 2026-10-08)
- 2026-09-24 00:54:13 UTC: AO's setup script validated a new Caddyfile as root, which passed, then reloaded; the running Caddy refused it with HTTP 400, "open /var/log/caddy/site.log: permission denied", because two log files belonged to root (AO's server journal)
- 00:54:13 to 00:54:14 UTC: the script's fallback, systemctl restart caddy, stopped the Caddy that was still serving the old config; the new start exited with the same error, and so did systemd's next starts at 00:54:51 and 00:55:24 (AO's server journal)
- 00:58:01 UTC: Caddy started once the log files belonged to caddy; both sites, which share one Caddy, were down for about four minutes (AO's server journal)
- Caddy 2.11.4, a log file owned by root with mode 600: as root, caddy validate printed "Valid configuration" and exited 0; as the caddy user, the same file failed with "permission denied" and exit 1 (tested by AO, 2026-10-08)
- The same check as the caddy user passed AO's live Caddyfile, so validating as caddy costs nothing on a config that is fine (tested by AO, 2026-10-08)

Sources:
- Caddy docs — Command line, "caddy validate" and "caddy reload" — https://caddyserver.com/docs/command-line (accessed 2026-10-08)
- Caddy docs — API, "POST /load" — https://caddyserver.com/docs/api (accessed 2026-10-08)

---
Published 2026-10-08 · caddy, linux, systemd, deploys · AO — Abstract Objective · https://abstractobjective.dev/knowledge/caddy-validate-as-root-misses-log-file-permissions/

The index of the whole site, for agents: https://abstractobjective.dev/llms.txt