Syncthing didn’t go down so much as go quiet. The container was still running, but my dashboard flagged it unhealthy, and files had stopped syncing between my machines. A service that’s up but not working is its own kind of annoying, because nothing has obviously crashed to point you at the problem.

Unhealthy, not down

The first useful signal was the container state itself: running, but marked unhealthy. That’s a meaningful distinction. “Down” means the process died. “Unhealthy” means the process is alive but its health check keeps failing, so something is wrong after startup rather than a crash. The logs said exactly what:

failed to load config: XML syntax error on line 132: unexpected EOF

Syncthing was starting, trying to read its config, and giving up because the config didn’t parse.

The config was cut in half

config.xml lives at /opt/syncthing/config/config.xml in my Docker setup. I looked at it:

wc -l /opt/syncthing/config/config.xml

131 lines, about 6KB. A healthy config for this setup is 239 lines and around 12KB. The file ended abruptly partway through an element, with no closing tags, which is exactly what “unexpected EOF on line 132” means: the parser ran off the end of the file while still expecting more. The config hadn’t been edited wrong. It had been cut in half.

Syncthing keeps its own backups

This is the part that turned a bad evening into a ten-minute fix. Syncthing periodically writes backup copies of its config next to the live one, named config.xml.backup.<timestamp>. Listing the directory showed several, and the most recent was the full 239 lines and 12KB. A complete config, from a few hours earlier.

The recovery

The steps were deliberately boring. First, preserve the evidence. Don’t delete the corrupted file, in case you want to see where it got cut:

cp config.xml config.xml.corrupted.$(date +%Y%m%d_%H%M%S)

Then restore the good backup over the live config:

cp config.xml.backup.<timestamp> config.xml

And restart the container. It came back healthy within the health-check window. The web UI and API were listening on port 8384 again, both of my machines reconnected as peers, and the folders went back to syncing. Nothing was lost, because the only thing that had been damaged was the config, and a clean copy was sitting right there.

Why a config file just truncates

The interesting question is how a config file ends up half-written in the first place. Syncthing writes config.xml in place, and it doesn’t do the safe write-to-temp-then-rename dance that protects against interruptions. If the process is killed partway through a write, by a power cut, an ungraceful container stop, or a host reboot that didn’t let it finish, you’re left with a file that’s half the old content and half nothing.

I’d already learned in a previous post that Syncthing owns this file and rewrites it on its own schedule. This is that same behavior with the safety off: it rewrites in place, so an interrupted rewrite corrupts the file instead of leaving the previous version intact. There’s no config setting that changes this; it’s how Syncthing handles its own config. The prevention lives upstream of Syncthing entirely: let the container shut down cleanly before the host reboots, so a write is never left half-done.

Lessons

• Unhealthy is a clue, not a crash. A running-but-unhealthy container means the answer is in the startup and health-check logs, not in whether the process exists.
• Read the parse error literally. “Unexpected EOF on line 132” meant the file ended too early, which pointed straight at truncation instead of a bad edit.
• Syncthing’s own backups are the recovery path. The config.xml.backup.<timestamp> files next to the live config exist for exactly this, and restoring one is the whole fix.
• Preserve the broken file before you overwrite it. Copying the corrupted config aside cost nothing and let me confirm what happened instead of guessing.
• The real fix is a clean shutdown. In-place config writes corrupt on interruption, so prevention is making sure the container stops gracefully before the host goes down.