The dangers of rsync into a full filesystem
Friday, December 10th, 2010I have an overly-complex and brittle chain of storage between my MythTV server and the machine that does playback onto the TV screen; one of the steps between them is a regular rsync of files from one machine to another — unfortunately disk usage on “the other” is not well controlled.
This means that I often end up with a full filesystem at one end, and I’m trying to stuff more files into it. And because I’m invoking rsync from cron instead of doing it properly and invoking it from a ‘while true; do …; sleep; done’ loop script, I end up with multiple rsync’s running.
I haven’t dissected the behaviour yet, but I’ve found a whole load of quite sizeable rsync temporary files (”.<filename>.randomsuffix”). I’m guessing that these exist because the filesystem keeps on filling during these rsync runs, and somehow rsync isn’t managing to delete the temp files when this happens.
I’d be happy to accept that rsync would normally try to delete temp files when things go wrong, but with potentially multiple simultaneous rsyncs banging on, over an NFS link to an embedded machine with low disk performance, I guess that there are some timeout issues; added to which occasionally I have to kill rsyncs at the top end of the chain.
So just a warning; if you rsync badly (i.e. over cron) into a filesystem that is prone to filling up, please remember to look out for temp files while you are cleaning up!