Time Machine Exclusions
So Time Machine is a pretty convenient way to backup your machine, and I use it to backup my laptop to an external FireWire drive. Although Time Machine backs up the “whole system,” I assumed there had to be some exclusions, such as cache files or /dev, for example. After a short bit of digging, I discovered that these paths are specified in a standard property list at /System/Library/CoreServices/backupd.bundle/Resources/StdExclusions.plist
The full list is some 57 items, and is available below. Besides the expected cache items, the list includes some items I thought interesting:
/home */Library/Logs /Users/Guest /Library/Safari/Icons.db /.Spotlight-V100
Of these, the only one I find odd is the exclusion of logs. If your system goes haywire such that you restore it completely from a backup, it might be nice to see what went wrong.
Here’s the full list of the excluded paths:
Contents Excluded
The contents of these paths are excluded, but the directories themselves are preserved as they are required for a successful restore.
/Volumes /Network /automount /.vol /tmp /cores /private/tmp /private/Network /private/tftpboot /private/var/automount /private/var/log /private/var/folders /private/var/log/apache2 /private/var/log/cups /private/var/log/fax /private/var/log/ppp /private/var/log/sa /private/var/log/samba /private/var/log/uucp /private/var/run /private/var/spool /private/var/tmp /private/var/vm /private/var/db/dhcpclient /private/var/db/fseventsd /Library/Caches /Library/Logs /System/Library/Caches /System/Library/Extensions/Caches
Paths Excluded
These directories and their contents are excluded completely.
/.Spotlight-V100 /.Trashes /.fseventsd /.hotfiles.btree /Backups.backupdb /Desktop DB /Desktop DF /Network/Servers /Previous Systems /Users/Shared/SC Info /Users/Guest /dev /home /net /private/var/db/Spotlight /private/var/db/Spotlight-V100
User Paths Excluded
These directories are and their contents are excluded per-user. The path is relative to the user’s home folder.
Library/Application Support/MobileSync Library/Application Support/SyncServices Library/Caches Library/Logs Library/Mail/Envelope Index Library/Mail/AvailableFeeds Library/Mirrors Library/PubSub/Database Library/PubSub/Downloads Library/PubSub/Feeds Library/Safari/Icons.db Library/Safari/HistoryIndex.sk
It’s even worse than that. Today I did a full restore of my boot disk (by booting from the installation CD and selecting Restore in the Tools menu), and after that Apache2 keeps spurning out error messages that it can find its log files. Indeed, there is no Apache2 subdirectory anymore in the /var/logs directory and apparently Apache can’t create it…
Comment by menneke — November 15, 2007 @ 4:59 pm
Is your /var a symlink to /private/var? The log directory for apache2 (at least according to TimeMachine) is /private/var/log/apache2, is that the directory that your apache2 is trying to create? Furthermore, are you using the built in apache, or a copy obtained elsewhere?
Comment by Devin Lane — November 16, 2007 @ 2:01 am
Same problem here. Yes, “/var -> private/var”. Apache, postfix, and I think some others fail after restore, due to missing directories they’re not willing to recreate. Indications here:
http://discussions.apple.com/thread.jspa?messageID=6367763#6367763
suggest this is an intermittent problem: some restores work, some fail. creepy.
Comment by jackr — January 18, 2008 @ 1:14 am
I see that StdExclusions.plist “COntentsExcluded” lists /provate/var/log. It also lists /private/var/log/apache2, as if that directory might be backed up and then restored, but looking into my backup volume, I don’t see it anywhere. I suspect that the exclusion of the contents of /private/var/log/ is excluding even the directories for things like /private/var/log/apache2.
Comment by jackr — January 18, 2008 @ 1:43 am
Well, at least they had the decency to put them in a plist for us sysadmins…not that these exclusions are a big deal for ma+pa on their Time Capsule.
I think they leave out the logs because they’re supposed to be backed up for a week or two anyway, albeit not on a secondary drive.
Comment by Bug — February 5, 2008 @ 1:02 am
/home
WTF?
Am I missing something because I’m thinking Apple is stoopid.
Comment by terris — February 13, 2008 @ 10:57 pm
Terris– /Users has all the account info.
Comment by Matt — March 5, 2008 @ 12:49 am
The file is actually at /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist (the “Contents” part of the path is missing from this blog post).
Comment by Barry K. Nathan — March 29, 2008 @ 8:01 pm
I just deleted the StdExclusions.plist. My backup is 600mb bigger. I am going to replace my HD on Monday, we’ll see when I restore it if I’ll have all my cache’s and DB’s.
Comment by Matthew Miller — April 19, 2008 @ 7:46 am
Update 1: I looked at my Time Machine Backup, all those paths/caches/DB’s now appear in my Time Machine Backup
Comment by Matthew Miller — April 19, 2008 @ 7:49 am
I just commented out all the exclusions in the plist file and the resulting backup.inProgress is currently 58GB of data. But I’d be wrong since it’s rewriting all the symlinks to the correct files. Right?
I’m assuming that this is the size of the (usually) excluded files on my system. Wow.
See progress image here: http://flickr.com/photos/luxuryluke/2438550718/
Comment by Luke Dorny — April 24, 2008 @ 4:02 am
[...]resource [...]
Comment by acer — July 4, 2008 @ 9:20 pm