I host a Minecraft server for myself and some friends. It’s a lot of fun; highlights include an extensive public transport network, several castles, a chalet-under-construction, and some cave networks that are pretty imposing, but still nothing close to what the real world has to offer (Mammoth Cave has 630km of explored passageways!)

Minecraft stores its state as a set of binary files in a directory named “world”:

Our world folder is currently just under 200MB. But there’s a problem – if the Minecraft server is halfway through a write when the box shuts down (maybe because a fuse blows), then you end up with corrupted world state, and have to restore from backup. In the four months or so I’ve been paying attention, this has happened four times.

The naïve approach would be a nightly backup of the full 200MB of world state to a single backup folder. The problem here is that if the worldstate becomes corrupted, but you don’t notice before the next backup runs, then you’re out of luck – your active worldstate is corrupted, and so is your only backup.

A nicer approach would be to retain your old backups – each night, you copy the current state to world_backup_20110728, then 29, then 30, etc. The problem here is that each week you chew up 1.4GB of hard drive space.

So then you could have a backup retention policy based on the good old backup tape rotation approach – keep the 6 most recent daily backups, 4 most recent Friday backups, 12 most recent first-of-the-month backups, and all of your first of January backups, say. But that would be a bit more Powershell excitement than I want a quick backup of a game server to extend to.

The approach I’ve settled with for now is this:

1. Download and install Mercurial (including TortoiseHg).
2. Create a backup folder d:\backups\minecraft\worldrepo
3. Run “hg init” in that folder to initialise the backup folder as a Mercurial repository.
4. Schedule a nightly backup script that copies my world state to that backup folder, and then tells Mercurial to update the repository to match the current working directory:

robocopy "c:\program files (x86)\minecraft server\world" d:\backups\minecraft\worldrepo /s
cd d:\backups\minecraft\worldrepo
d:
hg commit --addremove --user backup --message backup

The result is this:

It’s a Mercurial repository that grows each night only by the size of the individual files that have actually changed in the Minecraft world state (plus around 1Kb of bookkeeping; in practice, if nobody’s done any building in Minecraft, I’m seeing the repo grow by 1 Kilobyte each backup). It’d be rare for one player to affect more than, say, 4 region files in a single session, causing around 12MB growth in the backup repo.

The internals of the Minecraft save format (there’s a step at the end of the persistence pipeline that ZLib compresses each 3d area of the map state) are such that a small change in world state (from the PoV of a player) can easily yield a new binary world file that’s quite different to the original. This means that Mercurial’s not going to be able to efficiently store deltas between worldstate versions for a given area of the map, so the size increase of the repository each time you modify an area of the map and add a new backup to the repo is going to be signficant.

One possible solution to this is counter-intuitive – I believe it would be much more space efficient to instead back up a de-compressed version of the individual *.mcr files, so that Mercurial could better exploit the high-inter-version coherence (on average, the change between versions of an area in Minecraft are very small – a tunnel dug here, a small hut here, etc. etc. - not much major terraforming work gets done unless you’re Chris). Under this approach, the first version you store under Mercurial would be much larger, but the increase in repo size with each version you back up would be significantly lower.

I’d like to play with this idea someday.

In conclusion: Mercurial – is there anything it doesn’t improve? The answer: no.