[BBLISA] Backing up sparse files ... VM's and TrueCrypt ... etc

[David Allan]

On Tue, 16 Feb 2010, Edward Ned Harvey wrote:

> Does nobody backup sparse files?  I can?t believe there?s no good way to 
> do it.  Of particular interest, I would like to backup:

Sparse files are sufficiently troublesome to program for that it's 
possible that there's no good way in widely used tools.  I've spent most 
of the last three weeks dealing with them, and they are a real PITA in a 
lot of ways.  Well defined PITA and extremely useful, but there are a lot 
of corner cases.

> I currently have Virtual Machines and TrueCrypt images excluded from the 
> regular Time Machine and Acronis True Image backups of peoples? 
> laptops.  But I?m not comfortable simply neglecting the VM?s and 
> TrueCrypt volumes, as if they?re not important.

Can you run the backup from within the VMs?  That's my preferred 
strategy after ignoring the disk files on the host.

> I?ve also tried rsync.  People all over the place say it should do well, 
> but in practice, I found that doing a single incremental takes 2x longer 
> than doing the whole image.  So again, IMHO, not useful.  Unless I am 
> simply using it wrong.  But I put plenty of effort into making sure I 
> was using it right, so I?m really pretty sure I didn?t get that wrong.

That sounds right, assuming that the size of the backup was the size of 
the data, not the size of the sparse file including the unmapped blocks. 
In order to determine if a particular block should be backed up, the block 
has to be read, then every byte has to be examined to determine if it's 
zero.  If the entire block is zero, then it's considered to be unmapped. 
I can see that taking 2x the time.  Unfortunately, the good answer is 
something like Linux's fiemap ioctl, but that's not supported even on all 
Linux filesystems, let alone Windows and Mac.  They may have an 
equivalent, but I don't know what it is, and whether any backup programs 
use it.

Dave