Date: Sat, 27 Jan 2007 14:41:20 -0800 From: Bakul Shah <bakul@bitblocks.com> To: Peter Jeremy <peterjeremy@optushome.com.au> Cc: freebsd-current@freebsd.org Subject: Re: Interesting speed benchmarks Message-ID: <20070127224120.6CFCE5B32@mail.bitblocks.com> In-Reply-To: Your message of "Sun, 28 Jan 2007 08:42:34 %2B1100." <20070127214234.GO927@turion.vk2pj.dyndns.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Peter Jeremy <peterjeremy@optushome.com.au> wrote:
> On Sat, 2007-Jan-27 11:54:37 -0500, Brian Reichert wrote:
> >On Sat, Jan 27, 2007 at 09:43:52AM +1100, Peter Jeremy wrote:
> >> Note that dump/restore is the only tool that can correctly reproduce
> >> sparse files. tar, cpio and pax also have filename and file size
> >> restrictions. I don't think that cpio or pax support ACLs or file
> >> flags.
> >
> >I thought 'star' handled sparse files and all the extra magic? Not
> >that this is germaine to the topic at hand...
>
> You can detect a sparse file by comparing the length of the file
> with the number of allocated blocks. You can't determine whether a
> specific block is all NULLs or not allocated without either bypassing
> the filesystem (which dump does) or using a syscall to retrieve a
> bitmap of allocated blocks (which FreeBSD doesn't have). =20
Both rsync and gtar create a sparse file by not writing
blocks of zeroes given -S option. You are correct that there
is no special help for reading sparse files. I recall
pjd@freebsd.org saying he plans to implement solaris specific
extensions to lseek() in zfs (lseek's whence arg can take
SEEK_HOLE/SEEK_DATA values, to seek to next null/nonnull
block). It would be nice if this was done for all
filesystems. Returning a bitmap is far more work than adding
SEEK_{HOLE,DATA}.
For backups I use vbackup & venti from plan9ports. Like dump
vbackup knows ufs1 & ufs2 structure to walk the filesytem and
write only live data to venti (a SHA1-addressed block storage
server) so it correctly handles sparse files. A full backup
is faster than dump. All subseqeuent backups are incremental
and generally run very fast (for example I can backup a UFS2
system at about 10MB/s to a USB2 ATA disk. dd shows write
speed of 23MB/s).
You can get at backed up data in a number of different ways:
1) vcat (makes an identical disk image given a snapshot's
`score')
2) vftp (you can browse/copy files individually)
3) vnfs + vmount (make one or more snapshots accessible in
filesystem namespace)
AFAIK flags, attributes & ACLs are maintained but are only
available if you use vcat. See http://swtch.com/plan9ports
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070127224120.6CFCE5B32>