Date: Wed, 18 Jul 2001 03:00:53 +0300 (EEST) From: Maxim Sobolev <sobomax@FreeBSD.org> To: alex@big.endian.de Cc: small@FreeBSD.org, arch@FreeBSD.org Subject: Re: Extending md(4) to allow it use pre-compressed disk image Message-ID: <200107180000.f6I00pk01057@mail.uic-in.net> In-Reply-To: <20010717194812.A804@zerogravity.kawo2.rwth-aachen.d>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 17 Jul 2001 19:48:12 +0200, Alexander Langer wrote: > Thus spake Maxim Sobolev (sobomax@FreeBSD.ORG): > > > size could vary), compresses each cluster using zlib and writes > > compressed clusters along with information about offset of each > > cluster into resulting image. After that compressed image could be put > > You sure that you don't produce that much overhead with all the > additional gzip headers that it doesn't worse the compression? That depends on how big your "much" is ;). It certainly produces some overhead compared to the plain gzip'ed file due to following reasons: additional index with offset of each compressed cluster (4 bytes per cluster) and clustering, which means that pieces of input treated by zlib as a fully independend data, that obviously limits the compression ratio. The former is rather small, as you can easily calculate with 16KB per cluster there will be only 1280 clusters for a 20MB image, which occupes only 5KB of data, while the latter is much bigger and it significantly depends on the size of cluster used. The following table compares results of compressing 10MB test image, which contains /bin and /etc subdirs from my workstation using different block sizes: Original size of the image: 10485760 bytes Size of the image when compressed with gzip -9: 3855949 bytes Cluster Size of Compressed Overhead Compared Amount ofRAM Size Image to gzip'ed Image Saved ------- ------------------ ----------------- ------------ 512 5,932,330 54% 4,553,430 1024 5,246,779 36% 5,238,981 2048 4,792,210 24% 5,693,550 4096 4,482,894 16% 6,002,866 8192 4,258,122 10% 6,227,638 16384 4,111,334 7% 6,374,426 32768 4,004,817 4% 6,480,943 65536 3,939,527 2% 6,546,233 Both the driver and compression utility support different block sizes, so it is really up to user whether he preffers performance over compression ratio or compression ratio over performance. Obviously that with the larger block sizes the driver will do more and more redundant work because to read some, arbitrary small portion of the cluster the driver has to decompress the whole cluster. However, provided simple MRU cahe effectively masks this effect by holding dozen of the most recently used clusters in decomressed form, so that from the performance standpoint difference between the cluster size of 512 bytes and cluster size of 16KB is ralatively small. > > I would like to know if there is enough interest in integrating this > > feature into base system, please let me know what do you think about > > it. > > Can you tell loader to load a mfs-compressed.gz? (I hope so) Yes, obviously you can, but unfortunately compression ratios achieved by compressing already compressed `mfs-compressed' images are rather low (typically less than 1%), so you wouldn't gain too much this way. :(( Please feel free to contact me with any other questions you might have. I think that among other things this new feature could be very useful to lower down minimal memory requirements during installation process, which could be a concern if we will move away from the plain old sysinstall(8) to a some GUI-based tool. -Maxim To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200107180000.f6I00pk01057>