From owner-svn-src-all@FreeBSD.ORG Wed Apr 2 16:40:16 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A7A26DF; Wed, 2 Apr 2014 16:40:16 +0000 (UTC) Received: from mail.lifanov.com (mail.lifanov.com [206.125.175.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0AD1D9C9; Wed, 2 Apr 2014 16:40:15 +0000 (UTC) Received: by mail.lifanov.com (Postfix, from userid 58) id 4F5BD1AF4D4; Wed, 2 Apr 2014 12:40:15 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mail.lifanov.com X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT shortcircuit=ham autolearn=disabled version=3.4.0 Received: from [10.1.3.5] (cnet520-windstream.mcclatchyinteractive.com [166.108.16.2]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.lifanov.com (Postfix) with ESMTPSA id 6C9F01A5943; Wed, 2 Apr 2014 12:40:14 -0400 (EDT) Message-ID: <533C3D6D.3060604@mail.lifanov.com> Date: Wed, 02 Apr 2014 12:40:13 -0400 From: Nikolai Lifanov User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Glen Barber Subject: Re: svn commit: r264027 - in head: release share/man/man7 References: <201404012241.s31MfRW6020684@svn.freebsd.org> <20140402154022.GA70867@lor.one-eyed-alien.net> <20140402155134.GG14379@glenbarber.us> <533C32F5.9050809@mail.lifanov.com> <20140402160650.GH14379@glenbarber.us> <533C3992.9030203@mail.lifanov.com> <20140402163350.GI14379@glenbarber.us> In-Reply-To: <20140402163350.GI14379@glenbarber.us> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, Brooks Davis , src-committers@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 16:40:16 -0000 On 04/02/14 12:33, Glen Barber wrote: > On Wed, Apr 02, 2014 at 12:23:46PM -0400, Nikolai Lifanov wrote: >> On 04/02/14 12:06, Glen Barber wrote: >>> On Wed, Apr 02, 2014 at 11:55:33AM -0400, Nikolai Lifanov wrote: >>>> On 04/02/14 11:51, Glen Barber wrote: >>>>> On Wed, Apr 02, 2014 at 10:40:22AM -0500, Brooks Davis wrote: >>>>>> On Tue, Apr 01, 2014 at 10:41:27PM +0000, Glen Barber wrote: >>>>>>> Author: gjb >>>>>>> Date: Tue Apr 1 22:41:26 2014 >>>>>>> New Revision: 264027 >>>>>>> URL: http://svnweb.freebsd.org/changeset/base/264027 >>>>>>> >>>>>>> Log: >>>>>>> Add a new release build variable, WITH_COMPRESSED_IMAGES. >>>>>>> >>>>>>> When set to a non-empty value, the installation medium is >>>>>>> compressed with gzip(1) as part of the 'install' target in >>>>>>> the release/ directory. >>>>>>> >>>>>>> With gzip(1) compression, downloadable image are reduced in >>>>>>> size quite significantly. Build test against head@263927 >>>>>>> shows the following: >>>>>>> >>>>>>> bootonly.iso: 64% smaller >>>>>>> disc1.iso: 44% smaller >>>>>>> memstick.img: 47% smaller >>>>>>> mini-memstick.img: 65% smaller >>>>>>> dvd1.iso: untested >>>>>>> >>>>>>> This option is off by default, I would eventually like to >>>>>>> turn it on by default, and remove the '-k' flag to gzip(1) >>>>>>> so only compressed images are published on FTP. >>>>>> >>>>>> I'd recommend testing xz compression as well. With UFS images of a full >>>>>> world the savings vs gzip are significant (more than 30% IIRC, but it's >>>>>> need more than a year since I checked so I'm a bit unsure of the exact >>>>>> numbers). >>>>>> >>>>> >>>>> delphij also brought this up. >>>>> >>>>> I have concerns with xz(1), since there was mention in IRC that Windows >>>>> users may have problems decompressing xz-compressed images. So, gzip(1) >>>>> is used because it seems to be the more commonly-supported archive >>>>> mechanisms. >>>>> >>>>> The benefit of xz(1) over gzip(1) was only 50M-ish. >>>>> >>>>> -rw-r--r-- 1 root wheel 601M Mar 28 20:18 disc1.iso >>>>> -rw-r--r-- 1 root wheel 381M Mar 28 20:18 disc1.iso.bz2 >>>>> -rw-r--r-- 1 root wheel 392M Mar 28 20:18 disc1.iso.gz >>>>> -rw-r--r-- 1 root wheel 348M Mar 28 20:18 disc1.iso.xz >>>>> >>>>> Glen >>>>> >>>> >>>> How about 7zip (Windows program, not file format)? What would a Windows >>>> user use that can decompress gzip and not xz? It was a problem around >>>> ~2007, but xz support is no longer rare or exotic. >>>> >>> >>> I don't know, to be honest. I have no Windows machines to test, so >>> I can only go by what I am told. >>> >>> Glen >>> >> >> I just verified it with 7zip for Windows version 9.22. It extracts >> .tar.xz archives and decompresses .xz images. >> > > Great, thank you for confirming. > > So, the question I have now is, considering the time it takes to > compress the image with xz(1) compared to gzip(1), is that worth saving > the 50MB space? > > I do not object to using xz(1), but the compression time difference was > quite significant. (I do not have numbers off-hand, but can run the > compression again.) > > Glen > Compression is much more expensive. There are different levels too, so xz -9e my16Gfile.img on a 4 core i7 can take up to 8 hours. Other levels are cheaper. De-compression (for the end user) is as cheap or cheaper than gzip. Depending on the chosen compression level, the minimum amount of RAM required for decompression can also grow. The xz(1) explains these properties. - Nikolai Lifanov