From owner-cvs-all@FreeBSD.ORG Wed Dec 21 21:06:01 2005 Return-Path: X-Original-To: cvs-all@FreeBSD.ORG Delivered-To: cvs-all@FreeBSD.ORG Received: from green.homeunix.org (freefall.freebsd.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 8F79516A41F; Wed, 21 Dec 2005 21:06:01 +0000 (GMT) (envelope-from green@green.homeunix.org) Received: from green.homeunix.org (green@localhost [127.0.0.1]) by green.homeunix.org (8.13.4/8.13.1) with ESMTP id jBLL60A1076049; Wed, 21 Dec 2005 16:06:00 -0500 (EST) (envelope-from green@green.homeunix.org) Received: (from green@localhost) by green.homeunix.org (8.13.4/8.13.1/Submit) id jBLL60oH076047; Wed, 21 Dec 2005 16:06:00 -0500 (EST) (envelope-from green) Date: Wed, 21 Dec 2005 16:05:59 -0500 From: Brian Fundakowski Feldman To: Maxim.Sobolev@portaone.com Message-ID: <20051221210559.GD35080@green.homeunix.org> References: <200512210217.jBL2HxJn057045@repoman.freebsd.org> <20051221044622.GC35080@green.homeunix.org> <43A9913F.3040106@portaone.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <43A9913F.3040106@portaone.com> User-Agent: Mutt/1.5.11 Cc: cvs-src@FreeBSD.ORG, src-committers@FreeBSD.ORG, cvs-all@FreeBSD.ORG Subject: Re: cvs commit: src/sys/boot/i386/libi386 biosmem.c i386_copy.c libi386.h src/sys/boot/i386/loader main.c X-BeenThere: cvs-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the entire tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2005 21:06:01 -0000 On Wed, Dec 21, 2005 at 09:30:39AM -0800, Maxim Sobolev wrote: > Brian Fundakowski Feldman wrote: > >On Wed, Dec 21, 2005 at 02:17:59AM +0000, Maxim Sobolev wrote: > >>sobomax 2005-12-21 02:17:58 UTC > >> > >> FreeBSD src repository > >> > >> Modified files: > >> sys/boot/i386/libi386 biosmem.c i386_copy.c libi386.h > >> sys/boot/i386/loader main.c > >> Log: > >> For the cases when loading bzip2-compressed kernels enabled use last > >> 3MB of physical memory for heap instead of range between 1MB and 4MB. > >> This makes this feature working with PAE and amd64 kernels, which are > >> loaded at 2MB. Teach i386_copyin() to avoid using range allocated by > >> heap in such case, so that it won't trash heap in the low memory > >> conditions. > >> > >> This should make loading bzip2-compressed kernels/modules/mfs images > >> generally useable, so that re@ team is welcome to evaluate merits > >> of using this feature in the installation CDs. > >> > >> Valuable suggestions by: jhb > > > >Should probably just be using 7-zip instead, though. > > Huh? Are you going to add appropriate loader(8) support? Besides, 7-zip > apparently even more memory hungry than bzip2: > > d{N}: Sets Dictionary size - [0, 28], default: 23 (8MB) > The maximum value for dictionary size is 256 MB = 2^28 bytes. > Dictionary size is calculated as DictionarySize = 2^N bytes. > For decompressing file compressed by LZMA method with dictionary > size D = 2^N you need about D bytes of memory (RAM). > > sobomax@bugor$ lzma e /tmp/kernel /tmp/kernel.lzma > > LZMA 4.17 Copyright (c) 1999-2004 Igor Pavlov 2005-04-18 > sobomax@bugor$ ls -l /tmp/kernel.lzma > -rw-r--r-- 1 sobomax wheel 1289963 21 ?????? 08:05 /tmp/kernel.lzma > sobomax@bugor$ bzip2 -9 /tmp/kernel > sobomax@bugor$ ls -l /tmp/kernel.bz2 > -r-xr-xr-x 1 sobomax wheel 1514551 21 ?????? 08:04 /tmp/kernel.bz2 > sobomax@bugor$ sudo bunzip2 /tmp/kernel.bz2 > sobomax@bugor$ lzma e /tmp/kernel /tmp/kernel.lzma -d11 > > LZMA 4.17 Copyright (c) 1999-2004 Igor Pavlov 2005-04-18 > sobomax@bugor$ ls -l /tmp/kernel.lzma > -rw-r--r-- 1 sobomax wheel 1550520 21 ?????? 08:10 /tmp/kernel.lzma > > As you can see, when memory size available for decompression is a > concern (roughly 2MB in this example) bzip2 provides comparable compression. 5.2 Single-call Decompressing with input stream callback -------------------------------------------------------- When to use: File->RAM or Flash->RAM decompressing. Compile files: LzmaDecode.h, LzmaDecode.c Compile defines: _LZMA_IN_CB Memory Requirements: - Buffer for input stream: any size (for example, 16 KB) - Output buffer: uncompressed size - LZMA Internal Structures (~16 KB for default settings) So adding up the dictionary sizes and limiting it to 2MB, using compression that optimizes x86 executable storage, LZMA instead of bzip2, solid archiving, we get: {"/home/green" green@green}$ du -sk kernel.* 5328 kernel.7z 8070 kernel.bzip2-4ed 8038 kernel.bzip2-8ed 19406 kernel.straight I would see those as considerable savings. -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\