From owner-freebsd-questions@FreeBSD.ORG Sat Apr 8 05:28:36 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A37216A400 for ; Sat, 8 Apr 2006 05:28:36 +0000 (UTC) (envelope-from peter@wemm.org) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id B004A43D49 for ; Sat, 8 Apr 2006 05:28:35 +0000 (GMT) (envelope-from peter@wemm.org) Received: from fw.wemm.org (canning.wemm.org [192.203.228.65]) by canning.wemm.org (Postfix) with ESMTP id 4B9982A909 for ; Fri, 7 Apr 2006 22:28:35 -0700 (PDT) (envelope-from peter@wemm.org) Received: from peter-laptop.wemm.org (dhcp96.wemm.org [10.0.0.96]) by fw.wemm.org (Postfix) with ESMTP id E3E8CE2B3 for ; Fri, 7 Apr 2006 22:28:34 -0700 (PDT) (envelope-from peter@wemm.org) Received: from peter-laptop.wemm.org (localhost [127.0.0.1]) by peter-laptop.wemm.org (8.13.4/8.13.4) with ESMTP id k385SDHG004292; Fri, 7 Apr 2006 22:28:13 -0700 (PDT) (envelope-from peter@wemm.org) Received: from localhost (localhost [[UNIX: localhost]]) by peter-laptop.wemm.org (8.13.4/8.13.3/Submit) id k385SCSV004291; Fri, 7 Apr 2006 22:28:12 -0700 (PDT) (envelope-from peter@wemm.org) X-Authentication-Warning: peter-laptop.wemm.org: peter set sender to peter@wemm.org using -f From: Peter Wemm To: Paul Marciano Date: Fri, 7 Apr 2006 22:28:12 -0700 User-Agent: KMail/1.8.3 References: <20060407230152.57316.qmail@web54004.mail.yahoo.com> In-Reply-To: <20060407230152.57316.qmail@web54004.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200604072228.12452.peter@wemm.org> Cc: freebsd-questions@freebsd.org Subject: Re: Does FreeBSD support sparse kernel crash dumps? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Apr 2006 05:28:36 -0000 On Friday 07 April 2006 04:01 pm, Paul Marciano wrote: > --- Peter Wemm wrote: > > On Friday 07 April 2006 12:47 pm, Paul Marciano > > > > wrote: > > > Hello. I read a while back about someone working > > > > on > > > > > supporting sparse kernel crash dumps (dumping only > > > > the > > > > > active kernel pages to the dump device as opposed > > > > to > > > > > all physical memory - for machines where the phys > > > > mem > > > > > is greater than the dump dev space.) > > > > > > Does anyone know the status of that project? Was > > > > it > > > > > committed, or are there plans to commit it? > > > > I have a working prototype as of last night. There > > should be something > > committable in the next week or two. > > > > When I boot my 2GB machine and force a crash dump > > from single user mode, the > > fully debuggable vmcore file is in the 40-50MB > > range. A busy machine with > > 12GB ram took about 150MB to dump. > > > > There are still some things to work out. It was > > written for the amd64 kernel, > > but can be ported to i386. > > > > -Peter > > That's very timely news Peter. Do you think your code > is easily back-portable to 5.4? Are the changes > limited to dump_machdep.c or otherwise not dependent > on a great deal of -current updates? My WIP is relative to 6.x. It will port forwards to current fairly easily, and will port to i386 trivially. The code would be simpler on i386 because direct map data doesn't exist there. But on the other hand, i386 has to deal with PAE mode and the dump code would be significantly impacted by PAE mode Changes are mostly in sys/amd64/amd64/dump_machdep.c (a rewrite actually), an 8 line addition to sys/vm/vm_page.c and some trivial patches to some other MD pmap files. libkvm is affected as a complete rewrite of lib/libkvm/kvm_amd64.c. (Porting libkvm changes to i386 will be easy and relatively immune to PAE kernel effects). > Do you compress the data stream at all (e.g. gzip)? No, but it could be done in theory.. if you were willing to set aside some memory for the compression algorithm to use. Or just do some sort of simple compression (run length encoding?) The problem is that the dump code cannot allocate memory after the machine has crashed. It has to be able to run as isolated from the rest of the kernel as possible in order to give a true snapshot of the undisturbed state. > I have a specific need on a CompactFlash based system. > I have a 256MB IDE mode card on a machine with 512MB > physical memory. I can probably commit 128MB of the > card as a dumpdev but I can't go beyond that. I truely do not know what to expect. Without compression, check the difference between sysctl vm.kvm_free and vm.kvm_size to know how much an uncompressed minidump would take. I don't recall if those are on 5.4 or not (the code for those sysctl's would backport from pmap.c really easily). > Good luck with it. > > Regards, > Paul. -Peter