From owner-freebsd-arch@FreeBSD.ORG Mon Dec 31 18:14:24 2007 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4522116A418; Mon, 31 Dec 2007 18:14:24 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from speedfactory.net (mail6.speedfactory.net [66.23.216.219]) by mx1.freebsd.org (Postfix) with ESMTP id D0D9F13C4CC; Mon, 31 Dec 2007 18:14:23 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from server.baldwin.cx (unverified [66.23.211.162]) by speedfactory.net (SurgeMail 3.8q) with ESMTP id 226821200-1834499 for multiple; Mon, 31 Dec 2007 13:16:47 -0500 Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.8/8.13.8) with ESMTP id lBVIEFmP098301; Mon, 31 Dec 2007 13:14:20 -0500 (EST) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: freebsd-arch@FreeBSD.org Date: Mon, 31 Dec 2007 11:28:41 -0500 User-Agent: KMail/1.9.6 References: <20071229133256.D957@desktop> In-Reply-To: <20071229133256.D957@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200712311128.42114.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Mon, 31 Dec 2007 13:14:20 -0500 (EST) X-Virus-Scanned: ClamAV 0.91.2/5311/Mon Dec 31 12:05:08 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: arch@FreeBSD.org Subject: Re: kvm_getfiles is badly broken X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2007 18:14:24 -0000 On Saturday 29 December 2007 06:44:50 pm Jeff Roberson wrote: > >From kvm_getfiles(3): > > The number of files found is returned in the reference parameter cnt. > The files are returned as a contiguous array of file structures, > preceded > by the address of the first file entry in the kernel. > > sysctl kern.file is used if the kernel is live. This code assumes the > kernel copies out a struct filelist before any files. It does not. I can > not find any consumers of this interface however. I also don't understand > why it supplies the address of the first file and what this would be used > for. > > There are other users of sysctl kern.file which assume it does not prepend > this address so it would be wrong to change that. Would it also be wrong > to change kvm to supply null as the first address? > > Other inconsistencies include live kernels returning strcut xfile and dead > kernels returning struct file. The interface in kvm_getfiles() claims to > return struct files. I can't imagine any code actually relies on this > routine. > > Any opinions on what we should do with this? It has been broken since > 2002 at least. I'm committing changes for my lockless struct file work. > As part of that I'll commit a broken but compiling implementation that > matches current bugs but causes the code to fail whenever it is called. As long as fstat(1) still works on coredumps I'm happy. :) Actually, it looks like fstat(1) doesn't use sysctl at all, but always uses kvm, even on live kernels. It would probably be nice to provide a kvm_getfiles(3) that exports 'struct xfile' objects by using the sysctl on live systems and by simulating the syscall on crashdumps to generate 'struct xfile' objects. You could then fix fstat(1) to use that instead of what it does now. Note that fstat(1) actually works now by walking each process' fdp table rather than by walking the global filelist. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Dec 31 18:14:24 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4522116A418; Mon, 31 Dec 2007 18:14:24 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from speedfactory.net (mail6.speedfactory.net [66.23.216.219]) by mx1.freebsd.org (Postfix) with ESMTP id D0D9F13C4CC; Mon, 31 Dec 2007 18:14:23 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from server.baldwin.cx (unverified [66.23.211.162]) by speedfactory.net (SurgeMail 3.8q) with ESMTP id 226821200-1834499 for multiple; Mon, 31 Dec 2007 13:16:47 -0500 Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.8/8.13.8) with ESMTP id lBVIEFmP098301; Mon, 31 Dec 2007 13:14:20 -0500 (EST) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: freebsd-arch@FreeBSD.org Date: Mon, 31 Dec 2007 11:28:41 -0500 User-Agent: KMail/1.9.6 References: <20071229133256.D957@desktop> In-Reply-To: <20071229133256.D957@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200712311128.42114.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Mon, 31 Dec 2007 13:14:20 -0500 (EST) X-Virus-Scanned: ClamAV 0.91.2/5311/Mon Dec 31 12:05:08 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: arch@FreeBSD.org Subject: Re: kvm_getfiles is badly broken X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2007 18:14:24 -0000 On Saturday 29 December 2007 06:44:50 pm Jeff Roberson wrote: > >From kvm_getfiles(3): > > The number of files found is returned in the reference parameter cnt. > The files are returned as a contiguous array of file structures, > preceded > by the address of the first file entry in the kernel. > > sysctl kern.file is used if the kernel is live. This code assumes the > kernel copies out a struct filelist before any files. It does not. I can > not find any consumers of this interface however. I also don't understand > why it supplies the address of the first file and what this would be used > for. > > There are other users of sysctl kern.file which assume it does not prepend > this address so it would be wrong to change that. Would it also be wrong > to change kvm to supply null as the first address? > > Other inconsistencies include live kernels returning strcut xfile and dead > kernels returning struct file. The interface in kvm_getfiles() claims to > return struct files. I can't imagine any code actually relies on this > routine. > > Any opinions on what we should do with this? It has been broken since > 2002 at least. I'm committing changes for my lockless struct file work. > As part of that I'll commit a broken but compiling implementation that > matches current bugs but causes the code to fail whenever it is called. As long as fstat(1) still works on coredumps I'm happy. :) Actually, it looks like fstat(1) doesn't use sysctl at all, but always uses kvm, even on live kernels. It would probably be nice to provide a kvm_getfiles(3) that exports 'struct xfile' objects by using the sysctl on live systems and by simulating the syscall on crashdumps to generate 'struct xfile' objects. You could then fix fstat(1) to use that instead of what it does now. Note that fstat(1) actually works now by walking each process' fdp table rather than by walking the global filelist. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Dec 31 18:18:58 2007 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6ED7116A418; Mon, 31 Dec 2007 18:18:58 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 450F113C50B; Mon, 31 Dec 2007 18:18:58 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 4EB6D485BD; Mon, 31 Dec 2007 13:18:57 -0500 (EST) Date: Mon, 31 Dec 2007 18:18:57 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: John Baldwin In-Reply-To: <200712311128.42114.jhb@freebsd.org> Message-ID: <20071231181758.I59781@fledge.watson.org> References: <20071229133256.D957@desktop> <200712311128.42114.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: kvm_getfiles is badly broken X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2007 18:18:58 -0000 On Mon, 31 Dec 2007, John Baldwin wrote: > As long as fstat(1) still works on coredumps I'm happy. :) Actually, it > looks like fstat(1) doesn't use sysctl at all, but always uses kvm, even on > live kernels. It would probably be nice to provide a kvm_getfiles(3) that > exports 'struct xfile' objects by using the sysctl on live systems and by > simulating the syscall on crashdumps to generate 'struct xfile' objects. > You could then fix fstat(1) to use that instead of what it does now. Note > that fstat(1) actually works now by walking each process' fdp table rather > than by walking the global filelist. FWIW, I found the kernel export of file descriptor information via sysctl a bit sparse, so I implemented a more extended interface for procstat -f. It could well be that fstat could be modified to use that interface and probably do quite a bit less work as a result. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Mon Dec 31 18:18:58 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6ED7116A418; Mon, 31 Dec 2007 18:18:58 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 450F113C50B; Mon, 31 Dec 2007 18:18:58 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 4EB6D485BD; Mon, 31 Dec 2007 13:18:57 -0500 (EST) Date: Mon, 31 Dec 2007 18:18:57 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: John Baldwin In-Reply-To: <200712311128.42114.jhb@freebsd.org> Message-ID: <20071231181758.I59781@fledge.watson.org> References: <20071229133256.D957@desktop> <200712311128.42114.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: kvm_getfiles is badly broken X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2007 18:18:58 -0000 On Mon, 31 Dec 2007, John Baldwin wrote: > As long as fstat(1) still works on coredumps I'm happy. :) Actually, it > looks like fstat(1) doesn't use sysctl at all, but always uses kvm, even on > live kernels. It would probably be nice to provide a kvm_getfiles(3) that > exports 'struct xfile' objects by using the sysctl on live systems and by > simulating the syscall on crashdumps to generate 'struct xfile' objects. > You could then fix fstat(1) to use that instead of what it does now. Note > that fstat(1) actually works now by walking each process' fdp table rather > than by walking the global filelist. FWIW, I found the kernel export of file descriptor information via sysctl a bit sparse, so I implemented a more extended interface for procstat -f. It could well be that fstat could be modified to use that interface and probably do quite a bit less work as a result. Robert N M Watson Computer Laboratory University of Cambridge