From owner-freebsd-arch Sun Oct 13 3:38:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 17FE837B401 for ; Sun, 13 Oct 2002 03:38:10 -0700 (PDT) Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by mx1.FreeBSD.org (Postfix) with ESMTP id D3C5E43E75 for ; Sun, 13 Oct 2002 03:38:08 -0700 (PDT) (envelope-from ticso@cicely8.cicely.de) Received: from cicely5.cicely.de (cicely5.cicely.de [IPv6:3ffe:400:8d0:301:200:92ff:fe9b:20e7]) by srv1.cosmo-project.de (8.12.5/8.12.5) with ESMTP id g9DAb06K003317 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Sun, 13 Oct 2002 12:37:35 +0200 (CEST) (envelope-from ticso@cicely8.cicely.de) Received: from cicely8.cicely.de (cicely8.cicely.de [10.1.1.10]) by cicely5.cicely.de (8.12.6/8.12.6) with ESMTP id g9DAaugK096974 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Sun, 13 Oct 2002 12:36:56 +0200 (CEST) (envelope-from ticso@cicely8.cicely.de) Received: from cicely8.cicely.de (localhost [127.0.0.1]) by cicely8.cicely.de (8.12.6/8.12.6) with ESMTP id g9DAatlv033775; Sun, 13 Oct 2002 12:36:55 +0200 (CEST) (envelope-from ticso@cicely8.cicely.de) Received: (from ticso@localhost) by cicely8.cicely.de (8.12.6/8.12.6/Submit) id g9DAZc6G033774; Sun, 13 Oct 2002 12:35:38 +0200 (CEST) Date: Sun, 13 Oct 2002 12:35:38 +0200 From: Bernd Walter To: "M. Warner Losh" Cc: hch@infradead.org, wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram Message-ID: <20021013103538.GG17920@cicely8.cicely.de> Reply-To: ticso@cicely.de References: <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20021012.150616.129769790.imp@bsdimp.com> X-Operating-System: FreeBSD cicely8.cicely.de 5.0-CURRENT i386 User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Oct 12, 2002 at 03:06:16PM -0600, M. Warner Losh wrote: > In message: <20021012135245.A16453@infradead.org> > Christoph Hellwig writes: > : On Wed, Oct 09, 2002 at 04:57:37PM -0700, Wes Peters wrote: > : > Linux solved this problem by refusing to do it. The candidates for DMA > : > transfers include skbufs and buffers from the disk buffer pool, both of > : > which are allocated from the lowest 4GB of physical ram when using PAE > : > mode. > : > : Umm, Linux _does_ DMA into any memory if the NIC/HBA/whatever supports > : it. > > Unless the card is 64bit, it can't DMA past 4G. Shouldn't all modern pci chips support two 32bit word addresses. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 4:13:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7D5CA37B401 for ; Sun, 13 Oct 2002 04:13:33 -0700 (PDT) Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0CDEA43E88 for ; Sun, 13 Oct 2002 04:13:33 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0024.cvx40-bradley.dialup.earthlink.net ([216.244.42.24] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 180gfX-0002TO-00; Sun, 13 Oct 2002 04:12:24 -0700 Message-ID: <3DA954CF.98B0891A@mindspring.com> Date: Sun, 13 Oct 2002 04:11:11 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: ticso@cicely.de Cc: "M. Warner Losh" , hch@infradead.org, wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> <20021013103538.GG17920@cicely8.cicely.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bernd Walter wrote: > > : Umm, Linux _does_ DMA into any memory if the NIC/HBA/whatever supports > > : it. > > > > Unless the card is 64bit, it can't DMA past 4G. > > Shouldn't all modern pci chips support two 32bit word addresses. Maybe. The issue is the connectors. That you can use 64 bit chips in nominally 32 bit cards actually makes the situation much worse: if we could tell 64 bit vs. 32 bit data path all the way to the DMA engine on the card (without needing to trigger a DMA and have a mirror area in physical memory both above and below the 32 bit mark, to see where the data actually goes), then it would be much less of a problem. In most cases, the DMA gets put directly into memory mapped by the VM of the kernel (the KVA). FreeBSD has a unified VM and buffer cache. It makes little to no sense to DMA directly into user space from hardware; among other things, it means that protection would be limited to page boundaries, and you would break cache coherency between mmap() and read/write I/O to files for which the DMA'ed data was the backing store. In effect, this means that all DMA'ed data has to be known to the kernel. I would be interested in knowing how Linux "knows" that the DMA target for a card, with a driver which has communicated a 64 bit DMA target address to it, has been modified, and how it deals with shared memory segments, mmap'ed data, and data that needs to be kept coherent between processes with different PAE/PSE-36 mappings (i.e. process P1 and P2, where P2 has memory below 4G, and P2 has memory above 8G, so the mappings do not coincide in a region of physical memory shared by both processes -- or that's even *possible* to have coresident in the KVA at the same time). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 5: 7:32 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2DB5B37B404 for ; Sun, 13 Oct 2002 05:07:31 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6095843E6A for ; Sun, 13 Oct 2002 05:07:30 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.3/8.12.3) with ESMTP id g9DC7Bpk013086; Sun, 13 Oct 2002 06:07:11 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Sun, 13 Oct 2002 06:06:48 -0600 (MDT) Message-Id: <20021013.060648.93373269.imp@bsdimp.com> To: ticso@cicely.de, ticso@cicely8.cicely.de Cc: hch@infradead.org, wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram From: "M. Warner Losh" In-Reply-To: <20021013103538.GG17920@cicely8.cicely.de> References: <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> <20021013103538.GG17920@cicely8.cicely.de> X-Mailer: Mew version 2.1 on Emacs 21.2 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <20021013103538.GG17920@cicely8.cicely.de> Bernd Walter writes: : On Sat, Oct 12, 2002 at 03:06:16PM -0600, M. Warner Losh wrote: : > In message: <20021012135245.A16453@infradead.org> : > Christoph Hellwig writes: : > : On Wed, Oct 09, 2002 at 04:57:37PM -0700, Wes Peters wrote: : > : > Linux solved this problem by refusing to do it. The candidates for DMA : > : > transfers include skbufs and buffers from the disk buffer pool, both of : > : > which are allocated from the lowest 4GB of physical ram when using PAE : > : > mode. : > : : > : Umm, Linux _does_ DMA into any memory if the NIC/HBA/whatever supports : > : it. : > : > Unless the card is 64bit, it can't DMA past 4G. : : Shouldn't all modern pci chips support two 32bit word addresses. I don't think so. There's only 32-bits on the bus for the address on most pci cards (not the 64-bit pci cards, which are special), so I don't see how they can go above 4G. Unless I'm being unusually dense this morning. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 5: 9:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D36B37B401 for ; Sun, 13 Oct 2002 05:09:25 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id DEA8D43E8A for ; Sun, 13 Oct 2002 05:09:24 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.3/8.12.3) with ESMTP id g9DC9Epk013099; Sun, 13 Oct 2002 06:09:15 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Sun, 13 Oct 2002 06:08:51 -0600 (MDT) Message-Id: <20021013.060851.113437955.imp@bsdimp.com> To: tlambert2@mindspring.com Cc: ticso@cicely.de, hch@infradead.org, wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram From: "M. Warner Losh" In-Reply-To: <3DA954CF.98B0891A@mindspring.com> References: <20021012.150616.129769790.imp@bsdimp.com> <20021013103538.GG17920@cicely8.cicely.de> <3DA954CF.98B0891A@mindspring.com> X-Mailer: Mew version 2.1 on Emacs 21.2 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <3DA954CF.98B0891A@mindspring.com> Terry Lambert writes: : In most cases, the DMA gets put directly into memory mapped by the : VM of the kernel (the KVA). Actually, in most cases on i386 the memory gets DMAd to a phyiscal address, which is why there is a 4G limit in the hardware. Since it is a phyiscal address, knowing VM tricks I don't think is relevant. PAE is basically a vm trick. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 5:31:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8F21737B401 for ; Sun, 13 Oct 2002 05:31:19 -0700 (PDT) Received: from phoenix.infradead.org (carisma.slowglass.com [195.224.96.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id D68D543E77 for ; Sun, 13 Oct 2002 05:31:18 -0700 (PDT) (envelope-from hch@infradead.org) Received: from hch by phoenix.infradead.org with local (Exim 4.10) id 180hti-0003yE-00; Sun, 13 Oct 2002 13:31:06 +0100 Date: Sun, 13 Oct 2002 13:31:06 +0100 From: Christoph Hellwig To: "M. Warner Losh" Cc: wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.ORG Subject: Re: Database indexes and ram Message-ID: <20021013133106.C15151@infradead.org> References: <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20021012.150616.129769790.imp@bsdimp.com>; from imp@bsdimp.com on Sat, Oct 12, 2002 at 03:06:16PM -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Oct 12, 2002 at 03:06:16PM -0600, M. Warner Losh wrote: > Unless the card is 64bit, it can't DMA past 4G. Right. But if you have a server with > 4GB memory you usually have thos cards. Dito for the non-PCI devices. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 11: 1:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8305D37B401 for ; Sun, 13 Oct 2002 11:01:53 -0700 (PDT) Received: from flamingo.mail.pas.earthlink.net (flamingo.mail.pas.earthlink.net [207.217.120.232]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1A19843E8A for ; Sun, 13 Oct 2002 11:01:53 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0507.cvx40-bradley.dialup.earthlink.net ([216.244.43.252] helo=mindspring.com) by flamingo.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 180n3K-0000T8-00; Sun, 13 Oct 2002 11:01:22 -0700 Message-ID: <3DA9B4A8.194A02FC@mindspring.com> Date: Sun, 13 Oct 2002 11:00:08 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "M. Warner Losh" Cc: ticso@cicely.de, hch@infradead.org, wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <20021012.150616.129769790.imp@bsdimp.com> <20021013103538.GG17920@cicely8.cicely.de> <3DA954CF.98B0891A@mindspring.com> <20021013.060851.113437955.imp@bsdimp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "M. Warner Losh" wrote: > In message: <3DA954CF.98B0891A@mindspring.com> > Terry Lambert writes: > : In most cases, the DMA gets put directly into memory mapped by the > : VM of the kernel (the KVA). > > Actually, in most cases on i386 the memory gets DMAd to a phyiscal > address, which is why there is a 4G limit in the hardware. Since it > is a phyiscal address, knowing VM tricks I don't think is relevant. > PAE is basically a vm trick. You've taken the argument out of context. The argument is about: 1) The interrupt handler for the completed DMA 2) The fact that most data which is DMA'ed ends up being shared between multiple processes 3) The fact that the VM and buffer cache are unified, so that even if you wanted to do explicit coherency between multiple copies of DMA'ed data, you would not be able to, unless they occurred into a region which was not replicated, which means one which was shared, which means "in the KVA", which means "not in the bank selected PAE/PSE-36 window". -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 11: 2:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D56A337B401 for ; Sun, 13 Oct 2002 11:02:27 -0700 (PDT) Received: from flamingo.mail.pas.earthlink.net (flamingo.mail.pas.earthlink.net [207.217.120.232]) by mx1.FreeBSD.org (Postfix) with ESMTP id 817E843E8A for ; Sun, 13 Oct 2002 11:02:27 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0507.cvx40-bradley.dialup.earthlink.net ([216.244.43.252] helo=mindspring.com) by flamingo.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 180n4D-0001UI-00; Sun, 13 Oct 2002 11:02:17 -0700 Message-ID: <3DA9B4E1.1257DAA8@mindspring.com> Date: Sun, 13 Oct 2002 11:01:05 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Christoph Hellwig Cc: "M. Warner Losh" , wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.ORG Subject: Re: Database indexes and ram References: <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> <20021013133106.C15151@infradead.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Christoph Hellwig wrote: > On Sat, Oct 12, 2002 at 03:06:16PM -0600, M. Warner Losh wrote: > > Unless the card is 64bit, it can't DMA past 4G. > > Right. But if you have a server with > 4GB memory you usually have > thos cards. Dito for the non-PCI devices. Name one motherboard with more than 2 64 bit PCI slots. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 11: 9:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1187C37B401 for ; Sun, 13 Oct 2002 11:09:23 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 339A143E7B for ; Sun, 13 Oct 2002 11:09:22 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.3/8.12.3) with ESMTP id g9DI9Dpk014384; Sun, 13 Oct 2002 12:09:13 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Sun, 13 Oct 2002 12:08:47 -0600 (MDT) Message-Id: <20021013.120847.31902907.imp@bsdimp.com> To: tlambert2@mindspring.com Cc: ticso@cicely.de, hch@infradead.org, wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram From: "M. Warner Losh" In-Reply-To: <3DA9B4A8.194A02FC@mindspring.com> References: <3DA954CF.98B0891A@mindspring.com> <20021013.060851.113437955.imp@bsdimp.com> <3DA9B4A8.194A02FC@mindspring.com> X-Mailer: Mew version 2.1 on Emacs 21.2 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <3DA9B4A8.194A02FC@mindspring.com> Terry Lambert writes: : "M. Warner Losh" wrote: : > In message: <3DA954CF.98B0891A@mindspring.com> : > Terry Lambert writes: : > : In most cases, the DMA gets put directly into memory mapped by the : > : VM of the kernel (the KVA). : > : > Actually, in most cases on i386 the memory gets DMAd to a phyiscal : > address, which is why there is a 4G limit in the hardware. Since it : > is a phyiscal address, knowing VM tricks I don't think is relevant. : > PAE is basically a vm trick. : : You've taken the argument out of context. The argument is about: : : 1) The interrupt handler for the completed DMA : : 2) The fact that most data which is DMA'ed ends up being shared : between multiple processes : : 3) The fact that the VM and buffer cache are unified, so that : even if you wanted to do explicit coherency between multiple : copies of DMA'ed data, you would not be able to, unless they : occurred into a region which was not replicated, which means : one which was shared, which means "in the KVA", which means : "not in the bank selected PAE/PSE-36 window". I think that's all irrelevant. Cards with 32bits can't go about 4GB. It is a far more fundamental problem. Even 32bit cards in 64bit slots can't do this. 64bit cards could DMA into anywhere in the first 64bits of RAM, of course. However, you are at least partially right about what you say, I think. When DMAing into high memory, you'd still have to arrange for those physical pages to be in the process that you are talking about (because each process still has a 4G size limit, and I think it has to be all in the same segment unless it is PAE/PSE aware). I was confusing what you said with "The DMA is based on a virtual address" which is not quite the same thing. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 11:16:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 51BCB37B401 for ; Sun, 13 Oct 2002 11:16:53 -0700 (PDT) Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by mx1.FreeBSD.org (Postfix) with ESMTP id 193E743E88 for ; Sun, 13 Oct 2002 11:16:52 -0700 (PDT) (envelope-from ticso@cicely8.cicely.de) Received: from cicely5.cicely.de (cicely5.cicely.de [IPv6:3ffe:400:8d0:301:200:92ff:fe9b:20e7]) by srv1.cosmo-project.de (8.12.5/8.12.5) with ESMTP id g9DIGd6K009230 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Sun, 13 Oct 2002 20:16:42 +0200 (CEST) (envelope-from ticso@cicely8.cicely.de) Received: from cicely8.cicely.de (cicely8.cicely.de [10.1.1.10]) by cicely5.cicely.de (8.12.6/8.12.6) with ESMTP id g9DIGbgK099335 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Sun, 13 Oct 2002 20:16:38 +0200 (CEST) (envelope-from ticso@cicely8.cicely.de) Received: from cicely8.cicely.de (localhost [127.0.0.1]) by cicely8.cicely.de (8.12.6/8.12.6) with ESMTP id g9DIGalv035246; Sun, 13 Oct 2002 20:16:37 +0200 (CEST) (envelope-from ticso@cicely8.cicely.de) Received: (from ticso@localhost) by cicely8.cicely.de (8.12.6/8.12.6/Submit) id g9DIGZQo035245; Sun, 13 Oct 2002 20:16:35 +0200 (CEST) Date: Sun, 13 Oct 2002 20:16:34 +0200 From: Bernd Walter To: "M. Warner Losh" Cc: tlambert2@mindspring.com, ticso@cicely.de, hch@infradead.org, wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram Message-ID: <20021013181633.GB34517@cicely8.cicely.de> Reply-To: ticso@cicely.de References: <3DA954CF.98B0891A@mindspring.com> <20021013.060851.113437955.imp@bsdimp.com> <3DA9B4A8.194A02FC@mindspring.com> <20021013.120847.31902907.imp@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20021013.120847.31902907.imp@bsdimp.com> X-Operating-System: FreeBSD cicely8.cicely.de 5.0-CURRENT i386 User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Oct 13, 2002 at 12:08:47PM -0600, M. Warner Losh wrote: > I think that's all irrelevant. Cards with 32bits can't go about 4GB. > It is a far more fundamental problem. Even 32bit cards in 64bit slots > can't do this. 64bit cards could DMA into anywhere in the first > 64bits of RAM, of course. Of course they can do. It's just a matter if the card and the board support 2 address cycles. Or if the board can map the pci reachable space - as alphas can do. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 11:56:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7064D37B401 for ; Sun, 13 Oct 2002 11:56:43 -0700 (PDT) Received: from conure.mail.pas.earthlink.net (conure.mail.pas.earthlink.net [207.217.120.54]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0C8A843E7B for ; Sun, 13 Oct 2002 11:56:43 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0507.cvx40-bradley.dialup.earthlink.net ([216.244.43.252] helo=mindspring.com) by conure.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 180nuR-0005qp-00; Sun, 13 Oct 2002 11:56:16 -0700 Message-ID: <3DA9C181.D36065CA@mindspring.com> Date: Sun, 13 Oct 2002 11:54:57 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "M. Warner Losh" Cc: ticso@cicely.de, hch@infradead.org, wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <3DA954CF.98B0891A@mindspring.com> <20021013.060851.113437955.imp@bsdimp.com> <3DA9B4A8.194A02FC@mindspring.com> <20021013.120847.31902907.imp@bsdimp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "M. Warner Losh" wrote: > I think that's all irrelevant. Cards with 32bits can't go about 4GB. > It is a far more fundamental problem. Even 32bit cards in 64bit slots > can't do this. 64bit cards could DMA into anywhere in the first > 64bits of RAM, of course. I think we are in violent agreement. 8-). > I was confusing what you said with "The DMA is based on a virtual > address" which is not quite the same thing. Sorry if this was confusing; it was meant to reference the mappability of the physical memory, not that the DMA targetted anything but physical memory. You *could* "check DMAability" by having a target area at some address Q above 32 bits, and then checking to see if the data went there, or to the address Q & 0x00000000ffffffff instead. This would require you to pick your areas carefully. Ugly, ugly. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 12: 0:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3B35037B401 for ; Sun, 13 Oct 2002 12:00:47 -0700 (PDT) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id D986E43E7B for ; Sun, 13 Oct 2002 12:00:46 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.5/8.12.4) with ESMTP id g9DJ0ZPQ054778; Sun, 13 Oct 2002 12:00:35 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.5/8.12.4/Submit) id g9DJ0ZAM054777; Sun, 13 Oct 2002 12:00:35 -0700 (PDT) (envelope-from dillon) Date: Sun, 13 Oct 2002 12:00:35 -0700 (PDT) From: Matthew Dillon Message-Id: <200210131900.g9DJ0ZAM054777@apollo.backplane.com> To: "M. Warner Losh" Cc: tlambert2@mindspring.com, ticso@cicely.de, hch@infradead.org, wes@softweyr.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <3DA954CF.98B0891A@mindspring.com> <20021013.060851.113437955.imp@bsdimp.com> <3DA9B4A8.194A02FC@mindspring.com> <20021013.120847.31902907.imp@bsdimp.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Google is your friend. I found a quick reference on the PCI bus. A 32 bit PCI bus can support 64 bit addresses through the use of two address cycles prefacing the data transfer. That said, we all know how shoddy a large chunk of the PCI market is (well, really the *entire* PC market, not just PCI). Just because the spec allows it doesn't mean the chipset/motherboards support it or, more importantly, support it reliably. Remember all the cache/MMU bugs that showed up in the 486 series when people started actually using the MMU? IMHO as much as I like the coolness of throwing more then 4G into a PC not really designed to take more then 4G, I personally believe that it is wiser to distribute processing at that point rather then spend money on more specialized (and finicky) configurations. You will see me throw more then 4G into a box when the next generation capable of dealing with more then 4G becomes commoditized. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 12: 5:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0115637B401 for ; Sun, 13 Oct 2002 12:05:58 -0700 (PDT) Received: from conure.mail.pas.earthlink.net (conure.mail.pas.earthlink.net [207.217.120.54]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8EA0743E88 for ; Sun, 13 Oct 2002 12:05:57 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0507.cvx40-bradley.dialup.earthlink.net ([216.244.43.252] helo=mindspring.com) by conure.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 180o3c-0000eg-00; Sun, 13 Oct 2002 12:05:44 -0700 Message-ID: <3DA9C3B9.E78BBFE6@mindspring.com> Date: Sun, 13 Oct 2002 12:04:25 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: ticso@cicely.de Cc: "M. Warner Losh" , hch@infradead.org, wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <3DA954CF.98B0891A@mindspring.com> <20021013.060851.113437955.imp@bsdimp.com> <3DA9B4A8.194A02FC@mindspring.com> <20021013.120847.31902907.imp@bsdimp.com> <20021013181633.GB34517@cicely8.cicely.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bernd Walter wrote: > Of course they can do. > It's just a matter if the card and the board support 2 address cycles. > Or if the board can map the pci reachable space - as alphas can do. The question is whether you can say reliably that all cards that will be sharing cached data space can do this, or whether you will have to bounce the data to below 4G. If you can't *know*, then to ensure operation, you *must* bounce the data to proactively guarantee that the physical address will be in range of the card's DMA engine. Among other things, this means recognizing a 32 bit card in a 64 bit slot, and a 64 bit card in a 32 bit slot, and a 64 bit card in a 64 bit slot, but which has only 32 bits worth of electrical connector on the physical card. If you can guarantee that, then you can do it without bouncing. Can you do that? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 12:35:57 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AC12A37B401 for ; Sun, 13 Oct 2002 12:35:55 -0700 (PDT) Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7B2E643E8A for ; Sun, 13 Oct 2002 12:35:54 -0700 (PDT) (envelope-from ticso@cicely8.cicely.de) Received: from cicely5.cicely.de (cicely5.cicely.de [IPv6:3ffe:400:8d0:301:200:92ff:fe9b:20e7]) by srv1.cosmo-project.de (8.12.5/8.12.5) with ESMTP id g9DJZa6K010268 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Sun, 13 Oct 2002 21:35:41 +0200 (CEST) (envelope-from ticso@cicely8.cicely.de) Received: from cicely8.cicely.de (cicely8.cicely.de [10.1.1.10]) by cicely5.cicely.de (8.12.6/8.12.6) with ESMTP id g9DJZZgK099683 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Sun, 13 Oct 2002 21:35:36 +0200 (CEST) (envelope-from ticso@cicely8.cicely.de) Received: from cicely8.cicely.de (localhost [127.0.0.1]) by cicely8.cicely.de (8.12.6/8.12.6) with ESMTP id g9DJZYlv035417; Sun, 13 Oct 2002 21:35:34 +0200 (CEST) (envelope-from ticso@cicely8.cicely.de) Received: (from ticso@localhost) by cicely8.cicely.de (8.12.6/8.12.6/Submit) id g9DJZX2u035416; Sun, 13 Oct 2002 21:35:33 +0200 (CEST) Date: Sun, 13 Oct 2002 21:35:33 +0200 From: Bernd Walter To: Matthew Dillon Cc: "M. Warner Losh" , tlambert2@mindspring.com, ticso@cicely.de, hch@infradead.org, wes@softweyr.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram Message-ID: <20021013193532.GC34517@cicely8.cicely.de> Reply-To: ticso@cicely.de References: <3DA954CF.98B0891A@mindspring.com> <20021013.060851.113437955.imp@bsdimp.com> <3DA9B4A8.194A02FC@mindspring.com> <20021013.120847.31902907.imp@bsdimp.com> <200210131900.g9DJ0ZAM054777@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200210131900.g9DJ0ZAM054777@apollo.backplane.com> X-Operating-System: FreeBSD cicely8.cicely.de 5.0-CURRENT i386 User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Oct 13, 2002 at 12:00:35PM -0700, Matthew Dillon wrote: > Google is your friend. I found a quick reference on the PCI bus. > A 32 bit PCI bus can support 64 bit addresses through the use of > two address cycles prefacing the data transfer. The main problem is that it's not there since day 1. As Terry said - you need to know if the address path is 64 bit capable or not. E.g. the DEC 2105x PCI-PCI bridges can't do 64 bit, but the 2115x can. > That said, we all know how shoddy a large chunk of the PCI market is > (well, really the *entire* PC market, not just PCI). Just because the > spec allows it doesn't mean the chipset/motherboards support it or, > more importantly, support it reliably. Remember all the cache/MMU bugs > that showed up in the 486 series when people started actually using the > MMU? The same problem I currently see in getting 3.3V capable PCI cards. Sometimes reading the chip specs I get the feeling that vendors just forget to make that slit. > IMHO as much as I like the coolness of throwing more then 4G into a PC > not really designed to take more then 4G, I personally believe that > it is wiser to distribute processing at that point rather then spend > money on more specialized (and finicky) configurations. You will see > me throw more then 4G into a box when the next generation capable of > dealing with more then 4G becomes commoditized. Unless you can map PCI space you always have to deal with older PCI cards which are only capable of addressing 32 - no matter what platform and generation you are using. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 15: 4:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3FFDD37B401 for ; Sun, 13 Oct 2002 15:04:43 -0700 (PDT) Received: from platon.gneto.com (as6-1-5.kr.m.bonet.se [217.215.84.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3FE0843E8A for ; Sun, 13 Oct 2002 15:04:42 -0700 (PDT) (envelope-from martin@mullet.se) Received: from mullet.se (unknown [192.168.1.10]) by platon.gneto.com (Postfix) with ESMTP id 52FA54D9F; Mon, 14 Oct 2002 00:04:40 +0200 (CEST) Message-ID: <3DA9EE01.6070701@mullet.se> Date: Mon, 14 Oct 2002 00:04:49 +0200 From: Martin Nilsson User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Terry Lambert Cc: arch@FreeBSD.ORG Subject: Re: Database indexes and ram References: <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> <20021013133106.C15151@infradead.org> <3DA9B4E1.1257DAA8@mindspring.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > Christoph Hellwig wrote: >>>Unless the card is 64bit, it can't DMA past 4G. >>Right. But if you have a server with > 4GB memory you usually have >>thos cards. Dito for the non-PCI devices. >=20 > Name one motherboard with more than 2 64 bit PCI slots. The high end of todays dual P4 Xeon boards come with only 64-bit PCI=20 slots, they also accept up to 16GB of memory. The low cost boards come with 2-3 32-bit PCI slots and a cople of 64-bit = ones they takes up to 8-12GB of memory. Why not take a look yourself: http://www.supermicro.com/Product_page/product-m1.htm /Martin --=20 Martin Nilsson, CTO & Founder, Mullet Scandinavia AB, Malm=F6, SWEDEN E-mail: martin@mullet.se, Phone: +46-(0)708-606170 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 15:36:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6873737B401 for ; Sun, 13 Oct 2002 15:36:30 -0700 (PDT) Received: from cimlogic.com.au (cimlog.lnk.telstra.net [139.130.51.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1ECAA43EA9 for ; Sun, 13 Oct 2002 15:36:28 -0700 (PDT) (envelope-from jb@cimlogic.com.au) Received: from freebsd1.cimlogic.com.au (localhost.cimlogic.com.au [127.0.0.1]) by cimlogic.com.au (8.12.6/8.12.6) with ESMTP id g9DMaNVe039192 for ; Mon, 14 Oct 2002 08:36:23 +1000 (EST) (envelope-from jb@cimlogic.com.au) Received: (from jb@localhost) by freebsd1.cimlogic.com.au (8.12.6/8.12.6/Submit) id g9DMaNhd039191 for arch@freebsd.org; Mon, 14 Oct 2002 08:36:23 +1000 (EST) Date: Mon, 14 Oct 2002 08:36:23 +1000 From: John Birrell To: arch@freebsd.org Subject: USB driver design issues Message-ID: <20021014083622.G261@freebsd1.cimlogic.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I have a few problems with the design of the USB drivers in FreeBSD. From the investigation I've done so far, the problems appear to arise from the inability of the code to handle kldload and kldunload. The result is that I can crash a RELENG_4 kernel (and probably a -current one too since the USB driver code is similar) simply by connecting or disconnecting the USB cable. Sub-optimal. I was working on a driver for the Belkin VideoBus II (a video capture thingy). Like other custom drivers I have, this one has a test harness that kldloads the driver, opens it, does a few things, closes and kldunloads it. Connecting the USB cable after the driver is loaded causes the USB drivers to correctly probe and attach the device. The probe and attachment process hangs a USB pointer off the device 'ivar' hook. When the device is closed and the driver unloaded, the kernel code NULLs the 'driver' pointer in the device, but leaves the 'ivar' pointer set. The USB code continues to remember that the cable is connected, but there is now no device for it. Disconnecting the cable at this point crashes the kernel as it tries to use the device to report the event to usbd. If the cable isn't disconnected, but the driver module is kldloaded again, the kernel calls the driver_module_handler code which tries to probe and attach the device, using the bogus 'ivar' pointer and this crashes the kernel. What I think the USB code /should/ do (the reason for posting to this list) is to use the driver_loaded callback to ensure that it's list of USB devices 'connected' to the hub is updated as driver modules are loaded and unloaded. For example, if a USB scanner device (say) is already connected and powered up, but the uscanner driver isn't loaded, then the device will appear as 'ugen'. If the uscanner driver is subsequently loaded, the USB code should re-discover that device as 'uscanner'. If the uscanner driver is unloaded, then the device should go back to 'ugen'. Comments? (BTW, I've tried emailing Nick Hibma about this, but I haven't had a reply yet. It's only been a few days though.) -- John Birrell To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 16:45:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DB93E37B401 for ; Sun, 13 Oct 2002 16:45:19 -0700 (PDT) Received: from smtp02.iprimus.net.au (smtp02.iprimus.net.au [210.50.76.70]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7DF7A43E9C for ; Sun, 13 Oct 2002 16:45:18 -0700 (PDT) (envelope-from tim@robbins.dropbear.id.au) Received: from dilbert.robbins.dropbear.id.au ([210.50.36.112]) by smtp02.iprimus.net.au with Microsoft SMTPSVC(5.0.2195.4617); Mon, 14 Oct 2002 09:45:04 +1000 Received: from dilbert.robbins.dropbear.id.au (lfd14pdlivpbvo8d@localhost [127.0.0.1]) by dilbert.robbins.dropbear.id.au (8.12.6/8.12.6) with ESMTP id g9DNj1bo026478 for ; Mon, 14 Oct 2002 09:45:02 +1000 (EST) (envelope-from tim@dilbert.robbins.dropbear.id.au) Received: (from tim@localhost) by dilbert.robbins.dropbear.id.au (8.12.6/8.12.6/Submit) id g9DNj0Eu026446 for freebsd-arch@freebsd.org; Mon, 14 Oct 2002 09:45:00 +1000 (EST) (envelope-from tim) Date: Mon, 14 Oct 2002 09:44:59 +1000 From: Tim Robbins To: freebsd-arch@freebsd.org Subject: Macros in Message-ID: <20021014094459.A25812@dilbert.robbins.dropbear.id.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i X-OriginalArrivalTime: 13 Oct 2002 23:45:04.0927 (UTC) FILETIME=[8E07B2F0:01C27312] Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Since revision 1.25 of src/include/stdio.h removed the macro versions of putc() and getc(), is there any reason why we shouldn't go the whole way and remove the rest of the macros (*_unlocked()) ? This would let us make FILE opaque and move the definition of struct __sFILE into src/lib/libc/stdio/local.h. This would also let us remove the __sFILEX hack. Tim To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 19: 0:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4E53E37B401 for ; Sun, 13 Oct 2002 19:00:27 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 888EC43E6E for ; Sun, 13 Oct 2002 19:00:26 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-9.access.nethere.net ([66.63.140.201] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 180uWd-0006ZN-00; Sun, 13 Oct 2002 20:00:07 -0600 Message-ID: <3DAA27F9.DBDA4EFE@softweyr.com> Date: Sun, 13 Oct 2002 19:12:10 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Christoph Hellwig Cc: Matthew Dillon , "Vladimir B. Grebenschikov" , Nate Lawson , arch@FreeBSD.ORG Subject: Re: Database indexes and ram (was Re: using mem above 4Gb was:swapon some regular file) References: <1034105993.913.1.camel@vbook.express.ru> <200210082015.g98KFFrq084625@apollo.backplane.com> <1034109053.913.7.camel@vbook.express.ru> <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Christoph Hellwig wrote: > > On Wed, Oct 09, 2002 at 04:57:37PM -0700, Wes Peters wrote: > > Linux solved this problem by refusing to do it. The candidates for DMA > > transfers include skbufs and buffers from the disk buffer pool, both of > > which are allocated from the lowest 4GB of physical ram when using PAE > > mode. > > Umm, Linux _does_ DMA into any memory if the NIC/HBA/whatever supports > it. And the driver has been modified. There are 3 or 4 drivers that have been, not a daunting majority by any stretch. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 19: 9:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D13E137B401; Sun, 13 Oct 2002 19:09:45 -0700 (PDT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9BCCF43E7B; Sun, 13 Oct 2002 19:09:45 -0700 (PDT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 5A575AE265; Sun, 13 Oct 2002 19:09:45 -0700 (PDT) Date: Sun, 13 Oct 2002 19:09:45 -0700 From: Alfred Perlstein To: Tim Robbins Cc: freebsd-arch@freebsd.org Subject: Re: Macros in Message-ID: <20021014020945.GI95327@elvis.mu.org> References: <20021014094459.A25812@dilbert.robbins.dropbear.id.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20021014094459.A25812@dilbert.robbins.dropbear.id.au> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Tim Robbins [021013 16:49] wrote: > Since revision 1.25 of src/include/stdio.h removed the macro versions of > putc() and getc(), is there any reason why we shouldn't go the whole way > and remove the rest of the macros (*_unlocked()) ? > > This would let us make FILE opaque and move the definition of > struct __sFILE into src/lib/libc/stdio/local.h. This would also let us > remove the __sFILEX hack. But don't we want some level of speed for stdio operations even at the expense of some transparency? -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 19:11:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0DB6737B401 for ; Sun, 13 Oct 2002 19:11:32 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C3F743E7B for ; Sun, 13 Oct 2002 19:11:31 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-8.access.nethere.net ([66.63.140.200] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 180uhW-0006c7-00; Sun, 13 Oct 2002 20:11:22 -0600 Message-ID: <3DAA2AAB.CC3A6D69@softweyr.com> Date: Sun, 13 Oct 2002 19:23:39 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Garance A Drosihn Cc: arch@FreeBSD.ORG Subject: Re: 6.0 branching (no longer: HEADS UP: 5.0 Feature FreezeOctober 16, 2002) References: <200210112056.g9BKuZEx041686@apollo.backplane.com> <3DA7C3DF.1CFD6978@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Garance A Drosihn wrote: > > At 11:40 PM -0700 10/11/02, Wes Peters wrote: > >I think we need to discuss when we will branch 6.x. I think we need > >to wait until we have a 5.x release that is stable enough to consider > >for production workstation usage levels, and hope we may reach that > >point by the 5.2 release. > > It is good to explicitly say something about this topic now, so people > don't think a 6.0-current branch is going to happen right away. > > [...] > > We should be clear that the criteria is "5.x is production quality", > and not "when .x reaches .2". At some point (which might be 5.1) it > will probably be helpful to have an explicit list of what issues need > to be fixed before we can make the new -current branch. With Warner's excellent summation and this, it seems the developer corps are pretty much prepared for this. I just wanted to make sure this had been discussed RECENTLY in public, so nobody is surprised when the 6.x branch doesn't immediately pop up. I agree it's a good idea for everyone to think about what their criteria for "enough success" for 5.x is as well. We've recently seen quite a bit of discussion on several features that people want to get into 5.0, or make default on 5.0, that may or may not be ready for prime time. Making these projects ready to be the default will be a big part of getting 5.x to the point where 6.0 can be branched, I suspect. Fortunately, this seems to have had the desired effect of creating a few moments of introspection, where everyone contemplates what their criteria for success are. That's a good sign, I'm quite glad to be able to report such success to the nay-sayers. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 19:12:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2F5F937B401 for ; Sun, 13 Oct 2002 19:12:57 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6E7A643EAC for ; Sun, 13 Oct 2002 19:12:56 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-10.access.nethere.net ([66.63.140.202] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 180uia-0006cY-00; Sun, 13 Oct 2002 20:12:29 -0600 Message-ID: <3DAA2AEC.EFF1C9F1@softweyr.com> Date: Sun, 13 Oct 2002 19:24:44 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: ticso@cicely.de Cc: "M. Warner Losh" , hch@infradead.org, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> <20021013103538.GG17920@cicely8.cicely.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bernd Walter wrote: > > On Sat, Oct 12, 2002 at 03:06:16PM -0600, M. Warner Losh wrote: > > In message: <20021012135245.A16453@infradead.org> > > Christoph Hellwig writes: > > : On Wed, Oct 09, 2002 at 04:57:37PM -0700, Wes Peters wrote: > > : > Linux solved this problem by refusing to do it. The candidates for DMA > > : > transfers include skbufs and buffers from the disk buffer pool, both of > > : > which are allocated from the lowest 4GB of physical ram when using PAE > > : > mode. > > : > > : Umm, Linux _does_ DMA into any memory if the NIC/HBA/whatever supports > > : it. > > > > Unless the card is 64bit, it can't DMA past 4G. > > Shouldn't all modern pci chips support two 32bit word addresses. Yeah, right. Find me a 64-bit ATAPI controller... -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 19:14:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 71ED637B401 for ; Sun, 13 Oct 2002 19:14:41 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA0B343EC2 for ; Sun, 13 Oct 2002 19:14:40 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-11.access.nethere.net ([66.63.140.203] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 180ukD-0006cz-00; Sun, 13 Oct 2002 20:14:09 -0600 Message-ID: <3DAA2B4A.477A6BBE@softweyr.com> Date: Sun, 13 Oct 2002 19:26:18 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: "M. Warner Losh" Cc: ticso@cicely.de, ticso@cicely8.cicely.de, hch@infradead.org, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> <20021013103538.GG17920@cicely8.cicely.de> <20021013.060648.93373269.imp@bsdimp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "M. Warner Losh" wrote: > > In message: <20021013103538.GG17920@cicely8.cicely.de> > Bernd Walter writes: > : On Sat, Oct 12, 2002 at 03:06:16PM -0600, M. Warner Losh wrote: > : > In message: <20021012135245.A16453@infradead.org> > : > Christoph Hellwig writes: > : > : On Wed, Oct 09, 2002 at 04:57:37PM -0700, Wes Peters wrote: > : > : > Linux solved this problem by refusing to do it. The candidates for DMA > : > : > transfers include skbufs and buffers from the disk buffer pool, both of > : > : > which are allocated from the lowest 4GB of physical ram when using PAE > : > : > mode. > : > : > : > : Umm, Linux _does_ DMA into any memory if the NIC/HBA/whatever supports > : > : it. > : > > : > Unless the card is 64bit, it can't DMA past 4G. > : > : Shouldn't all modern pci chips support two 32bit word addresses. > > I don't think so. There's only 32-bits on the bus for the address on > most pci cards (not the 64-bit pci cards, which are special), so I > don't see how they can go above 4G. Unless I'm being unusually dense > this morning. The PCI spec has a feature for doing two address word transfers so 32-bit cards can DMA/address 64-bit memory regions, but few if any chipsets support it. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 19:14:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A826E37B401 for ; Sun, 13 Oct 2002 19:14:53 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id E28CA43ECD for ; Sun, 13 Oct 2002 19:14:52 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.3/8.12.3) with ESMTP id g9E2Empk021273; Sun, 13 Oct 2002 20:14:48 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Sun, 13 Oct 2002 20:14:27 -0600 (MDT) Message-Id: <20021013.201427.98861313.imp@bsdimp.com> To: dillon@apollo.backplane.com Cc: arch@FreeBSD.ORG Subject: Re: Database indexes and ram From: "M. Warner Losh" In-Reply-To: <200210131900.g9DJ0ZAM054777@apollo.backplane.com> References: <3DA9B4A8.194A02FC@mindspring.com> <20021013.120847.31902907.imp@bsdimp.com> <200210131900.g9DJ0ZAM054777@apollo.backplane.com> X-Mailer: Mew version 2.1 on Emacs 21.2 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <200210131900.g9DJ0ZAM054777@apollo.backplane.com> Matthew Dillon writes: : Google is your friend. I found a quick reference on the PCI bus. : A 32 bit PCI bus can support 64 bit addresses through the use of : two address cycles prefacing the data transfer. Right, but the cards have to support it as well, and most of the pci cards extant today simply do not support 64-bit operations. The 64-bit bit in BARS is set to 0 (well, more accuratelly, the decode size field is set to 00 or 01, meaning 4G or 1M of decode logic respectively). This means that most of them are incapible of generating the 64-bit addresses needed to do dma above 4G. Maybe the bridge chipsets support it, but very few cards do (and none of the ones that I've brushed up against, but I tend to get cards at the low end of the spectrum). Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 19:19:57 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7267F37B401 for ; Sun, 13 Oct 2002 19:19:56 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id A4E9443EB7 for ; Sun, 13 Oct 2002 19:19:55 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-11.access.nethere.net ([66.63.140.203] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 180uoP-0006e0-00; Sun, 13 Oct 2002 20:18:29 -0600 Message-ID: <3DAA2C4F.9E15CA75@softweyr.com> Date: Sun, 13 Oct 2002 19:30:39 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: ticso@cicely.de, "M. Warner Losh" , hch@infradead.org, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <3DA954CF.98B0891A@mindspring.com> <20021013.060851.113437955.imp@bsdimp.com> <3DA9B4A8.194A02FC@mindspring.com> <20021013.120847.31902907.imp@bsdimp.com> <20021013181633.GB34517@cicely8.cicely.de> <3DA9C3B9.E78BBFE6@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > > Bernd Walter wrote: > > Of course they can do. > > It's just a matter if the card and the board support 2 address cycles. > > Or if the board can map the pci reachable space - as alphas can do. > > The question is whether you can say reliably that all cards that > will be sharing cached data space can do this, or whether you > will have to bounce the data to below 4G. > > If you can't *know*, then to ensure operation, you *must* bounce > the data to proactively guarantee that the physical address will > be in range of the card's DMA engine. > > Among other things, this means recognizing a 32 bit card in a 64 > bit slot, and a 64 bit card in a 32 bit slot, and a 64 bit card > in a 64 bit slot, but which has only 32 bits worth of electrical > connector on the physical card. > > If you can guarantee that, then you can do it without bouncing. > > Can you do that? No, and that's exactly why the Linux developers took the tack they did: all of the DMA targets are allocated in the lower 4GB of physical address space. It was quite an intelligent decision, one that made me grin when I "got it." -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 20:11:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B1E8A37B401 for ; Sun, 13 Oct 2002 20:11:27 -0700 (PDT) Received: from snipe.mail.pas.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 43D2A43EB1 for ; Sun, 13 Oct 2002 20:11:27 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0015.cvx22-bradley.dialup.earthlink.net ([209.179.198.15] helo=mindspring.com) by snipe.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 180vdG-0007g2-00; Sun, 13 Oct 2002 20:11:03 -0700 Message-ID: <3DAA357D.B0E51BB5@mindspring.com> Date: Sun, 13 Oct 2002 20:09:49 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: ticso@cicely.de, "M. Warner Losh" , hch@infradead.org, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <3DA954CF.98B0891A@mindspring.com> <20021013.060851.113437955.imp@bsdimp.com> <3DA9B4A8.194A02FC@mindspring.com> <20021013.120847.31902907.imp@bsdimp.com> <20021013181633.GB34517@cicely8.cicely.de> <3DA9C3B9.E78BBFE6@mindspring.com> <3DAA2C4F.9E15CA75@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > No, and that's exactly why the Linux developers took the tack they did: > all of the DMA targets are allocated in the lower 4GB of physical address > space. It was quite an intelligent decision, one that made me grin when > I "got it." Did you ever have Dr. Tripp, e.g. for Analytical Mechanics? I think this falls into what he called IOTTMCO: Intuitively Obvious To The Most Casual Observer. 8-). It's reasonable to bounce the memory below 4G. It's the same deal for ISA drivers today. The Problem with the Alpha 2G limit is that it has the same type of issue -- basically, the code does not use the Bus Space code. In theory, the interface that's already there could handle all the necessary bouncing automatically (assuming you could tell if you could get away without the bouncing, without attributing the driver, like Linux does). I'm still not sure that, with the VM limitations, etc., that the ability to use the extra memory is worth all that much. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 21:31: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E8E3037B401; Sun, 13 Oct 2002 21:31:03 -0700 (PDT) Received: from mail.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6060D43EAC; Sun, 13 Oct 2002 21:31:03 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from localhost (eischen@localhost) by mail.pcnet.com (8.12.3/8.12.1) with ESMTP id g9E4UvsI015443; Mon, 14 Oct 2002 00:30:57 -0400 (EDT) Date: Mon, 14 Oct 2002 00:30:57 -0400 (EDT) From: Daniel Eischen To: Tim Robbins Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Macros in In-Reply-To: <20021014094459.A25812@dilbert.robbins.dropbear.id.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 14 Oct 2002, Tim Robbins wrote: > Since revision 1.25 of src/include/stdio.h removed the macro versions of > putc() and getc(), is there any reason why we shouldn't go the whole way > and remove the rest of the macros (*_unlocked()) ? These functions are defined by POSIX as thread-safe functions. The are allowed to be implemented as macros, but not required to be. You need to add them as functions before you can remove them as macros. I am not sure whether the performance gain warrants keeping them as macros or not. > This would let us make FILE opaque and move the definition of > struct __sFILE into src/lib/libc/stdio/local.h. This would also let us > remove the __sFILEX hack. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 23:34:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 782BE37B401 for ; Sun, 13 Oct 2002 23:34:22 -0700 (PDT) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3704D43EB2 for ; Sun, 13 Oct 2002 23:34:22 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by canning.wemm.org (Postfix) with ESMTP id 1056D2A88D; Sun, 13 Oct 2002 23:34:22 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Wes Peters Cc: Terry Lambert , ticso@cicely.de, "M. Warner Losh" , hch@infradead.org, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram In-Reply-To: <3DAA2C4F.9E15CA75@softweyr.com> Date: Sun, 13 Oct 2002 23:34:22 -0700 From: Peter Wemm Message-Id: <20021014063422.1056D2A88D@canning.wemm.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > Terry Lambert wrote: > > > > Bernd Walter wrote: > > > Of course they can do. > > > It's just a matter if the card and the board support 2 address cycles. > > > Or if the board can map the pci reachable space - as alphas can do. > > > > The question is whether you can say reliably that all cards that > > will be sharing cached data space can do this, or whether you > > will have to bounce the data to below 4G. > > > > If you can't *know*, then to ensure operation, you *must* bounce > > the data to proactively guarantee that the physical address will > > be in range of the card's DMA engine. > > > > Among other things, this means recognizing a 32 bit card in a 64 > > bit slot, and a 64 bit card in a 32 bit slot, and a 64 bit card > > in a 64 bit slot, but which has only 32 bits worth of electrical > > connector on the physical card. > > > > If you can guarantee that, then you can do it without bouncing. > > > > Can you do that? > > No, and that's exactly why the Linux developers took the tack they did: > all of the DMA targets are allocated in the lower 4GB of physical address > space. It was quite an intelligent decision, one that made me grin when > I "got it." And then there's the AGP remap table stuff to provide a window from anywhere in memory into the lower 4G of space that's within reach of 32 bit PCI devices... Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 13 23:53:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9CF3537B401 for ; Sun, 13 Oct 2002 23:53:14 -0700 (PDT) Received: from swan.mail.pas.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F22243EB2 for ; Sun, 13 Oct 2002 23:53:14 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0276.cvx21-bradley.dialup.earthlink.net ([209.179.193.21] helo=mindspring.com) by swan.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 180z5U-0005Mo-00; Sun, 13 Oct 2002 23:52:24 -0700 Message-ID: <3DAA695F.96E345E6@mindspring.com> Date: Sun, 13 Oct 2002 23:51:11 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Peter Wemm Cc: Wes Peters , ticso@cicely.de, "M. Warner Losh" , hch@infradead.org, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram References: <20021014063422.1056D2A88D@canning.wemm.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter Wemm wrote: > And then there's the AGP remap table stuff to provide a window from > anywhere in memory into the lower 4G of space that's within reach of 32 bit > PCI devices... Peter, you are sneaky... in a good way. Do you really think this could be abused this way? How much, in the way of windows, can you have lying around at one time? I'm not very AGP-aware, other than "I need it for my X server". If you could have one per outstanding request to a card, that would be enough, I think... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 14 0:34: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9F2CC37B401; Mon, 14 Oct 2002 00:34:08 -0700 (PDT) Received: from vbook.express.ru (asplinux.ru [195.133.213.194]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F51743EB2; Mon, 14 Oct 2002 00:34:08 -0700 (PDT) (envelope-from vova@express.ru) Received: from vova by vbook.express.ru with local (Exim 3.36 #1) id 180zjq-000OrL-00; Mon, 14 Oct 2002 11:34:06 +0400 To: freebsd-arch@freebsd.org, freebsd-current@freebsd.org Subject: short uid/gid Message-Id: From: "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" Date: Mon, 14 Oct 2002 11:34:06 +0400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi I have found that SysVIPC functions uses structure with short uid/gid types. What is valid solution ? Change types to uid_t/gid_t (but this will broke binary compatibility) Change syscalls to old_* and add new with "right" structures, or something else ? struct ipc_perm { ushort cuid; /* creator user id */ ushort cgid; /* creator group id */ ushort uid; /* user id */ ushort gid; /* group id */ ushort mode; /* r/w permission */ ushort seq; /* sequence # (to generate unique msg/sem/shm id) */ key_t key; /* user specified msg/sem/shm key */ }; -- Vladimir B. Grebenschikov vova@sw.ru, SWsoft, Inc. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 14 7:31:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2791B37B401 for ; Mon, 14 Oct 2002 07:31:57 -0700 (PDT) Received: from phoenix.infradead.org (carisma.slowglass.com [195.224.96.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id D5B9543E9C for ; Mon, 14 Oct 2002 07:31:51 -0700 (PDT) (envelope-from hch@infradead.org) Received: from hch by phoenix.infradead.org with local (Exim 4.10) id 1816Dt-0004Kg-00; Mon, 14 Oct 2002 15:29:33 +0100 Date: Mon, 14 Oct 2002 15:29:33 +0100 From: Christoph Hellwig To: Terry Lambert Cc: Christoph Hellwig , "M. Warner Losh" , wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.ORG Subject: Re: Database indexes and ram Message-ID: <20021014152933.A16492@infradead.org> References: <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> <20021013133106.C15151@infradead.org> <3DA9B4E1.1257DAA8@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <3DA9B4E1.1257DAA8@mindspring.com>; from tlambert2@mindspring.com on Sun, Oct 13, 2002 at 11:01:05AM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Oct 13, 2002 at 11:01:05AM -0700, Terry Lambert wrote: > Christoph Hellwig wrote: > > On Sat, Oct 12, 2002 at 03:06:16PM -0600, M. Warner Losh wrote: > > > Unless the card is 64bit, it can't DMA past 4G. > > > > Right. But if you have a server with > 4GB memory you usually have > > thos cards. Dito for the non-PCI devices. > > Name one motherboard with more than 2 64 bit PCI slots. IBM summit (x440), IBM/Sequent NUMAQ, Unisys es7xxx, many serverworks-base big boards. Apples' current G4 PowerMACs, etc for the ones with 32bit CPUs. For 64bit CPUs: most PCI-based UltraSparcs, SGI SN0/SN1/SN2 (mips and ia64), many AlphaServers, many Power4-based IBM pSeries machiens, NEC AzusA, and I"m sure I forgot a few. Not to mention other designs on which Linux doesn't even run. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 14 8:37:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 33A2D37B401 for ; Mon, 14 Oct 2002 08:37:52 -0700 (PDT) Received: from phoenix.infradead.org (carisma.slowglass.com [195.224.96.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7883143E8A for ; Mon, 14 Oct 2002 08:37:51 -0700 (PDT) (envelope-from hch@infradead.org) Received: from hch by phoenix.infradead.org with local (Exim 4.10) id 1817Hb-0004zA-00; Mon, 14 Oct 2002 16:37:27 +0100 Date: Mon, 14 Oct 2002 16:37:27 +0100 From: Christoph Hellwig To: Wes Peters Cc: Matthew Dillon , "Vladimir B. Grebenschikov" , Nate Lawson , arch@FreeBSD.ORG Subject: Re: Database indexes and ram (was Re: using mem above 4Gb was:swapon some regular file) Message-ID: <20021014163727.A18798@infradead.org> References: <1034105993.913.1.camel@vbook.express.ru> <200210082015.g98KFFrq084625@apollo.backplane.com> <1034109053.913.7.camel@vbook.express.ru> <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> <3DAA27F9.DBDA4EFE@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <3DAA27F9.DBDA4EFE@softweyr.com>; from wes@softweyr.com on Sun, Oct 13, 2002 at 07:12:10PM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Oct 13, 2002 at 07:12:10PM -0700, Wes Peters wrote: > And the driver has been modified. There are 3 or 4 drivers that have > been, not a daunting majority by any stretch. There are about each 20 NIC and HBA and a few other drivers. I.e. the drivers for hardware that has this feature usually supports it. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 14 10:10: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1AE7C37B401 for ; Mon, 14 Oct 2002 10:10:08 -0700 (PDT) Received: from genius.tao.org.uk (genius.tao.org.uk [212.135.162.51]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A3EC43EAC for ; Mon, 14 Oct 2002 10:10:07 -0700 (PDT) (envelope-from joe@genius.tao.org.uk) Received: by genius.tao.org.uk (Postfix, from userid 100) id 3FD264637; Mon, 14 Oct 2002 18:09:49 +0100 (BST) Date: Mon, 14 Oct 2002 18:09:49 +0100 From: Josef Karthauser To: John Birrell Cc: arch@freebsd.org Subject: Re: USB driver design issues Message-ID: <20021014170949.GB25180@genius.tao.org.uk> References: <20021014083622.G261@freebsd1.cimlogic.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20021014083622.G261@freebsd1.cimlogic.com.au> User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Oct 14, 2002 at 08:36:23AM +1000, John Birrell wrote: > I have a few problems with the design of the USB drivers in > FreeBSD. From the investigation I've done so far, the problems > appear to arise from the inability of the code to handle > kldload and kldunload. The result is that I can crash a > RELENG_4 kernel (and probably a -current one too since the USB > driver code is similar) simply by connecting or disconnecting > the USB cable. Sub-optimal. There are known bugs in -stable that I imagine won't ever get fixed. The stack has been reengineered in NetBSD though and ported to -current, but it remains to be seen whether anyone's got the time to port it to -stable. There was a chance that I was going to do it last month, but the sponsor pulled out at the last minute and I'm busy enough to not have the time to attempt this in my free-time. I don't know about the issue that you mention particularly, but it does appear that device disconnection is now very stable in -current. It may be worth trying to put a current box together to try it, or finding someone who can test your driver on a -current box. Joe. -- "As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality." - Albert Einstein, 1921 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 14 10:28:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 993BE37B401; Mon, 14 Oct 2002 10:28:40 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7307843EA9; Mon, 14 Oct 2002 10:28:39 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.12.4/8.12.4) with SMTP id g9EHRwOo062016; Mon, 14 Oct 2002 13:27:58 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Mon, 14 Oct 2002 13:27:57 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" Cc: freebsd-arch@freebsd.org, freebsd-current@freebsd.org Subject: Re: short uid/gid In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Yeah, this is a Known Problem, and it's quite unfortunate, actually. I looked at trying to solve it -- changing the types respectively to uid_t, gid_t, and mode_t, but it involved a lot of ABI munging because the structures are shared between the userland interface and the kernel implementation. The result is that any change in the kernel structure really requires you to break out the user structures from the kernel structure, which requires a lot of work. Also, ipc_perm is shared between all three SysVIPC services, and to compound things, there are already ofoo() interfaces for older versions of the structures. My belief is that seperating the user and kernel structures is really necessary -- making a kipc_perm, etc, so we can better support fine-grained locking and extensible security. However, someone has to do the grunt work, and last time I tried, I spent several days and only made a bit of progress. If you want to give a first pass at breaking out the user and kernel structures and send a patch, I'll be happy to work with you to get it integrated. I think the steps are: (1) Divorce user and kernel structures for all of the SysVIPC interfaces, and provide functions to map between them as necessary. (2) Remove the original compatibility interfaces left over from eons ago (and figure out how many eons ago so we know what ABIs we're finally removing). (3) Define new userland versions of necessary structures and create a new set of ofoo() and foo() interfaces based on the change. (4) Go back through and dribble the kernel structures with new toys, such as 'struct label', etc. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories On Mon, 14 Oct 2002, Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info wrote: > > Hi > > I have found that SysVIPC functions uses structure with short uid/gid types. > > What is valid solution ? > > Change types to uid_t/gid_t (but this will broke binary compatibility) > Change syscalls to old_* and add new with "right" structures, > or something else ? > > struct ipc_perm { > ushort cuid; /* creator user id */ > ushort cgid; /* creator group id */ > ushort uid; /* user id */ > ushort gid; /* group id */ > ushort mode; /* r/w permission */ > ushort seq; /* sequence # (to generate unique msg/sem/shm id) */ > key_t key; /* user specified msg/sem/shm key */ > }; > > -- > Vladimir B. Grebenschikov > vova@sw.ru, SWsoft, Inc. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 14 11:30:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 55F3237B401 for ; Mon, 14 Oct 2002 11:30:39 -0700 (PDT) Received: from hawk.mail.pas.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7859D43EB2 for ; Mon, 14 Oct 2002 11:30:37 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0078.cvx40-bradley.dialup.earthlink.net ([216.244.42.78] helo=mindspring.com) by hawk.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 1819yf-0000jo-00; Mon, 14 Oct 2002 11:30:05 -0700 Message-ID: <3DAB0CE4.642A949E@mindspring.com> Date: Mon, 14 Oct 2002 11:28:52 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Christoph Hellwig Cc: "M. Warner Losh" , wes@softweyr.com, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.ORG Subject: Re: Database indexes and ram References: <200210082051.g98KpjU1084793@apollo.backplane.com> <3DA4C271.37AACAA3@softweyr.com> <20021012135245.A16453@infradead.org> <20021012.150616.129769790.imp@bsdimp.com> <20021013133106.C15151@infradead.org> <3DA9B4E1.1257DAA8@mindspring.com> <20021014152933.A16492@infradead.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Christoph Hellwig wrote: > > Name one motherboard with more than 2 64 bit PCI slots. > > IBM summit (x440), IBM/Sequent NUMAQ, Unisys es7xxx, many serverworks-base > big boards. Apples' current G4 PowerMACs, etc for the ones with 32bit CPUs. > > For 64bit CPUs: most PCI-based UltraSparcs, SGI SN0/SN1/SN2 (mips and ia64), > many AlphaServers, many Power4-based IBM pSeries machiens, NEC AzusA, > and I"m sure I forgot a few. Not to mention other designs > on which Linux doesn't even run. Non-OEM 32 bit Intel processor using boards are the ones of interest. Someone else pointed out that SuperMicro now sells a couple of them. Most of the 64 bit slots are limited to 66MHz on most of the boards on the page theiy pointed to, even for the OEM boards, with only 2 slots at 100MHz. The interesting question is bandwidth, for the problems we've been talking about. Probably, I should have asked for "non-OEM 32 bit Intel processor boards with 64 bit PCI-PCI bridge chips, no 32 bit slots, and a guaranteed 64 bit datapath all the way to all the cards". PCI-X for all is interesting, too, in that it raises the bandwith limit to a burst rate of 8Gbit. IMO, if the UVA space isn't enlarged, you are going to be paying the cost in I/O for the database applications we've been discussing. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 14 12:18:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8F12537B418; Mon, 14 Oct 2002 12:18:32 -0700 (PDT) Received: from baraca.united.net.ua (ns.united.net.ua [193.111.8.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 086F443E77; Mon, 14 Oct 2002 12:18:30 -0700 (PDT) (envelope-from max@vega.com) Received: from vega.vega.com (xDSL-2-2.united.net.ua [193.111.9.226]) by baraca.united.net.ua (8.11.6/8.11.6) with ESMTP id g9EJILs47634; Mon, 14 Oct 2002 22:18:23 +0300 (EEST) (envelope-from max@vega.com) Received: from vega.vega.com (max@localhost [127.0.0.1]) by vega.vega.com (8.12.6/8.12.5) with ESMTP id g9EJIFab000561; Mon, 14 Oct 2002 22:18:15 +0300 (EEST) (envelope-from sobomax@FreeBSD.org) Received: (from max@localhost) by vega.vega.com (8.12.6/8.12.5/Submit) id g9EJIANV000560; Mon, 14 Oct 2002 22:18:10 +0300 (EEST) Date: Mon, 14 Oct 2002 22:18:10 +0300 From: Maxim Sobolev To: Robert Watson Cc: "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid Message-ID: <20021014191810.GA338@vega.vega.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Linux solved the problem by introducing a new flag for {msg,shm,sem}ctl(3) interfaces (IPC_64), which if set tells the kernel that user supplies new version of the structure. The kernel itself internally keeps all relevant information already in IPC_64 format, doing conversion before returning it to user if IPC_64 isn't set. I think that for portability reasons we should at least consider the same or similar route. -Maxim On Mon, Oct 14, 2002 at 01:27:57PM -0400, Robert Watson wrote: > Yeah, this is a Known Problem, and it's quite unfortunate, actually. I > looked at trying to solve it -- changing the types respectively to uid_t, > gid_t, and mode_t, but it involved a lot of ABI munging because the > structures are shared between the userland interface and the kernel > implementation. The result is that any change in the kernel structure > really requires you to break out the user structures from the kernel > structure, which requires a lot of work. Also, ipc_perm is shared between > all three SysVIPC services, and to compound things, there are already > ofoo() interfaces for older versions of the structures. My belief is that > seperating the user and kernel structures is really necessary -- making a > kipc_perm, etc, so we can better support fine-grained locking and > extensible security. However, someone has to do the grunt work, and last > time I tried, I spent several days and only made a bit of progress. If > you want to give a first pass at breaking out the user and kernel > structures and send a patch, I'll be happy to work with you to get it > integrated. I think the steps are: > > (1) Divorce user and kernel structures for all of the SysVIPC interfaces, > and provide functions to map between them as necessary. > > (2) Remove the original compatibility interfaces left over from eons ago > (and figure out how many eons ago so we know what ABIs we're finally > removing). > > (3) Define new userland versions of necessary structures and create a new > set of ofoo() and foo() interfaces based on the change. > > (4) Go back through and dribble the kernel structures with new toys, such > as 'struct label', etc. > > Robert N M Watson FreeBSD Core Team, TrustedBSD Projects > robert@fledge.watson.org Network Associates Laboratories > > On Mon, 14 Oct 2002, Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info wrote: > > > > > Hi > > > > I have found that SysVIPC functions uses structure with short uid/gid types. > > > > What is valid solution ? > > > > Change types to uid_t/gid_t (but this will broke binary compatibility) > > Change syscalls to old_* and add new with "right" structures, > > or something else ? > > > > struct ipc_perm { > > ushort cuid; /* creator user id */ > > ushort cgid; /* creator group id */ > > ushort uid; /* user id */ > > ushort gid; /* group id */ > > ushort mode; /* r/w permission */ > > ushort seq; /* sequence # (to generate unique msg/sem/shm id) */ > > key_t key; /* user specified msg/sem/shm key */ > > }; > > > > -- > > Vladimir B. Grebenschikov > > vova@sw.ru, SWsoft, Inc. > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-arch" in the body of the message > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-current" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 14 13:37: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 54C7B37B401; Mon, 14 Oct 2002 13:37:03 -0700 (PDT) Received: from cimlogic.com.au (cimlog.lnk.telstra.net [139.130.51.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2251043EAA; Mon, 14 Oct 2002 13:37:01 -0700 (PDT) (envelope-from jb@cimlogic.com.au) Received: from freebsd1.cimlogic.com.au (localhost.cimlogic.com.au [127.0.0.1]) by cimlogic.com.au (8.12.6/8.12.6) with ESMTP id g9EKarVe042150; Tue, 15 Oct 2002 06:36:53 +1000 (EST) (envelope-from jb@cimlogic.com.au) Received: (from jb@localhost) by freebsd1.cimlogic.com.au (8.12.6/8.12.6/Submit) id g9EKaqR8042149; Tue, 15 Oct 2002 06:36:52 +1000 (EST) Date: Tue, 15 Oct 2002 06:36:52 +1000 From: John Birrell To: Josef Karthauser Cc: John Birrell , arch@FreeBSD.org Subject: Re: USB driver design issues Message-ID: <20021015063651.I261@freebsd1.cimlogic.com.au> References: <20021014083622.G261@freebsd1.cimlogic.com.au> <20021014170949.GB25180@genius.tao.org.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20021014170949.GB25180@genius.tao.org.uk>; from joe@FreeBSD.org on Mon, Oct 14, 2002 at 06:09:49PM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Oct 14, 2002 at 06:09:49PM +0100, Josef Karthauser wrote: > There are known bugs in -stable that I imagine won't ever get fixed. > The stack has been reengineered in NetBSD though and ported to > -current, but it remains to be seen whether anyone's got the time > to port it to -stable. I might try that. A potential problem is that I only have a few USB devices. > I don't know about the issue that you mention particularly, but it does > appear that device disconnection is now very stable in -current. It may > be worth trying to put a current box together to try it, or finding > someone who can test your driver on a -current box. No it doesn't (on inspection). The uhub driver is missing DEVMETHOD(bus_child_detached, uhub_child_detached) in the uhubroot so that device isn't cleared when a driver is kldunloaded. The -current version has DEVMETHOD(bus_driver_added, uhub_driver_added), but the method is empty. This is required in -stable to stop the bus_generic_driver_added from being called and probing the devices without setting up the attach_arg structure. And empty uhub_driver_added method means that a USB device has to be disconnected and re-connected before the device is probed as ugen after a driver is kldunloaded. -- John Birrell To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 14 18:14:37 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E17E637B401 for ; Mon, 14 Oct 2002 18:14:35 -0700 (PDT) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A98143EB3 for ; Mon, 14 Oct 2002 18:14:35 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by canning.wemm.org (Postfix) with ESMTP id 763AE2A88D; Mon, 14 Oct 2002 18:14:35 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Terry Lambert Cc: Wes Peters , ticso@cicely.de, "M. Warner Losh" , hch@infradead.org, dillon@apollo.backplane.com, vova@sw.ru, nate@root.org, arch@FreeBSD.org Subject: Re: Database indexes and ram In-Reply-To: <3DAA695F.96E345E6@mindspring.com> Date: Mon, 14 Oct 2002 18:14:35 -0700 From: Peter Wemm Message-Id: <20021015011435.763AE2A88D@canning.wemm.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > Peter Wemm wrote: > > And then there's the AGP remap table stuff to provide a window from > > anywhere in memory into the lower 4G of space that's within reach of 32 bit > > PCI devices... > > Peter, you are sneaky... in a good way. > > Do you really think this could be abused this way? How much, in the > way of windows, can you have lying around at one time? I'm not very > AGP-aware, other than "I need it for my X server". I cant claim credit for this.. The linux folks supposedly have it working - David O'Brien told me about it. Anyway, with this you then are required to have the kernel manage the aperture space. For a desktop, the real question is whether you want to slow down the agp-using X server in order to avoid bounces by using resources that could otherwise be used for doing graphics. But for a server with no agp card in use, it would presumably be a no-brainer. The way that I understand that it works is that the northbridge/memory controller/whatever provides an alternative "view" of system memory. > If you could have one per outstanding request to a card, that would > be enough, I think... I believe this is a function of the agp/memory subsystem (which often share the same silicon), not the graphics card itself. I mentioned this before and one of my misunderstandings was corrected. I'm still not sure that I have the details correct. If you read some of the pci/agp_* drivers (I read agp_amd.c), you will see that the chips have things like page tables, tlb's etc. And you can connect arbitary memory pages to the mapping tables to make them visible in the aperture (below 4G) so that busmaster devices can get to them. I'm not entirely sure what the graphics subsystem *does* with this stuff, but I assume it is useful for something. Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 7:20: 8 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 94CF237B401; Tue, 15 Oct 2002 07:20:07 -0700 (PDT) Received: from corbulon.video-collage.com (corbulon.video-collage.com [64.35.99.179]) by mx1.FreeBSD.org (Postfix) with ESMTP id AE77943EAF; Tue, 15 Oct 2002 07:20:01 -0700 (PDT) (envelope-from mi+celsius@aldan.algebra.com) Received: from 250-217.customer.cloud9.net (250-217.customer.cloud9.net [168.100.250.217]) by corbulon.video-collage.com (8.12.6/8.12.6) with ESMTP id g9FEJrGx060458 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=FAIL); Tue, 15 Oct 2002 10:19:55 -0400 (EDT) (envelope-from mi+celsius@aldan.algebra.com) Received: from celsius.virtual-estates.com ([172.21.19.243]) by 250-217.customer.cloud9.net (8.12.6/8.12.6) with ESMTP id g9FELexm050084; Tue, 15 Oct 2002 10:21:53 -0400 (EDT) (envelope-from mi+celsius@aldan.algebra.com) Content-Type: text/plain; charset="koi8-u" From: Mikhail Teterin To: "Daniel C. Sobral" Subject: Re: cvs commit: src/usr.sbin/syslogd syslog.conf.5 syslogd.c Date: Tue, 15 Oct 2002 16:22:16 +0200 User-Agent: KMail/1.4.3 References: <200210111114.g9BBExUw083255@freefall.freebsd.org> In-Reply-To: <200210111114.g9BBExUw083255@freefall.freebsd.org> Cc: arch@FreeBSD.org MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200210151622.16764.mi+celsius@aldan.algebra.com> X-Scanned-By: MIMEDefang 2.21 (www . roaringpenguin . com / mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Friday 11 October 2002 01:14 pm, Daniel C. Sobral wrote: = dcs 2002/10/11 04:14:58 PDT = = Modified files: (Branch: RELENG_4) = usr.sbin/syslogd syslog.conf.5 syslogd.c = Log: = MFC: rev 1.107. Introduce !-program and !+program, so you can specify the = opposite behavior of !program. This reminds me -- a while ago I needed to configure a machine to act as the "loghost", and wanted to save messages from different machines into different files. Can such a thing be added too? -mi To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 7:42:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BEC5537B401; Tue, 15 Oct 2002 07:42:42 -0700 (PDT) Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id B3C9143EAA; Tue, 15 Oct 2002 07:42:28 -0700 (PDT) (envelope-from ru@whale.sunbay.crimea.ua) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g9FEdHT19775; Tue, 15 Oct 2002 17:39:17 +0300 (EEST) (envelope-from ru) Date: Tue, 15 Oct 2002 17:39:17 +0300 From: Ruslan Ermilov To: Mikhail Teterin Cc: "Daniel C. Sobral" , arch@FreeBSD.ORG Subject: Re: cvs commit: src/usr.sbin/syslogd syslog.conf.5 syslogd.c Message-ID: <20021015143917.GA18943@sunbay.com> References: <200210111114.g9BBExUw083255@freefall.freebsd.org> <200210151622.16764.mi+celsius@aldan.algebra.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pf9I7BMVVzbSWLtt" Content-Disposition: inline In-Reply-To: <200210151622.16764.mi+celsius@aldan.algebra.com> User-Agent: Mutt/1.3.99i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --pf9I7BMVVzbSWLtt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Oct 15, 2002 at 04:22:16PM +0200, Mikhail Teterin wrote: > On Friday 11 October 2002 01:14 pm, Daniel C. Sobral wrote: > =3D dcs 2002/10/11 04:14:58 PDT > =3D > =3D Modified files: (Branch: RELENG_4) > =3D usr.sbin/syslogd syslog.conf.5 syslogd.c > =3D Log: > =3D MFC: rev 1.107. Introduce !-program and !+program, so you can speci= fy the > =3D opposite behavior of !program. >=20 > This reminds me -- a while ago I needed to configure a machine to act as = the > "loghost", and wanted to save messages from different machines into diffe= rent =20 > files. Can such a thing be added too? >=20 Once again? : A hostname specification of the form `#+hostname' or : `+hostname' and the following blocks will be applied to messages received : from the specified hostname. Alternatively, a hostname specification : `#-hostname' or `-hostname' causes the following blocks to be applied to : messages from any host but the one specified. If the hostname is given : as `@', the local hostname will be used. A program or hostname specifi- : cation may be reset by giving the program or hostname as `*'. Cheers, --=20 Ruslan Ermilov Sysadmin and DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age --pf9I7BMVVzbSWLtt Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (FreeBSD) iD8DBQE9rCiVUkv4P6juNwoRAsQLAKCFI1M1T3K9ICmAcXE9Kefu4krndACeLwE8 bWVdsLyi28HiTJ3sEpjmvSo= =re2I -----END PGP SIGNATURE----- --pf9I7BMVVzbSWLtt-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 8: 5:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BB76A37B401; Tue, 15 Oct 2002 08:05:37 -0700 (PDT) Received: from corbulon.video-collage.com (corbulon.video-collage.com [64.35.99.179]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F78943E97; Tue, 15 Oct 2002 08:05:36 -0700 (PDT) (envelope-from mi+celsius@aldan.algebra.com) Received: from 250-217.customer.cloud9.net (250-217.customer.cloud9.net [168.100.250.217]) by corbulon.video-collage.com (8.12.6/8.12.6) with ESMTP id g9FF5XGx060970 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=FAIL); Tue, 15 Oct 2002 11:05:34 -0400 (EDT) (envelope-from mi+celsius@aldan.algebra.com) Received: from celsius.virtual-estates.com ([172.21.19.243]) by 250-217.customer.cloud9.net (8.12.6/8.12.6) with ESMTP id g9FF7Vxm050197; Tue, 15 Oct 2002 11:07:32 -0400 (EDT) (envelope-from mi+celsius@aldan.algebra.com) Content-Type: text/plain; charset="iso-8859-1" From: Mikhail Teterin To: Ruslan Ermilov Subject: Re: cvs commit: src/usr.sbin/syslogd syslog.conf.5 syslogd.c Date: Tue, 15 Oct 2002 17:08:07 +0200 User-Agent: KMail/1.4.3 Cc: "Daniel C. Sobral" , arch@FreeBSD.ORG References: <200210111114.g9BBExUw083255@freefall.freebsd.org> <200210151622.16764.mi+celsius@aldan.algebra.com> <20021015143917.GA18943@sunbay.com> In-Reply-To: <20021015143917.GA18943@sunbay.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200210151708.07772.mi+celsius@aldan.algebra.com> X-Scanned-By: MIMEDefang 2.21 (www . roaringpenguin . com / mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tuesday 15 October 2002 04:39 pm, Ruslan Ermilov wrote: = Once again? = : A hostname specification of the form `#+hostname' or = : `+hostname' and the following blocks will be applied to messages received = : from the specified hostname. Alternatively, a hostname specification = : `#-hostname' or `-hostname' causes the following blocks to be applied to = : messages from any host but the one specified. If the hostname is given = : as `@', the local hostname will be used. A program or hostname specifi- = : cation may be reset by giving the program or hostname as `*'. Ouch! My appologies :-) I remember raising this issue years ago, but, I guess, by the time it was implemented, I had some workaround. Time for some memory enhancing drugs, I guess... -mi To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 8:52:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D78FA37B401 for ; Tue, 15 Oct 2002 08:52:29 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id E2AEE43EAA for ; Tue, 15 Oct 2002 08:52:28 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id LAA04413; Tue, 15 Oct 2002 11:52:28 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g9FFpw655630; Tue, 15 Oct 2002 11:51:58 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15788.14750.474882.99633@grasshopper.cs.duke.edu> Date: Tue, 15 Oct 2002 11:51:58 -0400 (EDT) To: Peter Wemm Cc: arch@FreeBSD.org Subject: Re: Database indexes and ram In-Reply-To: <20021015011435.763AE2A88D@canning.wemm.org> References: <3DAA695F.96E345E6@mindspring.com> <20021015011435.763AE2A88D@canning.wemm.org> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter Wemm writes: > Terry Lambert wrote: > > Peter Wemm wrote: > > > And then there's the AGP remap table stuff to provide a window from > > > anywhere in memory into the lower 4G of space that's within reach of 32 bit > > > PCI devices... > > > > Peter, you are sneaky... in a good way. > > > > Do you really think this could be abused this way? How much, in the > > way of windows, can you have lying around at one time? I'm not very > > AGP-aware, other than "I need it for my X server". > > I cant claim credit for this.. The linux folks supposedly have it working - > David O'Brien told me about it. > FWIW, Doug Rabson thought of this a few years ago when we first ported FreeBSD/alpha to the API UP1x000. Unlike most alphas, it has an AGP slot, and it does not have a pci chipset capable of doing scatter/gather dma mapping. Doug suggested that I use the AGP remap table for ISA dma, but I could never make it work. I ended up just doing bounce buffers. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 11:45:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E626337B401 for ; Tue, 15 Oct 2002 11:45:44 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 554F643EB3 for ; Tue, 15 Oct 2002 11:45:44 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id OAA12141 for ; Tue, 15 Oct 2002 14:45:43 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g9FIjDX55815; Tue, 15 Oct 2002 14:45:13 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15788.25145.767671.511786@grasshopper.cs.duke.edu> Date: Tue, 15 Oct 2002 14:45:13 -0400 (EDT) To: freebsd-arch@freebsd.org Subject: when will malloc(9) be giant-free? X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I'm trying to call malloc from a mostly giant-free driver, and dying like this: panic(c0346b0e,c0346c57,c03584e6,138,c037caa0) at panic+0x11f _mtx_assert(c037cc60,1,c03584e6,138,158) at _mtx_assert+0xbc kmem_malloc(c0832078,21000,1,d8e287ec,c02db8ea) at kmem_malloc+0x2d page_alloc(0,21000,d8e287df,1,23459e4) at page_alloc+0x27 uma_large_malloc(21000,1,c02db064,6,c4435cfc) at uma_large_malloc+0x5a malloc(20fff,c037af20,1,d8e2883c,c4741873) at malloc+0xcd __gm_arch_kernel_malloc(20fff,0,d8e2884c,c47435d1,20fff) at __gm_arch_kernel_malloc+0x12 So, yes, I'll grab giant around calls to malloc and free, but when do we anticipate that kmem_malloc() will be giant-free? Yes, I whined about this a few months ago in relation to mbuf allocation at interrupt time (I was porting the ethernet emulation side of our driver to -current). Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 13:18:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D112137B401 for ; Tue, 15 Oct 2002 13:18:54 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id C9A6C43E6E for ; Tue, 15 Oct 2002 13:18:53 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id g9FKIjIw039166; Tue, 15 Oct 2002 22:18:46 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Andrew Gallatin Cc: freebsd-arch@FreeBSD.ORG Subject: Re: when will malloc(9) be giant-free? In-Reply-To: Your message of "Tue, 15 Oct 2002 14:45:13 EDT." <15788.25145.767671.511786@grasshopper.cs.duke.edu> Date: Tue, 15 Oct 2002 22:18:45 +0200 Message-ID: <39165.1034713125@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <15788.25145.767671.511786@grasshopper.cs.duke.edu>, Andrew Gallatin writes: > >I'm trying to call malloc from a mostly giant-free driver, and dying >like this: > >panic(c0346b0e,c0346c57,c03584e6,138,c037caa0) at panic+0x11f >_mtx_assert(c037cc60,1,c03584e6,138,158) at _mtx_assert+0xbc >kmem_malloc(c0832078,21000,1,d8e287ec,c02db8ea) at kmem_malloc+0x2d >page_alloc(0,21000,d8e287df,1,23459e4) at page_alloc+0x27 >uma_large_malloc(21000,1,c02db064,6,c4435cfc) at uma_large_malloc+0x5a >malloc(20fff,c037af20,1,d8e2883c,c4741873) at malloc+0xcd >__gm_arch_kernel_malloc(20fff,0,d8e2884c,c47435d1,20fff) at __gm_arch_kernel_malloc+0x12 >[...] >Yes, I whined about this a few months ago in relation to mbuf allocation >at interrupt time (I was porting the ethernet emulation side of our >driver to -current). /me clears his throat and puts a bass-voice on the choir. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 15:17:30 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E27BC37B401; Tue, 15 Oct 2002 15:17:18 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF67643E6A; Tue, 15 Oct 2002 15:17:17 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id g9FMHDIw060638; Wed, 16 Oct 2002 00:17:13 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: net@freebsd.org, arch@freebsd.org Subject: RFC: eliminating the _IP_VHL hack. From: Poul-Henning Kamp Date: Wed, 16 Oct 2002 00:17:13 +0200 Message-ID: <60637.1034720233@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG almost 7 years ago, this commit introduced the _IP_VHL hack in our IP-stack: ] revision 1.7 ] date: 1995/12/21 21:20:27; author: wollman; state: Exp; lines: +5 -1 ] If _IP_VHL is defined, declare a single ip_vhl member in struct ip rather ] than separate ip_v and ip_hl members. Should have no effect on current code, ] but I'd eventually like to get rid of those obnoxious bitfields completely. We can argue a lot about how long time we should wait for "eventually", but I would say that 7 years is far too long, considering the status: In the meantime absolutely no code has picked up on this idea, and source files using the _IP_VHL hack in in the minority in our tree. The side effect of having some source-files using the _IP_VHL hack and some not is that sizeof(struct ip) varies from file to file, which at best is confusing an at worst the source of some really evil bugs. I would therefore propose to eliminate the _IP_VHL hack from the kernel to end this state of (potential) confusion, and invite comments to the following patch: Index: ip.h =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip.h,v retrieving revision 1.19 diff -u -r1.19 ip.h --- ip.h 14 Dec 2001 19:37:32 -0000 1.19 +++ ip.h 15 Oct 2002 22:02:26 -0000 @@ -47,9 +47,6 @@ * Structure of an internet header, naked of options. */ struct ip { -#ifdef _IP_VHL - u_char ip_vhl; /* version << 4 | header length >> 2 */ -#else #if BYTE_ORDER == LITTLE_ENDIAN u_int ip_hl:4, /* header length */ ip_v:4; /* version */ @@ -58,7 +55,6 @@ u_int ip_v:4, /* version */ ip_hl:4; /* header length */ #endif -#endif /* not _IP_VHL */ u_char ip_tos; /* type of service */ u_short ip_len; /* total length */ u_short ip_id; /* identification */ @@ -72,13 +68,6 @@ u_short ip_sum; /* checksum */ struct in_addr ip_src,ip_dst; /* source and dest address */ }; - -#ifdef _IP_VHL -#define IP_MAKE_VHL(v, hl) ((v) << 4 | (hl)) -#define IP_VHL_HL(vhl) ((vhl) & 0x0f) -#define IP_VHL_V(vhl) ((vhl) >> 4) -#define IP_VHL_BORING 0x45 -#endif #define IP_MAXPACKET 65535 /* maximum packet size */ Index: ip_icmp.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_icmp.c,v retrieving revision 1.70 diff -u -r1.70 ip_icmp.c --- ip_icmp.c 1 Aug 2002 03:53:04 -0000 1.70 +++ ip_icmp.c 15 Oct 2002 22:05:23 -0000 @@ -51,7 +51,6 @@ #include #include -#define _IP_VHL #include #include #include @@ -128,7 +127,7 @@ struct ifnet *destifp; { register struct ip *oip = mtod(n, struct ip *), *nip; - register unsigned oiplen = IP_VHL_HL(oip->ip_vhl) << 2; + register unsigned oiplen = oip->ip_hl << 2; register struct icmp *icp; register struct mbuf *m; unsigned icmplen; @@ -214,7 +213,8 @@ nip = mtod(m, struct ip *); bcopy((caddr_t)oip, (caddr_t)nip, sizeof(struct ip)); nip->ip_len = m->m_len; - nip->ip_vhl = IP_VHL_BORING; + nip->ip_v = IPVERSION; + nip->ip_hl = 5; nip->ip_p = IPPROTO_ICMP; nip->ip_tos = 0; icmp_reflect(m); @@ -364,7 +364,7 @@ * Problem with datagram; advise higher level routines. */ if (icmplen < ICMP_ADVLENMIN || icmplen < ICMP_ADVLEN(icp) || - IP_VHL_HL(icp->icmp_ip.ip_vhl) < (sizeof(struct ip) >> 2)) { + icp->icmp_ip.ip_hl < (sizeof(struct ip) >> 2)) { icmpstat.icps_badlen++; goto freeit; } @@ -526,7 +526,7 @@ if (code > 3) goto badcode; if (icmplen < ICMP_ADVLENMIN || icmplen < ICMP_ADVLEN(icp) || - IP_VHL_HL(icp->icmp_ip.ip_vhl) < (sizeof(struct ip) >> 2)) { + icp->icmp_ip.ip_hl < (sizeof(struct ip) >> 2)) { icmpstat.icps_badlen++; break; } @@ -593,7 +593,7 @@ struct in_ifaddr *ia; struct in_addr t; struct mbuf *opts = 0; - int optlen = (IP_VHL_HL(ip->ip_vhl) << 2) - sizeof(struct ip); + int optlen = (ip->ip_hl << 2) - sizeof(struct ip); struct route *ro = NULL, rt; if (!in_canforward(ip->ip_src) && @@ -703,7 +703,8 @@ * mbuf's data back, and adjust the IP length. */ ip->ip_len -= optlen; - ip->ip_vhl = IP_VHL_BORING; + ip->ip_v = IPVERSION; + ip->ip_hl = 5; m->m_len -= optlen; if (m->m_flags & M_PKTHDR) m->m_pkthdr.len -= optlen; @@ -734,7 +735,7 @@ register int hlen; register struct icmp *icp; - hlen = IP_VHL_HL(ip->ip_vhl) << 2; + hlen = ip->ip_hl << 2; m->m_data += hlen; m->m_len -= hlen; icp = mtod(m, struct icmp *); Index: ip_icmp.h =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_icmp.h,v retrieving revision 1.18 diff -u -r1.18 ip_icmp.h --- ip_icmp.h 19 Mar 2002 21:25:46 -0000 1.18 +++ ip_icmp.h 15 Oct 2002 22:02:52 -0000 @@ -123,13 +123,8 @@ #define ICMP_TSLEN (8 + 3 * sizeof (n_time)) /* timestamp */ #define ICMP_MASKLEN 12 /* address mask */ #define ICMP_ADVLENMIN (8 + sizeof (struct ip) + 8) /* min */ -#ifndef _IP_VHL #define ICMP_ADVLEN(p) (8 + ((p)->icmp_ip.ip_hl << 2) + 8) /* N.B.: must separately check that ip_hl >= 5 */ -#else -#define ICMP_ADVLEN(p) (8 + (IP_VHL_HL((p)->icmp_ip.ip_vhl) << 2) + 8) - /* N.B.: must separately check that header length >= 5 */ -#endif /* * Definition of type and code field values. Index: ip_input.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_input.c,v retrieving revision 1.211 diff -u -r1.211 ip_input.c --- ip_input.c 10 Oct 2002 12:03:36 -0000 1.211 +++ ip_input.c 15 Oct 2002 22:02:57 -0000 @@ -34,8 +34,6 @@ * $FreeBSD: src/sys/netinet/ip_input.c,v 1.211 2002/10/10 12:03:36 maxim Exp $ */ -#define _IP_VHL - #include "opt_bootp.h" #include "opt_ipfw.h" #include "opt_ipdn.h" @@ -324,7 +322,7 @@ if (args.rule) { /* dummynet already filtered us */ ip = mtod(m, struct ip *); - hlen = IP_VHL_HL(ip->ip_vhl) << 2; + hlen = ip->ip_hl << 2; goto iphack ; } @@ -340,12 +338,12 @@ } ip = mtod(m, struct ip *); - if (IP_VHL_V(ip->ip_vhl) != IPVERSION) { + if (ip->ip_v != IPVERSION) { ipstat.ips_badvers++; goto bad; } - hlen = IP_VHL_HL(ip->ip_vhl) << 2; + hlen = ip->ip_hl << 2; if (hlen < sizeof(struct ip)) { /* minimum header length */ ipstat.ips_badhlen++; goto bad; @@ -755,7 +753,7 @@ ipstat.ips_reassembled++; ip = mtod(m, struct ip *); /* Get the header length of the reassembled packet */ - hlen = IP_VHL_HL(ip->ip_vhl) << 2; + hlen = ip->ip_hl << 2; #ifdef IPDIVERT /* Restore original checksum before diverting packet */ if (divert_info != 0) { @@ -882,7 +880,7 @@ struct ip *ip = mtod(m, struct ip *); register struct mbuf *p, *q, *nq; struct mbuf *t; - int hlen = IP_VHL_HL(ip->ip_vhl) << 2; + int hlen = ip->ip_hl << 2; int i, next; /* @@ -1020,7 +1018,7 @@ */ q = fp->ipq_frags; ip = GETIP(q); - if (next + (IP_VHL_HL(ip->ip_vhl) << 2) > IP_MAXPACKET) { + if (next + (ip->ip_hl << 2) > IP_MAXPACKET) { ipstat.ips_toolong++; ip_freef(head, fp); return (0); @@ -1068,8 +1066,8 @@ nipq--; (void) m_free(dtom(fp)); ip_nfragpackets--; - m->m_len += (IP_VHL_HL(ip->ip_vhl) << 2); - m->m_data -= (IP_VHL_HL(ip->ip_vhl) << 2); + m->m_len += (ip->ip_hl << 2); + m->m_data -= (ip->ip_hl << 2); /* some debugging cruft by sklower, below, will go away soon */ if (m->m_flags & M_PKTHDR) /* XXX this should be done elsewhere */ m_fixhdr(m); @@ -1193,7 +1191,7 @@ dst = ip->ip_dst; cp = (u_char *)(ip + 1); - cnt = (IP_VHL_HL(ip->ip_vhl) << 2) - sizeof (struct ip); + cnt = (ip->ip_hl << 2) - sizeof (struct ip); for (; cnt > 0; cnt -= optlen, cp += optlen) { opt = cp[IPOPT_OPTVAL]; if (opt == IPOPT_EOL) @@ -1582,14 +1580,15 @@ register caddr_t opts; int olen; - olen = (IP_VHL_HL(ip->ip_vhl) << 2) - sizeof (struct ip); + olen = (ip->ip_hl << 2) - sizeof (struct ip); opts = (caddr_t)(ip + 1); i = m->m_len - (sizeof (struct ip) + olen); bcopy(opts + olen, opts, (unsigned)i); m->m_len -= olen; if (m->m_flags & M_PKTHDR) m->m_pkthdr.len -= olen; - ip->ip_vhl = IP_MAKE_VHL(IPVERSION, sizeof(struct ip) >> 2); + ip->ip_v = IPVERSION; + ip->ip_hl = sizeof(struct ip) >> 2; } u_char inetctlerrmap[PRC_NCMDS] = { @@ -1686,7 +1685,7 @@ MGET(mcopy, M_DONTWAIT, m->m_type); if (mcopy != NULL) { M_COPY_PKTHDR(mcopy, m); - mcopy->m_len = imin((IP_VHL_HL(ip->ip_vhl) << 2) + 8, + mcopy->m_len = imin((ip->ip_hl << 2) + 8, (int)ip->ip_len); m_copydata(m, 0, mcopy->m_len, mtod(mcopy, caddr_t)); #ifdef MAC Index: ip_output.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_output.c,v retrieving revision 1.165 diff -u -r1.165 ip_output.c --- ip_output.c 23 Sep 2002 08:56:24 -0000 1.165 +++ ip_output.c 15 Oct 2002 22:03:41 -0000 @@ -34,8 +34,6 @@ * $FreeBSD: src/sys/netinet/ip_output.c,v 1.165 2002/09/23 08:56:24 maxim Exp $ */ -#define _IP_VHL - #include "opt_ipfw.h" #include "opt_ipdn.h" #include "opt_ipdivert.h" @@ -191,7 +189,7 @@ #endif if (args.rule != NULL) { /* dummynet already saw us */ ip = mtod(m, struct ip *); - hlen = IP_VHL_HL(ip->ip_vhl) << 2 ; + hlen = ip->ip_hl << 2 ; if (ro->ro_rt) ia = ifatoia(ro->ro_rt->rt_ifa); goto sendit; @@ -210,7 +208,8 @@ * Fill in IP header. */ if ((flags & (IP_FORWARDING|IP_RAWOUTPUT)) == 0) { - ip->ip_vhl = IP_MAKE_VHL(IPVERSION, hlen >> 2); + ip->ip_v = IPVERSION; + ip->ip_hl = hlen >> 2; ip->ip_off &= IP_DF; #ifdef RANDOM_IP_ID ip->ip_id = ip_randomid(); @@ -219,7 +218,7 @@ #endif ipstat.ips_localout++; } else { - hlen = IP_VHL_HL(ip->ip_vhl) << 2; + hlen = ip->ip_hl << 2; } dst = (struct sockaddr_in *)&ro->ro_dst; @@ -553,11 +552,7 @@ /* be sure to update variables that are affected by ipsec4_output() */ ip = mtod(m, struct ip *); -#ifdef _IP_VHL - hlen = IP_VHL_HL(ip->ip_vhl) << 2; -#else hlen = ip->ip_hl << 2; -#endif if (ro->ro_rt == NULL) { if ((flags & IP_ROUTETOIF) == 0) { printf("ip_output: " @@ -862,13 +857,8 @@ ip->ip_len = htons(ip->ip_len); ip->ip_off = htons(ip->ip_off); ip->ip_sum = 0; - if (sw_csum & CSUM_DELAY_IP) { - if (ip->ip_vhl == IP_VHL_BORING) { - ip->ip_sum = in_cksum_hdr(ip); - } else { - ip->ip_sum = in_cksum(m, hlen); - } - } + if (sw_csum & CSUM_DELAY_IP) + ip->ip_sum = in_cksum(m, hlen); /* Record statistics for this interface address. */ if (!(flags & IP_FORWARDING) && ia) { @@ -988,7 +978,8 @@ *mhip = *ip; if (hlen > sizeof (struct ip)) { mhlen = ip_optcopy(ip, mhip) + sizeof (struct ip); - mhip->ip_vhl = IP_MAKE_VHL(IPVERSION, mhlen >> 2); + mhip->ip_v = IPVERSION; + mhip->ip_hl = mhlen >> 2; } m->m_len = mhlen; mhip->ip_off = ((off - hlen) >> 3) + ip->ip_off; @@ -1012,13 +1003,8 @@ m->m_pkthdr.csum_flags = m0->m_pkthdr.csum_flags; mhip->ip_off = htons(mhip->ip_off); mhip->ip_sum = 0; - if (sw_csum & CSUM_DELAY_IP) { - if (mhip->ip_vhl == IP_VHL_BORING) { - mhip->ip_sum = in_cksum_hdr(mhip); - } else { - mhip->ip_sum = in_cksum(m, mhlen); - } - } + if (sw_csum & CSUM_DELAY_IP) + mhip->ip_sum = in_cksum(m, mhlen); *mnext = m; mnext = &m->m_nextpkt; nfrags++; @@ -1041,13 +1027,8 @@ ip->ip_off |= IP_MF; ip->ip_off = htons(ip->ip_off); ip->ip_sum = 0; - if (sw_csum & CSUM_DELAY_IP) { - if (ip->ip_vhl == IP_VHL_BORING) { - ip->ip_sum = in_cksum_hdr(ip); - } else { - ip->ip_sum = in_cksum(m, hlen); - } - } + if (sw_csum & CSUM_DELAY_IP) + ip->ip_sum = in_cksum(m, hlen); sendorfree: for (m = m0; m; m = m0) { m0 = m->m_nextpkt; @@ -1097,7 +1078,7 @@ u_short csum, offset; ip = mtod(m, struct ip *); - offset = IP_VHL_HL(ip->ip_vhl) << 2 ; + offset = ip->ip_hl << 2 ; csum = in_cksum_skip(m, ip->ip_len, offset); if (m->m_pkthdr.csum_flags & CSUM_UDP && csum == 0) csum = 0xffff; @@ -1169,7 +1150,8 @@ ip = mtod(m, struct ip *); bcopy(p->ipopt_list, ip + 1, optlen); *phlen = sizeof(struct ip) + optlen; - ip->ip_vhl = IP_MAKE_VHL(IPVERSION, *phlen >> 2); + ip->ip_v = IPVERSION; + ip->ip_hl = *phlen >> 2; ip->ip_len += optlen; return (m); } @@ -1187,7 +1169,7 @@ cp = (u_char *)(ip + 1); dp = (u_char *)(jp + 1); - cnt = (IP_VHL_HL(ip->ip_vhl) << 2) - sizeof (struct ip); + cnt = (ip->ip_hl << 2) - sizeof (struct ip); for (; cnt > 0; cnt -= optlen, cp += optlen) { opt = cp[0]; if (opt == IPOPT_EOL) @@ -2025,11 +2007,7 @@ ip->ip_len = htons(ip->ip_len); ip->ip_off = htons(ip->ip_off); ip->ip_sum = 0; - if (ip->ip_vhl == IP_VHL_BORING) { - ip->ip_sum = in_cksum_hdr(ip); - } else { - ip->ip_sum = in_cksum(copym, hlen); - } + ip->ip_sum = in_cksum(copym, hlen); /* * NB: * It's not clear whether there are any lingering Index: raw_ip.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/raw_ip.c,v retrieving revision 1.100 diff -u -r1.100 raw_ip.c --- raw_ip.c 15 Aug 2002 18:51:26 -0000 1.100 +++ raw_ip.c 15 Oct 2002 22:06:09 -0000 @@ -59,7 +59,6 @@ #include #include -#define _IP_VHL #include #include #include @@ -263,10 +262,10 @@ ip = mtod(m, struct ip *); /* don't allow both user specified and setsockopt options, and don't allow packet length sizes that will crash */ - if (((IP_VHL_HL(ip->ip_vhl) != (sizeof (*ip) >> 2)) + if (((ip->ip_hl != (sizeof (*ip) >> 2)) && inp->inp_options) || (ip->ip_len > m->m_pkthdr.len) - || (ip->ip_len < (IP_VHL_HL(ip->ip_vhl) << 2))) { + || (ip->ip_len < (ip->ip_hl << 2))) { m_freem(m); return EINVAL; } Index: tcp_subr.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/tcp_subr.c,v retrieving revision 1.143 diff -u -r1.143 tcp_subr.c --- tcp_subr.c 10 Oct 2002 21:41:30 -0000 1.143 +++ tcp_subr.c 15 Oct 2002 22:06:27 -0000 @@ -62,7 +62,6 @@ #include #include -#define _IP_VHL #include #include #include @@ -284,7 +283,8 @@ { struct ip *ip = (struct ip *) ip_ptr; - ip->ip_vhl = IP_VHL_BORING; + ip->ip_v = IPVERSION; + ip->ip_hl = 5; ip->ip_tos = 0; ip->ip_len = 0; ip->ip_id = 0; @@ -371,7 +371,7 @@ KASSERT(tp != NULL || m != NULL, ("tcp_respond: tp and m both NULL")); #ifdef INET6 - isipv6 = IP_VHL_V(((struct ip *)ipgen)->ip_vhl) == 6; + isipv6 = ((struct ip *)ipgen)->ip_v == 6; ip6 = ipgen; #endif /* INET6 */ ip = ipgen; @@ -1102,7 +1102,7 @@ if (ip) { s = splnet(); th = (struct tcphdr *)((caddr_t)ip - + (IP_VHL_HL(ip->ip_vhl) << 2)); + + (ip->ip_hl << 2)); INP_INFO_WLOCK(&tcbinfo); inp = in_pcblookup_hash(&tcbinfo, faddr, th->th_dport, ip->ip_src, th->th_sport, 0, NULL); -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 15:26:25 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BB40237B401; Tue, 15 Oct 2002 15:26:24 -0700 (PDT) Received: from carp.icir.org (carp.icir.org [192.150.187.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id 59F4D43E6A; Tue, 15 Oct 2002 15:26:24 -0700 (PDT) (envelope-from rizzo@carp.icir.org) Received: from carp.icir.org (localhost [127.0.0.1]) by carp.icir.org (8.12.3/8.12.3) with ESMTP id g9FMQOpJ027014; Tue, 15 Oct 2002 15:26:24 -0700 (PDT) (envelope-from rizzo@carp.icir.org) Received: (from rizzo@localhost) by carp.icir.org (8.12.3/8.12.3/Submit) id g9FMQOAH027013; Tue, 15 Oct 2002 15:26:24 -0700 (PDT) (envelope-from rizzo) Date: Tue, 15 Oct 2002 15:26:24 -0700 From: Luigi Rizzo To: Poul-Henning Kamp Cc: net@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: RFC: eliminating the _IP_VHL hack. Message-ID: <20021015152624.A26977@carp.icir.org> References: <60637.1034720233@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <60637.1034720233@critter.freebsd.dk>; from phk@FreeBSD.ORG on Wed, Oct 16, 2002 at 12:17:13AM +0200 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Oct 16, 2002 at 12:17:13AM +0200, Poul-Henning Kamp wrote: ... > I would therefore propose to eliminate the _IP_VHL hack from the kernel yes, go for it. cheers luigi To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 15:46:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F3DC237B401; Tue, 15 Oct 2002 15:46:38 -0700 (PDT) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1CB8D43E65; Tue, 15 Oct 2002 15:46:38 -0700 (PDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.3/8.12.5) with ESMTP id g9FMkagQ007724 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Tue, 15 Oct 2002 18:46:37 -0400 (EDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.3/8.12.5/Submit) id g9FMka0o007721; Tue, 15 Oct 2002 18:46:36 -0400 (EDT) (envelope-from wollman) Date: Tue, 15 Oct 2002 18:46:36 -0400 (EDT) From: Garrett Wollman Message-Id: <200210152246.g9FMka0o007721@khavrinen.lcs.mit.edu> To: Poul-Henning Kamp Cc: net@FreeBSD.ORG, arch@FreeBSD.ORG Subject: RFC: eliminating the _IP_VHL hack. In-Reply-To: <60637.1034720233@critter.freebsd.dk> References: <60637.1034720233@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > In the meantime absolutely no code has picked up on this idea, It was copied in spirit from OSF/1. > The side effect of having some source-files using the _IP_VHL hack and > some not is that sizeof(struct ip) varies from file to file, Not so. Any compiler which allocates different amounts of storage to one eight-bit member versus two four-bit bitfield members is seriously broken (and would defeat the whole purpose). > I would therefore propose to eliminate the _IP_VHL hack from the kernel > to end this state of (potential) confusion, and invite comments to the > following patch: Much better to delete the bogus BYTE_ORDER kluge from ip.h. (Note that the definition of the bitfields in question has nothing whatsoever to do with the actual byte order in use; it simply relies on the historical behavior of compilers which allocated space for bitfields in BYTE_ORDER order.) -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 16:22:57 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D65EB37B406 for ; Tue, 15 Oct 2002 16:22:55 -0700 (PDT) Received: from rootlabs.com (root.org [67.118.192.226]) by mx1.FreeBSD.org (Postfix) with SMTP id 00F2543E77 for ; Tue, 15 Oct 2002 16:22:55 -0700 (PDT) (envelope-from nate@rootlabs.com) Received: (qmail 37692 invoked by uid 1000); 15 Oct 2002 23:22:56 -0000 Date: Tue, 15 Oct 2002 16:22:56 -0700 (PDT) From: Nate Lawson To: Poul-Henning Kamp Cc: net@freebsd.org, arch@freebsd.org Subject: Re: RFC: eliminating the _IP_VHL hack. In-Reply-To: <60637.1034720233@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Oct 2002, Poul-Henning Kamp wrote: > almost 7 years ago, this commit introduced the _IP_VHL hack in our > IP-stack: > > ] revision 1.7 > ] date: 1995/12/21 21:20:27; author: wollman; state: Exp; lines: +5 -1 > ] If _IP_VHL is defined, declare a single ip_vhl member in struct ip rather > ] than separate ip_v and ip_hl members. Should have no effect on current code, > ] but I'd eventually like to get rid of those obnoxious bitfields completely. > > We can argue a lot about how long time we should wait for "eventually", > but I would say that 7 years is far too long, considering the status: Fine by me. > RCS file: /home/ncvs/src/sys/netinet/ip_icmp.c,v > retrieving revision 1.70 > diff -u -r1.70 ip_icmp.c > --- ip_icmp.c 1 Aug 2002 03:53:04 -0000 1.70 > +++ ip_icmp.c 15 Oct 2002 22:05:23 -0000 > @@ -51,7 +51,6 @@ > #include > #include > > -#define _IP_VHL > #include > #include > #include > @@ -128,7 +127,7 @@ > struct ifnet *destifp; > { > register struct ip *oip = mtod(n, struct ip *), *nip; > - register unsigned oiplen = IP_VHL_HL(oip->ip_vhl) << 2; > + register unsigned oiplen = oip->ip_hl << 2; > register struct icmp *icp; > register struct mbuf *m; > unsigned icmplen; > @@ -214,7 +213,8 @@ > nip = mtod(m, struct ip *); > bcopy((caddr_t)oip, (caddr_t)nip, sizeof(struct ip)); > nip->ip_len = m->m_len; > - nip->ip_vhl = IP_VHL_BORING; > + nip->ip_v = IPVERSION; > + nip->ip_hl = 5; I think there is a manifest constant for the default ipv4 header size but can't remember it right now. -Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 18:22:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C635437B401; Tue, 15 Oct 2002 18:22:54 -0700 (PDT) Received: from mta5.snfc21.pbi.net (mta5.snfc21.pbi.net [206.13.28.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 819B643E7B; Tue, 15 Oct 2002 18:22:54 -0700 (PDT) (envelope-from hsu@FreeBSD.org) Received: from FreeBSD.org ([63.193.112.125]) by mta5.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0H41001UMVU5KX@mta5.snfc21.pbi.net>; Tue, 15 Oct 2002 18:22:54 -0700 (PDT) Date: Tue, 15 Oct 2002 18:23:22 -0700 From: Jeffrey Hsu Subject: Re: RFC: eliminating the _IP_VHL hack. In-reply-to: Message from Poul-Henning Kamp "of Wed, 16 Oct 2002 00:17:13 +0200." <60637.1034720233@critter.freebsd.dk> To: Poul-Henning Kamp Cc: net@freebsd.org, arch@freebsd.org Message-id: <0H41001UNVU5KX@mta5.snfc21.pbi.net> MIME-version: 1.0 X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > The side effect of having some source-files using the _IP_VHL hack and > some not is that sizeof(struct ip) varies from file to file, which at > best is confusing an at worst the source of some really evil bugs. > I would therefore propose to eliminate the _IP_VHL hack from the kernel > to end this state of (potential) confusion This problem could be solved more easily by changing the u_int back to an u_char, as it used to be before rev 1.15: Index: ip.h =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip.h,v retrieving revision 1.19 diff -u -r1.19 ip.h --- ip.h 14 Dec 2001 19:37:32 -0000 1.19 +++ ip.h 16 Oct 2002 01:15:48 -0000 @@ -51,11 +51,11 @@ u_char ip_vhl; /* version << 4 | header length >> 2 */ #else #if BYTE_ORDER == LITTLE_ENDIAN - u_int ip_hl:4, /* header length */ + u_char ip_hl:4, /* header length */ ip_v:4; /* version */ #endif #if BYTE_ORDER == BIG_ENDIAN - u_int ip_v:4, /* version */ + u_char ip_v:4, /* version */ ip_hl:4; /* header length */ #endif #endif /* not _IP_VHL */ But, if we were to pick one or the other to discard, I would keep the IP_VHL because that field really is a byte in the IP header To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 19:45:10 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B1B0937B401; Tue, 15 Oct 2002 19:45:07 -0700 (PDT) Received: from clmboh1-smtp5.columbus.rr.com (clmboh1-smtp5.columbus.rr.com [65.24.0.116]) by mx1.FreeBSD.org (Postfix) with ESMTP id 80CE743E77; Tue, 15 Oct 2002 19:45:06 -0700 (PDT) (envelope-from dzerkel@columbus.rr.com) Received: from zoomer (dhcp065-024-067-163.columbus.rr.com [65.24.67.163]) by clmboh1-smtp5.columbus.rr.com (8.11.2/8.11.2) with ESMTP id g9G2j4023824; Tue, 15 Oct 2002 22:45:04 -0400 (EDT) Content-Type: text/plain; charset="koi8-r" From: "Danny J. Zerkel" Organization: Zerkular Systems To: Maxim Sobolev , Robert Watson Subject: Re: short uid/gid Date: Tue, 15 Oct 2002 22:47:38 -0400 User-Agent: KMail/1.4.3 Cc: "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG References: <20021014191810.GA338@vega.vega.com> In-Reply-To: <20021014191810.GA338@vega.vega.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200210152247.38939.dzerkel@columbus.rr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At least for our Linux emulation layer, supporting IPC_64 would be one of the pieces (probably the main one) keeping The Sims from running. The other thing we are missing is the Linux ioctl() interface for reading MSDOSFS directories, but that may be optional. I will eventually take a look at this (IPC_64) if the scheduling fairy brings me the time. Danny dzerkel@columbus.rr.com On Monday 14 October 2002 15:18, Maxim Sobolev wrote: > Linux solved the problem by introducing a new flag for {msg,shm,sem}ctl(3) > interfaces (IPC_64), which if set tells the kernel that user supplies > new version of the structure. The kernel itself internally keeps all > relevant information already in IPC_64 format, doing conversion before > returning it to user if IPC_64 isn't set. I think that for portability > reasons we should at least consider the same or similar route. > > -Maxim > > On Mon, Oct 14, 2002 at 01:27:57PM -0400, Robert Watson wrote: > > Yeah, this is a Known Problem, and it's quite unfortunate, actually. I > > looked at trying to solve it -- changing the types respectively to uid_t, > > gid_t, and mode_t, but it involved a lot of ABI munging because the > > structures are shared between the userland interface and the kernel > > implementation. The result is that any change in the kernel structure > > really requires you to break out the user structures from the kernel > > structure, which requires a lot of work. Also, ipc_perm is shared > > between all three SysVIPC services, and to compound things, there are > > already ofoo() interfaces for older versions of the structures. My > > belief is that seperating the user and kernel structures is really > > necessary -- making a kipc_perm, etc, so we can better support > > fine-grained locking and extensible security. However, someone has to do > > the grunt work, and last time I tried, I spent several days and only made > > a bit of progress. If you want to give a first pass at breaking out the > > user and kernel structures and send a patch, I'll be happy to work with > > you to get it integrated. I think the steps are: > > > > (1) Divorce user and kernel structures for all of the SysVIPC interfaces, > > and provide functions to map between them as necessary. > > > > (2) Remove the original compatibility interfaces left over from eons ago > > (and figure out how many eons ago so we know what ABIs we're finally > > removing). > > > > (3) Define new userland versions of necessary structures and create a new > > set of ofoo() and foo() interfaces based on the change. > > > > (4) Go back through and dribble the kernel structures with new toys, such > > as 'struct label', etc. > > > > Robert N M Watson FreeBSD Core Team, TrustedBSD Projects > > robert@fledge.watson.org Network Associates Laboratories > > > > On Mon, 14 Oct 2002, Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info wrote: > > > Hi > > > > > > I have found that SysVIPC functions uses structure with short uid/gid > > > types. > > > > > > What is valid solution ? > > > > > > Change types to uid_t/gid_t (but this will broke binary compatibility) > > > Change syscalls to old_* and add new with "right" structures, > > > or something else ? > > > > > > struct ipc_perm { > > > ushort cuid; /* creator user id */ > > > ushort cgid; /* creator group id */ > > > ushort uid; /* user id */ > > > ushort gid; /* group id */ > > > ushort mode; /* r/w permission */ > > > ushort seq; /* sequence # (to generate unique msg/sem/shm id) */ > > > key_t key; /* user specified msg/sem/shm key */ > > > }; > > > > > > -- > > > Vladimir B. Grebenschikov > > > vova@sw.ru, SWsoft, Inc. > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > > with "unsubscribe freebsd-arch" in the body of the message > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-current" in the body of the message > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-current" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 21:37: 1 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7898937B401; Tue, 15 Oct 2002 21:37:00 -0700 (PDT) Received: from gull.mail.pas.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 02F6443E75; Tue, 15 Oct 2002 21:37:00 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0283.cvx22-bradley.dialup.earthlink.net ([209.179.199.28] helo=mindspring.com) by gull.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 181fvQ-0001bG-00; Tue, 15 Oct 2002 21:36:53 -0700 Message-ID: <3DACEC96.D307070A@mindspring.com> Date: Tue, 15 Oct 2002 21:35:34 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Danny J. Zerkel" Cc: Maxim Sobolev , Robert Watson , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid References: <20021014191810.GA338@vega.vega.com> <200210152247.38939.dzerkel@columbus.rr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Danny J. Zerkel" wrote: > At least for our Linux emulation layer, supporting IPC_64 would be one > of the pieces (probably the main one) keeping The Sims from running. > The other thing we are missing is the Linux ioctl() interface for reading > MSDOSFS directories, but that may be optional. > > I will eventually take a look at this (IPC_64) if the scheduling fairy brings > me the time. Basically, they convert a lot of short fields to what the manual page calls "longs", but which are actually 32 bit values; makes you wonder where they obtained the name "IPC_64", when it should probably more properly be "IPC_32". They pad it out to 36 bytes by adding two unsigned long fields (32 bit longs); gotta wonder why they didn't stop at 32 bytes... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 21:43:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9703F37B401; Tue, 15 Oct 2002 21:43:43 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id C1E6B43E6A; Tue, 15 Oct 2002 21:43:42 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.12.4/8.12.4) with SMTP id g9G4h5Oo095614; Wed, 16 Oct 2002 00:43:05 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 16 Oct 2002 00:43:05 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: "Danny J. Zerkel" Cc: Maxim Sobolev , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid In-Reply-To: <200210152247.38939.dzerkel@columbus.rr.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 15 Oct 2002, Danny J. Zerkel wrote: > At least for our Linux emulation layer, supporting IPC_64 would be one > of the pieces (probably the main one) keeping The Sims from running. > The other thing we are missing is the Linux ioctl() interface for > reading MSDOSFS directories, but that may be optional. > > I will eventually take a look at this (IPC_64) if the scheduling fairy > brings me the time. While I think support for the IPC_64 flag under emulation is useful, I'd rather make use of compatibility system calls and type improvements for the base FreeBSD implementation of the System V IPC APIs. Most of the work necessary to support those changes is required in order to better support fine-grained locking and MAC (breaking out the user and kernel structures). Also, it means future applications will make use of the improved APIs by default. In addition, other systems, such as Solaris, have already made this change in this way, avoiding the IPC_64 flag design. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 15 23:39:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 04E3137B401; Tue, 15 Oct 2002 23:39:48 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 278AC43E6E; Tue, 15 Oct 2002 23:39:47 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id g9G6deIw051939; Wed, 16 Oct 2002 08:39:41 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Garrett Wollman Cc: net@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: RFC: eliminating the _IP_VHL hack. In-Reply-To: Your message of "Tue, 15 Oct 2002 18:46:36 EDT." <200210152246.g9FMka0o007721@khavrinen.lcs.mit.edu> Date: Wed, 16 Oct 2002 08:39:40 +0200 Message-ID: <51938.1034750380@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <200210152246.g9FMka0o007721@khavrinen.lcs.mit.edu>, Garrett Wollman writes: >Much better to delete the bogus BYTE_ORDER kluge from ip.h. (Note >that the definition of the bitfields in question has nothing >whatsoever to do with the actual byte order in use; it simply relies >on the historical behavior of compilers which allocated space for >bitfields in BYTE_ORDER order.) Do you intend to complete the task you originally started ? What is your plan for 3rd party software ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 0:52:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 03A5437B401; Wed, 16 Oct 2002 00:52:53 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6747B43EAF; Wed, 16 Oct 2002 00:52:51 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id RAA05653; Wed, 16 Oct 2002 17:52:48 +1000 Date: Wed, 16 Oct 2002 18:03:24 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Jeffrey Hsu Cc: Poul-Henning Kamp , , Subject: Re: RFC: eliminating the _IP_VHL hack. In-Reply-To: <0H41001UNVU5KX@mta5.snfc21.pbi.net> Message-ID: <20021016172019.W4358-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 15 Oct 2002, Jeffrey Hsu wrote: > > The side effect of having some source-files using the _IP_VHL hack and > > some not is that sizeof(struct ip) varies from file to file, which at > > best is confusing an at worst the source of some really evil bugs. There is no such effect, or ip would not work. Bitfields are inherently unportable, but here everything is carefully packed so that there is a good chance that both versions give the same struct layout for all non-broken compilers (the layout needs to be compiler-independent too). > > I would therefore propose to eliminate the _IP_VHL hack from the kernel > > to end this state of (potential) confusion This would remove the least unportable version. > This problem could be solved more easily by changing the u_int back > to an u_char, as it used to be before rev 1.15: > > Index: ip.h > =================================================================== > RCS file: /home/ncvs/src/sys/netinet/ip.h,v > retrieving revision 1.19 > diff -u -r1.19 ip.h > --- ip.h 14 Dec 2001 19:37:32 -0000 1.19 > +++ ip.h 16 Oct 2002 01:15:48 -0000 > @@ -51,11 +51,11 @@ > u_char ip_vhl; /* version << 4 | header length >> 2 */ > #else > #if BYTE_ORDER == LITTLE_ENDIAN > - u_int ip_hl:4, /* header length */ > + u_char ip_hl:4, /* header length */ > ip_v:4; /* version */ > #endif > #if BYTE_ORDER == BIG_ENDIAN > - u_int ip_v:4, /* version */ > + u_char ip_v:4, /* version */ > ip_hl:4; /* header length */ > #endif > #endif /* not _IP_VHL */ > > But, if we were to pick one or the other to discard, I would keep the > IP_VHL because that field really is a byte in the IP header This would just guarantee unportable behaviour by untranslating from C. C compilers reject bit-fields that are declared to have u_char, since u_char is not a permitted type for a bit-field. E.g., from the C99 draft n869.txt: 6.7.2.1 Language 6.7.2.1 WG14/N869 Committee Draft -- January 18, 1999 115 [#8] A bit-field shall have a type that is a qualified or unqualified version of _Bool, signed int, or unsigned int. ... Little is lost by restricting the type of a bitfield in this way, since the width of a bitfield can be determined without lokking at its type. Bit-fields that are declared to have types other than the standard types for them are a poorly documented gcc extension. The type of a bit-field has little or no affect on packing of bits near the bitfield, at least if everything is packed manually. However, the types of bit-fields within a struct affects padding at the end of a struct in a (usually) less than useful way: if a standard standard type is used, then structs get padded if necessary to give alignment of that type. E.g., on i386's the following structs have size 4 although you probably want them to have size 1: struct foo1 { int bar: 1; }; struct foo2 { int bar: 8; }; TenDRA is bug-for-bug compatible with gcc here. This can be worked around if the compiler is gcc by declaring the struct as __packed (__aligned(1) doesn't seem to work), or by declaring the bit-field type as char, or on some systems by using -mno-bit-align. These workarounds are even more unportable than bit-fields. However, there is no problem with struct ip yet, because everything is manually packed to work with "all" C compilers on "all" systems. This will break on systems with 64-bit ints, since sizeof(struct ip) == 20 is not a multiple of 8. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 1:16:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4FA1737B401; Wed, 16 Oct 2002 01:16:54 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 81E3C43E77; Wed, 16 Oct 2002 01:16:53 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id g9G8GeIw069414; Wed, 16 Oct 2002 10:16:47 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Bruce Evans Cc: Jeffrey Hsu , net@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: RFC: eliminating the _IP_VHL hack. In-Reply-To: Your message of "Wed, 16 Oct 2002 18:03:24 +1000." <20021016172019.W4358-100000@gamplex.bde.org> Date: Wed, 16 Oct 2002 10:16:40 +0200 Message-ID: <69413.1034756200@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20021016172019.W4358-100000@gamplex.bde.org>, Bruce Evans writes: >On Tue, 15 Oct 2002, Jeffrey Hsu wrote: > >> > The side effect of having some source-files using the _IP_VHL hack and >> > some not is that sizeof(struct ip) varies from file to file, which at >> > best is confusing an at worst the source of some really evil bugs. > >There is no such effect, or ip would not work. s/,/ with our current compilers and architectures,/ >> > I would therefore propose to eliminate the _IP_VHL hack from the kernel >> > to end this state of (potential) confusion > >This would remove the least unportable version. Could be, but there is no point in us having two different versions, neither of which is guaranteed to be portable, in particular when one of the two is private to FreeBSD and not even consistently used there. For reference: NetBSD hasn't got the _IP_VHL but has a __packed__ on the structure. OpenBSD hasn't got the _IP_VHL either, but hasn't adopted the __packed__ from NetBSD. Linux also uses bitfields and endianess #ifdefs. >These workarounds are even more unportable than bit-fields. However, >there is no problem with struct ip yet, because everything is manually >packed to work with "all" C compilers on "all" systems. This will >break on systems with 64-bit ints, since sizeof(struct ip) == 20 is >not a multiple of 8. Good point. I'll ammend my proposal to include a __packed__ and a CTASSERT on the size of struct ip == 20. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 2: 8:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F128A37B401; Wed, 16 Oct 2002 02:08:18 -0700 (PDT) Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8436F43E8A; Wed, 16 Oct 2002 02:08:18 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0009.cvx21-bradley.dialup.earthlink.net ([209.179.192.9] helo=mindspring.com) by pintail.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 181kA5-0000o5-00; Wed, 16 Oct 2002 02:08:17 -0700 Message-ID: <3DAD2B79.AD5412CA@mindspring.com> Date: Wed, 16 Oct 2002 02:03:53 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Robert Watson Cc: "Danny J. Zerkel" , Maxim Sobolev , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Robert Watson wrote: > While I think support for the IPC_64 flag under emulation is useful, I'd > rather make use of compatibility system calls and type improvements for > the base FreeBSD implementation of the System V IPC APIs. Most of the > work necessary to support those changes is required in order to better > support fine-grained locking and MAC (breaking out the user and kernel > structures). Also, it means future applications will make use of the > improved APIs by default. In addition, other systems, such as Solaris, > have already made this change in this way, avoiding the IPC_64 flag > design. You could also simply use non-intersecting cmd parameter values for the new calls, which avaids the special flag, and leaves the backward compatability without adding grundles of new system calls. I thought that everything was going to 64 bits and we were getting a new system call entry vector in 5.x, anyway... wasn't Matt going to do something like that? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 2:31:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C298F37B401; Wed, 16 Oct 2002 02:31:09 -0700 (PDT) Received: from baraca.united.net.ua (ns.united.net.ua [193.111.8.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8465F43EB3; Wed, 16 Oct 2002 02:31:05 -0700 (PDT) (envelope-from max@vega.com) Received: from vega.vega.com (xDSL-2-2.united.net.ua [193.111.9.226]) by baraca.united.net.ua (8.11.6/8.11.6) with ESMTP id g9G9Usu39477; Wed, 16 Oct 2002 12:30:55 +0300 (EEST) (envelope-from max@vega.com) Received: from vega.vega.com (max@localhost [127.0.0.1]) by vega.vega.com (8.12.6/8.12.5) with ESMTP id g9G9Up6N011223; Wed, 16 Oct 2002 12:30:51 +0300 (EEST) (envelope-from sobomax@FreeBSD.org) Received: (from max@localhost) by vega.vega.com (8.12.6/8.12.5/Submit) id g9G9UnZb011222; Wed, 16 Oct 2002 12:30:49 +0300 (EEST) Date: Wed, 16 Oct 2002 12:30:48 +0300 From: Maxim Sobolev To: Terry Lambert Cc: Robert Watson , "Danny J. Zerkel" , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid Message-ID: <20021016093048.GB10908@vega.vega.com> References: <3DAD2B79.AD5412CA@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <3DAD2B79.AD5412CA@mindspring.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Oct 16, 2002 at 02:03:53AM -0700, Terry Lambert wrote: > Robert Watson wrote: > > While I think support for the IPC_64 flag under emulation is useful, I'd > > rather make use of compatibility system calls and type improvements for > > the base FreeBSD implementation of the System V IPC APIs. Most of the > > work necessary to support those changes is required in order to better > > support fine-grained locking and MAC (breaking out the user and kernel > > structures). Also, it means future applications will make use of the > > improved APIs by default. In addition, other systems, such as Solaris, > > have already made this change in this way, avoiding the IPC_64 flag > > design. > > You could also simply use non-intersecting cmd parameter values > for the new calls, which avaids the special flag, and leaves the > backward compatability without adding grundles of new system calls. What about source-level compatibility, which IMO is a good thing, at least if it doesn't add too much complexity (it clearly doesn't in this case)? Also, handling single flag should be easier from the coding perspective than a load of new values, after all we can do something like: #define IPC_STAT_OLD 0xXY #define IPC_SET_OLD 0xZW [...] #define IPC_64 0x100 #define IPC_STAT (IPC_STAT_OLD | IPC_64) #define IPC_SET (IPC_SET_OLD | IPC_64) [...] Which automagically will bring 64-version of syscalls after recompilation, while retaining ABI compatibility. -Maxim > I thought that everything was going to 64 bits and we were getting > a new system call entry vector in 5.x, anyway... wasn't Matt going > to do something like that? > > -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 3: 3:30 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D06B37B404; Wed, 16 Oct 2002 03:03:27 -0700 (PDT) Received: from shuttle.wide.toshiba.co.jp (shuttle.wide.toshiba.co.jp [202.249.10.124]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2437F43EAA; Wed, 16 Oct 2002 03:03:26 -0700 (PDT) (envelope-from jinmei@isl.rdc.toshiba.co.jp) Received: from localhost ([3ffe:501:4819:2000:14e:a4d1:f898:6ed5]) by shuttle.wide.toshiba.co.jp (8.11.6/8.9.1) with ESMTP id g9GA2mt66864; Wed, 16 Oct 2002 19:02:48 +0900 (JST) Date: Wed, 16 Oct 2002 19:03:33 +0900 Message-ID: From: JINMEI Tatuya / =?ISO-2022-JP?B?GyRCP0BMQEMjOkgbKEI=?= To: "Sam Leffler" Cc: "Julian Elischer" , , Subject: Re: CFR: m_tag patch In-Reply-To: <080101c27151$b2e92a30$52557f42@errno.com> References: <18d301c26e5e$8b5c7a30$52557f42@errno.com> <080101c27151$b2e92a30$52557f42@errno.com> User-Agent: Wanderlust/2.6.1 (Upside Down) Emacs/21.2 Mule/5.0 (SAKAKI) Organization: Research & Development Center, Toshiba Corp., Kawasaki, Japan. MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII X-Dispatcher: imput version 20000228(IM140) Lines: 59 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> On Fri, 11 Oct 2002 11:12:02 -0700, >>>>> "Sam Leffler" said: >> > struct m_tag { >> > SLIST_ENTRY(m_tag) m_tag_link; /* List of packet tags > */ >> > u_int16_t m_tag_id; /* Tag ID */ >> > u_int16_t m_tag_len; /* Length of data */ >> > u_int32_t m_tag_cookie; /* Module/ABI */ >> > }; (snip) >> Sorry for interrupting, but please let me make it sure. Do you intend >> to hide the additional member from other modules than the m_tag >> internal? I'm afraid a story that (e.g.) some code fragments in the >> network layer directly refers to m_tag_cookie, which will break source >> level compatibility with other BSDs (when the code fragments are >> shared with others). As suz said before, we (KAME) are very much >> afraid of this kind of story. > The changes I'm proposing for KAME code make no references to m_tag_cookie. > Things should be clear when you have a patch to look at. I know that, but what I'm worrying about is a story that *.c under netinet[6] will have a direct reference to m_tag_cookie *in the future". Then we'll need to separate the code fragments like this: #if defined(__FreeBSD__) && && __FreeBSD__ >= 5 if (mtag->m_tag_cookie != PACKET_TAG_IPSEC_OUT_DONE && mtag->m_tag_cookie != PACKET_TAG_IPSEC_OUT_CRYPTO_NEEDED) continue; #else if (mtag->m_tag_id != PACKET_TAG_IPSEC_OUT_DONE && mtag->m_tag_id != PACKET_TAG_IPSEC_OUT_CRYPTO_NEEDED) continue; #endif (derived from the current KAME's ip6_output.c) We've experienced a lot of headaches due to this type of incompatibility. I fully understand that once some changes are incorporated to a particular BSD, the BSD developers are free to modify the code based on their local policy, even if the result introduces the incompatibility with other BSDs. Of course, there should be a reason for the modification, and the change may provide a better behavior. So, I can only ask, "please understand our position (that needs to handle all *BSDs) and consider a compromise." > I'm working on > getting that to you. Yes, I noticed that, thanks. I'll take a closer look at it later. JINMEI, Tatuya Communication Platform Lab. Corporate R&D Center, Toshiba Corp. jinmei@isl.rdc.toshiba.co.jp To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 5:29:37 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4324137B401; Wed, 16 Oct 2002 05:29:35 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2661843EB1; Wed, 16 Oct 2002 05:29:33 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id g9GCTMIw014911; Wed, 16 Oct 2002 14:29:24 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Bruce Evans Cc: net@freebsd.org, arch@freebsd.org Subject: Re: cvs commit: src/sys/dev/kbd atkbdcreg.h In-Reply-To: Your message of "Wed, 16 Oct 2002 22:23:18 +1000." <20021016221657.I5365-100000@gamplex.bde.org> Date: Wed, 16 Oct 2002 14:29:22 +0200 Message-ID: <14910.1034771362@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20021016221657.I5365-100000@gamplex.bde.org>, Bruce Evans writes: >> This kind of bug is a _really_ horribly thing as we end up with one bit >> of code thinking a particular structure is 136 bytes and another that it >> is only 112 bytes. >> >> Ideally all places would remember to #include the right "opt_foo.h" file, >> but I think in practice file containing the variable sized struct should >> #include it explicitly as a precaution. > >Ideally, header files wouldn't have any "variable sized structs" or >anything else that depends on options. Core headers have to be much >more careful about this because including an options header nested >would break most modules and everything outside of the kernel (apart >from the bug that modules and things outside of the kernel have no >way to determine the correct value for the option). I agree. Suggestions for the best way of fixing "struct ipq" in ip_var.h hereby solicited. I can see that MAC just unceremonously put their own field in there, and I guess that is the most sensible thing to do, despite the RAM this may cost during a DoS attack. Objections to this patch: Index: ip_var.h =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_var.h,v retrieving revision 1.66 diff -u -r1.66 ip_var.h --- ip_var.h 16 Oct 2002 01:54:44 -0000 1.66 +++ ip_var.h 16 Oct 2002 07:35:44 -0000 @@ -68,10 +68,8 @@ u_short ipq_id; /* sequence id for reassembly */ struct mbuf *ipq_frags; /* to ip headers of fragments */ struct in_addr ipq_src,ipq_dst; -#ifdef IPDIVERT u_int32_t ipq_div_info; /* ipfw divert port & flags */ u_int16_t ipq_div_cookie; /* ipfw divert cookie */ -#endif struct label ipq_label; /* MAC label */ }; #endif /* _KERNEL */ -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 6: 4:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C87D437B401; Wed, 16 Oct 2002 06:04:22 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2A7E243E88; Wed, 16 Oct 2002 06:04:22 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.12.4/8.12.4) with SMTP id g9GD3lOo004743; Wed, 16 Oct 2002 09:03:47 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 16 Oct 2002 09:03:47 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Maxim Sobolev Cc: Terry Lambert , "Danny J. Zerkel" , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid In-Reply-To: <20021016093048.GB10908@vega.vega.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Oct 2002, Maxim Sobolev wrote: > What about source-level compatibility, which IMO is a good thing, at > least if it doesn't add too much complexity (it clearly doesn't in this > case)? Also, handling single flag should be easier from the coding > perspective than a load of new values, after all we can do something > like: I'm not convinced there's any value to providing the backward compatibility that has to be asked for: the only benefit to the current short-based API is that it allow serious security holes while not following the standard API offered by other platforms (except Linux). Freshly compiled applications should be using the proper types to represent uid's and gid's -- if they're not doing that in the existing code, they'll get truncated to the right size for "bug compatibility". If they are using the correct size, they'll work correctly. To be able to run properly on other platforms (vis Solaris), they already should be using those types. And it's not like the approach you've described makes it any easier to implement: you still have to break out the old and new structures since changing ipc_perm breaks the ABI for all of the System V interfaces, rewrite the kernel code, etc. You might as well have added the compatibility system calls since you still have to do all the mapping. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 7:46:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 88ECB37B401; Wed, 16 Oct 2002 07:46:15 -0700 (PDT) Received: from carp.icir.org (carp.icir.org [192.150.187.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F26543E8A; Wed, 16 Oct 2002 07:46:15 -0700 (PDT) (envelope-from rizzo@carp.icir.org) Received: from carp.icir.org (localhost [127.0.0.1]) by carp.icir.org (8.12.3/8.12.3) with ESMTP id g9GEkBpJ034883; Wed, 16 Oct 2002 07:46:11 -0700 (PDT) (envelope-from rizzo@carp.icir.org) Received: (from rizzo@localhost) by carp.icir.org (8.12.3/8.12.3/Submit) id g9GEkAG2034882; Wed, 16 Oct 2002 07:46:10 -0700 (PDT) (envelope-from rizzo) Date: Wed, 16 Oct 2002 07:46:10 -0700 From: Luigi Rizzo To: "JINMEI Tatuya / ?$B?@L@C#:H?(B" Cc: Sam Leffler , Julian Elischer , freebsd-arch@FreeBSD.ORG, freebsd-net@FreeBSD.ORG Subject: Re: CFR: m_tag patch Message-ID: <20021016074610.C34626@carp.icir.org> References: <18d301c26e5e$8b5c7a30$52557f42@errno.com> <080101c27151$b2e92a30$52557f42@errno.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from jinmei@isl.rdc.toshiba.co.jp on Wed, Oct 16, 2002 at 07:03:33PM +0900 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Actually from what i have read on previous postings on this thread, the only additional check that you might/will need is to make sure that m_tag_cookie corresponds to the GENERIC ABI. Also note that in your example the code should be conditional on __FreeBSD_version and not on __FreeBSD__ cheers luigi On Wed, Oct 16, 2002 at 07:03:33PM +0900, JINMEI Tatuya / ?$B?@L@C#:H?(B wrote: ... > >>>>> "Sam Leffler" said: > > >> > struct m_tag { > >> > SLIST_ENTRY(m_tag) m_tag_link; /* List of packet tags > > */ > >> > u_int16_t m_tag_id; /* Tag ID */ > >> > u_int16_t m_tag_len; /* Length of data */ > >> > u_int32_t m_tag_cookie; /* Module/ABI */ > >> > }; > > (snip) > > >> Sorry for interrupting, but please let me make it sure. Do you intend > >> to hide the additional member from other modules than the m_tag > >> internal? I'm afraid a story that (e.g.) some code fragments in the > >> network layer directly refers to m_tag_cookie, which will break source > >> level compatibility with other BSDs (when the code fragments are > >> shared with others). As suz said before, we (KAME) are very much > >> afraid of this kind of story. > > > The changes I'm proposing for KAME code make no references to m_tag_cookie. > > Things should be clear when you have a patch to look at. > > I know that, but what I'm worrying about is a story that *.c under > netinet[6] will have a direct reference to m_tag_cookie *in the > future". Then we'll need to separate the code fragments like this: > > #if defined(__FreeBSD__) && && __FreeBSD__ >= 5 > if (mtag->m_tag_cookie != PACKET_TAG_IPSEC_OUT_DONE && > mtag->m_tag_cookie != > PACKET_TAG_IPSEC_OUT_CRYPTO_NEEDED) > continue; > #else > if (mtag->m_tag_id != PACKET_TAG_IPSEC_OUT_DONE && > mtag->m_tag_id != > PACKET_TAG_IPSEC_OUT_CRYPTO_NEEDED) > continue; > #endif > (derived from the current KAME's ip6_output.c) > > We've experienced a lot of headaches due to this type of > incompatibility. I fully understand that once some changes are > incorporated to a particular BSD, the BSD developers are free to > modify the code based on their local policy, even if the result > introduces the incompatibility with other BSDs. Of course, there > should be a reason for the modification, and the change may provide a > better behavior. So, I can only ask, "please understand our position > (that needs to handle all *BSDs) and consider a compromise." > > > I'm working on > > getting that to you. > > Yes, I noticed that, thanks. I'll take a closer look at it later. > > JINMEI, Tatuya > Communication Platform Lab. > Corporate R&D Center, Toshiba Corp. > jinmei@isl.rdc.toshiba.co.jp > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-net" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 7:52:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7494537B401; Wed, 16 Oct 2002 07:52:57 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3712543EAA; Wed, 16 Oct 2002 07:52:56 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id AAA13232; Thu, 17 Oct 2002 00:52:43 +1000 Date: Thu, 17 Oct 2002 01:03:20 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Poul-Henning Kamp Cc: Jeffrey Hsu , , Subject: Re: RFC: eliminating the _IP_VHL hack. In-Reply-To: <69413.1034756200@critter.freebsd.dk> Message-ID: <20021017004627.O5865-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Oct 2002, Poul-Henning Kamp wrote: > In message <20021016172019.W4358-100000@gamplex.bde.org>, Bruce Evans writes: > >On Tue, 15 Oct 2002, Jeffrey Hsu wrote: > > > >> > The side effect of having some source-files using the _IP_VHL hack and > >> > some not is that sizeof(struct ip) varies from file to file, which at > >> > best is confusing an at worst the source of some really evil bugs. > > > >There is no such effect, or ip would not work. > > s/,/ with our current compilers and architectures,/ It is the non-_IP_VHL case that will break. The _IP_VHL case requires little more than the existence of uint8_t, which is now required by POSIX. > >> > I would therefore propose to eliminate the _IP_VHL hack from the kernel > >> > to end this state of (potential) confusion > > > >This would remove the least unportable version. > > Could be, but there is no point in us having two different versions, > neither of which is guaranteed to be portable, in particular when > one of the two is private to FreeBSD and not even consistently used > there. I agree that source oce portability is a problem and wasn't helped by not completing this change. > For reference: > > NetBSD hasn't got the _IP_VHL but has a __packed__ on the > structure. As you may know, I don't like gccisms like __packed__. NetBSD actually uses __attribute__(__packed__) like we used to in other places. Our __packed doesn't exactly help portability, since it is a syntax error on systems that don't #define it. > OpenBSD hasn't got the _IP_VHL either, but hasn't adopted > the __packed__ from NetBSD. > > Linux also uses bitfields and endianess #ifdefs. There are also an amazing number of unportable bit-fields in , and . The declarations of these don't even use endianness #ifdefs or __packed in most cases. Some unportabilities are avoided using the gcc extension of small bit-field types. > I'll ammend my proposal to include a __packed__ and a CTASSERT on > the size of struct ip == 20. I prefer just the CTASSERT until __packed is found to be necessary. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 8: 4:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2241437B401; Wed, 16 Oct 2002 08:04:47 -0700 (PDT) Received: from baraca.united.net.ua (ns.united.net.ua [193.111.8.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 67E2B43E9E; Wed, 16 Oct 2002 08:04:36 -0700 (PDT) (envelope-from sobomax@FreeBSD.org) Received: from vega.vega.com (xDSL-2-2.united.net.ua [193.111.9.226]) by baraca.united.net.ua (8.11.6/8.11.6) with ESMTP id g9GF4Nu55665; Wed, 16 Oct 2002 18:04:27 +0300 (EEST) (envelope-from sobomax@FreeBSD.org) Received: from FreeBSD.org (big_brother.vega.com [192.168.1.1]) by vega.vega.com (8.12.6/8.12.5) with ESMTP id g9GF4J6N012473; Wed, 16 Oct 2002 18:04:19 +0300 (EEST) (envelope-from sobomax@FreeBSD.org) Message-ID: <3DAD8084.C60C5C0F@FreeBSD.org> Date: Wed, 16 Oct 2002 18:06:44 +0300 From: Maxim Sobolev Organization: Vega International Capital X-Mailer: Mozilla 4.8 [en] (Windows NT 5.0; U) X-Accept-Language: en,uk,ru MIME-Version: 1.0 To: Robert Watson Cc: Terry Lambert , "Danny J. Zerkel" , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: short uid/gid References: Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Robert Watson wrote: > > On Wed, 16 Oct 2002, Maxim Sobolev wrote: > > > What about source-level compatibility, which IMO is a good thing, at > > least if it doesn't add too much complexity (it clearly doesn't in this > > case)? Also, handling single flag should be easier from the coding > > perspective than a load of new values, after all we can do something > > like: > > I'm not convinced there's any value to providing the backward > compatibility that has to be asked for: the only benefit to the current > short-based API is that it allow serious security holes while not > following the standard API offered by other platforms (except Linux). > Freshly compiled applications should be using the proper types to > represent uid's and gid's -- if they're not doing that in the existing > code, they'll get truncated to the right size for "bug compatibility". If > they are using the correct size, they'll work correctly. To be able to > run properly on other platforms (vis Solaris), they already should be > using those types. Actually I am not proposing to keep the current short-based API. As I stated in my previous message, with such system in place all freshly-recompiled apps will use the new ABI automagically. > And it's not like the approach you've described makes it any easier to > implement: you still have to break out the old and new structures since > changing ipc_perm breaks the ABI for all of the System V interfaces, > rewrite the kernel code, etc. You might as well have added the > compatibility system calls since you still have to do all the mapping. I don't quite understand what you are trying to say. Linux does the following on the beginning and the end of each {shm,sem,msg}ctl call: {shm,sem,msg}ctl(...) { if (IPC_64 flag is not set) { convert user-supplied structure to a new 64-bit format } proceed as usual using 64-bit structure if (IPC_64 flag is not set) { convert result to be transferred back to user into old format } return } This is apparently much more compact, ABI-safe and easier to implement and maintain since we don't have two versions of mostly the same code. -Maxim To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 8:46:37 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 895BF37B4D0 for ; Wed, 16 Oct 2002 08:46:35 -0700 (PDT) Received: from srvr20.engin.umich.edu (srvr20.engin.umich.edu [141.213.75.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id B25ED43E9C for ; Wed, 16 Oct 2002 08:46:32 -0700 (PDT) (envelope-from wingc@engin.umich.edu) Received: from pi.engin.umich.edu (IDENT:90lqYZULV6Km21uS+YDNCYnyFbrel0gW@pi.engin.umich.edu [141.213.40.176]) by srvr20.engin.umich.edu (8.9.3/8.9.1) with ESMTP id LAA13616 for ; Wed, 16 Oct 2002 11:46:19 -0400 (EDT) Received: from localhost (IDENT:aGK1SjXfBXsatibUMDacs7dztfTz6XQg@localhost [127.0.0.1]) by pi.engin.umich.edu (8.12.5/8.12.5) with ESMTP id g9GFifot017662 for ; Wed, 16 Oct 2002 11:44:41 -0400 Date: Wed, 16 Oct 2002 11:44:41 -0400 (EDT) From: Christopher Allen Wing To: freebsd-arch@freebsd.org Subject: why IPC_64 (Re: short uid/gid) Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, I was the person responsible for IPC_64 in Linux. Let me summarize what happened so that you get the history behind this. IPC_64 is basically intended as a kernel-libc private interface. It is not exposed to user-side applications through the standard include files. At the time that the kernel changes were made, the Linux kernel hackers preferred the IPC_64 modifier flag to a new system call entry point. The kernel changes included new system calls for 32-bit uid/gid, including: {,l,f}stat() {,l,f}chown() {get,set}{,e,re,res}{uid,gid}() In addition to the kernel changes, the user-side SysV IPC structure definitions were decoupled from the kernel's definitions. glibc in Linux wraps the IPC system call, and handles all the backward compatibility issues. Starting with glibc 2.2, glibc's headers (which are what user-side compiles include) contain a new definition of the SysV IPC structures with 32-bit UIDs, 64-bit sizes for shared memory, increased message queue limits, etc. Binary compatibility with old applications is handled by 'versioned symbols'. If you compile a program that uses SysV IPC against a newer glibc, you get symbol references that look like this: linux-glibc22% nm ./ipcs |grep ctl U msgctl@@GLIBC_2.2 U semctl@@GLIBC_2.2 U shmctl@@GLIBC_2.2 Programs built against older versions of glibc will have older symbol references: linux-glibc21% nm ./ipcs |grep ctl U msgctl@@GLIBC_2.0 U semctl@@GLIBC_2.0 U shmctl@@GLIBC_2.0 The older symbols use the older structure definitions. glibc 2.2 and above contains both sets of symbols. User-side compiles do not specify IPC_64. They just get the newer structure definitions when compiled against glibc 2.2+ because we changed the include files. glibc handles backward compatibility when running on an older kernel. If the kernel doesn't support IPC_64, glibc emulates it by translating the older structure definitions to and from the new ones. The upshot of this is that FreeBSD doesn't have to implement IPC_64 for Linux programs to work under emulation. (although glibc will fall back to using the older, 16-bit uid system calls) The FreeBSD Linux emulator should implement IPC_64, and the other 32-bit UID Linux system calls, for full functionality with newer Linux software. The padding in the structures was an attempt to leave room for future changes without creating a new version of the IPC system call. For instance, you can see the padding for future 64-bit time. In retrospect, I don't particularly like this decision. My gut feeling is that we are going to end up needing another version of the IPC call anyway. The extra padding shouldn't be a performance problem though, since the structures with padding are only used for setting and examining permissions on IPC objects, which I would expect to happen infrequently. Regards, Chris Wing wingc@engin.umich.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 9: 8: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7260C37B401; Wed, 16 Oct 2002 09:07:59 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9FBF943EAF; Wed, 16 Oct 2002 09:07:57 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.12.4/8.12.4) with SMTP id g9GG7HOo045971; Wed, 16 Oct 2002 12:07:17 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 16 Oct 2002 12:07:16 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Maxim Sobolev Cc: Terry Lambert , "Danny J. Zerkel" , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: short uid/gid In-Reply-To: <3DAD8084.C60C5C0F@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Oct 2002, Maxim Sobolev wrote: > > And it's not like the approach you've described makes it any easier to > > implement: you still have to break out the old and new structures since > > changing ipc_perm breaks the ABI for all of the System V interfaces, > > rewrite the kernel code, etc. You might as well have added the > > compatibility system calls since you still have to do all the mapping. > > I don't quite understand what you are trying to say. Linux does the > following on the beginning and the end of each {shm,sem,msg}ctl call: > > {shm,sem,msg}ctl(...) > { > if (IPC_64 flag is not set) { > convert user-supplied structure to a new 64-bit format > } > proceed as usual using 64-bit structure > if (IPC_64 flag is not set) { > convert result to be transferred back to user into old format > } > return > } > > This is apparently much more compact, ABI-safe and easier to implement > and maintain since we don't have two versions of mostly the same code. What you are missing here is that the same structures used in kernel are used in userspace, so an important step to "fixing" the SysVIPC problem is to divorce those structures, which means you need two sets of internalize and externalize functions. Your proposed solution involves exactly as much code as the other solution, it just happens to do the compatibility above the system call layer rather than at the system call layer. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 9:31: 1 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6402137B401; Wed, 16 Oct 2002 09:31:00 -0700 (PDT) Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id C3CF743EAC; Wed, 16 Oct 2002 09:30:59 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0308.cvx40-bradley.dialup.earthlink.net ([216.244.43.53] helo=mindspring.com) by pintail.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 181r4U-0005Yw-00; Wed, 16 Oct 2002 09:30:58 -0700 Message-ID: <3DAD93FB.78324E0E@mindspring.com> Date: Wed, 16 Oct 2002 09:29:47 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Maxim Sobolev Cc: Robert Watson , "Danny J. Zerkel" , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid References: <3DAD2B79.AD5412CA@mindspring.com> <20021016093048.GB10908@vega.vega.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Maxim Sobolev wrote: > > You could also simply use non-intersecting cmd parameter values > > for the new calls, which avaids the special flag, and leaves the > > backward compatability without adding grundles of new system calls. > > What about source-level compatibility, which IMO is a good thing, > at least if it doesn't add too much complexity (it clearly doesn't > in this case)? Also, handling single flag should be easier from > the coding perspective than a load of new values, after all > we can do something like: > > #define IPC_STAT_OLD 0xXY > #define IPC_SET_OLD 0xZW > [...] > > #define IPC_64 0x100 > > #define IPC_STAT (IPC_STAT_OLD | IPC_64) > #define IPC_SET (IPC_SET_OLD | IPC_64) > [...] > > Which automagically will bring 64-version of syscalls after > recompilation, while retaining ABI compatibility. This is exactly what I was suggesting. In Linux, you do this manually (read their header files). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 9:47:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A4C6C37B401; Wed, 16 Oct 2002 09:47:31 -0700 (PDT) Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F87043EAC; Wed, 16 Oct 2002 09:47:31 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0308.cvx40-bradley.dialup.earthlink.net ([216.244.43.53] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 181rKL-0004fT-00; Wed, 16 Oct 2002 09:47:22 -0700 Message-ID: <3DAD97D1.DB378D6B@mindspring.com> Date: Wed, 16 Oct 2002 09:46:09 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Robert Watson Cc: Maxim Sobolev , "Danny J. Zerkel" , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Robert Watson wrote: > I'm not convinced there's any value to providing the backward > compatibility that has to be asked for: the only benefit to the current > short-based API is that it allow serious security holes while not > following the standard API offered by other platforms (except Linux). The main benefit, to my mind, is standards compliance. The secondary benefit is ABI similarity for the purposes of supporting ABIs to run non-native software (e.g. Linux prior to 2.4). Binary backward compatability *demands* that you limit the range of your UIDs and GIDs to what will fit in a uint16_t, just as the recent issue with a pid_t exceeeding this size has damaged binary backward compatability with third party a.out binaries. > Freshly compiled applications should be using the proper types to > represent uid's and gid's -- if they're not doing that in the existing > code, they'll get truncated to the right size for "bug compatibility". If > they are using the correct size, they'll work correctly. To be able to > run properly on other platforms (vis Solaris), they already should be > using those types. Truncating of oversized values for credentials is Bad(tm), in that what you get is a different credential. The constraint has to be a priori enforced, or it's meaningless. > And it's not like the approach you've described makes it any easier to > implement: you still have to break out the old and new structures since > changing ipc_perm breaks the ABI for all of the System V interfaces, > rewrite the kernel code, etc. You might as well have added the > compatibility system calls since you still have to do all the mapping. His approach avoids the proliferation of system call entry points, which may conflict with those of other BSDs (among other things). BSDI limited the number of available additional FreeBSD system calls in a standardization effort a while back, if you'll remember, so a proliferation of calls is Bad. In addition, the compatability required this proliferation going forward. FWIW, I would agree with you, if you were advocating a cross-OS ABI definition of manifest constants, so that FreeBSD's new ABI, and, say, Solaris or Linux's ABI, were more congruent (you could get this without a new sstem call vector entry point, by using a different "ELF brand" value), but that's not what you are advocating. Also FWIW, my personal preference would be to follow the Solaris ABI; it's the least volatile of all ABI's in this area. In terms of the available applications, Linux is probably a better choice, but in terms of the amount of maintenance work required, Solaris is far and away the leader. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 10: 2:37 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 978DB37B401; Wed, 16 Oct 2002 10:02:33 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id C978D43E3B; Wed, 16 Oct 2002 10:02:31 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.12.4/8.12.4) with SMTP id g9GH1oOo058816; Wed, 16 Oct 2002 13:01:51 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 16 Oct 2002 13:01:50 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Terry Lambert Cc: Maxim Sobolev , "Danny J. Zerkel" , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid In-Reply-To: <3DAD97D1.DB378D6B@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Oct 2002, Terry Lambert wrote: > Robert Watson wrote: > > I'm not convinced there's any value to providing the backward > > compatibility that has to be asked for: the only benefit to the current > > short-based API is that it allow serious security holes while not > > following the standard API offered by other platforms (except Linux). > > The main benefit, to my mind, is standards compliance. The secondary > benefit is ABI similarity for the purposes of supporting ABIs to run > non-native software (e.g. Linux prior to 2.4). Could you point me at the standard that indicates these fields should be shorts? We're all advocating switching to uid_t/gid_t/id_t here, the only question is with regard to compatibility. Platforms such as Solaris already use uid_t/gid_t for these fields, and we break portable applications because applications assume that uid_t fits into ipc_perm.uid (for example). > Binary backward compatability *demands* that you limit the range of your > UIDs and GIDs to what will fit in a uint16_t, just as the recent issue > with a pid_t exceeeding this size has damaged binary backward > compatability with third party a.out binaries. This has to do with the size of the field. uid_t and gid_t are 32-bit on most relevant platforms, although there are lingering software compatibility issues (such as the Sysem V IPC issue). The goal now is to correct lingering incompatibilities so that we can properly support these ranges. > > Freshly compiled applications should be using the proper types to > > represent uid's and gid's -- if they're not doing that in the existing > > code, they'll get truncated to the right size for "bug compatibility". If > > they are using the correct size, they'll work correctly. To be able to > > run properly on other platforms (vis Solaris), they already should be > > using those types. > > Truncating of oversized values for credentials is Bad(tm), in that what > you get is a different credential. The constraint has to be a priori > enforced, or it's meaningless. I think we're in agreement here. Unfortunately, because of the "short" uid/gid/cuid/cgid fields, truncation is *already* happening. That's what we're trying to fix. If you try to crush a uid_t into ipc_perm.uid on FreeBSD, the field is truncated, resulting in a vulnerability. What pushing the type checking into the application space does is generate warnings for applications making bogus assumptions, and corrects the truncation performed by the kernel in the current scenario. I.e., when a SysVIPC shm segment is created by a user with a uid that doesn't fit in short, the kernel gets it "wrong". The change we're talking about will fix the kernel, and make type problems visible to the applications if they make incorrect assumptions. Of course, past applications assuming 16-bit uids and gids will still be wrong, nothing changes that, but we do fix current applications. > > And it's not like the approach you've described makes it any easier to > > implement: you still have to break out the old and new structures since > > changing ipc_perm breaks the ABI for all of the System V interfaces, > > rewrite the kernel code, etc. You might as well have added the > > compatibility system calls since you still have to do all the mapping. > > His approach avoids the proliferation of system call entry points, which > may conflict with those of other BSDs (among other things). BSDI > limited the number of available additional FreeBSD system calls in a > standardization effort a while back, if you'll remember, so a > proliferation of calls is Bad. In addition, the compatability required > this proliferation going forward. I suspect that this may end up being a red herring. You do realize that OpenBSD and NetBSD have *already* made precisely the change I am describing? Here's the exerpt from OpenBSD's ipc.h: struct ipc_perm { uid_t cuid; /* creator user id */ gid_t cgid; /* creator group id */ uid_t uid; /* user id */ gid_t gid; /* group id */ mode_t mode; /* r/w permission */ unsigned short seq; /* sequence # (to generate unique msg/sem/shm id) */ key_t key; /* user specified msg/sem/shm key */ }; #ifdef _KERNEL struct oipc_perm { unsigned short cuid; /* creator user id */ unsigned short cgid; /* creator group id */ unsigned short uid; /* user id */ unsigned short gid; /* group id */ unsigned short mode; /* r/w permission */ unsigned short seq; /* sequence # (to generate unique msg/sem/shm id) */ key_t key; /* user specified msg/sem/shm key */ }; #endif We're *improve* our compatibility with other BSD platforms by making these changes. > FWIW, I would agree with you, if you were advocating a cross-OS ABI > definition of manifest constants, so that FreeBSD's new ABI, and, say, > Solaris or Linux's ABI, were more congruent (you could get this without > a new sstem call vector entry point, by using a different "ELF brand" > value), but that's not what you are advocating. I'm advocating we adopt pretty much the ABI for these calls that Solaris uses. If you inspect their ipc.h, you'll see that they define two versions of the structure, ipc_perm, and ipc_perm32. ipc_perm is the version that makes use of types such as uid_t and gid_t; ipc_perm32 hard-codes these fields to 32-bit and can be used for compatibility. > Also FWIW, my personal preference would be to follow the Solaris ABI; > it's the least volatile of all ABI's in this area. In terms of the > available applications, Linux is probably a better choice, but in terms > of the amount of maintenance work required, Solaris is far and away the > leader. We already have the SysVIPC wrappers for the Linux code, but those are actually broken because they don't take into account our truncation bugs. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 10: 5: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 44F7337B401; Wed, 16 Oct 2002 10:04:59 -0700 (PDT) Received: from swan.mail.pas.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id D887C43E8A; Wed, 16 Oct 2002 10:04:58 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0308.cvx40-bradley.dialup.earthlink.net ([216.244.43.53] helo=mindspring.com) by swan.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 181rbC-00027m-00; Wed, 16 Oct 2002 10:04:46 -0700 Message-ID: <3DAD9BE6.530F8466@mindspring.com> Date: Wed, 16 Oct 2002 10:03:34 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Bruce Evans , net@freebsd.org, arch@freebsd.org Subject: Re: cvs commit: src/sys/dev/kbd atkbdcreg.h References: <14910.1034771362@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > In message <20021016221657.I5365-100000@gamplex.bde.org>, Bruce Evans writes: > >Ideally, header files wouldn't have any "variable sized structs" or > >anything else that depends on options. Core headers have to be much > >more careful about this because including an options header nested > >would break most modules and everything outside of the kernel (apart > >from the bug that modules and things outside of the kernel have no > >way to determine the correct value for the option). > > I agree. > > Suggestions for the best way of fixing "struct ipq" in ip_var.h > hereby solicited. Please use a union; things like diversion information result in other information not being used. It makes no sense to uniformly bloat all memory allocations for minority features, merely for the sake of uniformity. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 10:27:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E2BB537B4E3 for ; Wed, 16 Oct 2002 10:27:49 -0700 (PDT) Received: from hm61.locaweb.com.br (hm61.locaweb.com.br [200.213.197.161]) by mx1.FreeBSD.org (Postfix) with SMTP id 8FBB743EB2 for ; Wed, 16 Oct 2002 10:27:48 -0700 (PDT) (envelope-from bill.coutinho@dextra.com.br) Received: (qmail 12424 invoked from network); 16 Oct 2002 17:27:24 -0000 Received: from unknown (200.246.179.93) by hm61.locaweb.com.br with QMTP; 16 Oct 2002 17:27:24 -0000 Received: (qmail 22041 invoked from network); 16 Oct 2002 17:27:24 -0000 Received: from unknown (HELO bill) (bill.coutinho@dextra.com.br@200.207.51.166) by hm28.locaweb.com.br with SMTP; 16 Oct 2002 17:27:24 -0000 From: "Bill Coutinho" To: Subject: Jail subsystem + 802.1Q VLANs Date: Wed, 16 Oct 2002 14:27:26 -0300 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I've read about the Jail subsystem, and learned that each jailed process is bound to an specific IP address ("ip_number" field in "struct jail"). That's fine, but my question is: Is it possible to associate a jailed process to a VLAN number in a 802.1Q enabled net interface? -- Regards, Bill Coutinho. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 10:39: 3 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F32B737B401 for ; Wed, 16 Oct 2002 10:39:01 -0700 (PDT) Received: from web11205.mail.yahoo.com (web11205.mail.yahoo.com [216.136.131.187]) by mx1.FreeBSD.org (Postfix) with SMTP id A3F7943E3B for ; Wed, 16 Oct 2002 10:39:01 -0700 (PDT) (envelope-from gathorpe79@yahoo.com) Message-ID: <20021016173858.98439.qmail@web11205.mail.yahoo.com> Received: from [149.99.112.16] by web11205.mail.yahoo.com via HTTP; Wed, 16 Oct 2002 13:38:58 EDT Date: Wed, 16 Oct 2002 13:38:58 -0400 (EDT) From: Gary Thorpe Subject: Re: short uid/gid To: Terry Lambert Cc: freebsd-arch@freebsd.org In-Reply-To: <3DACEC96.D307070A@mindspring.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I am not aware of the latest ISO C standards, but I believe things like 'int' and 'long' are not defined in their lengths, but the minimum values they can represent: 'int' has to be able to do -65536 to +65536 approximately and long has to be able to do -2 billion to +2 billon approximately. On gcc x86, a 'long' may in fact be the same lentgh as an 'int'. The only types in C that *may* have their lengths defined are 'char' (8-bit value) and 'wchar' (16-bit value) as far as I know. I believe Java has the strict length interpritations. Has this changed with the latest C standards (and was it ever true in the first place)? --- Terry Lambert wrote: > "Danny J. Zerkel" wrote: > > At least for our Linux emulation layer, supporting IPC_64 would be > one > > of the pieces (probably the main one) keeping The Sims from > running. > > The other thing we are missing is the Linux ioctl() interface for > reading > > MSDOSFS directories, but that may be optional. > > > > I will eventually take a look at this (IPC_64) if the scheduling > fairy brings > > me the time. > > Basically, they convert a lot of short fields to what the manual > page calls "longs", but which are actually 32 bit values; makes > you wonder where they obtained the name "IPC_64", when it should > probably more properly be "IPC_32". > > They pad it out to 36 bytes by adding two unsigned long fields > (32 bit longs); gotta wonder why they didn't stop at 32 bytes... > > -- Terry > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message ______________________________________________________________________ Post your free ad now! http://personals.yahoo.ca To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 11: 4:16 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 401D737B401; Wed, 16 Oct 2002 11:04:09 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 26DE943E9C; Wed, 16 Oct 2002 11:04:09 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0308.cvx40-bradley.dialup.earthlink.net ([216.244.43.53] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 181sWa-0002CC-00; Wed, 16 Oct 2002 11:04:05 -0700 Message-ID: <3DADA9CC.CBF9E1E6@mindspring.com> Date: Wed, 16 Oct 2002 11:02:52 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Robert Watson Cc: Maxim Sobolev , "Danny J. Zerkel" , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Robert Watson wrote: > On Wed, 16 Oct 2002, Terry Lambert wrote: > > Robert Watson wrote: > > > I'm not convinced there's any value to providing the backward > > > compatibility that has to be asked for: the only benefit to the current > > > short-based API is that it allow serious security holes while not > > > following the standard API offered by other platforms (except Linux). > > > > The main benefit, to my mind, is standards compliance. The secondary > > benefit is ABI similarity for the purposes of supporting ABIs to run > > non-native software (e.g. Linux prior to 2.4). > > Could you point me at the standard that indicates these fields should be > shorts? We're all advocating switching to uid_t/gid_t/id_t here, the only > question is with regard to compatibility. Platforms such as Solaris > already use uid_t/gid_t for these fields, and we break portable > applications because applications assume that uid_t fits into ipc_perm.uid > (for example). The field sizes have, unfortunately, been documented in manual pages on a lot of platforms. Rather than survey them all, we can look at the FreeBSD manual page for shmctl(), which has an overflow on shm_nattch, as well as shm_atime/shm_dtime/shm_ctime; by extension, it also includes an struct ipc_perm as its first element (rather than a pointer to an ipc_perm struct with a version number field), which has ushort cuid/cgid/uid/gid/mode/seq members. If we look at the shmctl() definition in the X/Open and POSIX documentation, we see that shm_perm.mode's significance is limited to "low-order nine bits". In semctl(), the same standard defined the arg.array pointer as pointing to an array of unsigned shorts. Similarly, the standard references struct ipc_perm in the sys/sem.h entry point. There is actually arguabley non-opaque data which has been exported by proxy, which should not have been exported at all; specifically, the only members defined to be exported by the standards in the ipc_perm are: uid/gid (owner), cuid/cgid (creator), and mode. The upshot is that the standards failed to make the size of the values in conforming implementations required to be opaque, and now we have data interfaces we have to live with for source code compatability reasons, whose definitions are not controlled by a standard. 8-(. > > Binary backward compatability *demands* that you limit the range of your > > UIDs and GIDs to what will fit in a uint16_t, just as the recent issue > > with a pid_t exceeeding this size has damaged binary backward > > compatability with third party a.out binaries. > > This has to do with the size of the field. uid_t and gid_t are 32-bit on > most relevant platforms, although there are lingering software > compatibility issues (such as the Sysem V IPC issue). The goal now is to > correct lingering incompatibilities so that we can properly support these > ranges. Julian recently exposted a binary compatability issue when PIDs exceeded a short. See the -current archives for details. There's a similar problem at work here, except that now it's a security issue, I think. > > Truncating of oversized values for credentials is Bad(tm), in that what > > you get is a different credential. The constraint has to be a priori > > enforced, or it's meaningless. > > I think we're in agreement here. Unfortunately, because of the "short" > uid/gid/cuid/cgid fields, truncation is *already* happening. That's what > we're trying to fix. If you try to crush a uid_t into ipc_perm.uid on > FreeBSD, the field is truncated, resulting in a vulnerability. This will happen automatically on any FreeBSD system where you are running an old binary. The crushing occurs in kernel space, not in user space. That's the issue that the Linux IPC_64 author stated that they were addressing. By doing it the way they did, the guaranteed that, so long as the kernel has a consistent view, then user space would have a consistent view. > What pushing the type checking into the application space does is > generate warnings for applications making bogus assumptions, and > corrects the truncation performed by the kernel in the current > scenario. It only generates warnings on new compilations, not for already compiled legacy applications, which must continue to function without modification. > I.e., when a SysVIPC shm segment is created by a user with a uid > that doesn't fit in short, the kernel gets it "wrong". The change > we're talking about will fix the kernel, and make type problems > visible to the applications if they make incorrect assumptions. Of > course, past applications assuming 16-bit uids and gids will still > be wrong, nothing changes that, but we do fix current applications. As the Linux approach demonstrates, it's possible to get the new applications right, without inherently making the old applications "wrong". I would not advise decorating symbols for this, though. Using the approach of a different cmd space is the correct approach, and can be done without symbol decoration. In fact, I used exactly this approach, when I defined the F_RGETLK, F_RSETLK, and F_RSETLKW commands in fcntl(), for support of NFS locking, back in 1996/1997. By using a seperate command space, you allow use of structures special to that command space. Unless you want to go into adaptive symbol decoration, like Linux does (which would be very hard for FreeBSD to follow), the user space conversion to the expected kernel ABI will not work... in particular, FreeBSD already mixes support for 16 vs. 32 bit values for these fields across kernel interfaces, which is Wrong(tm). At the place one was converted, *all* should have been converted, and the ABI versioned, and this didn't happen. We are already screwed, and the question is what's going to be done about it. Unlike me as this is, my suggestion would be to wait until later, and fix everything at once, so that at least there was internal and external consistency in place. By everything, I include *everything*, including the on-disk quota structure members, which also have shorts for certain fields, and have a binary forward compatability conversion problem facing them. From the Linux ABI argument, which was the first argument put forward, the actual fix is to update the Linux port to provide the Linux 2.4 glibc_2_2 decorations as well as the glibc_2_0 decorations, and to support whatever interface it is that the Linux library is using to determine if the kernel supports the new or the old interface. That will fix the problem that originated the current discussion. In a lot of ways, the Linux approach is superior, although their maintaining the old interfaces in the newer version of the glibc is actually a bad thing: this is the same problem that Novell had wth it's API's, and I rather expect that Linux will see the same result: everyone will code for the older API's, since it will give them the largest possible potential user base. However, the beauty of the Linux approach is that they can rev. the ABI in the kernel to support only a single paradigm, and the conversion is handled in user space at the library level. For FreeBSD, the moral equivalent would be the ability to build an older version of a shared library, with the ability to use a newer version of the host ABI. FreeBSD doesn't really support that, at this point, unfortunately, because FreeBSD still supports static binaries, and static binaries imply a static ABI support that must be carried forward. > > His approach avoids the proliferation of system call entry points, which > > may conflict with those of other BSDs (among other things). BSDI > > limited the number of available additional FreeBSD system calls in a > > standardization effort a while back, if you'll remember, so a > > proliferation of calls is Bad. In addition, the compatability required > > this proliferation going forward. > > I suspect that this may end up being a red herring. You do realize that > OpenBSD and NetBSD have *already* made precisely the change I am > describing? Here's the exerpt from OpenBSD's ipc.h: Yes, I understand that they have made this change. I'm not really objecting to changing ipc_perm. > We're *improve* our compatibility with other BSD platforms by making these > changes. With respect, this will only become relevent when we rely on OpenBSD or NetBSD ABI support for access to applications. As to the general argument of "we should share code where possible", one could argue that we should start with the VFS interfaces, since NetBSD has working VFS stacking. There are a lot of other interfaces in this category, e.g. NetBSD's support for running ABI's from a much larger number of foreign platforms for a CPU architecture, and the fact that the FreeBSD and NetBSD and OpenBSD SYN cache code are not totally congruent. The FreeBSD and NetBSD "cookie" interface for restart of directory traversals are similarly inverted. There are millions of examples of gratuitous differences that could be corrected -- and should be, before trying to make this argument. > > FWIW, I would agree with you, if you were advocating a cross-OS ABI > > definition of manifest constants, so that FreeBSD's new ABI, and, say, > > Solaris or Linux's ABI, were more congruent (you could get this without > > a new sstem call vector entry point, by using a different "ELF brand" > > value), but that's not what you are advocating. > > I'm advocating we adopt pretty much the ABI for these calls that Solaris > uses. If you inspect their ipc.h, you'll see that they define two > versions of the structure, ipc_perm, and ipc_perm32. ipc_perm is the > version that makes use of types such as uid_t and gid_t; ipc_perm32 > hard-codes these fields to 32-bit and can be used for compatibility. No, actually you are suggesting adopting the API, with a similar looking, but inequal ABI. That's very different than the point I was trying to make here. If you are going to follow, rather than lead, then you would be best served to follow the system with the largest software based that is most similar to your own trarget market, in hopes of being able to capture converts on the basis of binary compatability with the least amount of pain. Right now, in order to run Solaris binaries, FreeBSD requires a lot of hacking, including the installation of commerciall licensed shared libraries. > > Also FWIW, my personal preference would be to follow the Solaris ABI; > > it's the least volatile of all ABI's in this area. In terms of the > > available applications, Linux is probably a better choice, but in terms > > of the amount of maintenance work required, Solaris is far and away the > > leader. > > We already have the SysVIPC wrappers for the Linux code, but those are > actually broken because they don't take into account our truncation bugs. ABI, not API. And out "truncation bugs" are a result of a partial conversion of the APIs. If you are suggesting a complete conversion, well, I will again point out that doing this job correctly touches a hell of a lot more code than we should be looking at touching, just prior to a major (not even a point!) release (starting with the on disk quota structures, etc.). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 11: 7:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A1FFA37B401; Wed, 16 Oct 2002 11:07:39 -0700 (PDT) Received: from shuttle.wide.toshiba.co.jp (shuttle.wide.toshiba.co.jp [202.249.10.124]) by mx1.FreeBSD.org (Postfix) with ESMTP id 34E5843EB2; Wed, 16 Oct 2002 11:07:37 -0700 (PDT) (envelope-from jinmei@isl.rdc.toshiba.co.jp) Received: from localhost ([3ffe:501:100f:f::6]) by shuttle.wide.toshiba.co.jp (8.11.6/8.9.1) with ESMTP id g9GI7Ht70328; Thu, 17 Oct 2002 03:07:18 +0900 (JST) Date: Thu, 17 Oct 2002 03:07:07 +0900 Message-ID: From: JINMEI Tatuya / =?ISO-2022-JP?B?GyRCP0BMQEMjOkgbKEI=?= To: Luigi Rizzo Cc: Sam Leffler , Julian Elischer , freebsd-arch@FreeBSD.ORG, freebsd-net@FreeBSD.ORG Subject: Re: CFR: m_tag patch In-Reply-To: <20021016074610.C34626@carp.icir.org> References: <18d301c26e5e$8b5c7a30$52557f42@errno.com> <080101c27151$b2e92a30$52557f42@errno.com> <20021016074610.C34626@carp.icir.org> User-Agent: Wanderlust/2.6.1 (Upside Down) Emacs/21.2 Mule/5.0 (SAKAKI) Organization: Research & Development Center, Toshiba Corp., Kawasaki, Japan. MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII X-Dispatcher: imput version 20000228(IM140) Lines: 33 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> On Wed, 16 Oct 2002 07:46:10 -0700, >>>>> Luigi Rizzo said: > Actually from what i have read on previous postings on this thread, > the only additional check that you might/will need is to make sure > that m_tag_cookie corresponds to the GENERIC ABI. (I re-read the thread) perhaps the example in my previous message wasn't good (and it was at least incorrect). According to the discussion on the thread, we'll probably keep m_tag_cookie being 0 and use m_tag_id in (e.g.) ip6_output.c. So, we'll be happy if this convention is kept (and will be kept) at least under netinet6. It would be good to make more explicit comments on the intended usage, not just a single comment for the m_tag_cookie member: u_int32_t m_tag_cookie; /* Module/ABI */ > Also note that in your example the code should be conditional > on __FreeBSD_version and not on __FreeBSD__ We (KAME) actually need __FreeBSD__, because we want to keep a single code base for all *BSDs (again, I know this is our local issue). But, whether or not we need __FreeBSD__ (in addition to __FreeBSD_version) is a minor point. From our point view, we do want to avoid any ifdef for separating different *BSDs. Thanks, JINMEI, Tatuya Communication Platform Lab. Corporate R&D Center, Toshiba Corp. jinmei@isl.rdc.toshiba.co.jp To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 11:15: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AF1F137B401 for ; Wed, 16 Oct 2002 11:15:03 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 49AD543E6E for ; Wed, 16 Oct 2002 11:15:03 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0308.cvx40-bradley.dialup.earthlink.net ([216.244.43.53] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 181shB-0003uo-00; Wed, 16 Oct 2002 11:15:01 -0700 Message-ID: <3DADAC5D.B3D9105B@mindspring.com> Date: Wed, 16 Oct 2002 11:13:49 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Gary Thorpe Cc: freebsd-arch@freebsd.org Subject: Re: short uid/gid References: <20021016173858.98439.qmail@web11205.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Gary Thorpe wrote: > I am not aware of the latest ISO C standards, but I believe things like > 'int' and 'long' are not defined in their lengths, but the minimum > values they can represent: 'int' has to be able to do -65536 to +65536 > approximately and long has to be able to do -2 billion to +2 billon > approximately. On gcc x86, a 'long' may in fact be the same lentgh as > an 'int'. The only types in C that *may* have their lengths defined are > 'char' (8-bit value) and 'wchar' (16-bit value) as far as I know. I > believe Java has the strict length interpritations. > > Has this changed with the latest C standards (and was it ever true in > the first place)? No. Definition of atomic types is not really relevent, in this case, since we can ignore it because of the compiler/platform we are using. It's an orthogonal issue. The problem at hand is one of historically exposed interfaces, and hostorical mixing of sizes for supposedly uniformly sized values in the FreeBSD ABI, as exposed data interfaces (in files and/or in data that gets copied in and out of the kernel). In particular, it's possible to drop legacy support from the ABI, using a non-intersection approach, but not possible, using the Linux approach, unless we also adopt the user space dual library ABI symbol decoration, at the same time (or do what Windows does, with their "IUnknown" mechanism for interface versioning of libraries). An approach that makes us carry legacy interfaces forward in user space for all time is a bad approach. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 11:19: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BB0E137B401; Wed, 16 Oct 2002 11:19:04 -0700 (PDT) Received: from carp.icir.org (carp.icir.org [192.150.187.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id CD80343E8A; Wed, 16 Oct 2002 11:19:03 -0700 (PDT) (envelope-from rizzo@carp.icir.org) Received: from carp.icir.org (localhost [127.0.0.1]) by carp.icir.org (8.12.3/8.12.3) with ESMTP id g9GIIvpJ038254; Wed, 16 Oct 2002 11:18:57 -0700 (PDT) (envelope-from rizzo@carp.icir.org) Received: (from rizzo@localhost) by carp.icir.org (8.12.3/8.12.3/Submit) id g9GIIvl8038253; Wed, 16 Oct 2002 11:18:57 -0700 (PDT) (envelope-from rizzo) Date: Wed, 16 Oct 2002 11:18:57 -0700 From: Luigi Rizzo To: "JINMEI Tatuya / ?$B?@L@C#:H?(B" Cc: Sam Leffler , Julian Elischer , freebsd-arch@FreeBSD.ORG, freebsd-net@FreeBSD.ORG Subject: Re: CFR: m_tag patch Message-ID: <20021016111857.A38181@carp.icir.org> References: <18d301c26e5e$8b5c7a30$52557f42@errno.com> <080101c27151$b2e92a30$52557f42@errno.com> <20021016074610.C34626@carp.icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from jinmei@isl.rdc.toshiba.co.jp on Thu, Oct 17, 2002 at 03:07:07AM +0900 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Oct 17, 2002 at 03:07:07AM +0900, JINMEI Tatuya / ?$B?@L@C#:H?(B wrote: ... > (I re-read the thread) perhaps the example in my previous message > wasn't good (and it was at least incorrect). According to the > discussion on the thread, we'll probably keep m_tag_cookie being 0 and > use m_tag_id in (e.g.) ip6_output.c. So, we'll be happy if this > convention is kept (and will be kept) at least under netinet6. unfortunately this will not prevent code from having to check that the m_tag_cookie actually corresponds to the value you want, to make sure that your code does not misinterpret as own tags generated/destined to other clients. Am I correct, Sam ? > > Also note that in your example the code should be conditional > > on __FreeBSD_version and not on __FreeBSD__ > > We (KAME) actually need __FreeBSD__, because we want to keep a single > code base for all *BSDs (again, I know this is our local issue). But, i know it is a minor point, but we had this discussion before (and hit related bugs during cross builds). __FreeBSD__ refers to the build environment, whereas __FreeBSD_version matches the source tree you are building, and it is #defined on all versions of FreeBSD that you care about. So, should you have FreeBSD-specific code (hopefully as little as possible), you should *never* use __FreeBSD__ but instead use only __FreeBSD_version. cheers luigi To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 11:29:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E9E5337B404; Wed, 16 Oct 2002 11:29:53 -0700 (PDT) Received: from ebb.errno.com (ebb.errno.com [66.127.85.87]) by mx1.FreeBSD.org (Postfix) with ESMTP id 500B943EE8; Wed, 16 Oct 2002 11:29:29 -0700 (PDT) (envelope-from sam@errno.com) Received: from melange (melange.errno.com [66.127.85.82]) (authenticated bits=0) by ebb.errno.com (8.12.5/8.12.1) with ESMTP id g9GITP1H005622 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO); Wed, 16 Oct 2002 11:29:26 -0700 (PDT)?g (envelope-from sam@errno.com)œ X-Authentication-Warning: ebb.errno.com: Host melange.errno.com [66.127.85.82] claimed to be melange Message-ID: <24db01c27541$f529da40$52557f42@errno.com> From: "Sam Leffler" To: "JINMEI Tatuya" , "Luigi Rizzo" Cc: , , "Julian Elischer" References: <18d301c26e5e$8b5c7a30$52557f42@errno.com> <080101c27151$b2e92a30$52557f42@errno.com> <20021016074610.C34626@carp.icir.org> <20021016111857.A38181@carp.icir.org> Subject: Re: CFR: m_tag patch Date: Wed, 16 Oct 2002 11:29:25 -0700 Organization: Errno Consulting MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > On Thu, Oct 17, 2002 at 03:07:07AM +0900, JINMEI Tatuya / ?$B?@L@C#:H?(B wrote: > ... > > (I re-read the thread) perhaps the example in my previous message > > wasn't good (and it was at least incorrect). According to the > > discussion on the thread, we'll probably keep m_tag_cookie being 0 and > > use m_tag_id in (e.g.) ip6_output.c. So, we'll be happy if this > > convention is kept (and will be kept) at least under netinet6. > > unfortunately this will not prevent code from having to check that > the m_tag_cookie actually corresponds to the value you want, to > make sure that your code does not misinterpret as own tags > generated/destined to other clients. > > Am I correct, Sam ? > Correct. If you explicitly look inside the m_tag structure instead of using one of the m_tag_* routines to search then you will need to validate m_tag_cookie to avoid interpreting tags created by other modules. The comments I wrote in mbuf.h for this stuff describe this and say that when writing code that is to be compatible with other systems one should always use MTAG_ABI_COMPAT and the openbsd-compatible m_tag_get and m_tag_find routines. m_tag_cookie was mainly added so that netgraph could piggyback on top of the facility and remove it's private code. At some point it may be worthwhile to talk with the openbsd+netbsd folks about adopting this new m_tag facility. Sam To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 11:41:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D4B9737B401; Wed, 16 Oct 2002 11:41:20 -0700 (PDT) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id 72E3A43E9C; Wed, 16 Oct 2002 11:41:20 -0700 (PDT) (envelope-from julian@elischer.org) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20021016184028.ZPUO11063.rwcrmhc52.attbi.com@InterJet.elischer.org>; Wed, 16 Oct 2002 18:40:28 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id LAA86833; Wed, 16 Oct 2002 11:24:35 -0700 (PDT) Date: Wed, 16 Oct 2002 11:24:34 -0700 (PDT) From: Julian Elischer To: Luigi Rizzo Cc: "JINMEI Tatuya / ?$B?@L@C#:H?(B" , Sam Leffler , freebsd-arch@FreeBSD.ORG, freebsd-net@FreeBSD.ORG Subject: Re: CFR: m_tag patch In-Reply-To: <20021016111857.A38181@carp.icir.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Oct 2002, Luigi Rizzo wrote: > On Thu, Oct 17, 2002 at 03:07:07AM +0900, JINMEI Tatuya / ?$B?@L@C#:H?(B wrote: > ... > > (I re-read the thread) perhaps the example in my previous message > > wasn't good (and it was at least incorrect). According to the > > discussion on the thread, we'll probably keep m_tag_cookie being 0 and > > use m_tag_id in (e.g.) ip6_output.c. So, we'll be happy if this > > convention is kept (and will be kept) at least under netinet6. > > unfortunately this will not prevent code from having to check that > the m_tag_cookie actually corresponds to the value you want, to > make sure that your code does not misinterpret as own tags > generated/destined to other clients. > > Am I correct, Sam ? yes but this is always done with macros, and the OpenBSD compatible macros also check for the correct cookie value. > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 16:48:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D98537B401; Wed, 16 Oct 2002 16:48:41 -0700 (PDT) Received: from clmboh1-smtp5.columbus.rr.com (clmboh1-smtp5.columbus.rr.com [65.24.0.116]) by mx1.FreeBSD.org (Postfix) with ESMTP id E72A143E8A; Wed, 16 Oct 2002 16:48:39 -0700 (PDT) (envelope-from dzerkel@columbus.rr.com) Received: from zoomer (dhcp065-024-067-163.columbus.rr.com [65.24.67.163]) by clmboh1-smtp5.columbus.rr.com (8.11.2/8.11.2) with ESMTP id g9GNm8004213; Wed, 16 Oct 2002 19:48:08 -0400 (EDT) Content-Type: text/plain; charset="iso-8859-1" From: "Danny J. Zerkel" Organization: Zerkular Systems To: Robert Watson Subject: Re: short uid/gid Date: Wed, 16 Oct 2002 19:50:45 -0400 User-Agent: KMail/1.4.3 Cc: Maxim Sobolev , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG References: In-Reply-To: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200210161950.45338.dzerkel@columbus.rr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wednesday 16 October 2002 00:43, Robert Watson wrote: > On Tue, 15 Oct 2002, Danny J. Zerkel wrote: > > At least for our Linux emulation layer, supporting IPC_64 would be one > > of the pieces (probably the main one) keeping The Sims from running. > > The other thing we are missing is the Linux ioctl() interface for > > reading MSDOSFS directories, but that may be optional. > > > > I will eventually take a look at this (IPC_64) if the scheduling fairy > > brings me the time. > > While I think support for the IPC_64 flag under emulation is useful, I'd > rather make use of compatibility system calls and type improvements for > the base FreeBSD implementation of the System V IPC APIs. Most of the > work necessary to support those changes is required in order to better > support fine-grained locking and MAC (breaking out the user and kernel > structures). Also, it means future applications will make use of the > improved APIs by default. In addition, other systems, such as Solaris, > have already made this change in this way, avoiding the IPC_64 flag > design. > > Robert N M Watson FreeBSD Core Team, TrustedBSD Projects > robert@fledge.watson.org Network Associates Laboratories Well, the scheduling fairy is very stingy. I haven't looked at what Solaris does for this. I have used there large file support mechanism in libraries, and something similar may be appropriate, to maintain binary compatibility. It could even be temporary until the 5.0-release, if it were done quickly. I guess that counts me out. After 5.0 it would probably be necessary to keep the compatibility stuff around until 6.0. Danny J. Zerkel dzerkel@columbus.rr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 20:41:53 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 32CD037B401; Wed, 16 Oct 2002 20:41:49 -0700 (PDT) Received: from clmboh1-smtp5.columbus.rr.com (clmboh1-smtp5.columbus.rr.com [65.24.0.116]) by mx1.FreeBSD.org (Postfix) with ESMTP id 110B443E6A; Wed, 16 Oct 2002 20:41:48 -0700 (PDT) (envelope-from dzerkel@columbus.rr.com) Received: from zoomer (dhcp065-024-067-163.columbus.rr.com [65.24.67.163]) by clmboh1-smtp5.columbus.rr.com (8.11.2/8.11.2) with ESMTP id g9H3fk025386; Wed, 16 Oct 2002 23:41:46 -0400 (EDT) Content-Type: text/plain; charset="iso-8859-1" From: "Danny J. Zerkel" Organization: Zerkular Systems To: Robert Watson , Terry Lambert Subject: Re: short uid/gid Date: Wed, 16 Oct 2002 23:44:22 -0400 User-Agent: KMail/1.4.3 Cc: Maxim Sobolev , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG References: In-Reply-To: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200210162344.22969.dzerkel@columbus.rr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wednesday 16 October 2002 13:01, Robert Watson wrote: > On Wed, 16 Oct 2002, Terry Lambert wrote: > > Robert Watson wrote: > > > I'm not convinced there's any value to providing the backward > > > compatibility that has to be asked for: the only benefit to the current > > > short-based API is that it allow serious security holes while not > > > following the standard API offered by other platforms (except Linux). > > > > The main benefit, to my mind, is standards compliance. The secondary > > benefit is ABI similarity for the purposes of supporting ABIs to run > > non-native software (e.g. Linux prior to 2.4). > > Could you point me at the standard that indicates these fields should be > shorts? We're all advocating switching to uid_t/gid_t/id_t here, the only > question is with regard to compatibility. Platforms such as Solaris > already use uid_t/gid_t for these fields, and we break portable > applications because applications assume that uid_t fits into ipc_perm.uid > (for example). > > > Binary backward compatability *demands* that you limit the range of your > > UIDs and GIDs to what will fit in a uint16_t, just as the recent issue > > with a pid_t exceeeding this size has damaged binary backward > > compatability with third party a.out binaries. > > This has to do with the size of the field. uid_t and gid_t are 32-bit on > most relevant platforms, although there are lingering software > compatibility issues (such as the Sysem V IPC issue). The goal now is to > correct lingering incompatibilities so that we can properly support these > ranges. > > > > Freshly compiled applications should be using the proper types to > > > represent uid's and gid's -- if they're not doing that in the existing > > > code, they'll get truncated to the right size for "bug compatibility". > > > If they are using the correct size, they'll work correctly. To be able > > > to run properly on other platforms (vis Solaris), they already should > > > be using those types. > > > > Truncating of oversized values for credentials is Bad(tm), in that what > > you get is a different credential. The constraint has to be a priori > > enforced, or it's meaningless. > > I think we're in agreement here. Unfortunately, because of the "short" > uid/gid/cuid/cgid fields, truncation is *already* happening. That's what > we're trying to fix. If you try to crush a uid_t into ipc_perm.uid on > FreeBSD, the field is truncated, resulting in a vulnerability. What > pushing the type checking into the application space does is generate > warnings for applications making bogus assumptions, and corrects the > truncation performed by the kernel in the current scenario. I.e., when a > SysVIPC shm segment is created by a user with a uid that doesn't fit in > short, the kernel gets it "wrong". The change we're talking about will > fix the kernel, and make type problems visible to the applications if they > make incorrect assumptions. Of course, past applications assuming 16-bit > uids and gids will still be wrong, nothing changes that, but we do fix > current applications. > > > > And it's not like the approach you've described makes it any easier to > > > implement: you still have to break out the old and new structures since > > > changing ipc_perm breaks the ABI for all of the System V interfaces, > > > rewrite the kernel code, etc. You might as well have added the > > > compatibility system calls since you still have to do all the mapping. > > > > His approach avoids the proliferation of system call entry points, which > > may conflict with those of other BSDs (among other things). BSDI > > limited the number of available additional FreeBSD system calls in a > > standardization effort a while back, if you'll remember, so a > > proliferation of calls is Bad. In addition, the compatability required > > this proliferation going forward. > > I suspect that this may end up being a red herring. You do realize that > OpenBSD and NetBSD have *already* made precisely the change I am > describing? Here's the exerpt from OpenBSD's ipc.h: > > struct ipc_perm { > uid_t cuid; /* creator user id */ > gid_t cgid; /* creator group id */ > uid_t uid; /* user id */ > gid_t gid; /* group id */ > mode_t mode; /* r/w permission */ > unsigned short seq; /* sequence # (to generate unique > msg/sem/shm id) */ > key_t key; /* user specified msg/sem/shm key */ > }; > > #ifdef _KERNEL > struct oipc_perm { > unsigned short cuid; /* creator user id */ > unsigned short cgid; /* creator group id */ > unsigned short uid; /* user id */ > unsigned short gid; /* group id */ > unsigned short mode; /* r/w permission */ > unsigned short seq; /* sequence # (to generate unique > msg/sem/shm id) */ > key_t key; /* user specified msg/sem/shm key */ > }; > #endif > > We're *improve* our compatibility with other BSD platforms by making these > changes. > > > FWIW, I would agree with you, if you were advocating a cross-OS ABI > > definition of manifest constants, so that FreeBSD's new ABI, and, say, > > Solaris or Linux's ABI, were more congruent (you could get this without > > a new sstem call vector entry point, by using a different "ELF brand" > > value), but that's not what you are advocating. > > I'm advocating we adopt pretty much the ABI for these calls that Solaris > uses. If you inspect their ipc.h, you'll see that they define two > versions of the structure, ipc_perm, and ipc_perm32. ipc_perm is the > version that makes use of types such as uid_t and gid_t; ipc_perm32 > hard-codes these fields to 32-bit and can be used for compatibility. > > > Also FWIW, my personal preference would be to follow the Solaris ABI; > > it's the least volatile of all ABI's in this area. In terms of the > > available applications, Linux is probably a better choice, but in terms > > of the amount of maintenance work required, Solaris is far and away the > > leader. > > We already have the SysVIPC wrappers for the Linux code, but those are > actually broken because they don't take into account our truncation bugs. > > Robert N M Watson FreeBSD Core Team, TrustedBSD Projects > robert@fledge.watson.org Network Associates Laboratories > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-current" in the body of the message To quote the POSIX standard: The ipc_perm structure shall contain the following members: uid_t uid Owner's user ID. gid_t gid Owner's group ID. uid_t cuid Creator's user ID. gid_t cgid Creator's group ID. mode_t mode Read/write permission. I don't think size is an issue. At least not within a given machine. I think OpenBSD and NetBSD have already taken the correct path. We just need to have a compatibility interface (automatic extra flag: pretty cheap; or alternate syscalls: probably a waste). Danny J. Zerkel dzerkel@columbus.rr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 16 23: 8:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 96B4D37B401; Wed, 16 Oct 2002 23:08:25 -0700 (PDT) Received: from harrier.mail.pas.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F10443E8A; Wed, 16 Oct 2002 23:08:25 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0045.cvx22-bradley.dialup.earthlink.net ([209.179.198.45] helo=mindspring.com) by harrier.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 1823pW-0005qI-00; Wed, 16 Oct 2002 23:08:23 -0700 Message-ID: <3DAE538D.848550B9@mindspring.com> Date: Wed, 16 Oct 2002 23:07:09 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Danny J. Zerkel" Cc: Robert Watson , Maxim Sobolev , "Vladimir B. Grebenschikov,Moscow,408-7227,123-4567,Some-info" , freebsd-arch@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: short uid/gid References: <200210162344.22969.dzerkel@columbus.rr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Danny J. Zerkel" wrote: > To quote the POSIX standard: > > The ipc_perm structure shall contain the following members: > > uid_t uid Owner's user ID. > gid_t gid Owner's group ID. > uid_t cuid Creator's user ID. > gid_t cgid Creator's group ID. > mode_t mode Read/write permission. > > I don't think size is an issue. At least not within a given machine. > I think OpenBSD and NetBSD have already taken the correct path. > We just need to have a compatibility interface (automatic extra > flag: pretty cheap; or alternate syscalls: probably a waste). If our only goal were POSIX compliance, you are correct. Since we also want to maintain binary compatability with third party software, that's not enough. You also have to look at interfaces published in FreeBSD documentation, or by reference in FreeBSD or POSIX documentation. Such as the contents of the ipc_perm structure, as declared, rather than by way of an enumeration of required fields. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 17 9:19:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D712237B401 for ; Thu, 17 Oct 2002 09:19:10 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id F17CD43E8A for ; Thu, 17 Oct 2002 09:19:09 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.3/8.12.3) with ESMTP id g9HGJ8pk043542 for ; Thu, 17 Oct 2002 10:19:08 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 17 Oct 2002 10:18:33 -0600 (MDT) Message-Id: <20021017.101833.110719994.imp@bsdimp.com> To: arch@freebsd.org Subject: color, again, in grotty From: "M. Warner Losh" X-Mailer: Mew version 2.1 on Emacs 21.2 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG OK. Something is really broken with the latest groff. nroff -mandoc devd.8 | more UNTITLED LOCAL UNTITLED ESC[1mNAMEESC[0m ESC[1mdevd ESC[22m- Device state change daemon etc Notice that it now bogusly and wrongly contains escape characters. This is wrong wrong wrong and must be fixed. Color is cool, but putting it in nroff is absolutely the worst and wrongest place one can put it. Why is it done this way? Can't it be fixed? Also, I know there was a long thread about this in cvs-committers, but that's a lame place to talk about it, so I'm restarting the conversation here. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 17 14:27:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0512637B401 for ; Thu, 17 Oct 2002 14:27:27 -0700 (PDT) Received: from PIKES.panasas.com (gw2.panasas.com [65.194.124.178]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7CB8D43E6A for ; Thu, 17 Oct 2002 14:27:26 -0700 (PDT) (envelope-from kbarnett@panasas.com) Received: by PIKES.panasas.com with Internet Mail Service (5.5.2653.19) id ; Thu, 17 Oct 2002 17:27:25 -0400 Message-ID: <30489F1321F5C343ACF6872B2CF7942A023E7B58@PIKES.panasas.com> From: kbarnett@panasas.com To: freebsd-arch@FreeBSD.org Subject: accessing RAM mapped above 4GB Date: Thu, 17 Oct 2002 17:27:25 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Does the x86 version of FreeBSD support accessing physical RAM mapped above 4GB? In other words, does FreeBSD support PAE (Physical Address Extension)? If not, are there any plans to add this support in the near future? Thanks, Kevin Barnett Software Engineer Panasas, Inc. (http://www.panasas.com) (832) 601-7570 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 17 14:40:10 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 644CE37B401 for ; Thu, 17 Oct 2002 14:40:09 -0700 (PDT) Received: from sccrmhc02.attbi.com (sccrmhc02.attbi.com [204.127.202.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id DD52343E3B for ; Thu, 17 Oct 2002 14:40:08 -0700 (PDT) (envelope-from julian@elischer.org) Received: from InterJet.elischer.org ([12.232.206.8]) by sccrmhc02.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20021017214008.DULQ26432.sccrmhc02.attbi.com@InterJet.elischer.org>; Thu, 17 Oct 2002 21:40:08 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA03124; Thu, 17 Oct 2002 14:35:39 -0700 (PDT) Date: Thu, 17 Oct 2002 14:35:38 -0700 (PDT) From: Julian Elischer To: kbarnett@panasas.com Cc: freebsd-arch@FreeBSD.org Subject: Re: accessing RAM mapped above 4GB In-Reply-To: <30489F1321F5C343ACF6872B2CF7942A023E7B58@PIKES.panasas.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 17 Oct 2002 kbarnett@panasas.com wrote: > Does the x86 version of FreeBSD support accessing physical RAM > mapped above 4GB? In other words, does FreeBSD support PAE (Physical > Address Extension)? If not, are there any plans to add this support > in the near future? "Not Yet" To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 17 14:40:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E1FEE37B401 for ; Thu, 17 Oct 2002 14:40:33 -0700 (PDT) Received: from carver.gumbysoft.com (carver.gumbysoft.com [66.220.23.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 803CE43E88 for ; Thu, 17 Oct 2002 14:40:33 -0700 (PDT) (envelope-from dwhite@gumbysoft.com) Received: by carver.gumbysoft.com (Postfix, from userid 1000) id 4D6AB72FD4; Thu, 17 Oct 2002 14:38:55 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by carver.gumbysoft.com (Postfix) with ESMTP id 4C3DA72FD1; Thu, 17 Oct 2002 14:38:55 -0700 (PDT) Date: Thu, 17 Oct 2002 14:38:55 -0700 (PDT) From: Doug White To: kbarnett@panasas.com Cc: freebsd-arch@FreeBSD.org Subject: Re: accessing RAM mapped above 4GB In-Reply-To: <30489F1321F5C343ACF6872B2CF7942A023E7B58@PIKES.panasas.com> Message-ID: <20021017143821.S88254-100000@carver.gumbysoft.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Long line wrapped. On Thu, 17 Oct 2002 kbarnett@panasas.com wrote: > Does the x86 version of FreeBSD support accessing physical RAM mapped > above 4GB? No. > In other words, does FreeBSD support PAE (Physical Address Extension)? Not yet. > If not, are there any plans to add this support in the near future? Yes, work is underway. -- Doug White | FreeBSD: The Power to Serve dwhite@gumbysoft.com | www.FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 17 16:12:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4C22937B401 for ; Thu, 17 Oct 2002 16:12:48 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D5C4443E65 for ; Thu, 17 Oct 2002 16:12:47 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0240.cvx40-bradley.dialup.earthlink.net ([216.244.42.240] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 182Jol-0007PC-00; Thu, 17 Oct 2002 16:12:40 -0700 Message-ID: <3DAF439F.52DB7AF3@mindspring.com> Date: Thu, 17 Oct 2002 16:11:27 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Doug White Cc: kbarnett@panasas.com, freebsd-arch@FreeBSD.org Subject: Re: accessing RAM mapped above 4GB References: <20021017143821.S88254-100000@carver.gumbysoft.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Doug White wrote: > On Thu, 17 Oct 2002 kbarnett@panasas.com wrote: [ ... PAE support ... ] > > If not, are there any plans to add this support in the near future? > > Yes, work is underway. You're writing code for this? I know Peter Wemm had discussed doing this for Yahoo, but the idea was shelved; is this a new effort? Will there be a way to get around the additional overhead for this in the <4G (most common) case? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 17 20: 5:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A664837B401 for ; Thu, 17 Oct 2002 20:05:45 -0700 (PDT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7062743E65 for ; Thu, 17 Oct 2002 20:05:45 -0700 (PDT) (envelope-from ps@mu.org) Received: by elvis.mu.org (Postfix, from userid 1000) id 374D5AE255; Thu, 17 Oct 2002 20:05:45 -0700 (PDT) Date: Thu, 17 Oct 2002 20:05:45 -0700 From: Paul Saab To: Terry Lambert Cc: Doug White , kbarnett@panasas.com, freebsd-arch@FreeBSD.org Subject: Re: accessing RAM mapped above 4GB Message-ID: <20021018030545.GA47024@elvis.mu.org> References: <20021017143821.S88254-100000@carver.gumbysoft.com> <3DAF439F.52DB7AF3@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3DAF439F.52DB7AF3@mindspring.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert (tlambert2@mindspring.com) wrote: > I know Peter Wemm had discussed doing this for Yahoo, but the > idea was shelved; is this a new effort? Since when? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 17 21:47: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D437537B4DF for ; Thu, 17 Oct 2002 21:46:53 -0700 (PDT) Received: from pirzyk.org (dsl-65-184-181-29.telocity.com [65.184.181.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9C05943E88 for ; Thu, 17 Oct 2002 21:46:52 -0700 (PDT) (envelope-from jim@pirzyk.org) Received: from snoopy (snoopy.pirzyk.org [10.26.0.4]) by pirzyk.org (8.12.3/8.12.3) with ESMTP id g9I4iFaw007027 for ; Thu, 17 Oct 2002 21:44:16 -0700 (PDT) (envelope-from jim@pirzyk.org) From: Jim Pirzyk Subject: Fwd: getnetbyname broken for DNS case? Date: Thu, 17 Oct 2002 21:46:48 -0700 User-Agent: KMail/1.4.3 To: freebsd-arch@freebsd.org MIME-Version: 1.0 Message-Id: <200210172143.59341.jim@pirzyk.org> Content-Type: Multipart/Mixed; boundary="------------Boundary-00=_0MU5PT6WIZBMNQMHKSST" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --------------Boundary-00=_0MU5PT6WIZBMNQMHKSST Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable So I have developed a patch to use gethostbydns* in the getnetbydns* case. I did not get any comments from -net on this original question, so I thought to change the audience. Comments, questions, etc. - JimP ---------- Forwarded Message ---------- Subject: getnetbyname broken for DNS case? Date: Wed, 18 Sep 2002 00:32:11 -0700 From: Jim Pirzyk To: freebsd-net@freebsd.org Hi, I am trying to debug a problem that I have with /etc/exports. I cannot put in the symbolic name for my network (10.26.0.0) without putting it in /etc/networks. It is currently in DNS, but it looks like the getnetby= name is not working correctly for the DNS case. The functions that are broken is _getnetbydnsname() and getnetanswer(). The _getnetbydnsname() is easily fixed by changing T_PTR to T_A in res_query(), but getnetanswer() is harder to solve. It is not parsing= the format of the result correctly. First I had to put some code in to test "type" being T_PTR only if "net_i" is set to BYADDR and T_A if "net_= i" is set to BYNAME. When I go into the res_hnok() test, it is assumed by the code that it IP address is in ascii format, which it is not (it is in network binar= y order). If I remove the res_hnok() test, then it does not load the netent.n_net address correctly. The code is broken for both -CURRENT and -STABLE. So my question is then, can I rewrite getnetbyname() to use gethostbyname= () call and massage the result to a netent entry? This would solve the DNS result parsing problem and would also get us T_AAA (IPv6) network name addresses. I would also implement getnetbyaddr() in terms of gethostbyad= dr() too. - JimP --- @(#) $Id: dot.signature,v 1.10 2001/05/17 23:38:49 Jim.Pirzyk Exp $ __o jim@pirzyk.org ----------------------------------------------- _'\<,_ (*)/ (*) ------------------------------------------------------- --=20 --- @(#) $Id: dot.signature,v 1.10 2001/05/17 23:38:49 Jim.Pirzyk Exp $ __o jim@pirzyk.org ----------------------------------------------- _'\<,_ =20 (*)/ (*) =20 --------------Boundary-00=_0MU5PT6WIZBMNQMHKSST Content-Type: text/x-diff; charset="iso-8859-1"; name="getnetbydns.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="getnetbydns.patch" *** lib/libc/net/getnetbydns.c.orig Thu Sep 19 08:58:31 2002 --- lib/libc/net/getnetbydns.c Sat Sep 21 09:52:14 2002 *************** *** 81,88 **** extern int h_errno; - #define BYADDR 0 - #define BYNAME 1 #define MAXALIASES 35 #if PACKETSZ > 1024 --- 81,86 ---- *************** *** 91,222 **** #define MAXPACKET 1024 #endif ! typedef union { ! HEADER hdr; ! u_char buf[MAXPACKET]; ! } querybuf; ! ! typedef union { ! long al; ! char ac; ! } align; static struct netent * ! getnetanswer(answer, anslen, net_i) ! querybuf *answer; ! int anslen; ! int net_i; { - register HEADER *hp; - register u_char *cp; - register int n; - u_char *eom; - int type, class, buflen, ancount, qdcount, haveanswer, i, nchar; - char aux1[MAXHOSTNAMELEN], aux2[MAXHOSTNAMELEN], ans[MAXHOSTNAMELEN]; - char *in, *st, *pauxt, *bp, **ap; - char *paux1 = &aux1[0], *paux2 = &aux2[0], flag = 0; static struct netent net_entry; ! static char *net_aliases[MAXALIASES], netbuf[PACKETSZ]; ! /* ! * find first satisfactory answer ! * ! * answer --> +------------+ ( MESSAGE ) ! * | Header | ! * +------------+ ! * | Question | the question for the name server ! * +------------+ ! * | Answer | RRs answering the question ! * +------------+ ! * | Authority | RRs pointing toward an authority ! * | Additional | RRs holding additional information ! * +------------+ ! */ ! eom = answer->buf + anslen; ! hp = &answer->hdr; ! ancount = ntohs(hp->ancount); /* #/records in the answer section */ ! qdcount = ntohs(hp->qdcount); /* #/entries in the question section */ ! bp = netbuf; ! buflen = sizeof(netbuf); ! cp = answer->buf + HFIXEDSZ; ! if (!qdcount) { ! if (hp->aa) ! h_errno = HOST_NOT_FOUND; ! else ! h_errno = TRY_AGAIN; ! return (NULL); } - while (qdcount-- > 0) - cp += __dn_skipname(cp, eom) + QFIXEDSZ; - ap = net_aliases; - *ap = NULL; net_entry.n_aliases = net_aliases; ! haveanswer = 0; ! while (--ancount >= 0 && cp < eom) { ! n = dn_expand(answer->buf, eom, cp, bp, buflen); ! if ((n < 0) || !res_dnok(bp)) ! break; ! cp += n; ! ans[0] = '\0'; ! (void)strncpy(&ans[0], bp, sizeof(ans) - 1); ! ans[sizeof(ans) - 1] = '\0'; ! GETSHORT(type, cp); ! GETSHORT(class, cp); ! cp += INT32SZ; /* TTL */ ! GETSHORT(n, cp); ! if (class == C_IN && type == T_PTR) { ! n = dn_expand(answer->buf, eom, cp, bp, buflen); ! if ((n < 0) || !res_hnok(bp)) { ! cp += n; ! return (NULL); ! } ! cp += n; ! *ap++ = bp; ! n = strlen(bp) + 1; ! bp += n; ! buflen -= n; ! net_entry.n_addrtype = ! (class == C_IN) ? AF_INET : AF_UNSPEC; ! haveanswer++; ! } ! } ! if (haveanswer) { ! *ap = NULL; ! switch (net_i) { ! case BYADDR: ! net_entry.n_name = *net_entry.n_aliases; ! net_entry.n_net = 0L; ! break; ! case BYNAME: ! in = *net_entry.n_aliases; ! net_entry.n_name = &ans[0]; ! aux2[0] = '\0'; ! for (i = 0; i < 4; i++) { ! for (st = in, nchar = 0; ! *st != '.'; ! st++, nchar++) ! ; ! if (nchar != 1 || *in != '0' || flag) { ! flag = 1; ! (void)strncpy(paux1, ! (i==0) ? in : in-1, ! (i==0) ?nchar : nchar+1); ! paux1[(i==0) ? nchar : nchar+1] = '\0'; ! pauxt = paux2; ! paux2 = strcat(paux1, paux2); ! paux1 = pauxt; ! } ! in = ++st; ! } ! net_entry.n_net = inet_network(paux2); ! break; ! } ! net_entry.n_aliases++; ! return (&net_entry); ! } ! h_errno = TRY_AGAIN; ! return (NULL); } struct netent * --- 89,132 ---- #define MAXPACKET 1024 #endif ! struct hostent * _gethostbydnsaddr(const char *, int, int); ! struct hostent * _gethostbydnsname(const char *, int); static struct netent * ! getnetanswer(answer) ! struct hostent *answer; { static struct netent net_entry; ! static char *net_aliases[MAXALIASES]; ! u_long net; ! int i; ! /* Check to make sure we found a hostent */ ! if ( !answer ) return (NULL); ! ! net_entry.n_name = answer->h_name; ! ! for (i = 0; answer->h_aliases[i] && i < MAXALIASES; i++) { ! net_aliases[i] = answer->h_aliases[i]; } net_entry.n_aliases = net_aliases; ! ! net_entry.n_addrtype = answer->h_addrtype; ! ! /* One difference between gethostby* and getnetby* is the */ ! /* address for the former is in network byte order and the */ ! /* latter is in machine byte order :( */ ! /* Do the memcpy instead of a cast to make sure we have */ ! /* aligned memory for a u_long */ ! memcpy (&net, answer->h_addr, sizeof (net)); ! net_entry.n_net = ntohl(net); ! ! /* Strip trailing zeros */ ! while ((net_entry.n_net & 0xff) == 0 && net_entry.n_net != 0) ! net_entry.n_net >>= 8; ! ! return (&net_entry); } struct netent * *************** *** 224,301 **** register unsigned long net; register int net_type; { ! unsigned int netbr[4]; ! int nn, anslen; ! querybuf buf; ! char qbuf[MAXDNAME]; ! unsigned long net2; ! struct netent *net_entry; ! ! if (net_type != AF_INET) ! return (NULL); ! ! for (nn = 4, net2 = net; net2; net2 >>= 8) ! netbr[--nn] = net2 & 0xff; ! switch (nn) { ! case 3: /* Class A */ ! sprintf(qbuf, "0.0.0.%u.in-addr.arpa", netbr[3]); ! break; ! case 2: /* Class B */ ! sprintf(qbuf, "0.0.%u.%u.in-addr.arpa", netbr[3], netbr[2]); ! break; ! case 1: /* Class C */ ! sprintf(qbuf, "0.%u.%u.%u.in-addr.arpa", netbr[3], netbr[2], ! netbr[1]); ! break; ! case 0: /* Class D - E */ ! sprintf(qbuf, "%u.%u.%u.%u.in-addr.arpa", netbr[3], netbr[2], ! netbr[1], netbr[0]); ! break; ! } ! anslen = res_query(qbuf, C_IN, T_PTR, (u_char *)&buf, sizeof(buf)); ! if (anslen < 0) { ! #ifdef DEBUG ! if (_res.options & RES_DEBUG) ! printf("res_query failed\n"); ! #endif ! return (NULL); ! } ! net_entry = getnetanswer(&buf, anslen, BYADDR); ! if (net_entry) { ! unsigned u_net = net; /* maybe net should be unsigned ? */ ! ! /* Strip trailing zeros */ ! while ((u_net & 0xff) == 0 && u_net != 0) ! u_net >>= 8; ! net_entry->n_net = u_net; ! return (net_entry); ! } ! return (NULL); } struct netent * _getnetbydnsname(net) register const char *net; { ! int anslen; ! querybuf buf; ! char qbuf[MAXDNAME]; ! ! if ((_res.options & RES_INIT) == 0 && res_init() == -1) { ! h_errno = NETDB_INTERNAL; ! return (NULL); ! } ! strncpy(qbuf, net, sizeof(qbuf) - 1); ! qbuf[sizeof(qbuf) - 1] = '\0'; ! anslen = res_search(qbuf, C_IN, T_PTR, (u_char *)&buf, sizeof(buf)); ! if (anslen < 0) { ! #ifdef DEBUG ! if (_res.options & RES_DEBUG) ! printf("res_query failed\n"); ! #endif ! return (NULL); ! } ! return getnetanswer(&buf, anslen, BYNAME); } void --- 134,194 ---- register unsigned long net; register int net_type; { ! unsigned int netbr[4]; ! int nn, anslen; ! char qbuf[MAXDNAME]; ! unsigned long net2; ! struct hostent *buf; ! ! for (nn = 4, net2 = net; net2; net2 >>= 8) ! netbr[--nn] = net2 & 0xff; ! switch (nn) { ! case 3: /* Class A */ ! sprintf(qbuf, "0.0.0.%u.in-addr.arpa", netbr[3]); ! break; ! case 2: /* Class B */ ! sprintf(qbuf, "0.0.%u.%u.in-addr.arpa", netbr[3], netbr[2]); ! break; ! case 1: /* Class C */ ! sprintf(qbuf, "0.%u.%u.%u.in-addr.arpa", netbr[3], netbr[2], ! netbr[1]); ! break; ! case 0: /* Class D - E */ ! sprintf(qbuf, "%u.%u.%u.%u.in-addr.arpa", netbr[3], netbr[2], ! netbr[1], netbr[0]); ! break; ! } ! ! anslen = strlen(qbuf); ! ! if ((_res.options & RES_INIT) == 0 && res_init() == -1) { ! h_errno = NETDB_INTERNAL; ! return (NULL); ! } ! if (_res.options & RES_USE_INET6) { /* XXX */ ! buf = _gethostbydnsaddr(qbuf, anslen, AF_INET6); /* XXX */ ! if (buf) /* XXX */ ! return (getnetanswer(buf)); /* XXX */ ! } /* XXX */ ! return getnetanswer(_gethostbydnsaddr(qbuf, anslen, AF_INET)); } struct netent * _getnetbydnsname(net) register const char *net; { ! struct hostent *hp; ! ! if ((_res.options & RES_INIT) == 0 && res_init() == -1) { ! h_errno = NETDB_INTERNAL; ! return (NULL); ! } ! if (_res.options & RES_USE_INET6) { /* XXX */ ! hp = _gethostbydnsname(net, AF_INET6); /* XXX */ ! if (hp) /* XXX */ ! return (getnetanswer(hp)); /* XXX */ ! } /* XXX */ ! return getnetanswer(_gethostbydnsname(net, AF_INET)); } void --------------Boundary-00=_0MU5PT6WIZBMNQMHKSST-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 17 23:13:30 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2DDC037B406 for ; Thu, 17 Oct 2002 23:13:29 -0700 (PDT) Received: from snipe.mail.pas.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0A20E43E9E for ; Thu, 17 Oct 2002 23:13:28 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0077.cvx21-bradley.dialup.earthlink.net ([209.179.192.77] helo=mindspring.com) by snipe.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 182QNw-0000Q3-00; Thu, 17 Oct 2002 23:13:25 -0700 Message-ID: <3DAFA63E.8E2335D7@mindspring.com> Date: Thu, 17 Oct 2002 23:12:14 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Paul Saab Cc: Doug White , kbarnett@panasas.com, freebsd-arch@FreeBSD.org Subject: Re: accessing RAM mapped above 4GB References: <20021017143821.S88254-100000@carver.gumbysoft.com> <3DAF439F.52DB7AF3@mindspring.com> <20021018030545.GA47024@elvis.mu.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Paul Saab wrote: > Terry Lambert (tlambert2@mindspring.com) wrote: > > I know Peter Wemm had discussed doing this for Yahoo, but the > > idea was shelved; is this a new effort? > > Since when? He said something about doing the code many months ago; he mentioned that it wasn't being worked on about 2 months ago. I guess we could search the -current/-arch archives for exact dates; it wasn't a position statement on behalf of Yahoo or anything. If you're doing the code, then I guess it's not shelved. Are you doing the code? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Oct 18 2:53:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EE94D37B401 for ; Fri, 18 Oct 2002 02:53:44 -0700 (PDT) Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id E8DF243E6E for ; Fri, 18 Oct 2002 02:53:34 -0700 (PDT) (envelope-from ru@whale.sunbay.crimea.ua) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g9I9oQ626710; Fri, 18 Oct 2002 12:50:26 +0300 (EEST) (envelope-from ru) Date: Fri, 18 Oct 2002 12:50:26 +0300 From: Ruslan Ermilov To: "M. Warner Losh" Cc: arch@FreeBSD.ORG Subject: Re: color, again, in grotty Message-ID: <20021018095026.GA3386@sunbay.com> References: <20021017.101833.110719994.imp@bsdimp.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="98e8jtXdkpgskNou" Content-Disposition: inline In-Reply-To: <20021017.101833.110719994.imp@bsdimp.com> User-Agent: Mutt/1.3.99i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --98e8jtXdkpgskNou Content-Type: multipart/mixed; boundary="HcAYCG3uE/tztfnV" Content-Disposition: inline --HcAYCG3uE/tztfnV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [Before I start my speech: I have disabled the SGR support in grotty(1) completely until we get the resolution here. This does not of course affect the non-TTY groff(1) devices like -Tps, -TX100, etc.] On Thu, Oct 17, 2002 at 10:18:33AM -0600, M. Warner Losh wrote: > OK. Something is really broken with the latest groff. >=20 > nroff -mandoc devd.8 | more > UNTITLED LOCAL UNT= ITLED >=20 > ESC[1mNAMEESC[0m > ESC[1mdevd ESC[22m- Device state change daemon >=20 >=20 > etc >=20 > Notice that it now bogusly and wrongly contains escape characters. > This is wrong wrong wrong and must be fixed. Color is cool, but > putting it in nroff is absolutely the worst and wrongest place one can > put it. Why is it done this way? Can't it be fixed? >=20 This is only one opinion, and there exist others I would like you to listen to before judging: http://people.freebsd.org/~ru/grotty-SGR.mbox.gz (25 messages thread) Only a few quotes in support of colors, if you permit: : > Well, if the SGR escapes are on by default, I'd think you do need to : > consider the effects that will have on popular terminals. AFAIK, : > even the majority of xterm versions don't support colors, outside : > the GNU/Linux world. :=20 : The eternal question whether we shall follow a standard or not... : ISO 6429 is from 1991 -- isn't eleven years enough to wait? :=20 : > Didn't you say that you also use SGR for bold and underline? That : > would produce similar problems for terminals whose bold and : > underline commands are not SGR, right? :=20 : Do you think there are terminals which have SGR sequences for : activating colors and non-SGR sequences for activating bold and : underline? If you read the thread, I am pretty sure you will get the same feeling as I am -- it is that the new ANSI SGR escapes became the new standard for TTY-type output devices serviced with grotty(1), and nroff(1). Our default pager (less/more) also supports this feature through the -R option, as does the new -R option in the new info(1). We have also learned one pitfall with this new approach. Our default terminal driver, syscons(4), renders the "underscore" attribute, SGR E[4m (see screen(4)), the same as bold, which is the (code documented) bug, so using this new SGR scheme directly as it comes with new Groff worsens the output on cons25* type terminals. On the other hand, classical nroff(1) filters like ul(1) (and not by accident less(1)) render underlining with the "reverse" attribute on terminals that do not support underlining, like cons25. So we must either fix syscons(4) or to somehow tell grotty(1) to emit old sequences to print bold and italic characters. I have prepared and sent the patch to Groff maintainers that tells grotty(1) to use the old scheme for printing bold and italic characters, but otherwise have color output enabled. Today I have tried a different approach ("fixing" the driver), and was surprised by the unexpected good results -- I have changed the syscons(4) so that it renders "underscore" with "reverse", and I can now differentiate all three bold (bold attr), italic (underline attr), and bold+italic (bold+underline attrs) fonts (and colors!) on a cons25 TTY. There are three ways we can go here: 1. Disable SGR support for grotty(1). It can be soft-disabled by setting GROFF_NO_SGR in environment, or hard-disabled system-wide (like it is now in -CURRENT as of this writing) in /usr/share/tmac/troffrc. 2. Stay compatible with the new standard (11 years old ANSI std). This means we fix syscons(4) so that it renders "underscore" distinguishable from "bold", and change the default pager to "more -R". There will still be an option of soft-disabling it for those who want it disabled. I suggest option 2 for 5.0-CURRENT and soft option 1 (soft version) for 4.7-STABLE (once I import the new Groff into RELENG_4). Patch for option 2 is included. Soft option 1 would also require modifying src/etc/login.conf to add GROFF_NO_SGR to environment. Cheers, --=20 Ruslan Ermilov Sysadmin and DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age --HcAYCG3uE/tztfnV Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=p Content-Transfer-Encoding: quoted-printable Index: sys/dev/syscons/scterm-sc.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/dev/syscons/scterm-sc.c,v retrieving revision 1.17 diff -u -r1.17 scterm-sc.c --- sys/dev/syscons/scterm-sc.c 12 Sep 2001 08:37:13 -0000 1.17 +++ sys/dev/syscons/scterm-sc.c 18 Oct 2002 09:50:24 -0000 @@ -797,7 +797,8 @@ { int attr, mask =3D tcp->attr_mask; =20 - if (mask & REVERSE_ATTR) { + /* XXX: underline mapping for Hercules adapter can be better */ + if (mask & (REVERSE_ATTR | UNDERLINE_ATTR)) { attr =3D ((mask & FG_CHANGED) ? tcp->cur_color.bg : tcp->rev_color.fg) | (((mask & BG_CHANGED) ? @@ -805,8 +806,7 @@ } else attr =3D tcp->cur_color.fg | (tcp->cur_color.bg << 4); =20 - /* XXX: underline mapping for Hercules adapter can be better */ - if (mask & (BOLD_ATTR | UNDERLINE_ATTR)) + if (mask & BOLD_ATTR) attr ^=3D 0x08; if (mask & BLINK_ATTR) attr ^=3D 0x80; Index: contrib/groff/tmac/troffrc =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/contrib/groff/tmac/troffrc,v retrieving revision 1.11 diff -u -r1.11 troffrc --- contrib/groff/tmac/troffrc 18 Oct 2002 09:10:44 -0000 1.11 +++ contrib/groff/tmac/troffrc 18 Oct 2002 09:50:24 -0000 @@ -45,7 +45,4 @@ . .\} . -.\" Disable SGR support in grotty(1). -.if n .do nop \X'tty: sgr 0'\c -. .\" Don't let blank lines creep in here. Index: etc/root/dot.cshrc =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/etc/root/dot.cshrc,v retrieving revision 1.28 diff -u -r1.28 dot.cshrc --- etc/root/dot.cshrc 10 Jan 2001 02:37:16 -0000 1.28 +++ etc/root/dot.cshrc 18 Oct 2002 09:50:19 -0000 @@ -17,7 +17,7 @@ set path =3D (/sbin /bin /usr/sbin /usr/bin /usr/games /usr/local/sbin /us= r/local/bin /usr/X11R6/bin $HOME/bin) =20 setenv EDITOR vi -setenv PAGER more +setenv PAGER "more -R" setenv BLOCKSIZE K =20 if ($?prompt) then Index: etc/root/dot.profile =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/etc/root/dot.profile,v retrieving revision 1.20 diff -u -r1.20 dot.profile --- etc/root/dot.profile 27 Aug 1999 23:24:09 -0000 1.20 +++ etc/root/dot.profile 18 Oct 2002 09:50:19 -0000 @@ -6,5 +6,5 @@ export HOME TERM=3D${TERM:-cons25} export TERM -PAGER=3Dmore +PAGER=3D"more -R" export PAGER Index: share/skel/dot.cshrc =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/share/skel/dot.cshrc,v retrieving revision 1.13 diff -u -r1.13 dot.cshrc --- share/skel/dot.cshrc 10 Jan 2001 17:35:28 -0000 1.13 +++ share/skel/dot.cshrc 18 Oct 2002 09:50:19 -0000 @@ -17,7 +17,7 @@ set path =3D (/sbin /bin /usr/sbin /usr/bin /usr/games /usr/local/sbin /us= r/local/bin /usr/X11R6/bin $HOME/bin) =20 setenv EDITOR vi -setenv PAGER more +setenv PAGER "more -R" setenv BLOCKSIZE K =20 if ($?prompt) then Index: share/skel/dot.profile =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/share/skel/dot.profile,v retrieving revision 1.21 diff -u -r1.21 dot.profile --- share/skel/dot.profile 7 Jul 2002 00:00:54 -0000 1.21 +++ share/skel/dot.profile 18 Oct 2002 09:50:19 -0000 @@ -16,7 +16,7 @@ =20 BLOCKSIZE=3DK; export BLOCKSIZE EDITOR=3Dvi; export EDITOR -PAGER=3Dmore; export PAGER +PAGER=3D"more -R"; export PAGER =20 # set ENV to a file invoked each time sh is started for interactive use. ENV=3D$HOME/.shrc; export ENV --HcAYCG3uE/tztfnV-- --98e8jtXdkpgskNou Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (FreeBSD) iD8DBQE9r9liUkv4P6juNwoRAgobAKCCAjApHRxBOMYXV5eFs+4EeU91/QCfb7Zk DULnn6uPbtohd0OR5r8k+cM= =nNDK -----END PGP SIGNATURE----- --98e8jtXdkpgskNou-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Oct 18 8:48:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B26A637B406; Fri, 18 Oct 2002 08:48:30 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id E0BEA43EB2; Fri, 18 Oct 2002 08:48:29 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.3/8.12.3) with ESMTP id g9IFmSpk049536; Fri, 18 Oct 2002 09:48:29 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Fri, 18 Oct 2002 09:48:01 -0600 (MDT) Message-Id: <20021018.094801.123456703.imp@bsdimp.com> To: ru@FreeBSD.ORG Cc: arch@FreeBSD.ORG Subject: Re: color, again, in grotty From: "M. Warner Losh" In-Reply-To: <20021018095026.GA3386@sunbay.com> References: <20021017.101833.110719994.imp@bsdimp.com> <20021018095026.GA3386@sunbay.com> X-Mailer: Mew version 2.1 on Emacs 21.2 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <20021018095026.GA3386@sunbay.com> Ruslan Ermilov writes: : : > Well, if the SGR escapes are on by default, I'd think you do need to : : > consider the effects that will have on popular terminals. AFAIK, : : > even the majority of xterm versions don't support colors, outside : : > the GNU/Linux world. : : : : The eternal question whether we shall follow a standard or not... : : ISO 6429 is from 1991 -- isn't eleven years enough to wait? vt100 escape sequences have been known since 1980, 20 years! They are the defact terminal emulation escape sequence, why not code all programs to use them? Because although standardized, they are still device dependent. They are just one way to encode that information, not the only way. : : > Didn't you say that you also use SGR for bold and underline? That : : > would produce similar problems for terminals whose bold and : : > underline commands are not SGR, right? : : : : Do you think there are terminals which have SGR sequences for : : activating colors and non-SGR sequences for activating bold and : : underline? : : If you read the thread, I am pretty sure you will get the same feeling : as I am -- it is that the new ANSI SGR escapes became the new standard : for TTY-type output devices serviced with grotty(1), and nroff(1). : Our default pager (less/more) also supports this feature through the : -R option, as does the new -R option in the new info(1). I know that there sre terminals for which the SGR sequences screw things up. I had to deal with a raft of Televideo terminals back in the day that we were constantly finding stupid programmers who hard coded things to vt100 and we had to keep fixing the stupid programs because those programmers couldn't be bothered to use curses. I find this to be an analogous case. Back in the day, one could have argued that since all terminals understood vt100 sequences, we should have stored our man pages in vt100 escape sequences? Well, vt100s are almost impossible to find and live on only as emulation and most terminals no longer are only vt100, but some are close (cons25, for example, dies on some vt100 sequences). Back in the day, vt100's were claimed to be ANSI compliant terminals, which is also a standard. Yet, we didn't standardize on them because the nice thing about standards is that there are so many to choose from. So this whole argument that it is standard is a little bogus. : We have also learned one pitfall with this new approach. Our default : terminal driver, syscons(4), renders the "underscore" attribute, SGR : E[4m (see screen(4)), the same as bold, which is the (code documented) : bug, so using this new SGR scheme directly as it comes with new Groff : worsens the output on cons25* type terminals. On the other hand, : classical nroff(1) filters like ul(1) (and not by accident less(1)) : render underlining with the "reverse" attribute on terminals that do : not support underlining, like cons25. So we must either fix syscons(4) : or to somehow tell grotty(1) to emit old sequences to print bold and : italic characters. I have prepared and sent the patch to Groff : maintainers that tells grotty(1) to use the old scheme for printing : bold and italic characters, but otherwise have color output enabled. : Today I have tried a different approach ("fixing" the driver), and : was surprised by the unexpected good results -- I have changed the : syscons(4) so that it renders "underscore" with "reverse", and I : can now differentiate all three bold (bold attr), italic (underline : attr), and bold+italic (bold+underline attrs) fonts (and colors!) : on a cons25 TTY. That proves my point more. Just because these escape sequences are a standard, they are not the only way to go. 'Fixing' syscons will actually break legacy programs, which is bad and not really an option. Especially because it is to be compatible with a dubious standard from the Linux world. Bah humbug. If you want a linux comaptible console, write one, don't coopt FreeBSD. : There are three ways we can go here: : : 1. Disable SGR support for grotty(1). : : It can be soft-disabled by setting GROFF_NO_SGR in environment, : or hard-disabled system-wide (like it is now in -CURRENT as of : this writing) in /usr/share/tmac/troffrc. I like this option. SGR belongs not in groff, but in ul, et al. : 2. Stay compatible with the new standard (11 years old ANSI std). : This means we fix syscons(4) so that it renders "underscore" : distinguishable from "bold", and change the default pager to : "more -R". There will still be an option of soft-disabling : it for those who want it disabled. I don't like this option. Fixing syscons is silly and will break compatibility with the past. The standard is just 'a' standard, not THE standard like you get with posix or ansi-c. There are many other legitimate ways of doing things and moving to device dependence because someone takes a shining to a standard is not a wise move. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Oct 18 9:15:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AE96337B401; Fri, 18 Oct 2002 09:15:19 -0700 (PDT) Received: from nagual.pp.ru (pobrecita.freebsd.ru [194.87.13.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id B6A1E43EA9; Fri, 18 Oct 2002 09:15:18 -0700 (PDT) (envelope-from ache@pobrecita.freebsd.ru) Received: from pobrecita.freebsd.ru (ache@localhost [127.0.0.1]) by nagual.pp.ru (8.12.6/8.12.6) with ESMTP id g9IGFBnP012779; Fri, 18 Oct 2002 20:15:11 +0400 (MSD) (envelope-from ache@pobrecita.freebsd.ru) Received: (from ache@localhost) by pobrecita.freebsd.ru (8.12.6/8.12.6/Submit) id g9IGFB0U012778; Fri, 18 Oct 2002 20:15:11 +0400 (MSD) (envelope-from ache) Date: Fri, 18 Oct 2002 20:15:11 +0400 From: "Andrey A. Chernov" To: "M. Warner Losh" Cc: ru@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: color, again, in grotty Message-ID: <20021018161511.GA12504@nagual.pp.ru> References: <20021017.101833.110719994.imp@bsdimp.com> <20021018095026.GA3386@sunbay.com> <20021018.094801.123456703.imp@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20021018.094801.123456703.imp@bsdimp.com> User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Oct 18, 2002 at 09:48:01 -0600, M. Warner Losh wrote: > > I know that there sre terminals for which the SGR sequences screw > things up. I had to deal with a raft of Televideo terminals back in Moreover, output device can be not terminal (f.e. printer). > : 2. Stay compatible with the new standard (11 years old ANSI std). > : This means we fix syscons(4) so that it renders "underscore" > : distinguishable from "bold", and change the default pager to We can't "fix" syscons without making bold the same like some other attribute or color. Current underline mapping is added for compatibility only, it not means to be used at all, i.e. if you check cons25 termcap entry, there is no underline. So think like underline not exist at all, so can't be fixed by definition. -- Andrey A. Chernov http://ache.pp.ru/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Oct 18 13:11:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F2E5837B41E; Fri, 18 Oct 2002 13:11:48 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id AAEE243EAC; Fri, 18 Oct 2002 13:11:46 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0209.cvx21-bradley.dialup.earthlink.net ([209.179.192.209] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 182dST-0006n0-00; Fri, 18 Oct 2002 13:10:58 -0700 Message-ID: <3DB06A8B.E40B3004@mindspring.com> Date: Fri, 18 Oct 2002 13:09:47 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "M. Warner Losh" Cc: ru@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: color, again, in grotty References: <20021017.101833.110719994.imp@bsdimp.com> <20021018095026.GA3386@sunbay.com> <20021018.094801.123456703.imp@bsdimp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "M. Warner Losh" wrote: > In message: <20021018095026.GA3386@sunbay.com> > Ruslan Ermilov writes: > I know that there sre terminals for which the SGR sequences screw > things up. I had to deal with a raft of Televideo terminals back in > the day that we were constantly finding stupid programmers who hard > coded things to vt100 and we had to keep fixing the stupid programs > because those programmers couldn't be bothered to use curses. I find > this to be an analogous case. Select Graphic Rendition can in fact lock the keyboard of Televideo terminals; also Wyse, ADM-31, Lear-Siegler, Hazeltine, etc.. > Back in the day, one could have argued that since all terminals > understood vt100 sequences, we should have stored our man pages in > vt100 escape sequences? Well, vt100s are almost impossible to find > and live on only as emulation and most terminals no longer are only > vt100, but some are close (cons25, for example, dies on some vt100 > sequences). Back in the day, vt100's were claimed to be ANSI > compliant terminals, which is also a standard. Yet, we didn't > standardize on them because the nice thing about standards is that > there are so many to choose from. ANSI 3.64 is a standard which defines which escape sequences are permissable, and what form they will take. It does not guarantee implementation. It's possible to be an ANSI 3.64 compliant terminal, and not implement many of the sequences. The ISO color escape sequences, and many of the ANSI 3.64 sequences were in fact first implemented on the Commodore Amiga console. The SCO added extra escape sequences for color, on top of the ISO, which were technically in violation of ANSI 3.64, because they permitted access to the 16 colors of the IBM CGA cards, instead of limiting them to 8, which was the limit of the ISO standard. > So this whole argument that it is standard is a little bogus. We can all agree with this: not all terminals are ANSI 3.64 compliant; nor should they be. > : We have also learned one pitfall with this new approach. Our default > : terminal driver, syscons(4), renders the "underscore" attribute, SGR > : E[4m (see screen(4)), the same as bold, which is the (code documented) > : bug, so using this new SGR scheme directly as it comes with new Groff > : worsens the output on cons25* type terminals. On the other hand, > : classical nroff(1) filters like ul(1) (and not by accident less(1)) > : render underlining with the "reverse" attribute on terminals that do > : not support underlining, like cons25. The controller hardware here is emulating an IBM CGA adapter. The correct approach is to set the bit that converts the high bit of the 4 background color bits, such that you are left with 8 background color bits, and another attribute bit. Real VT100's, without AVO (Advanced Video Option) also do not support multiple attributes simultaneously, and only support 12 screen lines at 132 columns. On a side note, the console is 25 lines -- and almost all ANSI 3.64 glass tty's -- terminals -- were only 24 lines. Assumptions are bad. > : So we must either fix syscons(4) or to somehow tell grotty(1) to > : emit old sequences to print bold and italic characters. Fixing Grotty to use the termcap ("TermCap" -- Terminal Capabilities) database is the correct approach, in almost all cases. > That proves my point more. Just because these escape sequences are a > standard, they are not the only way to go. 'Fixing' syscons will > actually break legacy programs, which is bad and not really an > option. Especially because it is to be compatible with a dubious > standard from the Linux world. Bah humbug. If you want a linux > comaptible console, write one, don't coopt FreeBSD. The SCO console is actually one of the best consoles. The only deficiency that I've seem is in it's treatment of the "AM" (automargin) attribute, where it wraps a line after character 80, rather than before what would be character 81. That basically means that in order to draw in the lower-right-hand corner of the screen, you have to write the character to 25,79, and insert the character that *was* at 25,79 at 25,79. This moves the written character right, and lets you draw the corner without wrapping the screen. Most curses packages get this wrong. 8-). > : There are three ways we can go here: > : > : 1. Disable SGR support for grotty(1). > : > : It can be soft-disabled by setting GROFF_NO_SGR in environment, > : or hard-disabled system-wide (like it is now in -CURRENT as of > : this writing) in /usr/share/tmac/troffrc. > > I like this option. SGR belongs not in groff, but in ul, et al. I agree. This is the best option. > : 2. Stay compatible with the new standard (11 years old ANSI std). > : This means we fix syscons(4) so that it renders "underscore" > : distinguishable from "bold", and change the default pager to > : "more -R". There will still be an option of soft-disabling > : it for those who want it disabled. > > I don't like this option. Fixing syscons is silly and will break > compatibility with the past. The standard is just 'a' standard, not > THE standard like you get with posix or ansi-c. There are many other > legitimate ways of doing things and moving to device dependence > because someone takes a shining to a standard is not a wise move. And to do this, you will lose access to 8 of your background colors. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Oct 18 13:19:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E6EE637B401; Fri, 18 Oct 2002 13:19:25 -0700 (PDT) Received: from nagual.pp.ru (pobrecita.freebsd.ru [194.87.13.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id EB47543EB2; Fri, 18 Oct 2002 13:19:24 -0700 (PDT) (envelope-from ache@pobrecita.freebsd.ru) Received: from pobrecita.freebsd.ru (ache@localhost [127.0.0.1]) by nagual.pp.ru (8.12.6/8.12.6) with ESMTP id g9IKJKnP015214; Sat, 19 Oct 2002 00:19:20 +0400 (MSD) (envelope-from ache@pobrecita.freebsd.ru) Received: (from ache@localhost) by pobrecita.freebsd.ru (8.12.6/8.12.6/Submit) id g9IKJJ0e015213; Sat, 19 Oct 2002 00:19:19 +0400 (MSD) (envelope-from ache) Date: Sat, 19 Oct 2002 00:19:19 +0400 From: "Andrey A. Chernov" To: Terry Lambert Cc: "M. Warner Losh" , ru@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: color, again, in grotty Message-ID: <20021018201919.GA15100@nagual.pp.ru> References: <20021017.101833.110719994.imp@bsdimp.com> <20021018095026.GA3386@sunbay.com> <20021018.094801.123456703.imp@bsdimp.com> <3DB06A8B.E40B3004@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3DB06A8B.E40B3004@mindspring.com> User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Oct 18, 2002 at 13:09:47 -0700, Terry Lambert wrote: > > > : So we must either fix syscons(4) or to somehow tell grotty(1) to > > : emit old sequences to print bold and italic characters. > > Fixing Grotty to use the termcap ("TermCap" -- Terminal Capabilities) > database is the correct approach, in almost all cases. I agree. Grotty should get termcap color capabilities using TERM env. variable, if output isatty(). That is termcap purpose. -- Andrey A. Chernov http://ache.pp.ru/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Oct 19 6:50:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7393837B401; Sat, 19 Oct 2002 06:50:45 -0700 (PDT) Received: from nagual.pp.ru (pobrecita.freebsd.ru [194.87.13.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E08C43E42; Sat, 19 Oct 2002 06:50:44 -0700 (PDT) (envelope-from ache@pobrecita.freebsd.ru) Received: from pobrecita.freebsd.ru (ache@localhost [127.0.0.1]) by nagual.pp.ru (8.12.6/8.12.6) with ESMTP id g9JDodnP038032; Sat, 19 Oct 2002 17:50:39 +0400 (MSD) (envelope-from ache@pobrecita.freebsd.ru) Received: (from ache@localhost) by pobrecita.freebsd.ru (8.12.6/8.12.6/Submit) id g9JDocZq038031; Sat, 19 Oct 2002 17:50:38 +0400 (MSD) (envelope-from ache) Date: Sat, 19 Oct 2002 17:50:38 +0400 From: "Andrey A. Chernov" To: Terry Lambert Cc: "M. Warner Losh" , ru@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: color, again, in grotty Message-ID: <20021019135038.GA37956@nagual.pp.ru> References: <20021017.101833.110719994.imp@bsdimp.com> <20021018095026.GA3386@sunbay.com> <20021018.094801.123456703.imp@bsdimp.com> <3DB06A8B.E40B3004@mindspring.com> <20021018201919.GA15100@nagual.pp.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20021018201919.GA15100@nagual.pp.ru> User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Full variant: Look at color 'ls' for example of proper colors implementation via termcap database. The same thing can be done via {n}curses if someone not want to deal with termcap/terminfo stuff. In addition of looking at TERM variable grotty should have an option to specify TERM variable at command line. Short variant: Grotty should have non-default option indicating that output device is capable to display ANSI colors. Making assumption that every thing support ANSI color is Linuxism, someone should explain this to groff developers. -- Andrey A. Chernov http://ache.pp.ru/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message