From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 05:06:17 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 920CA16A4BF for ; Mon, 6 Oct 2003 05:06:17 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 780D243FE5 for ; Mon, 6 Oct 2003 05:06:16 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h96C692B021230 for ; Mon, 6 Oct 2003 14:06:10 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: arch@freebsd.org From: Poul-Henning Kamp Date: Mon, 06 Oct 2003 14:06:09 +0200 Message-ID: <21229.1065441969@critter.freebsd.dk> Subject: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 12:06:17 -0000 Some disk controllers require that the memory buffer for a DMA transfer be aligned on a particular bounday, typically four bytes, but in some cases larger than that. If a userland program does a read or write on a disk device (/dev/ad0 for instance), we do some VM swizzle in physio and the request is in essence a zero-copy thing directly into the process' pages, and as an sideeffect of this, the pointer passed to the device driver has the same offset into a page as the users request (the lower 12 bits on i386). That means that userland programs can send down I/O requests which are not properly aligned. This program, in other words, would be a problem for some of our disk controllers: #include int main(int argc, char **argv) { char *p; p = malloc(8192); /* page aligned */ read (0, p + 1, 2048); /* not */ } Most code which does disk I/O from userland uses malloc'ed buffers for the sectors and the alignment comes for free, but we currently have no requirement that it be so. How much code (apart from newfs(8)!) would we break by enforcing a minimum alignment of for instance 16 bytes on disk-I/O requests from userland ? It would be a trivial matter to do the "bounce-buffering" somewhere along the road, but I am not convinced it is the right thing to do because all of sudden some requests would take a performance hit to do a copy and some would not... Qualified, informed input requested. No Bikesheds please. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 05:35:39 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 34E5F16A4B3 for ; Mon, 6 Oct 2003 05:35:39 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 66B1743FE1 for ; Mon, 6 Oct 2003 05:35:38 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.9p1/8.12.3) with ESMTP id h96CZbsd009556; Mon, 6 Oct 2003 05:35:37 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.9p1/8.12.3/Submit) id h96CZbFQ009555; Mon, 6 Oct 2003 05:35:37 -0700 (PDT) (envelope-from rizzo) Date: Mon, 6 Oct 2003 05:35:37 -0700 From: Luigi Rizzo To: Poul-Henning Kamp Message-ID: <20031006053537.A9428@xorpc.icir.org> References: <21229.1065441969@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <21229.1065441969@critter.freebsd.dk>; from phk@phk.freebsd.dk on Mon, Oct 06, 2003 at 02:06:09PM +0200 cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 12:35:39 -0000 On Mon, Oct 06, 2003 at 02:06:09PM +0200, Poul-Henning Kamp wrote: ... > If a userland program does a read or write on a disk device (/dev/ad0 > for instance), we do some VM swizzle in physio and the request is ^^^^^ I don't completely follow -- is this something you plan to do, or it is what we do now ? In the latter case then the code you show below is already causing trouble, and enforcing (or should we say "checking") that the alignment is respected would just make the problem evident (which is clearly desirable). In any case i guess it is unavoidable to implement some form of bounce buffers -- sometimes you just can't modify the binary that issues the read/write request. And if you can provide some optimization to bypass the bounce buffer after checking that the alignment is the correct one, then you are improving performance in some cases and not harming in the remaining ones, so i would say go for it... cheers luigi > in essence a zero-copy thing directly into the process' pages, and > as an sideeffect of this, the pointer passed to the device driver > has the same offset into a page as the users request (the lower 12 > bits on i386). > > That means that userland programs can send down I/O requests which > are not properly aligned. > > This program, in other words, would be a problem for some of our > disk controllers: > > #include > > int > main(int argc, char **argv) > { > char *p; > > p = malloc(8192); /* page aligned */ > read (0, p + 1, 2048); /* not */ > } > > Most code which does disk I/O from userland uses malloc'ed buffers for > the sectors and the alignment comes for free, but we currently have > no requirement that it be so. > > How much code (apart from newfs(8)!) would we break by enforcing a > minimum alignment of for instance 16 bytes on disk-I/O requests > from userland ? > > It would be a trivial matter to do the "bounce-buffering" somewhere > along the road, but I am not convinced it is the right thing to do > because all of sudden some requests would take a performance hit to > do a copy and some would not... > > Qualified, informed input requested. > > No Bikesheds please. > > Poul-Henning > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 14:46:54 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 769DC16A4B3 for ; Mon, 6 Oct 2003 14:46:54 -0700 (PDT) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 92A4B43FD7 for ; Mon, 6 Oct 2003 14:46:53 -0700 (PDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.9/8.12.9) with ESMTP id h96Lkpgk093487 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK CN=khavrinen.lcs.mit.edu issuer=SSL+20Client+20CA); Mon, 6 Oct 2003 17:46:52 -0400 (EDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.9/8.12.9/Submit) id h96Lkpx0093486; Mon, 6 Oct 2003 17:46:51 -0400 (EDT) (envelope-from wollman) Date: Mon, 6 Oct 2003 17:46:51 -0400 (EDT) From: Garrett Wollman Message-Id: <200310062146.h96Lkpx0093486@khavrinen.lcs.mit.edu> To: phk@phk.freebsd.dk X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: <21229.1065441969@critter.freebsd.dk> Organization: MIT Laboratory for Computer Science X-Spam-Score: -3.3 () IN_REP_TO X-Scanned-By: MIMEDefang 2.37 cc: arch@FreeBSD.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 21:46:54 -0000 PHK writes: >Most code which does disk I/O from userland uses malloc'ed buffers for >the sectors and the alignment comes for free, but we currently have >no requirement that it be so. I believe that the Standard allows practically arbitrary restrictions on what may be done with devices (since most devices, aside from ttys, are outside the scope of the Standard anyway). It says that read() may fail if: [ENXIO] A request was made of a nonexistent device, or the request was outside the capabilities of the device. (I'm assuming you're talking about reads from devices, as opposed to reads from files.) I think that gives us plenary authority to require appropriate alignment of data buffers used to access disk devices directly. -GAWollman From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 14:55:22 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F179E16A4BF for ; Mon, 6 Oct 2003 14:55:22 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF28743FE9 for ; Mon, 6 Oct 2003 14:55:21 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h96LtH2B026370; Mon, 6 Oct 2003 23:55:18 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Garrett Wollman From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 06 Oct 2003 17:46:51 EDT." <200310062146.h96Lkpx0093486@khavrinen.lcs.mit.edu> Date: Mon, 06 Oct 2003 23:55:17 +0200 Message-ID: <26369.1065477317@critter.freebsd.dk> cc: arch@FreeBSD.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 21:55:23 -0000 In message <200310062146.h96Lkpx0093486@khavrinen.lcs.mit.edu>, Garrett Wollman writes: >I think that gives us plenary authority to require appropriate >alignment of data buffers used to access disk devices directly. Well, apart from us wedging or botching the request rather than return a consistent error we're standards compliant then. It has been mentioned on #thatchannel that busdma should take care of this by copying the request as necessary. Faced with the prospect of a request several megabytes in size, I don't like the prospect of malloc/bcopy too much. All in all, I would advocate that we decide that disks in FreeBSD are allowed to return ENXIO if they get insufficiently aligned requests. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 15:20:11 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9038616A4BF for ; Mon, 6 Oct 2003 15:20:11 -0700 (PDT) Received: from shockwave.systems.pipex.net (shockwave.systems.pipex.net [62.241.160.9]) by mx1.FreeBSD.org (Postfix) with ESMTP id A72C843F75 for ; Mon, 6 Oct 2003 15:20:10 -0700 (PDT) (envelope-from mark@thuvia.org) Received: from dotar.thuvia.org (81-86-228-29.dsl.pipex.com [81.86.228.29]) by shockwave.systems.pipex.net (Postfix) with ESMTP id 4106A1C0095B; Mon, 6 Oct 2003 23:20:08 +0100 (BST) Received: from dotar.thuvia.org (localhost [127.0.0.1]) by dotar.thuvia.org (8.12.9/8.12.9) with ESMTP id h96MK7PA061402; Mon, 6 Oct 2003 23:20:07 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.9/8.12.9/Submit) id h96MK7PI061345; Mon, 6 Oct 2003 23:20:07 +0100 (BST) (envelope-from mark) Message-Id: <200310062220.h96MK7PI061345@dotar.thuvia.org> From: Mark Valentine Date: Mon, 6 Oct 2003 23:20:06 +0000 In-Reply-To: X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: wollman@khavrinen.lcs.mit.edu (Garrett Wollman), phk@phk.freebsd.dk cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 22:20:11 -0000 > From: wollman@khavrinen.lcs.mit.edu (Garrett Wollman) > Date: Mon 6 Oct, 2003 > Subject: Re: Alignment of disk-I/O from userland. > I believe that the Standard allows practically arbitrary restrictions > on what may be done with devices (since most devices, aside from ttys, > are outside the scope of the Standard anyway). It says that read() > may fail if: > > [ENXIO] A request was made of a nonexistent device, or the request > was outside the capabilities of the device. Hmm, but it doesn't say: [EDOOFUS] A request was made of a nonexistent device, or the request was outside the capabilities of the device driver writer. It would be reasonable to enforce such restrictions on a raw device if we still had block devices around, but it doesn't seem reasonable now. Cheers, Mark. -- "Tigers will do ANYTHING for a tuna fish sandwich." "We're kind of stupid that way." *munch* *munch* -- From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 15:22:11 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3665E16A4B3 for ; Mon, 6 Oct 2003 15:22:11 -0700 (PDT) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 47A1543FE3 for ; Mon, 6 Oct 2003 15:22:08 -0700 (PDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.9/8.12.9) with ESMTP id h96MM6gk093686 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK CN=khavrinen.lcs.mit.edu issuer=SSL+20Client+20CA); Mon, 6 Oct 2003 18:22:07 -0400 (EDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.9/8.12.9/Submit) id h96MM6MO093683; Mon, 6 Oct 2003 18:22:06 -0400 (EDT) (envelope-from wollman) Date: Mon, 6 Oct 2003 18:22:06 -0400 (EDT) From: Garrett Wollman Message-Id: <200310062222.h96MM6MO093683@khavrinen.lcs.mit.edu> To: Mark Valentine In-Reply-To: <200310062220.h96MK7PI061345@dotar.thuvia.org> References: <200310062220.h96MK7PI061345@dotar.thuvia.org> X-Spam-Score: -19.8 () IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,REPLY_WITH_QUOTES X-Scanned-By: MIMEDefang 2.37 cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 22:22:11 -0000 < said: > It would be reasonable to enforce such restrictions on a raw device if > we still had block devices around, but it doesn't seem reasonable now. I think you've got that backwards. When we had block devices, they would provide extra buffering to avoid I/O-size breakage. Character devices, which are all we have now, never made any promises. -GAWollman From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 15:27:43 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E3D4316A4B3 for ; Mon, 6 Oct 2003 15:27:43 -0700 (PDT) Received: from colossus.systems.pipex.net (colossus.systems.pipex.net [62.241.160.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 73AB743FEC for ; Mon, 6 Oct 2003 15:27:42 -0700 (PDT) (envelope-from mark@thuvia.org) Received: from dotar.thuvia.org (81-86-228-29.dsl.pipex.com [81.86.228.29]) by colossus.systems.pipex.net (Postfix) with ESMTP id 566CA1600032F; Mon, 6 Oct 2003 23:27:40 +0100 (BST) Received: from dotar.thuvia.org (localhost [127.0.0.1]) by dotar.thuvia.org (8.12.9/8.12.9) with ESMTP id h96MRdPA075171; Mon, 6 Oct 2003 23:27:39 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.9/8.12.9/Submit) id h96MRdUf075168; Mon, 6 Oct 2003 23:27:39 +0100 (BST) (envelope-from mark) Message-Id: <200310062227.h96MRdUf075168@dotar.thuvia.org> From: Mark Valentine Date: Mon, 6 Oct 2003 23:27:39 +0000 In-Reply-To: <200310062222.h96MM6MO093683@khavrinen.lcs.mit.edu> X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Garrett Wollman cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 22:27:44 -0000 > From: Garrett Wollman > Date: Mon 6 Oct, 2003 > Subject: Re: Alignment of disk-I/O from userland. > I think you've got that backwards. When we had block devices, they > would provide extra buffering to avoid I/O-size breakage. Character > devices, which are all we have now, never made any promises. My point was that we did have block devices which could make such promises, and that without them the raw device must offer an equivalent promise (especially when you consider binary compatibility - a program opening /dev/da0c was written assuming a block device interface). Cheers, Mark. -- "Tigers will do ANYTHING for a tuna fish sandwich." "We're kind of stupid that way." *munch* *munch* -- From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 15:36:38 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3873116A4B3 for ; Mon, 6 Oct 2003 15:36:38 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4378343FD7 for ; Mon, 6 Oct 2003 15:36:32 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h96MaU2B026940; Tue, 7 Oct 2003 00:36:30 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Mark Valentine From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 06 Oct 2003 23:20:06 -0000." <200310062220.h96MK7PI061345@dotar.thuvia.org> Date: Tue, 07 Oct 2003 00:36:30 +0200 Message-ID: <26939.1065479790@critter.freebsd.dk> cc: arch@freebsd.org cc: Garrett Wollman Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 22:36:38 -0000 In message <200310062220.h96MK7PI061345@dotar.thuvia.org>, Mark Valentine write s: >It would be reasonable to enforce such restrictions on a raw device if >we still had block devices around, but it doesn't seem reasonable now. It would be reasonable to make such a statement if you could demonstrate an actual application which must depend on this to work. The fact is that we currently do not offer any guarantee for disk-I/O even correctly reporting failure, unless your memory buffer is aligned according to driver specific requirements. And yet things still work. If I thought there would be any significant breakage (of non-shitty code), I would not be in doubt as to what the right thing to do would be :-) If shitty code breaks, I don't care. We're trying to raise the standard in and with FreeBSD, we're not trying to lower the bar to make any visual basic programmer pass. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 15:41:36 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3646016A4F3 for ; Mon, 6 Oct 2003 15:41:36 -0700 (PDT) Received: from colossus.systems.pipex.net (colossus.systems.pipex.net [62.241.160.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 289F3440B5 for ; Mon, 6 Oct 2003 15:40:52 -0700 (PDT) (envelope-from mark@thuvia.org) Received: from dotar.thuvia.org (81-86-228-29.dsl.pipex.com [81.86.228.29]) by colossus.systems.pipex.net (Postfix) with ESMTP id A7ED1160003A0; Mon, 6 Oct 2003 23:40:25 +0100 (BST) Received: from dotar.thuvia.org (localhost [127.0.0.1]) by dotar.thuvia.org (8.12.9/8.12.9) with ESMTP id h96MeOPA095803; Mon, 6 Oct 2003 23:40:24 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.9/8.12.9/Submit) id h96MeOWY095792; Mon, 6 Oct 2003 23:40:24 +0100 (BST) (envelope-from mark) Message-Id: <200310062240.h96MeOWY095792@dotar.thuvia.org> From: Mark Valentine Date: Mon, 6 Oct 2003 23:40:24 +0000 In-Reply-To: <26939.1065479790@critter.freebsd.dk> X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: "Poul-Henning Kamp" cc: arch@freebsd.org cc: Garrett Wollman Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 22:41:36 -0000 > From: "Poul-Henning Kamp" > Date: Tue 7 Oct, 2003 > Subject: Re: Alignment of disk-I/O from userland. > If shitty code breaks, I don't care. Me neither. However, "code written to yesterday's rules" != "shitty code today". Cheers, Mark. -- "Tigers will do ANYTHING for a tuna fish sandwich." "We're kind of stupid that way." *munch* *munch* -- From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 15:44:05 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 74B2016A4B3 for ; Mon, 6 Oct 2003 15:44:05 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [208.142.252.6]) by mx1.FreeBSD.org (Postfix) with ESMTP id E822F43F3F for ; Mon, 6 Oct 2003 15:44:03 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h96Mhls06679; Mon, 6 Oct 2003 18:43:47 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Mon, 6 Oct 2003 18:43:47 -0400 (EDT) From: Jeff Roberson To: Poul-Henning Kamp In-Reply-To: <26369.1065477317@critter.freebsd.dk> Message-ID: <20031006183611.B99666-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: Garrett Wollman Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 22:44:05 -0000 X-List-Received-Date: Mon, 06 Oct 2003 22:44:05 -0000 On Mon, 6 Oct 2003, Poul-Henning Kamp wrote: > In message <200310062146.h96Lkpx0093486@khavrinen.lcs.mit.edu>, Garrett Wollman > writes: > > >I think that gives us plenary authority to require appropriate > >alignment of data buffers used to access disk devices directly. > > Well, apart from us wedging or botching the request rather than > return a consistent error we're standards compliant then. > > It has been mentioned on #thatchannel that busdma should take care > of this by copying the request as necessary. Faced with the prospect > of a request several megabytes in size, I don't like the prospect > of malloc/bcopy too much. > > All in all, I would advocate that we decide that disks in FreeBSD > are allowed to return ENXIO if they get insufficiently aligned > requests. This could break things like cat, grep, strings, etc. except that they probably malloc sufficiently large chunks to get their own page, or at least heavily aligned data due to the properties of power of two allocators. I would be awfully sad if we broke the notion of character devices being regular files though. How are we going to express the alignment requirements to the upper layers? Does busdma provide this now? If not, do we know which devices have these requirements? I think that we should provide the bouncing purely for POLA. I doubt any software that ships with FreeBSD will need it so there should be no performance issues in practice. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 15:44:18 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7728D16A4B3 for ; Mon, 6 Oct 2003 15:44:18 -0700 (PDT) Received: from smtp.mho.com (smtp.mho.net [64.58.4.6]) by mx1.FreeBSD.org (Postfix) with SMTP id 4F17344001 for ; Mon, 6 Oct 2003 15:44:17 -0700 (PDT) (envelope-from scottl@freebsd.org) Received: (qmail 82585 invoked by uid 1002); 6 Oct 2003 22:44:16 -0000 Received: from unknown (HELO ?10.4.1.5?) (64.58.1.252) by smtp.mho.net with SMTP; 6 Oct 2003 22:44:16 -0000 Date: Mon, 6 Oct 2003 16:44:32 -0600 (MDT) From: Scott Long X-X-Sender: scottl@pooker.samsco.home To: Poul-Henning Kamp In-Reply-To: <26369.1065477317@critter.freebsd.dk> Message-ID: <20031006163218.L55190@pooker.samsco.home> References: <26369.1065477317@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: Garrett Wollman Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 22:44:18 -0000 On Mon, 6 Oct 2003, Poul-Henning Kamp wrote: > In message <200310062146.h96Lkpx0093486@khavrinen.lcs.mit.edu>, Garrett Wollman > writes: > > >I think that gives us plenary authority to require appropriate > >alignment of data buffers used to access disk devices directly. > > Well, apart from us wedging or botching the request rather than > return a consistent error we're standards compliant then. > > It has been mentioned on #thatchannel that busdma should take care > of this by copying the request as necessary. Faced with the prospect > of a request several megabytes in size, I don't like the prospect > of malloc/bcopy too much. Working around hardware requirements from the block layer gets messy. You obviously don't want to manually align buffers that are distined for hardware that doesn't care about the alignment. How do you communicate this property upwards from the driver? Callbacks? Extra flags in disk_create()? It's all rather messy that way. We already have the busdma interface whose sole purpose is to take system buffers and prepare them for transfer to/from hardware. It already understands the concept of alignment, though it only applies it to buffers that it allocates, not buffers that are handed to it. Fixing that would be easy, and would allow for optimizations based on the bus (GART and IOMMU remapping). It also keeps the property down at the device driver where it belongs. As for returning an error code for a buffer that we (arbitrarily) believe to be too big to align, I'm not sure that it's a valid thing to worry about. People expect their hardware to Just Work, regardless of how cheap it is. If someone buys a cheap card with a cheap DMA engine, their cost comes in from higher system time per I/O. If they don't like that, they can complain to the manufacturer and/or buy a better card. I'm not aware of Linux nor OSX rejecting I/O based on arbitrary size limits. Scott From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 16:11:16 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2518316A4DC; Mon, 6 Oct 2003 16:11:16 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 699B443FEC; Mon, 6 Oct 2003 16:11:14 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h96NBB2B027375; Tue, 7 Oct 2003 01:11:11 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Scott Long From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 06 Oct 2003 16:44:32 MDT." <20031006163218.L55190@pooker.samsco.home> Date: Tue, 07 Oct 2003 01:11:11 +0200 Message-ID: <27374.1065481871@critter.freebsd.dk> cc: arch@freebsd.org cc: Garrett Wollman Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 23:11:16 -0000 In message <20031006163218.L55190@pooker.samsco.home>, Scott Long writes: >We already >have the busdma interface whose sole purpose is to take system >buffers and prepare them for transfer to/from hardware [...] I certainly do agree that _if_ we do want to do copy/align busdma would be a good place for it. >As for returning an error code for a buffer that we (arbitrarily) believe >to be too big to align, [...] I have never advocated returning an error based on "alignment and size", only based on alignment alone. But I also just realized a complication I had not thought of earlier, and which may modify our thinking further: This is an issue for all physread()/physwrite() drivers, not just disks. In other words, if I want to write to 1MB blocks to a SCSI tape, and I don't align my in memory buffer sufficiently for the hardware, busdma would have to allocate 1MB of memory (it may _possibly_ be able to do so as disjunct pages rather than consequtively) and copy the entire request over. For disks we can chop the request at sector boundaries or multiple thereoff and deal with it that way, but we don't have that option for scsi_sa or even scsi_pt devices. Currently we impose a 128k upper limit on I/O requests, but we have already more or less agreed that needs to grow into the 4-16MB range soon. The more I think about it, there more arguments I find for retaining the status quo of requiring userland to do proper alignment (but with better error-checking). Particularly since the only unaligned case I know of yet, newfs(8), is by trivial accident rather than need or intent. The question of how to communicate the alignment required to userland has been raised. I propose this answer: Suffient alignment can be obtained by any one of these methods: 1. Allocate your buffer with malloc(3). 2. Align it to the request size. 3. Align it to a page. (The first is somewhat dependent on the behaviour of phkmalloc, and can be removed, but it offers a nice clean shortcut for most programmers.) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 17:31:21 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CB53716A4B3 for ; Mon, 6 Oct 2003 17:31:21 -0700 (PDT) Received: from ozlabs.org (ozlabs.org [203.10.76.45]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5C92C43FF5 for ; Mon, 6 Oct 2003 17:31:20 -0700 (PDT) (envelope-from grog@lemis.com) Received: from blackwater.lemis.com (blackwater.lemis.com [192.109.197.80]) by ozlabs.org (Postfix) with ESMTP id BC6412BC0D for ; Tue, 7 Oct 2003 10:31:17 +1000 (EST) Received: by blackwater.lemis.com (Postfix, from userid 1004) id D057451836; Tue, 7 Oct 2003 10:01:14 +0930 (CST) Date: Tue, 7 Oct 2003 10:01:14 +0930 From: Greg 'groggy' Lehey To: Poul-Henning Kamp Message-ID: <20031007003114.GF47054@wantadilla.lemis.com> References: <200310062220.h96MK7PI061345@dotar.thuvia.org> <26939.1065479790@critter.freebsd.dk> <21229.1065441969@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rWhLK7VZz0iBluhq" Content-Disposition: inline In-Reply-To: <26939.1065479790@critter.freebsd.dk> <21229.1065441969@critter.freebsd.dk> User-Agent: Mutt/1.4i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 cc: arch@freebsd.org cc: Garrett Wollman cc: Mark Valentine Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 00:31:22 -0000 --rWhLK7VZz0iBluhq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Monday, 6 October 2003 at 14:06:09 +0200, Poul-Henning Kamp wrote: > > Some disk controllers require that the memory buffer for a DMA > transfer be aligned on a particular bounday, typically four bytes, > but in some cases larger than that. > > This program, in other words, would be a problem for some of our > disk controllers: > > #include > > int > main(int argc, char **argv) > { > char *p; > > p = malloc(8192); /* page aligned */ > read (0, p + 1, 2048); /* not */ > } This kind of code is explicitly allowed in K&R 2 (page 170 et seq). > Most code which does disk I/O from userland uses malloc'ed buffers for > the sectors and the alignment comes for free, but we currently have > no requirement that it be so. > > How much code (apart from newfs(8)!) would we break by enforcing a > minimum alignment of for instance 16 bytes on disk-I/O requests > from userland ? Quite possibly a lot of third party code over which we have no control. I'm thinking of ibcs2 and Linux stuff, for example. > It would be a trivial matter to do the "bounce-buffering" somewhere > along the road, but I am not convinced it is the right thing to do > because all of sudden some requests would take a performance hit to > do a copy and some would not... I think the bounce buffering is necessary, but should be an emergency measure. It could generate a console message, for example. On Tuesday, 7 October 2003 at 0:36:30 +0200, Poul-Henning Kamp wrote: > > If shitty code breaks, I don't care. We're trying to raise the > standard in and with FreeBSD, we're not trying to lower the bar > to make any visual basic programmer pass. That depends on whether the programmer is a FreeBSD developer or a user. I agree that our own code should be aligned correctly. I don't think it's reasonable to expect all users to adhere to a higher coding standard than they would for, say, Linux. It's not difficult to cater for them, and if you like we can still tell them off when they do it, but I think the code should work nevertheless. People who get bitten by this "bug" won't see the clean design intentions behind it. They'll just know that FreeBSD is broken. Greg -- See complete headers for address and phone numbers. --rWhLK7VZz0iBluhq Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQE/gglSIubykFB6QiMRAjl+AJ9oYwEvWYvXVyR/TbjWZFe28bLilwCfeLBH KYVKdvQvOLv0gVX1O/D1p4s= =T/Eq -----END PGP SIGNATURE----- --rWhLK7VZz0iBluhq-- From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 17:49:09 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1135616A4B3; Mon, 6 Oct 2003 17:49:09 -0700 (PDT) Received: from gw.catspoiler.org (217-ip-163.nccn.net [209.79.217.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id E8D3643FBD; Mon, 6 Oct 2003 17:49:07 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.12.9/8.12.9) with ESMTP id h970mvN1052990; Mon, 6 Oct 2003 17:49:01 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <200310070049.h970mvN1052990@gw.catspoiler.org> Date: Mon, 6 Oct 2003 17:48:57 -0700 (PDT) From: Don Lewis To: phk@phk.freebsd.dk In-Reply-To: <27374.1065481871@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii cc: arch@FreeBSD.org cc: scottl@FreeBSD.org cc: wollman@khavrinen.lcs.mit.edu Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 00:49:09 -0000 On 7 Oct, Poul-Henning Kamp wrote: > But I also just realized a complication I had not thought of earlier, > and which may modify our thinking further: This is an issue for > all physread()/physwrite() drivers, not just disks. > > In other words, if I want to write to 1MB blocks to a SCSI tape, > and I don't align my in memory buffer sufficiently for the hardware, > busdma would have to allocate 1MB of memory (it may _possibly_ be > able to do so as disjunct pages rather than consequtively) and copy > the entire request over. > > For disks we can chop the request at sector boundaries or multiple > thereoff and deal with it that way, but we don't have that option > for scsi_sa or even scsi_pt devices. > > Currently we impose a 128k upper limit on I/O requests, but we have > already more or less agreed that needs to grow into the 4-16MB range > soon. There might be another reason to other than alignment to copy the data. What are the limits on the size of the scatter-gather lists that are supported by the DMA engines of commonly available controllers? It is unlikely that the user's buffer will be in contiguous memory even if it is properly aligned. It might be necessary to copy the the memory into some number of buffers, where each buffer is in contiguous physical memory, so that reasonable size I/O requests can be supported on commonly available controllers. A 4MB request might require 1K scatter-gather list entries on a machine with a 4K page size. Ick ... From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 17:55:19 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E019C16A4B3; Mon, 6 Oct 2003 17:55:19 -0700 (PDT) Received: from ebb.errno.com (ebb.errno.com [66.127.85.87]) by mx1.FreeBSD.org (Postfix) with ESMTP id 218EA43F85; Mon, 6 Oct 2003 17:55:19 -0700 (PDT) (envelope-from sam@errno.com) Received: from 66.127.85.91 ([66.127.85.91]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.9) with ESMTP id h970tH0x012579 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO); Mon, 6 Oct 2003 17:55:18 -0700 (PDT) (envelope-from sam@errno.com) From: Sam Leffler Organization: Errno Consulting To: "Poul-Henning Kamp" , Scott Long Date: Mon, 6 Oct 2003 17:53:28 -0700 User-Agent: KMail/1.5.2 References: <27374.1065481871@critter.freebsd.dk> In-Reply-To: <27374.1065481871@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200310061753.28562.sam@errno.com> cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 00:55:20 -0000 On Monday 06 October 2003 04:11 pm, Poul-Henning Kamp wrote: > In message <20031006163218.L55190@pooker.samsco.home>, Scott Long writes: ...stuff deleted... > >As for returning an error code for a buffer that we (arbitrarily) believe > >to be too big to align, [...] > > I have never advocated returning an error based on "alignment and size", > only based on alignment alone. Imposing this restriction is a major semantic change that I consider a very bad idea. You are basically imposing the semantics of O_DIRECT on all i/o operations going to a device. I think it is important to give best effort to support unaligned operations `by default. I can imagine restricting this to some upper size bound but existing applications, regardless of how well you consider them to be written, must continue to work. Sam From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 19:33:17 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AE0F816A4B3 for ; Mon, 6 Oct 2003 19:33:17 -0700 (PDT) Received: from smtp.mho.com (smtp.mho.net [64.58.4.6]) by mx1.FreeBSD.org (Postfix) with SMTP id 9878F43FDF for ; Mon, 6 Oct 2003 19:33:16 -0700 (PDT) (envelope-from scottl@freebsd.org) Received: (qmail 99772 invoked by uid 1002); 7 Oct 2003 02:33:15 -0000 Received: from unknown (HELO freebsd.org) (64.58.1.252) by smtp.mho.net with SMTP; 7 Oct 2003 02:33:15 -0000 Message-ID: <3F8225EB.5020902@freebsd.org> Date: Mon, 06 Oct 2003 20:33:15 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3) Gecko/20030425 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Don Lewis References: <200310070049.h970mvN1052990@gw.catspoiler.org> In-Reply-To: <200310070049.h970mvN1052990@gw.catspoiler.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: arch@FreeBSD.org cc: phk@phk.freebsd.dk cc: wollman@khavrinen.lcs.mit.edu Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 02:33:17 -0000 Don Lewis wrote: > On 7 Oct, Poul-Henning Kamp wrote: > > >>But I also just realized a complication I had not thought of earlier, >>and which may modify our thinking further: This is an issue for >>all physread()/physwrite() drivers, not just disks. >> >>In other words, if I want to write to 1MB blocks to a SCSI tape, >>and I don't align my in memory buffer sufficiently for the hardware, >>busdma would have to allocate 1MB of memory (it may _possibly_ be >>able to do so as disjunct pages rather than consequtively) and copy >>the entire request over. >> >>For disks we can chop the request at sector boundaries or multiple >>thereoff and deal with it that way, but we don't have that option >>for scsi_sa or even scsi_pt devices. >> >>Currently we impose a 128k upper limit on I/O requests, but we have >>already more or less agreed that needs to grow into the 4-16MB range >>soon. > > > There might be another reason to other than alignment to copy the data. > What are the limits on the size of the scatter-gather lists that are > supported by the DMA engines of commonly available controllers? It is > unlikely that the user's buffer will be in contiguous memory even if it > is properly aligned. It might be necessary to copy the the memory into > some number of buffers, where each buffer is in contiguous physical > memory, so that reasonable size I/O requests can be supported on > commonly available controllers. A 4MB request might require 1K > scatter-gather list entries on a machine with a 4K page size. Ick ... > This again is a job for busdma. Again, it isn't implemented, but the API details are already defined and waiting for the back-end code. With more and more drivers switching to busdma, it really makes much more sense to finish the implementation there rather than invent several new implementations elsewhere. Scott From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 23:29:26 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 32CCC16A4B3; Mon, 6 Oct 2003 23:29:26 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id F297343FCB; Mon, 6 Oct 2003 23:29:24 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h976TH2B032325; Tue, 7 Oct 2003 08:29:18 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Sam Leffler From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 06 Oct 2003 17:53:28 PDT." <200310061753.28562.sam@errno.com> Date: Tue, 07 Oct 2003 08:29:17 +0200 Message-ID: <32324.1065508157@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 06:29:26 -0000 In message <200310061753.28562.sam@errno.com>, Sam Leffler writes: >On Monday 06 October 2003 04:11 pm, Poul-Henning Kamp wrote: >> In message <20031006163218.L55190@pooker.samsco.home>, Scott Long writes: > ...stuff deleted... >> >As for returning an error code for a buffer that we (arbitrarily) believe >> >to be too big to align, [...] >> >> I have never advocated returning an error based on "alignment and size", >> only based on alignment alone. > >Imposing this restriction is a major semantic change that I consider a very >bad idea. You are basically imposing the semantics of O_DIRECT on all i/o >operations going to a device. I think it is important to give best effort to >support unaligned operations `by default. I can imagine restricting this to >some upper size bound but existing applications, regardless of how well you >consider them to be written, must continue to work. Now now, you are missing two of the finer points: 1: Not "on all i/o operations going to a device", but rather "on i/o operations which take the physread/write fast-path to avoid a copyin/out overhead." (disks and tapes mostly). Ttys, /dev/null and all the "typical" devices are unaffected. 2: Right now we _do_ impose this restriction, but our error-reporting is wildly inaccurate. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 23:54:58 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C275C16A4B3 for ; Mon, 6 Oct 2003 23:54:58 -0700 (PDT) Received: from ozlabs.org (ozlabs.org [203.10.76.45]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D17943FBD for ; Mon, 6 Oct 2003 23:54:57 -0700 (PDT) (envelope-from grog@lemis.com) Received: from blackwater.lemis.com (blackwater.lemis.com [192.109.197.80]) by ozlabs.org (Postfix) with ESMTP id 3485B2BC0D for ; Tue, 7 Oct 2003 16:54:54 +1000 (EST) Received: by blackwater.lemis.com (Postfix, from userid 1004) id 38CF551836; Tue, 7 Oct 2003 16:24:51 +0930 (CST) Date: Tue, 7 Oct 2003 16:24:51 +0930 From: Greg 'groggy' Lehey To: Poul-Henning Kamp Message-ID: <20031007065451.GE47054@wantadilla.lemis.com> References: <200310061753.28562.sam@errno.com> <32324.1065508157@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="QXAv++6zoyBcX2gv" Content-Disposition: inline In-Reply-To: <32324.1065508157@critter.freebsd.dk> User-Agent: Mutt/1.4i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 cc: Sam Leffler cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 06:54:58 -0000 --QXAv++6zoyBcX2gv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tuesday, 7 October 2003 at 8:29:17 +0200, Poul-Henning Kamp wrote: > In message <200310061753.28562.sam@errno.com>, Sam Leffler writes: >> On Monday 06 October 2003 04:11 pm, Poul-Henning Kamp wrote: >>> In message <20031006163218.L55190@pooker.samsco.home>, Scott Long writes: >> ...stuff deleted... >>>> As for returning an error code for a buffer that we (arbitrarily) believe >>>> to be too big to align, [...] >>> >>> I have never advocated returning an error based on "alignment and size", >>> only based on alignment alone. >> >> Imposing this restriction is a major semantic change that I consider a very >> bad idea. You are basically imposing the semantics of O_DIRECT on all i/o >> operations going to a device. I think it is important to give best effort to >> support unaligned operations `by default. I can imagine restricting this to >> some upper size bound but existing applications, regardless of how well you >> consider them to be written, must continue to work. > > Now now, you are missing two of the finer points: > > 1: Not "on all i/o operations going to a device", but rather "on i/o > operations which take the physread/write fast-path to avoid a copyin/out > overhead." (disks and tapes mostly). Ah. That's a whole different story. > Ttys, /dev/null and all the "typical" devices are unaffected. This is what I thought you meant at first, but your example using stdin suggested differently. Yes, that's not such a serious restriction. > 2: Right now we _do_ impose this restriction, but our > error-reporting is wildly inaccurate. And this is a pretty good indication that we're not going to break a lot of things with it. If this only applies to special devices, I don't see a problem with it. Greg -- See complete headers for address and phone numbers. --QXAv++6zoyBcX2gv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQE/gmM7IubykFB6QiMRAvO9AJ4uNuXIv8e9agMhJrguUCfG4AmP/ACgnL6G azbAbdTwu+3gjjIIDsEbjrs= =NDyj -----END PGP SIGNATURE----- --QXAv++6zoyBcX2gv-- From owner-freebsd-arch@FreeBSD.ORG Tue Oct 7 00:25:21 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5EE0816A4B3; Tue, 7 Oct 2003 00:25:21 -0700 (PDT) Received: from mailhub.fokus.fraunhofer.de (mailhub.fokus.fraunhofer.de [193.174.154.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 585D643FE1; Tue, 7 Oct 2003 00:25:19 -0700 (PDT) (envelope-from brandt@fokus.fraunhofer.de) Received: from beagle (beagle [193.175.132.100])h977PHS29653; Tue, 7 Oct 2003 09:25:17 +0200 (MEST) Date: Tue, 7 Oct 2003 09:25:16 +0200 (CEST) From: Harti Brandt To: Scott Long In-Reply-To: <20031006163218.L55190@pooker.samsco.home> Message-ID: <20031007091948.N57124@beagle.fokus.fraunhofer.de> References: <26369.1065477317@critter.freebsd.dk> <20031006163218.L55190@pooker.samsco.home> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: Poul-Henning Kamp cc: Garrett Wollman Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 07:25:21 -0000 On Mon, 6 Oct 2003, Scott Long wrote: SL>On Mon, 6 Oct 2003, Poul-Henning Kamp wrote: SL>> In message <200310062146.h96Lkpx0093486@khavrinen.lcs.mit.edu>, Garrett Wollman SL>> writes: SL>> SL>> >I think that gives us plenary authority to require appropriate SL>> >alignment of data buffers used to access disk devices directly. SL>> SL>> Well, apart from us wedging or botching the request rather than SL>> return a consistent error we're standards compliant then. SL>> SL>> It has been mentioned on #thatchannel that busdma should take care SL>> of this by copying the request as necessary. Faced with the prospect SL>> of a request several megabytes in size, I don't like the prospect SL>> of malloc/bcopy too much. SL> SL>Working around hardware requirements from the block layer gets messy. SL>You obviously don't want to manually align buffers that are distined SL>for hardware that doesn't care about the alignment. How do you SL>communicate this property upwards from the driver? Callbacks? Extra SL>flags in disk_create()? It's all rather messy that way. We already SL>have the busdma interface whose sole purpose is to take system SL>buffers and prepare them for transfer to/from hardware. It already SL>understands the concept of alignment, though it only applies it to SL>buffers that it allocates, not buffers that are handed to it. Fixing This seems to be not true. The alignment parameter for the busdma tag is more or less ignored in all backends. With the old malloc code you could simply make sure that the buffer you request is at least the size of alignment you need. With UMA this doesn't work anymore. I bumped into this problem while writing several ATM drivers which need alignment of 16 or 32 byte. At the moment the driver does this alignment itself by allocating a large enough dmamem buffer and fiddling with the pointers. harti SL>that would be easy, and would allow for optimizations based on the SL>bus (GART and IOMMU remapping). It also keeps the property down at SL>the device driver where it belongs. SL> SL>As for returning an error code for a buffer that we (arbitrarily) believe SL>to be too big to align, I'm not sure that it's a valid thing to worry SL>about. People expect their hardware to Just Work, regardless of how cheap SL>it is. If someone buys a cheap card with a cheap DMA engine, their cost SL>comes in from higher system time per I/O. If they don't like that, they SL>can complain to the manufacturer and/or buy a better card. I'm not aware SL>of Linux nor OSX rejecting I/O based on arbitrary size limits. SL> SL>Scott SL>_______________________________________________ SL>freebsd-arch@freebsd.org mailing list SL>http://lists.freebsd.org/mailman/listinfo/freebsd-arch SL>To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" SL> -- harti brandt, http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private brandt@fokus.fraunhofer.de, harti@freebsd.org From owner-freebsd-arch@FreeBSD.ORG Tue Oct 7 03:35:40 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ABA6D16A4B3; Tue, 7 Oct 2003 03:35:40 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [208.142.252.6]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8DE0543F3F; Tue, 7 Oct 2003 03:35:39 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h97AWu868742; Tue, 7 Oct 2003 06:32:56 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Tue, 7 Oct 2003 06:32:56 -0400 (EDT) From: Jeff Roberson To: Harti Brandt In-Reply-To: <20031007091948.N57124@beagle.fokus.fraunhofer.de> Message-ID: <20031007063207.V99666-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: Poul-Henning Kamp cc: Garrett Wollman Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 10:35:40 -0000 On Tue, 7 Oct 2003, Harti Brandt wrote: > On Mon, 6 Oct 2003, Scott Long wrote: > > SL>On Mon, 6 Oct 2003, Poul-Henning Kamp wrote: > SL>> In message <200310062146.h96Lkpx0093486@khavrinen.lcs.mit.edu>, Garrett Wollman > SL>> writes: > SL>> > SL>> >I think that gives us plenary authority to require appropriate > SL>> >alignment of data buffers used to access disk devices directly. > SL>> > SL>> Well, apart from us wedging or botching the request rather than > SL>> return a consistent error we're standards compliant then. > SL>> > SL>> It has been mentioned on #thatchannel that busdma should take care > SL>> of this by copying the request as necessary. Faced with the prospect > SL>> of a request several megabytes in size, I don't like the prospect > SL>> of malloc/bcopy too much. > SL> > SL>Working around hardware requirements from the block layer gets messy. > SL>You obviously don't want to manually align buffers that are distined > SL>for hardware that doesn't care about the alignment. How do you > SL>communicate this property upwards from the driver? Callbacks? Extra > SL>flags in disk_create()? It's all rather messy that way. We already > SL>have the busdma interface whose sole purpose is to take system > SL>buffers and prepare them for transfer to/from hardware. It already > SL>understands the concept of alignment, though it only applies it to > SL>buffers that it allocates, not buffers that are handed to it. Fixing > > This seems to be not true. The alignment parameter for the busdma tag is > more or less ignored in all backends. With the old malloc code you could > simply make sure that the buffer you request is at least the size of > alignment you need. With UMA this doesn't work anymore. I bumped into this > problem while writing several ATM drivers which need alignment of 16 or 32 > byte. At the moment the driver does this alignment itself by allocating a > large enough dmamem buffer and fiddling with the pointers. I don't believe that this is the case with UMA. Too many things depended on this behavior and so I left it in. > > harti > > SL>that would be easy, and would allow for optimizations based on the > SL>bus (GART and IOMMU remapping). It also keeps the property down at > SL>the device driver where it belongs. > SL> > SL>As for returning an error code for a buffer that we (arbitrarily) believe > SL>to be too big to align, I'm not sure that it's a valid thing to worry > SL>about. People expect their hardware to Just Work, regardless of how cheap > SL>it is. If someone buys a cheap card with a cheap DMA engine, their cost > SL>comes in from higher system time per I/O. If they don't like that, they > SL>can complain to the manufacturer and/or buy a better card. I'm not aware > SL>of Linux nor OSX rejecting I/O based on arbitrary size limits. > SL> > SL>Scott > SL>_______________________________________________ > SL>freebsd-arch@freebsd.org mailing list > SL>http://lists.freebsd.org/mailman/listinfo/freebsd-arch > SL>To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > SL> > > -- > harti brandt, > http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private > brandt@fokus.fraunhofer.de, harti@freebsd.org > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Tue Oct 7 04:04:36 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F33216A4B3; Tue, 7 Oct 2003 04:04:36 -0700 (PDT) Received: from mailhub.fokus.fraunhofer.de (mailhub.fokus.fraunhofer.de [193.174.154.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9D18B43FDF; Tue, 7 Oct 2003 04:04:30 -0700 (PDT) (envelope-from brandt@fokus.fraunhofer.de) Received: from beagle (beagle [193.175.132.100])h97B4MS09078; Tue, 7 Oct 2003 13:04:22 +0200 (MEST) Date: Tue, 7 Oct 2003 13:04:22 +0200 (CEST) From: Harti Brandt To: Jeff Roberson In-Reply-To: <20031007063207.V99666-100000@mail.chesapeake.net> Message-ID: <20031007125450.V63760@beagle.fokus.fraunhofer.de> References: <20031007063207.V99666-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: Poul-Henning Kamp cc: Garrett Wollman Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 11:04:36 -0000 On Tue, 7 Oct 2003, Jeff Roberson wrote: JR>On Tue, 7 Oct 2003, Harti Brandt wrote: JR> JR>> On Mon, 6 Oct 2003, Scott Long wrote: JR>> JR>> SL>buffers and prepare them for transfer to/from hardware. It already JR>> SL>understands the concept of alignment, though it only applies it to JR>> SL>buffers that it allocates, not buffers that are handed to it. Fixing JR>> JR>> This seems to be not true. The alignment parameter for the busdma tag is JR>> more or less ignored in all backends. With the old malloc code you could JR>> simply make sure that the buffer you request is at least the size of JR>> alignment you need. With UMA this doesn't work anymore. I bumped into this JR>> problem while writing several ATM drivers which need alignment of 16 or 32 JR>> byte. At the moment the driver does this alignment itself by allocating a JR>> large enough dmamem buffer and fiddling with the pointers. JR> JR>I don't believe that this is the case with UMA. Too many things depended JR>on this behavior and so I left it in. Well, when UMA went in I suddenly got panics (because I checked the alignment via KASSERT() so I added the address twiddeling code. This was some time ago, but I remember that a buffer was aligned to half of its size. That was when UMA was new, maybe that this was a problem only a short time - anyway I did not remove that code just to be on the safe side. In any case it should make sense to document in malloc(9) that one either can or cannot rely on this feature. Anyway, even with the guarantee that UMA aligns to at least the size of the buffer. bus_dmamem_alloc does not allocate a 256 byte buffer to a 1k boundary even if I specify the 1k alignment in the dma tag. harti -- harti brandt, http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private brandt@fokus.fraunhofer.de, harti@freebsd.org From owner-freebsd-arch@FreeBSD.ORG Tue Oct 7 08:41:21 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BEF3016A4B3 for ; Tue, 7 Oct 2003 08:41:21 -0700 (PDT) Received: from Daffy.timing.com (ns1int.timing.com [206.168.13.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id D459C43F75 for ; Tue, 7 Oct 2003 08:41:20 -0700 (PDT) (envelope-from ben@timing.com) Received: from piglet.timing.com (oink@piglet.timing.com [206.168.13.178]) by Daffy.timing.com (8.12.8p2/8.12.8) with ESMTP id h97FfI3R007432; Tue, 7 Oct 2003 09:41:18 -0600 (MDT) (envelope-from ben@timing.com) Received: from piglet.timing.com (oink@localhost.timing.com [127.0.0.1]) by piglet.timing.com (8.12.6p3/8.12.6) with ESMTP id h97FfII9034088; Tue, 7 Oct 2003 09:41:18 -0600 (MDT) (envelope-from ben@piglet.timing.com) Received: (from ben@localhost) by piglet.timing.com (8.12.6p3/8.12.6/Submit) id h97FfHNF034085; Tue, 7 Oct 2003 09:41:17 -0600 (MDT) From: Ben Mesander MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16258.56989.685473.900678@piglet.timing.com> Date: Tue, 7 Oct 2003 09:41:17 -0600 To: "Poul-Henning Kamp" In-Reply-To: <32324.1065508157@critter.freebsd.dk> References: <200310061753.28562.sam@errno.com> <32324.1065508157@critter.freebsd.dk> X-Mailer: VM 7.00 under Emacs 21.2.95.2 cc: Sam Leffler cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 15:41:21 -0000 Poul-Henning Kamp writes: > 1: Not "on all i/o operations going to a device", but rather "on i/o > operations which take the physread/write fast-path to avoid a copyin/out > overhead." (disks and tapes mostly). Ttys, /dev/null and all the > "typical" devices are unaffected. How about a flag for open(2), such as O_DIRECT, which would indicate if the fd in question is such a device, and you did not sufficiently align, etc. your buffers, then you get an error when you attempt the I/O operation. If you did not specify this flag when opening the device, then do the bounce buffer dance for the I/O operation, and lose efficiency. --Ben From owner-freebsd-arch@FreeBSD.ORG Tue Oct 7 09:02:46 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B542616A4B3 for ; Tue, 7 Oct 2003 09:02:46 -0700 (PDT) Received: from arginine.spc.org (arginine.spc.org [195.206.69.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 859E043FA3 for ; Tue, 7 Oct 2003 09:02:45 -0700 (PDT) (envelope-from bms@spc.org) Received: from localhost (localhost [127.0.0.1]) by arginine.spc.org (Postfix) with ESMTP id 98C0F65419; Tue, 7 Oct 2003 17:02:44 +0100 (BST) Received: from arginine.spc.org ([127.0.0.1]) by localhost (arginine.spc.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 42769-05-4; Tue, 7 Oct 2003 17:02:44 +0100 (BST) Received: from saboteur.dek.spc.org (unknown [81.3.72.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by arginine.spc.org (Postfix) with ESMTP id 6C11865292; Tue, 7 Oct 2003 17:02:43 +0100 (BST) Received: by saboteur.dek.spc.org (Postfix, from userid 1001) id C870D47; Tue, 7 Oct 2003 17:02:31 +0100 (BST) Date: Tue, 7 Oct 2003 17:02:31 +0100 From: Bruce M Simpson To: Ben Mesander Message-ID: <20031007160231.GB73063@saboteur.dek.spc.org> References: <200310061753.28562.sam@errno.com> <32324.1065508157@critter.freebsd.dk> <16258.56989.685473.900678@piglet.timing.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <16258.56989.685473.900678@piglet.timing.com> cc: Sam Leffler cc: Poul-Henning Kamp cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 16:02:46 -0000 On Tue, Oct 07, 2003 at 09:41:17AM -0600, Ben Mesander wrote: > How about a flag for open(2), such as O_DIRECT, which would indicate > if the fd in question is such a device, and you did not sufficiently > align, etc. your buffers, then you get an error when you attempt the > I/O operation. O_DIRECT is already used to indicate that the application wishes to bypass the buffer cache. I think Scott's proposal, that the busdma API gets re-used, creating bounce buffers where necessary, makes sense. I think that if phk intends to 'raise the bar', he'd better be buying the drinks at EuroBSDCon. :-) BMS From owner-freebsd-arch@FreeBSD.ORG Tue Oct 7 09:05:33 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9C05216A4B3 for ; Tue, 7 Oct 2003 09:05:33 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id D31B043FE0 for ; Tue, 7 Oct 2003 09:05:31 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h97G5M2B039628; Tue, 7 Oct 2003 18:05:23 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Bruce M Simpson From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 07 Oct 2003 17:02:31 BST." <20031007160231.GB73063@saboteur.dek.spc.org> Date: Tue, 07 Oct 2003 18:05:22 +0200 Message-ID: <39627.1065542722@critter.freebsd.dk> cc: Sam Leffler cc: arch@freebsd.org Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 16:05:33 -0000 In message <20031007160231.GB73063@saboteur.dek.spc.org>, Bruce M Simpson write s: >I think that if phk intends to 'raise the bar', he'd better be buying the >drinks at EuroBSDCon. :-) You seem to miss the fact that I merely want to nail the bar down and document where we decided to put it... If Scottl wants to make busdma do copy/align, then fine with me. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Oct 7 09:44:34 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 34BFB16A4B3; Tue, 7 Oct 2003 09:44:34 -0700 (PDT) Received: from mail.tcoip.com.br (erato.tco.net.br [200.220.254.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id B823243FBD; Tue, 7 Oct 2003 09:44:31 -0700 (PDT) (envelope-from dcs@tcoip.com.br) Received: from tcoip.com.br ([10.0.2.6]) by mail.tcoip.com.br (8.11.6/8.11.6) with ESMTP id h97GiDj19377; Tue, 7 Oct 2003 13:44:14 -0300 Message-ID: <3F82ED5D.3060803@tcoip.com.br> Date: Tue, 07 Oct 2003 13:44:13 -0300 From: "Daniel C. Sobral" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5b) Gecko/20030827 X-Accept-Language: en-us, en, pt-br, ja MIME-Version: 1.0 To: Brooks Davis References: <20030930171535.GA31908@Odin.AC.HMC.Edu> <13247.1064944601@critter.freebsd.dk> <20030930182359.GD31908@Odin.AC.HMC.Edu> In-Reply-To: <20030930182359.GD31908@Odin.AC.HMC.Edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable cc: arch@freebsd.org cc: Poul-Henning Kamp cc: Vincent Jardin cc: net@freebsd.org Subject: Re: adding if_dev member to struct ifnet X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 16:44:34 -0000 Brooks Davis wrote: >=20 > Not today, since none of them get used in the paths that do this. In > general the network code doesn't care what you call an interface. Ther= e > are a few corners where it does, but nothing that isn't specific to > a certain set of drivers. Additionally, it is necessary to not have > members called if_name and if_unit if we have if_xname as the primary > driver name. It's also worth noting that one of the things I want to d= o > is break the driver+unit mapping for certain types of pseudo devices. > Specifically vlan devices should be allocatable by creating an interfac= e > with a name like fxp0.100 so while you could synthesize a unit number, > it wouldn't have any useful meaning. I wonder how that works for vlans over bridges... --=20 Daniel C. Sobral Ger=EAncia de Opera=E7=F5es Divis=E3o de Comunica=E7=E3o de Dados Coordena=E7=E3o de Seguran=E7a VIVO Centro Oeste Norte Fones: 55-61-313-7654/Cel: 55-61-9618-0904 E-mail: Daniel.Capo@tco.net.br Daniel.Sobral@tcoip.com.br dcs@tcoip.com.br From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 04:47:28 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6F70516A4B3; Wed, 8 Oct 2003 04:47:28 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id AC7A443FD7; Wed, 8 Oct 2003 04:47:26 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with ESMTP id VAA11652; Wed, 8 Oct 2003 21:47:09 +1000 Date: Wed, 8 Oct 2003 21:45:47 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: "Adam C. Migus" In-Reply-To: <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> Message-ID: <20031008212302.T4729@gamplex.bde.org> References: <20030925092319.H5418@gamplex.bde.org><49939.204.254.155.35.1064593320.squirrel@mail.migus.org> <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Stefan Farfeleder cc: arch@FreeBSD.org cc: John Baldwin Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 11:47:28 -0000 On Thu, 2 Oct 2003, Adam C. Migus wrote: > > On Fri, 26 Sep 2003, Adam C. Migus wrote: > > This patch works for me, please let me know if there's any problems > with it or you'd like a PR. % ==== //depot/user/amigus/freebsd-amigus/src/usr.sbin/config/config.y#1 - /src/p4/user/amigus/perforce.freebsd.org/freebsd-amigus/usr.sbin/config/config.y ==== % @@ -118,6 +118,8 @@ % | % Config_spec SEMICOLON % | % + Include % + | % SEMICOLON % | % error SEMICOLON % @@ -164,9 +166,7 @@ % = { % hints = $2; % hintmode = 1; % - } | % - INCLUDE ID % - = { include($2, 0); }; % + }; % % System_spec: % CONFIG System_id System_parameter_list % @@ -265,6 +265,11 @@ % rmdev($2); % } ; % % +Include: % + INCLUDE ID % + = { include($2, 0); }; % + % + % %% % % void I found 1 problem with this: it doesn't require SEMICOLON after "INCLUDE ID", so parsing resumes after "ID" when the input stream is switched back to the includer. You can say things like include FOO device foo and then the "device foo" directive actually works. This can be considered a feature, but the following is not: include GENERIC.local at the end of GENERIC should be a syntax error, but it actually matches ID = GENERIC and includes GENERIC recursively, which gives a confusing error message. There would be a syntax error on switching back but config aborts before then. (Filenames other than single identifiers must be quoted to avoid problems like this. This is not very obvious since config was changed to not require quotes in most contexts.) Stefan's version requires SEMICOLON: % Index: src/usr.sbin/config/config.y % =================================================================== % RCS file: /usr/home/ncvs/src/usr.sbin/config/config.y,v % retrieving revision 1.61 % diff -u -r1.61 config.y % --- src/usr.sbin/config/config.y 6 Jul 2003 02:00:52 -0000 1.61 % +++ src/usr.sbin/config/config.y 27 Sep 2003 10:39:13 -0000 % @@ -118,6 +118,9 @@ % | % Config_spec SEMICOLON % | % + INCLUDE ID SEMICOLON % + = { include($2, 0); }; % + | % SEMICOLON % | % error SEMICOLON % @@ -164,9 +167,7 @@ % = { % hints = $2; % hintmode = 1; % - } | % - INCLUDE ID % - = { include($2, 0); }; % + } % % System_spec: % CONFIG System_id System_parameter_list I lost your examples showing that this doesn't quite work. What is the problem with it? Another old bug is that using "include" defeats the point of the INCLUDE_CONFIG_FILE option (since included files aren't expanded). Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 07:12:16 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1486C16A4B3; Wed, 8 Oct 2003 07:12:16 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id DB07C43FB1; Wed, 8 Oct 2003 07:12:13 -0700 (PDT) (envelope-from adam@migus.org) Received: from garple.migus.org ([68.55.83.94]) by comcast.net (rwcrmhc13) with ESMTP id <2003100814121301500liqsue>; Wed, 8 Oct 2003 14:12:13 +0000 Received: by garple.migus.org (Postfix, from userid 80) id AB08A8FC3D; Wed, 8 Oct 2003 10:12:12 -0400 (EDT) Received: from 204.254.155.35 (SquirrelMail authenticated user adam) by mail.migus.org with HTTP; Wed, 8 Oct 2003 10:12:12 -0400 (EDT) Message-ID: <51310.204.254.155.35.1065622332.squirrel@mail.migus.org> In-Reply-To: <20031008212302.T4729@gamplex.bde.org> References: <20030925092319.H5418@gamplex.bde.org><49939.204.254.155.35.1064593320.squirrel@mail.migus.org> <20030927080420.N18558@gamplex.bde.org> <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> <20031008212302.T4729@gamplex.bde.org> Date: Wed, 8 Oct 2003 10:12:12 -0400 (EDT) From: "Adam C. Migus" To: "Bruce Evans" User-Agent: SquirrelMail/1.4.1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal cc: Stefan Farfeleder cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 14:12:16 -0000 Bruce Evans said: > On Thu, 2 Oct 2003, Adam C. Migus wrote: > >> > On Fri, 26 Sep 2003, Adam C. Migus wrote: >> >> This patch works for me, please let me know if there's any >> problems >> with it or you'd like a PR. > > % ==== > //depot/user/amigus/freebsd-amigus/src/usr.sbin/config/config.y#1 - > /src/p4/user/amigus/perforce.freebsd.org/freebsd-amigus/usr.sbin/config/config.y > ==== > % @@ -118,6 +118,8 @@ > % | > % Config_spec SEMICOLON > % | > % + Include > % + | > % SEMICOLON > % | > % error SEMICOLON > % @@ -164,9 +166,7 @@ > % = { > % hints = $2; > % hintmode = 1; > % - } | > % - INCLUDE ID > % - = { include($2, 0); }; > % + }; > % > % System_spec: > % CONFIG System_id System_parameter_list > % @@ -265,6 +265,11 @@ > % rmdev($2); > % } ; > % > % +Include: > % + INCLUDE ID > % + = { include($2, 0); }; > % + > % + > % %% > % > % void > > I found 1 problem with this: it doesn't require SEMICOLON after > "INCLUDE ID", so parsing resumes after "ID" when the input stream > is switched back to the includer. You can say things like > > include FOO device foo > > and then the "device foo" directive actually works. This can be > considered a feature, but the following is not: > > include GENERIC.local > > at the end of GENERIC should be a syntax error, but it actually > matches ID = GENERIC and includes GENERIC recursively, which gives > a confusing error message. There would be a syntax error on > switching > back but config aborts before then. (Filenames other than single > identifiers must be quoted to avoid problems like this. This is not > very obvious since config was changed to not require quotes in most > contexts.) > > Stefan's version requires SEMICOLON: > > % Index: src/usr.sbin/config/config.y > % > =================================================================== > % RCS file: /usr/home/ncvs/src/usr.sbin/config/config.y,v > % retrieving revision 1.61 > % diff -u -r1.61 config.y > % --- src/usr.sbin/config/config.y 6 Jul 2003 02:00:52 -0000 1.61 > % +++ src/usr.sbin/config/config.y 27 Sep 2003 10:39:13 -0000 > % @@ -118,6 +118,9 @@ > % | > % Config_spec SEMICOLON > % | > % + INCLUDE ID SEMICOLON > % + = { include($2, 0); }; > % + | > % SEMICOLON > % | > % error SEMICOLON > % @@ -164,9 +167,7 @@ > % = { > % hints = $2; > % hintmode = 1; > % - } | > % - INCLUDE ID > % - = { include($2, 0); }; > % + } > % > % System_spec: > % CONFIG System_id System_parameter_list > > I lost your examples showing that this doesn't quite work. What is > the > problem with it? > > Another old bug is that using "include" defeats the point of the > INCLUDE_CONFIG_FILE option (since included files aren't expanded). > > Bruce > Bruce, I was half aware of this issue but viewed the bug/feature as a 'necessary evil' given the complications in dealing with it. Your point about defeating the INCLUDE_CONFIG_FILE option, which I admit I like but no longer use, furthers this point -- the 'include' option is messy, period. WRT Stefan's solution, simply nest two includes on the first line, like so: head -1 KERNEL include SUB1 head -1 SUB1 include SUB2 This will cause config to fail in the same manner it did before, with his patch. If you'd like, I can work on a more elegant solution that addresses the bugs you've outlined as well as (possibly, maybe) making INCLUDE_CONFIG_FILE useful again, with 'include.' Based on what I am reading I think people like 'include,' it's been there for a while and I don't think it's going away, so, if I'm correct fixing it right is the way to go. Let me know if you guys are interested, if so I'll happily make fixing /usr/sbin/config a side project. Otherwise I guess you'd be left with pick the broken you like best, document it and leave it that way or pull the include option entirely. -- Adam - (http://people.migus.org/~amigus/) Migus Dot Org - (http://www.migus.org/) From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 08:53:44 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0AE4016A4B3; Wed, 8 Oct 2003 08:53:44 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F55A43FEA; Wed, 8 Oct 2003 08:53:42 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with ESMTP id BAA03398; Thu, 9 Oct 2003 01:53:36 +1000 Date: Thu, 9 Oct 2003 01:52:14 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: "Adam C. Migus" In-Reply-To: <51310.204.254.155.35.1065622332.squirrel@mail.migus.org> Message-ID: <20031009014522.L5476@gamplex.bde.org> References: <20030925092319.H5418@gamplex.bde.org><49939.204.254.155.35.1064593320.squirrel@mail.migus.org> <20030927080420.N18558@gamplex.bde.org> <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> <51310.204.254.155.35.1065622332.squirrel@mail.migus.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Stefan Farfeleder cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 15:53:44 -0000 On Wed, 8 Oct 2003, Adam C. Migus wrote: > Bruce Evans said: > > Stefan's version requires SEMICOLON: > > % + INCLUDE ID SEMICOLON > > % + = { include($2, 0); }; > > % + | > > ... > > I lost your examples showing that this doesn't quite work. What is > > the > > problem with it? > ... > WRT Stefan's solution, simply nest two includes on the first line, > like so: > > head -1 KERNEL > include SUB1 > > head -1 SUB1 > include SUB2 > > This will cause config to fail in the same manner it did before, > with his patch. Er, but this works for me. I test with SUB2 having the following: 1) include GENERIC 2) same contents as GENERIC 3) same contents as GENERIC except for no comment lines at beginning. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 09:33:19 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BCBB316A4B3; Wed, 8 Oct 2003 09:33:19 -0700 (PDT) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id A361443FCB; Wed, 8 Oct 2003 09:33:14 -0700 (PDT) (envelope-from adam@migus.org) Received: from garple.migus.org ([68.55.83.94]) by comcast.net (rwcrmhc11) with ESMTP id <2003100816331401300qttede>; Wed, 8 Oct 2003 16:33:14 +0000 Received: by garple.migus.org (Postfix, from userid 80) id C84558FC40; Wed, 8 Oct 2003 12:33:13 -0400 (EDT) Received: from 204.254.155.35 (SquirrelMail authenticated user adam) by mail.migus.org with HTTP; Wed, 8 Oct 2003 12:33:13 -0400 (EDT) Message-ID: <51457.204.254.155.35.1065630793.squirrel@mail.migus.org> In-Reply-To: <20031009014522.L5476@gamplex.bde.org> References: <20030925092319.H5418@gamplex.bde.org><49939.204.254.155.35.1064593320.squirrel@mail.migus.org> <20030927080420.N18558@gamplex.bde.org> <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> <20031008212302.T4729@gamplex.bde.org> <51310.204.254.155.35.1065622332.squirrel@mail.migus.org> <20031009014522.L5476@gamplex.bde.org> Date: Wed, 8 Oct 2003 12:33:13 -0400 (EDT) From: "Adam C. Migus" To: "Bruce Evans" User-Agent: SquirrelMail/1.4.1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal cc: Stefan Farfeleder cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 16:33:19 -0000 Bruce Evans said: > On Wed, 8 Oct 2003, Adam C. Migus wrote: > >> Bruce Evans said: >> > Stefan's version requires SEMICOLON: >> > % + INCLUDE ID SEMICOLON >> > % + = { include($2, 0); }; >> > % + | >> > ... >> > I lost your examples showing that this doesn't quite work. What >> is >> > the >> > problem with it? >> ... >> WRT Stefan's solution, simply nest two includes on the first line, >> like so: >> >> head -1 KERNEL >> include SUB1 >> >> head -1 SUB1 >> include SUB2 >> >> This will cause config to fail in the same manner it did before, >> with his patch. > > Er, but this works for me. I test with SUB2 having the following: > 1) include GENERIC > 2) same contents as GENERIC > 3) same contents as GENERIC except for no comment lines at > beginning. > > Bruce > Here's my broken config (with Stefan's patch) compile, run, error, output and config exerpts: root@caster:ttyp4:config# make clean rm -f config config.o main.o lang.o mkmakefile.o mkheaders.o mkoptions.o config.8.gz config.8.cat.gz lang.c config.c y.tab.c y.tab.h root@caster:ttyp4:config# make CFLAGS="`make -V CFLAGS` -DYYDEBUG=1" Warning: Object directory not changed from original /src/freebsd/CURRENT/src/usr.sbin/config yacc -d config.y cp y.tab.c config.c cc -O -pipe -mcpu=pentiumpro -I. -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers -Werror -DYYDEBUG=1 -c config.c cc -O -pipe -mcpu=pentiumpro -I. -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers -Werror -DYYDEBUG=1 -c main.c lex -t lang.l > lang.c cc -O -pipe -mcpu=pentiumpro -I. -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers -Werror -DYYDEBUG=1 -c lang.c cc -O -pipe -mcpu=pentiumpro -I. -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers -Werror -DYYDEBUG=1 -c mkmakefile.c cc -O -pipe -mcpu=pentiumpro -I. -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers -Werror -DYYDEBUG=1 -c mkheaders.c cc -O -pipe -mcpu=pentiumpro -I. -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers -Werror -DYYDEBUG=1 -c mkoptions.c cc -O -pipe -mcpu=pentiumpro -I. -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers -Werror -DYYDEBUG=1 -o config config.o main.o lang.o mkmakefile.o mkheaders.o mkoptions.o -ll gzip -cn config.8 > config.8.gz root@caster:ttyp4:config# cp config /tmp root@caster:ttyp4:config# cd ../../sys/i386/conf root@caster:ttyp4:conf# export YYDEBUG=1 root@caster:ttyp4:conf# /tmp/config DISKLESS_SMP_MAC yydebug: state 0, reducing by rule 3 (Many_specs :) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 274 (INCLUDE) yydebug: state 2, shifting to state 19 yydebug: state 19, reading 275 (ID) yydebug: state 19, shifting to state 46 yydebug: state 46, reading 273 (SEMICOLON) yydebug: state 46, shifting to state 56 yydebug: state 56, reducing by rule 20 (Config_spec : INCLUDE ID SEMICOLON) yydebug: after reduction, shifting from state 2 to state 22 yydebug: state 22, reading 274 (INCLUDE) config: SMP_MAC:1: syntax error root@caster:ttyp4:conf# head -3 DISKLESS_SMP_MAC SMP_MAC MAC ==> DISKLESS_SMP_MAC <== include SMP_MAC ident DISKLESS_SMP_MAC ==> SMP_MAC <== include MAC #ident SMP_MAC ==> MAC <== # # MAC -- Generic kernel configuration file for FreeBSD/i386 w/MAC # root@caster:ttyp4:conf# -- Adam - (http://people.migus.org/~amigus/) Migus Dot Org - (http://www.migus.org/) From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 10:38:08 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C96B016A4B3; Wed, 8 Oct 2003 10:38:08 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id B899F43F3F; Wed, 8 Oct 2003 10:38:05 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with ESMTP id DAA12014; Thu, 9 Oct 2003 03:37:58 +1000 Date: Thu, 9 Oct 2003 03:36:35 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: "Adam C. Migus" In-Reply-To: <51457.204.254.155.35.1065630793.squirrel@mail.migus.org> Message-ID: <20031009032431.Y6050@gamplex.bde.org> References: <20030925092319.H5418@gamplex.bde.org><49939.204.254.155.35.1064593320.squirrel@mail.migus.org> <20030927080420.N18558@gamplex.bde.org> <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> <51310.204.254.155.35.1065622332.squirrel@mail.migus.org> <51457.204.254.155.35.1065630793.squirrel@mail.migus.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Stefan Farfeleder cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 17:38:09 -0000 On Wed, 8 Oct 2003, Adam C. Migus wrote: > Bruce Evans said: > > On Wed, 8 Oct 2003, Adam C. Migus wrote: > >> ... > >> WRT Stefan's solution, simply nest two includes on the first line, > >> like so: > >> > >> head -1 KERNEL > >> include SUB1 > >> > >> head -1 SUB1 > >> include SUB2 > >> > >> This will cause config to fail in the same manner it did before, > >> with his patch. > > > > Er, but this works for me. I test with SUB2 having the following: > > 1) include GENERIC > > 2) same contents as GENERIC > > 3) same contents as GENERIC except for no comment lines at > > beginning. > Here's my broken config (with Stefan's patch) compile, run, error, > output and config exerpts: This still works for me :-). > root@caster:ttyp4:config# make clean > rm -f config config.o main.o lang.o mkmakefile.o mkheaders.o > mkoptions.o config.8.gz config.8.cat.gz lang.c config.c y.tab.c > y.tab.h > root@caster:ttyp4:config# make CFLAGS="`make -V CFLAGS` -DYYDEBUG=1" > Warning: Object directory not changed from original > /src/freebsd/CURRENT/src/usr.sbin/config > yacc -d config.y > cp y.tab.c config.c > cc -O -pipe -mcpu=pentiumpro -I. > -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers > -Werror -DYYDEBUG=1 -c config.c > cc -O -pipe -mcpu=pentiumpro -I. > -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers > -Werror -DYYDEBUG=1 -c main.c > lex -t lang.l > lang.c > cc -O -pipe -mcpu=pentiumpro -I. > -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers > -Werror -DYYDEBUG=1 -c lang.c > cc -O -pipe -mcpu=pentiumpro -I. > -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers > -Werror -DYYDEBUG=1 -c mkmakefile.c > cc -O -pipe -mcpu=pentiumpro -I. > -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers > -Werror -DYYDEBUG=1 -c mkheaders.c > cc -O -pipe -mcpu=pentiumpro -I. > -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers > -Werror -DYYDEBUG=1 -c mkoptions.c > cc -O -pipe -mcpu=pentiumpro -I. > -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers > -Werror -DYYDEBUG=1 -o config config.o main.o lang.o mkmakefile.o > mkheaders.o mkoptions.o -ll > gzip -cn config.8 > config.8.gz > root@caster:ttyp4:config# cp config /tmp > root@caster:ttyp4:config# cd ../../sys/i386/conf > root@caster:ttyp4:conf# export YYDEBUG=1 > root@caster:ttyp4:conf# /tmp/config DISKLESS_SMP_MAC > yydebug: state 0, reducing by rule 3 (Many_specs :) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 274 (INCLUDE) > yydebug: state 2, shifting to state 19 > yydebug: state 19, reading 275 (ID) > yydebug: state 19, shifting to state 46 > yydebug: state 46, reading 273 (SEMICOLON) > yydebug: state 46, shifting to state 56 > yydebug: state 56, reducing by rule 20 (Config_spec : INCLUDE ID > SEMICOLON) This seems to be from a patching error. Stefan's patch moved "INCLUDE ID SEMICOLON" from Config_spec to Spec. > yydebug: after reduction, shifting from state 2 to state 22 > yydebug: state 22, reading 274 (INCLUDE) > config: SMP_MAC:1: syntax error > root@caster:ttyp4:conf# head -3 DISKLESS_SMP_MAC SMP_MAC MAC > ==> DISKLESS_SMP_MAC <== > include SMP_MAC > > ident DISKLESS_SMP_MAC > > ==> SMP_MAC <== > include MAC > > #ident SMP_MAC > > ==> MAC <== > # > # MAC -- Generic kernel configuration file for FreeBSD/i386 w/MAC > # My debugging output shows the move: %%% yydebug: state 0, reducing by rule 3 (Many_specs :) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 274 (INCLUDE) yydebug: state 2, shifting to state 19 yydebug: state 19, reading 275 (ID) yydebug: state 19, shifting to state 46 yydebug: state 46, reading 273 (SEMICOLON) yydebug: state 46, shifting to state 56 yydebug: state 56, reducing by rule 6 (Spec : INCLUDE ID SEMICOLON) ^^^^^^ ^^^^ [The first difference is here] yydebug: after reduction, shifting from state 2 to state 20 yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 274 (INCLUDE) yydebug: state 2, shifting to state 19 yydebug: state 19, reading 275 (ID) yydebug: state 19, shifting to state 46 yydebug: state 46, reading 273 (SEMICOLON) yydebug: state 46, shifting to state 56 yydebug: state 56, reducing by rule 6 (Spec : INCLUDE ID SEMICOLON) yydebug: after reduction, shifting from state 2 to state 20 yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 273 (SEMICOLON) yydebug: state 2, shifting to state 18 yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) yydebug: after reduction, shifting from state 2 to state 20 yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 273 (SEMICOLON) yydebug: state 2, shifting to state 18 yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) yydebug: after reduction, shifting from state 2 to state 20 yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 273 (SEMICOLON) yydebug: state 2, shifting to state 18 yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) yydebug: after reduction, shifting from state 2 to state 20 yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 273 (SEMICOLON) yydebug: state 2, shifting to state 18 yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) yydebug: after reduction, shifting from state 2 to state 20 yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 273 (SEMICOLON) yydebug: state 2, shifting to state 18 yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) yydebug: after reduction, shifting from state 2 to state 20 yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 273 (SEMICOLON) yydebug: state 2, shifting to state 18 yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) yydebug: after reduction, shifting from state 2 to state 20 yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 266 (IDENT) yydebug: state 2, shifting to state 11 yydebug: state 11, reading 275 (ID) yydebug: state 11, shifting to state 35 yydebug: state 35, reducing by rule 15 (Config_spec : IDENT ID) yydebug: after reduction, shifting from state 2 to state 22 yydebug: state 22, reading 273 (SEMICOLON) yydebug: state 22, shifting to state 48 yydebug: state 48, reducing by rule 5 (Spec : Config_spec SEMICOLON) yydebug: after reduction, shifting from state 2 to state 20 yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) yydebug: after reduction, shifting from state 0 to state 2 yydebug: state 2, reading 0 (end-of-file) yydebug: state 2, reducing by rule 1 (Configuration : Many_specs) yydebug: after reduction, shifting from state 0 to state 1 Specify machine type, e.g. ``machine i386'' %%% Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 12:07:25 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3E61D16A4B3; Wed, 8 Oct 2003 12:07:25 -0700 (PDT) Received: from sccrmhc13.comcast.net (sccrmhc13.comcast.net [204.127.202.64]) by mx1.FreeBSD.org (Postfix) with ESMTP id C2D1343FBF; Wed, 8 Oct 2003 12:07:23 -0700 (PDT) (envelope-from adam@migus.org) Received: from garple.migus.org ([68.55.83.94]) by comcast.net (sccrmhc13) with ESMTP id <2003100819072201600b0nshe>; Wed, 8 Oct 2003 19:07:22 +0000 Received: by garple.migus.org (Postfix, from userid 80) id 1D7F48FC40; Wed, 8 Oct 2003 15:07:22 -0400 (EDT) Received: from 204.254.155.35 (SquirrelMail authenticated user adam) by mail.migus.org with HTTP; Wed, 8 Oct 2003 15:07:22 -0400 (EDT) Message-ID: <51574.204.254.155.35.1065640042.squirrel@mail.migus.org> In-Reply-To: <20031008212302.T4729@gamplex.bde.org> References: <20030925092319.H5418@gamplex.bde.org><49939.204.254.155.35.1064593320.squirrel@mail.migus.org> <20030927080420.N18558@gamplex.bde.org> <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> <20031008212302.T4729@gamplex.bde.org> Date: Wed, 8 Oct 2003 15:07:22 -0400 (EDT) From: "Adam C. Migus" To: "Bruce Evans" User-Agent: SquirrelMail/1.4.1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal cc: Stefan Farfeleder cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 19:07:25 -0000 Bruce Evans said: > On Thu, 2 Oct 2003, Adam C. Migus wrote: > >> > On Fri, 26 Sep 2003, Adam C. Migus wrote: >> >> This patch works for me, please let me know if there's any >> problems >> with it or you'd like a PR. > > % ==== > //depot/user/amigus/freebsd-amigus/src/usr.sbin/config/config.y#1 - > /src/p4/user/amigus/perforce.freebsd.org/freebsd-amigus/usr.sbin/config/config.y > ==== > % @@ -118,6 +118,8 @@ > % | > % Config_spec SEMICOLON > % | > % + Include > % + | > % SEMICOLON > % | > % error SEMICOLON > % @@ -164,9 +166,7 @@ > % = { > % hints = $2; > % hintmode = 1; > % - } | > % - INCLUDE ID > % - = { include($2, 0); }; > % + }; > % > % System_spec: > % CONFIG System_id System_parameter_list > % @@ -265,6 +265,11 @@ > % rmdev($2); > % } ; > % > % +Include: > % + INCLUDE ID > % + = { include($2, 0); }; > % + > % + > % %% > % > % void > > I found 1 problem with this: it doesn't require SEMICOLON after > "INCLUDE ID", so parsing resumes after "ID" when the input stream > is switched back to the includer. You can say things like > > include FOO device foo > > and then the "device foo" directive actually works. This can be > considered a feature, but the following is not: > > include GENERIC.local > > at the end of GENERIC should be a syntax error, but it actually > matches ID = GENERIC and includes GENERIC recursively, which gives > a confusing error message. There would be a syntax error on > switching > back but config aborts before then. (Filenames other than single > identifiers must be quoted to avoid problems like this. This is not > very obvious since config was changed to not require quotes in most > contexts.) > > Stefan's version requires SEMICOLON: > > % Index: src/usr.sbin/config/config.y > % > =================================================================== > % RCS file: /usr/home/ncvs/src/usr.sbin/config/config.y,v > % retrieving revision 1.61 > % diff -u -r1.61 config.y > % --- src/usr.sbin/config/config.y 6 Jul 2003 02:00:52 -0000 1.61 > % +++ src/usr.sbin/config/config.y 27 Sep 2003 10:39:13 -0000 > % @@ -118,6 +118,9 @@ > % | > % Config_spec SEMICOLON > % | > % + INCLUDE ID SEMICOLON > % + = { include($2, 0); }; > % + | > % SEMICOLON > % | > % error SEMICOLON > % @@ -164,9 +167,7 @@ > % = { > % hints = $2; > % hintmode = 1; > % - } | > % - INCLUDE ID > % - = { include($2, 0); }; > % + } > % > % System_spec: > % CONFIG System_id System_parameter_list > > I lost your examples showing that this doesn't quite work. What is > the > problem with it? > > Another old bug is that using "include" defeats the point of the > INCLUDE_CONFIG_FILE option (since included files aren't expanded). > > Bruce > Bruce, Ah yes, my bad. It was a patching error. Specifically my error in not applying the patch but rather reading and carelessly assuming that Stefan had simply appended a SEMICOLON to the existing code, as his patch. :-) When I correctly apply Stefan's patch it fixes this problem however with Stefan's patch the code is essentially equivalent to mine, so, any caveats except those involving the absent SEMICOLON, still apply. For example the recursive include case you pointed out. Apologies for the misunderstanding. The offer to fix, or, perhaps in lieu of this thread, enhance /usr/sbin/config still stands if you'd like. It might be fun to take the suck out of /usr/sbin/config if people think it's warrented. :-) -- Adam - (http://people.migus.org/~amigus/) Migus Dot Org - (http://www.migus.org/) From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 12:30:13 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C12216A4BF for ; Wed, 8 Oct 2003 12:30:13 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 18B7043F85 for ; Wed, 8 Oct 2003 12:30:12 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from dhcp01.pn.xcllnt.net (dhcp01.pn.xcllnt.net [192.168.4.201]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h98JU6be051175; Wed, 8 Oct 2003 12:30:06 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) h98JU6Bl003253; Wed, 8 Oct 2003 12:30:06 -0700 (PDT) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.10/8.12.10/Submit) id h98JU6CR003252; Wed, 8 Oct 2003 12:30:06 -0700 (PDT) (envelope-from marcel) Date: Wed, 8 Oct 2003 12:30:06 -0700 From: Marcel Moolenaar To: "Adam C. Migus" Message-ID: <20031008193006.GD3007@dhcp01.pn.xcllnt.net> References: <20030927080420.N18558@gamplex.bde.org> <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> <20031008212302.T4729@gamplex.bde.org> <51574.204.254.155.35.1065640042.squirrel@mail.migus.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51574.204.254.155.35.1065640042.squirrel@mail.migus.org> User-Agent: Mutt/1.5.4i cc: Stefan Farfeleder cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 19:30:13 -0000 On Wed, Oct 08, 2003 at 03:07:22PM -0400, Adam C. Migus wrote: > > Apologies for the misunderstanding. The offer to fix, or, perhaps > in lieu of this thread, enhance /usr/sbin/config still stands if > you'd like. It might be fun to take the suck out of > /usr/sbin/config if people think it's warrented. :-) There's on old thread in the -current archives about dealing with mips and variations of an architecture (in the same sense that pc98 is a variation of i386). It would be cool to have that nailed down and have a config(8) that groks it. If you're working on config, maybe you can solve that problem too? :-) -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 12:47:56 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C87F16A4B3 for ; Wed, 8 Oct 2003 12:47:56 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 36A8343FDF for ; Wed, 8 Oct 2003 12:47:54 -0700 (PDT) (envelope-from adam@migus.org) Received: from garple.migus.org ([68.55.83.94]) by comcast.net (rwcrmhc13) with ESMTP id <2003100819475401500kf9fve>; Wed, 8 Oct 2003 19:47:54 +0000 Received: by garple.migus.org (Postfix, from userid 80) id ADBBE8FC51; Wed, 8 Oct 2003 15:47:53 -0400 (EDT) Received: from 204.254.155.35 (SquirrelMail authenticated user adam) by mail.migus.org with HTTP; Wed, 8 Oct 2003 15:47:53 -0400 (EDT) Message-ID: <51617.204.254.155.35.1065642473.squirrel@mail.migus.org> In-Reply-To: <20031008193006.GD3007@dhcp01.pn.xcllnt.net> References: <20030927080420.N18558@gamplex.bde.org> <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> <20031008212302.T4729@gamplex.bde.org> <51574.204.254.155.35.1065640042.squirrel@mail.migus.org> <20031008193006.GD3007@dhcp01.pn.xcllnt.net> Date: Wed, 8 Oct 2003 15:47:53 -0400 (EDT) From: "Adam C. Migus" To: "Marcel Moolenaar" User-Agent: SquirrelMail/1.4.1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal cc: Stefan Farfeleder cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 19:47:56 -0000 Marcel Moolenaar said: > On Wed, Oct 08, 2003 at 03:07:22PM -0400, Adam C. Migus wrote: >> >> Apologies for the misunderstanding. The offer to fix, or, perhaps >> in lieu of this thread, enhance /usr/sbin/config still stands if >> you'd like. It might be fun to take the suck out of >> /usr/sbin/config if people think it's warrented. :-) > > There's on old thread in the -current archives about dealing with > mips and variations of an architecture (in the same sense that pc98 > is a variation of i386). It would be cool to have that nailed down > and have a config(8) that groks it. If you're working on config, > maybe you can solve that problem too? :-) > > -- > Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net > Well I guess there are motivations to do some work on /usr/sbin/config then. Perhaps I should just start a new thread on what people think a reworked /usr/sbin/config should do and go from there. -- Adam - (http://people.migus.org/~amigus/) Migus Dot Org - (http://www.migus.org/) From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 14:36:28 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 39B2716A4B3 for ; Wed, 8 Oct 2003 14:36:28 -0700 (PDT) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9D4CC43FB1 for ; Wed, 8 Oct 2003 14:36:27 -0700 (PDT) (envelope-from adam@migus.org) Received: from garple.migus.org ([68.55.83.94]) by comcast.net (rwcrmhc11) with ESMTP id <2003100821362501300stvjce>; Wed, 8 Oct 2003 21:36:25 +0000 Received: by garple.migus.org (Postfix, from userid 80) id 849E98FC40; Wed, 8 Oct 2003 17:36:25 -0400 (EDT) Received: from 192.168.4.2 (SquirrelMail authenticated user adam) by mail.migus.org with HTTP; Wed, 8 Oct 2003 17:36:25 -0400 (EDT) Message-ID: <50402.192.168.4.2.1065648985.squirrel@mail.migus.org> Date: Wed, 8 Oct 2003 17:36:25 -0400 (EDT) From: "Adam C. Migus" To: arch@freebsd.org User-Agent: SquirrelMail/1.4.1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal Subject: update /usr/sbin/config? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 21:36:28 -0000 All, Based on discussions in the following thread: http://lists.freebsd.org/pipermail/freebsd-arch/2003-September/001319.html http://lists.freebsd.org/pipermail/freebsd-arch/2003-October/001362.html I'm requesting input for changes to /usr/sbin/config. While the aforementioned thread does contain a patch to fix an immediate bug, /usr/sbin/config does contain some other issues and short-comings (a couple of which are outlined in existing threads on arch@ and current@) that a minor reworking could fix. If I'm going to attack these already known issues I figure I should ask now if there are any others anyone else would like to address or if there are any features anyone would like to propose for discussion. As long as the idea doesn't involve a spot watch, python or contain the words "to just know," I'm open to pretty much anything anyone else is... :-) Ideally I'd like to pull a list of requirements from this thread when everyone's done. -- Adam - (http://people.migus.org/~amigus/) Migus Dot Org - (http://www.migus.org/) From owner-freebsd-arch@FreeBSD.ORG Thu Oct 9 04:13:48 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 12FEC16A4B3 for ; Thu, 9 Oct 2003 04:13:48 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1042043FDF for ; Thu, 9 Oct 2003 04:13:47 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9p1/8.12.9) with ESMTP id h99BDaAD077436; Thu, 9 Oct 2003 05:13:37 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 09 Oct 2003 05:13:38 -0600 (MDT) Message-Id: <20031009.051338.97397249.imp@bsdimp.com> To: marcel@xcllnt.net From: "M. Warner Losh" In-Reply-To: <20031008193006.GD3007@dhcp01.pn.xcllnt.net> References: <20031008212302.T4729@gamplex.bde.org> <51574.204.254.155.35.1065640042.squirrel@mail.migus.org> <20031008193006.GD3007@dhcp01.pn.xcllnt.net> X-Mailer: Mew version 2.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: stefan@fafoe.narf.at cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Oct 2003 11:13:48 -0000 In message: <20031008193006.GD3007@dhcp01.pn.xcllnt.net> Marcel Moolenaar writes: : On Wed, Oct 08, 2003 at 03:07:22PM -0400, Adam C. Migus wrote: : > : > Apologies for the misunderstanding. The offer to fix, or, perhaps : > in lieu of this thread, enhance /usr/sbin/config still stands if : > you'd like. It might be fun to take the suck out of : > /usr/sbin/config if people think it's warrented. :-) : : There's on old thread in the -current archives about dealing with : mips and variations of an architecture (in the same sense that pc98 : is a variation of i386). It would be cool to have that nailed down : and have a config(8) that groks it. If you're working on config, : maybe you can solve that problem too? :-) i386 is very special in that it has so few variants (pc98, sequent and roadrunner are the only ones I know, and outside of japan, all are rare). mips has boatloads, as does power pc, arm and sh[345]. alpha, amd64, sparc, sparc64 are very regular beasts in comparison. Also, it is more than a config issue. There will need to be some kernel source changes too. About 1/4 of the pc98 ifdefs could go away if the MACHINE/MACHINE_ARCH split were done more correctly for i386. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Oct 9 07:20:01 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 80EF016A4B3 for ; Thu, 9 Oct 2003 07:20:01 -0700 (PDT) Received: from sccrmhc13.comcast.net (sccrmhc13.comcast.net [204.127.202.64]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7B49C43FF5 for ; Thu, 9 Oct 2003 07:20:00 -0700 (PDT) (envelope-from adam@migus.org) Received: from garple.migus.org ([68.55.83.94]) by comcast.net (sccrmhc13) with ESMTP id <2003100914195901600a8kqie>; Thu, 9 Oct 2003 14:19:59 +0000 Received: by garple.migus.org (Postfix, from userid 80) id 54F208FC30; Thu, 9 Oct 2003 10:19:59 -0400 (EDT) Received: from 204.254.155.35 (SquirrelMail authenticated user adam) by mail.migus.org with HTTP; Thu, 9 Oct 2003 10:19:59 -0400 (EDT) Message-ID: <51676.204.254.155.35.1065709199.squirrel@mail.migus.org> In-Reply-To: <20031009.051338.97397249.imp@bsdimp.com> References: <20031008212302.T4729@gamplex.bde.org><51574.204.254.155.35.1065640042.squirrel@mail.migus.org><20031008193006.GD3007@dhcp01.pn.xcllnt.net> <20031009.051338.97397249.imp@bsdimp.com> Date: Thu, 9 Oct 2003 10:19:59 -0400 (EDT) From: "Adam C. Migus" To: "M. Warner Losh" User-Agent: SquirrelMail/1.4.2 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal cc: stefan@fafoe.narf.at cc: arch@freebsd.org cc: marcel@xcllnt.net Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Oct 2003 14:20:01 -0000 M. Warner Losh said: > In message: <20031008193006.GD3007@dhcp01.pn.xcllnt.net> > Marcel Moolenaar writes: > : On Wed, Oct 08, 2003 at 03:07:22PM -0400, Adam C. Migus wrote: > : > > : > Apologies for the misunderstanding. The offer to fix, or, > perhaps > : > in lieu of this thread, enhance /usr/sbin/config still stands if > : > you'd like. It might be fun to take the suck out of > : > /usr/sbin/config if people think it's warrented. :-) > : > : There's on old thread in the -current archives about dealing with > : mips and variations of an architecture (in the same sense that > pc98 > : is a variation of i386). It would be cool to have that nailed down > : and have a config(8) that groks it. If you're working on config, > : maybe you can solve that problem too? :-) > > i386 is very special in that it has so few variants (pc98, sequent > and > roadrunner are the only ones I know, and outside of japan, all are > rare). mips has boatloads, as does power pc, arm and sh[345]. > alpha, > amd64, sparc, sparc64 are very regular beasts in comparison. > > Also, it is more than a config issue. There will need to be some > kernel source changes too. About 1/4 of the pc98 ifdefs could go > away > if the MACHINE/MACHINE_ARCH split were done more correctly for i386. > > Warner > If someone wants to step up and coordinate efforts with me on this? It sounds reasonable to address in parallel. Is there anyone who'd volunteer to work with me on the non-config end or at least provide direction and expertise? I don't mind being the coordinator so as to minimize the person(s) time in dealing with the issue. -- Adam - (http://people.migus.org/~amigus/) Migus Dot Org - (http://www.migus.org/) From owner-freebsd-arch@FreeBSD.ORG Thu Oct 9 10:35:13 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5828A16A4B3 for ; Thu, 9 Oct 2003 10:35:13 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.bayarea.net [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E2F143F85 for ; Thu, 9 Oct 2003 10:35:12 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from dhcp01.pn.xcllnt.net (dhcp01.pn.xcllnt.net [192.168.4.201]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h99HYtbe057585; Thu, 9 Oct 2003 10:34:55 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) h99HYtBl006772; Thu, 9 Oct 2003 10:34:55 -0700 (PDT) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.10/8.12.10/Submit) id h99HYtS9006771; Thu, 9 Oct 2003 10:34:55 -0700 (PDT) (envelope-from marcel) Date: Thu, 9 Oct 2003 10:34:55 -0700 From: Marcel Moolenaar To: "Adam C. Migus" Message-ID: <20031009173455.GB6603@dhcp01.pn.xcllnt.net> References: <20031009.051338.97397249.imp@bsdimp.com> <51676.204.254.155.35.1065709199.squirrel@mail.migus.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51676.204.254.155.35.1065709199.squirrel@mail.migus.org> User-Agent: Mutt/1.5.4i cc: stefan@fafoe.narf.at cc: arch@freebsd.org cc: "M. Warner Losh" Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Oct 2003 17:35:13 -0000 On Thu, Oct 09, 2003 at 10:19:59AM -0400, Adam C. Migus wrote: > > > > Also, it is more than a config issue. There will need to be some > > kernel source changes too. About 1/4 of the pc98 ifdefs could go > > away > > if the MACHINE/MACHINE_ARCH split were done more correctly for i386. > > > > If someone wants to step up and coordinate efforts with me on this? > It sounds reasonable to address in parallel. Is there anyone who'd > volunteer to work with me on the non-config end or at least provide > direction and expertise? I don't mind being the coordinator so as > to minimize the person(s) time in dealing with the issue. I suggest you focus on the config(8) part first. Find out how best to specify what to build given that (MACHINE_ARCH,MACHINE) doesn't quite capture it. If it's backwards compatible then we don't actually have to reorganize the source tree yet, which allows you to finish your initial goal and also allows mips and powerpc to play with it a bit before we commit to it by moving pc98 code around. I can help out, but more on a theoretical front. You probably want people like nyan@, jmallet@, benno@, grehan@ or imp@ to validate concepts with reality or provide you with the input you need to sketch the problem space. -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-arch@FreeBSD.ORG Fri Oct 10 18:10:19 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 397F216A4C0 for ; Fri, 10 Oct 2003 18:10:19 -0700 (PDT) Received: from sccrmhc12.comcast.net (sccrmhc12.comcast.net [204.127.202.56]) by mx1.FreeBSD.org (Postfix) with ESMTP id A223643FE5 for ; Fri, 10 Oct 2003 18:10:16 -0700 (PDT) (envelope-from adam@migus.org) Received: from garple.migus.org ([68.55.83.94]) by comcast.net (sccrmhc12) with ESMTP id <20031011011015012004orfpe>; Sat, 11 Oct 2003 01:10:15 +0000 Received: by garple.migus.org (Postfix, from userid 80) id E31C28FC37; Fri, 10 Oct 2003 21:10:14 -0400 (EDT) Received: from 192.168.4.2 (SquirrelMail authenticated user adam) by mail.migus.org with HTTP; Fri, 10 Oct 2003 21:10:14 -0400 (EDT) Message-ID: <53701.192.168.4.2.1065834614.squirrel@mail.migus.org> In-Reply-To: <20031009032431.Y6050@gamplex.bde.org> References: <20030925092319.H5418@gamplex.bde.org><49939.204.254.155.35.1064593320.squirrel@mail.migus.org><20030927080420.N18558@gamplex.bde.org> <49955.192.168.4.2.1065074430.squirrel@mail.migus.org> <51310.204.254.155.35.1065622332.squirrel@mail.migus.org><51457.204.254.155.35.1065630793.squirrel@mail.migus.org> <20031009032431.Y6050@gamplex.bde.org> Date: Fri, 10 Oct 2003 21:10:14 -0400 (EDT) From: "Adam C. Migus" To: "Bruce Evans" User-Agent: SquirrelMail/1.4.2 MIME-Version: 1.0 Content-Type: multipart/mixed;boundary="----=_20031010211014_76716" X-Priority: 3 Importance: Normal cc: arch@freebsd.org Subject: Re: sys/conf/DEFAULT[S] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Oct 2003 01:10:19 -0000 ------=_20031010211014_76716 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit Bruce Evans said: > On Wed, 8 Oct 2003, Adam C. Migus wrote: > >> Bruce Evans said: >> > On Wed, 8 Oct 2003, Adam C. Migus wrote: >> >> ... >> >> WRT Stefan's solution, simply nest two includes on the first >> line, >> >> like so: >> >> >> >> head -1 KERNEL >> >> include SUB1 >> >> >> >> head -1 SUB1 >> >> include SUB2 >> >> >> >> This will cause config to fail in the same manner it did >> before, >> >> with his patch. >> > >> > Er, but this works for me. I test with SUB2 having the >> following: >> > 1) include GENERIC >> > 2) same contents as GENERIC >> > 3) same contents as GENERIC except for no comment lines at >> > beginning. > >> Here's my broken config (with Stefan's patch) compile, run, error, >> output and config exerpts: > > This still works for me :-). > >> root@caster:ttyp4:config# make clean >> rm -f config config.o main.o lang.o mkmakefile.o mkheaders.o >> mkoptions.o config.8.gz config.8.cat.gz lang.c config.c y.tab.c >> y.tab.h >> root@caster:ttyp4:config# make CFLAGS="`make -V CFLAGS` >> -DYYDEBUG=1" >> Warning: Object directory not changed from original >> /src/freebsd/CURRENT/src/usr.sbin/config >> yacc -d config.y >> cp y.tab.c config.c >> cc -O -pipe -mcpu=pentiumpro -I. >> -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers >> -Werror -DYYDEBUG=1 -c config.c >> cc -O -pipe -mcpu=pentiumpro -I. >> -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers >> -Werror -DYYDEBUG=1 -c main.c >> lex -t lang.l > lang.c >> cc -O -pipe -mcpu=pentiumpro -I. >> -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers >> -Werror -DYYDEBUG=1 -c lang.c >> cc -O -pipe -mcpu=pentiumpro -I. >> -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers >> -Werror -DYYDEBUG=1 -c mkmakefile.c >> cc -O -pipe -mcpu=pentiumpro -I. >> -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers >> -Werror -DYYDEBUG=1 -c mkheaders.c >> cc -O -pipe -mcpu=pentiumpro -I. >> -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers >> -Werror -DYYDEBUG=1 -c mkoptions.c >> cc -O -pipe -mcpu=pentiumpro -I. >> -I/src/freebsd/CURRENT/src/usr.sbin/config -Wsystem-headers >> -Werror -DYYDEBUG=1 -o config config.o main.o lang.o >> mkmakefile.o >> mkheaders.o mkoptions.o -ll >> gzip -cn config.8 > config.8.gz >> root@caster:ttyp4:config# cp config /tmp >> root@caster:ttyp4:config# cd ../../sys/i386/conf >> root@caster:ttyp4:conf# export YYDEBUG=1 >> root@caster:ttyp4:conf# /tmp/config DISKLESS_SMP_MAC >> yydebug: state 0, reducing by rule 3 (Many_specs :) >> yydebug: after reduction, shifting from state 0 to state 2 >> yydebug: state 2, reading 274 (INCLUDE) >> yydebug: state 2, shifting to state 19 >> yydebug: state 19, reading 275 (ID) >> yydebug: state 19, shifting to state 46 >> yydebug: state 46, reading 273 (SEMICOLON) >> yydebug: state 46, shifting to state 56 >> yydebug: state 56, reducing by rule 20 (Config_spec : INCLUDE ID >> SEMICOLON) > > This seems to be from a patching error. Stefan's patch moved > "INCLUDE ID SEMICOLON" from Config_spec to Spec. > >> yydebug: after reduction, shifting from state 2 to state 22 >> yydebug: state 22, reading 274 (INCLUDE) >> config: SMP_MAC:1: syntax error >> root@caster:ttyp4:conf# head -3 DISKLESS_SMP_MAC SMP_MAC MAC >> ==> DISKLESS_SMP_MAC <== >> include SMP_MAC >> >> ident DISKLESS_SMP_MAC >> >> ==> SMP_MAC <== >> include MAC >> >> #ident SMP_MAC >> >> ==> MAC <== >> # >> # MAC -- Generic kernel configuration file for FreeBSD/i386 w/MAC >> # > > My debugging output shows the move: > > %%% > yydebug: state 0, reducing by rule 3 (Many_specs :) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 274 (INCLUDE) > yydebug: state 2, shifting to state 19 > yydebug: state 19, reading 275 (ID) > yydebug: state 19, shifting to state 46 > yydebug: state 46, reading 273 (SEMICOLON) > yydebug: state 46, shifting to state 56 > yydebug: state 56, reducing by rule 6 (Spec : INCLUDE ID SEMICOLON) > ^^^^^^ ^^^^ > > [The first difference is here] > > yydebug: after reduction, shifting from state 2 to state 20 > yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 274 (INCLUDE) > yydebug: state 2, shifting to state 19 > yydebug: state 19, reading 275 (ID) > yydebug: state 19, shifting to state 46 > yydebug: state 46, reading 273 (SEMICOLON) > yydebug: state 46, shifting to state 56 > yydebug: state 56, reducing by rule 6 (Spec : INCLUDE ID SEMICOLON) > yydebug: after reduction, shifting from state 2 to state 20 > yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 273 (SEMICOLON) > yydebug: state 2, shifting to state 18 > yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) > yydebug: after reduction, shifting from state 2 to state 20 > yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 273 (SEMICOLON) > yydebug: state 2, shifting to state 18 > yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) > yydebug: after reduction, shifting from state 2 to state 20 > yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 273 (SEMICOLON) > yydebug: state 2, shifting to state 18 > yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) > yydebug: after reduction, shifting from state 2 to state 20 > yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 273 (SEMICOLON) > yydebug: state 2, shifting to state 18 > yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) > yydebug: after reduction, shifting from state 2 to state 20 > yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 273 (SEMICOLON) > yydebug: state 2, shifting to state 18 > yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) > yydebug: after reduction, shifting from state 2 to state 20 > yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 273 (SEMICOLON) > yydebug: state 2, shifting to state 18 > yydebug: state 18, reducing by rule 7 (Spec : SEMICOLON) > yydebug: after reduction, shifting from state 2 to state 20 > yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 266 (IDENT) > yydebug: state 2, shifting to state 11 > yydebug: state 11, reading 275 (ID) > yydebug: state 11, shifting to state 35 > yydebug: state 35, reducing by rule 15 (Config_spec : IDENT ID) > yydebug: after reduction, shifting from state 2 to state 22 > yydebug: state 22, reading 273 (SEMICOLON) > yydebug: state 22, shifting to state 48 > yydebug: state 48, reducing by rule 5 (Spec : Config_spec SEMICOLON) > yydebug: after reduction, shifting from state 2 to state 20 > yydebug: state 20, reducing by rule 2 (Many_specs : Many_specs Spec) > yydebug: after reduction, shifting from state 0 to state 2 > yydebug: state 2, reading 0 (end-of-file) > yydebug: state 2, reducing by rule 1 (Configuration : Many_specs) > yydebug: after reduction, shifting from state 0 to state 1 > Specify machine type, e.g. ``machine i386'' > %%% > > Bruce > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to > "freebsd-arch-unsubscribe@freebsd.org" > Bruce, After all the confusion here's a patch that will address and fix the the SEMICOLON and nesting issues and in addition fix the problem you had with file names. It does so by introducing a PATH type. So 'include' can now be used for things like "DEFAULTS.i386" (with or without quotes). -- Adam - (http://people.migus.org/~amigus/) Migus Dot Org - (http://www.migus.org/) ------=_20031010211014_76716 Content-Type: application/octet-stream; name="config.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="config.diff" ZGlmZiAtcnUzIGNvbmZpZy9jb25maWcueSBjb25maWcucGF0Y2gvY29uZmlnLnkKLS0tIGNvbmZp Zy9jb25maWcueQlTYXQgSnVsICA1IDIyOjAwOjUyIDIwMDMKKysrIGNvbmZpZy5wYXRjaC9jb25m aWcueQlGcmkgT2N0IDEwIDIwOjU5OjE3IDIwMDMKQEAgLTIzLDYgKzIzLDcgQEAKICV0b2tlbglT RU1JQ09MT04KICV0b2tlbglJTkNMVURFCiAKKyV0b2tlbgk8c3RyPglQQVRICiAldG9rZW4JPHN0 cj4JSUQKICV0b2tlbgk8dmFsPglOVU1CRVIKIApAQCAtMTE4LDYgKzExOSw4IEBACiAJCXwKIAlD b25maWdfc3BlYyBTRU1JQ09MT04KIAkJfAorCUluY2x1ZGUKKwkJfAogCVNFTUlDT0xPTgogCQl8 CiAJZXJyb3IgU0VNSUNPTE9OCkBAIC0xNjQsOSArMTY3LDcgQEAKIAkgICAgICA9IHsKIAkJICAg ICAgaGludHMgPSAkMjsKIAkJICAgICAgaGludG1vZGUgPSAxOwotCSAgICAgICAgfSB8Ci0JSU5D TFVERSBJRAotCSAgICAgID0geyBpbmNsdWRlKCQyLCAwKTsgfTsKKwkgICAgICAgIH07CiAKIFN5 c3RlbV9zcGVjOgogCUNPTkZJRyBTeXN0ZW1faWQgU3lzdGVtX3BhcmFtZXRlcl9saXN0CkBAIC0y NjQsNiArMjY1LDEyIEBACiAJCS8qIGFuZCB0aGUgZGV2aWNlIHBhcnQgKi8KIAkJcm1kZXYoJDIp OwogCQl9IDsKKworSW5jbHVkZToKKwlJTkNMVURFIFBBVEggU0VNSUNPTE9OCisJICAgICAgPSB7 IGluY2x1ZGUoJDIsIDApOyB9IHwKKwlJTkNMVURFIElEIFNFTUlDT0xPTgorCSAgICAgID0geyBp bmNsdWRlKCQyLCAwKTsgfTsKIAogJSUKIApkaWZmIC1ydTMgY29uZmlnL2xhbmcubCBjb25maWcu cGF0Y2gvbGFuZy5sCi0tLSBjb25maWcvbGFuZy5sCVNhdCBTZXAgMjcgMDI6Mzk6MTYgMjAwMwor KysgY29uZmlnLnBhdGNoL2xhbmcubAlGcmkgT2N0IDEwIDIwOjU5OjMwIDIwMDMKQEAgLTkyLDYg KzkyLDcgQEAKIGludCB5eWVycm9yKGNvbnN0IGNoYXIgKik7CiAKICV9CitQQVRICVtBLVphLXpf MC05XSpbLi9dWy1BLVphLXpfMC05Li9dKgogV09SRAlbQS1aYS16X11bLUEtWmEtel9dKgogSUQJ W0EtWmEtel9dWy1BLVphLXpfMC05XSoKICVTVEFSVCBOT05VTSBUT0VPTApAQCAtMTE2LDEwICsx MTcsMjggQEAKIAkJCQlCRUdJTiBOT05VTTsKIAkJCXJldHVybiBpOwogCQl9Cis8SU5JVElBTD57 UEFUSH0geworCQkJQkVHSU4gMDsKKwkJCXl5bHZhbC5zdHIgPSBzdHJkdXAoeXl0ZXh0KTsKKwkJ CXJldHVybiBQQVRIOworCQl9CiA8SU5JVElBTD57SUR9CXsKIAkJCUJFR0lOIDA7CiAJCQl5eWx2 YWwuc3RyID0gc3RyZHVwKHl5dGV4dCk7CiAJCQlyZXR1cm4gSUQ7CisJCX0KK1xcXCJ7UEFUSH1c XFwiCXsKKwkJCUJFR0lOIDA7CisJCQl5eXRleHRbeXlsZW5nLTJdID0gJyInOworCQkJeXl0ZXh0 W3l5bGVuZy0xXSA9ICdcMCc7CisJCQl5eWx2YWwuc3RyID0gc3RyZHVwKHl5dGV4dCArIDEpOwor CQkJcmV0dXJuIFBBVEg7CisJCX0KK1wie1BBVEh9XCIJeworCQkJQkVHSU4gMDsKKwkJCXl5dGV4 dFt5eWxlbmctMV0gPSAnXDAnOworCQkJeXlsdmFsLnN0ciA9IHN0cmR1cCh5eXRleHQgKyAxKTsK KwkJCXJldHVybiBQQVRIOwogCQl9CiBcXFwiW14iXStcXFwiCXsKIAkJCUJFR0lOIDA7Cg== ------=_20031010211014_76716--