From owner-freebsd-arch@FreeBSD.ORG Sun Jan 10 01:09:50 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0FBF106568D; Sun, 10 Jan 2010 01:09:50 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-fx0-f227.google.com (mail-fx0-f227.google.com [209.85.220.227]) by mx1.freebsd.org (Postfix) with ESMTP id D7F938FC15; Sun, 10 Jan 2010 01:09:49 +0000 (UTC) Received: by fxm27 with SMTP id 27so5720415fxm.3 for ; Sat, 09 Jan 2010 17:09:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=HBAhMQU4OjwEd5hNy/YRwKWDD4m97B5kvV0mBe3cqFg=; b=WspFmjHnOsrzpzoThKL+/55BEtVK7pUP13B6EjaE7lg1Lj+ZfOJ7KQxIOj8c7D8YtT 78tUxY2nBFYxnWCzbFuEyPtt/yiqr4/gpMSZQCEPmBvury8H4qLsTp0JxzHtSzZZQUzc FKxQXOC8g2oRzBhiAcLGNNo5cOaz/KcmBehuE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=XmXZX7XQbGZxBNjbB1RHkPeLRwxHoyXrQzE2UT4xJJg78fm+iiXX/sqyXP43+jsahy LXsF+4pDniTgDhyGxt25fJTOhBC/EwRQRG/M1qr60PS5J1E7d8rbdk9A4ib54jAWEFyP FHNu6zfr/76LlMF+RndHMyWKTalkbcq4CazRk= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.57.66 with SMTP id b2mr4625771fah.33.1263085783975; Sat, 09 Jan 2010 17:09:43 -0800 (PST) Date: Sun, 10 Jan 2010 02:09:43 +0100 X-Google-Sender-Auth: 1b92f40240ce688b Message-ID: <3bbf2fe11001091709t228c48cft1c17af686e9e9c46@mail.gmail.com> From: Attilio Rao To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: [PATCH] kthread_{suspend, resume, suspend_check} locking bugs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jan 2010 01:09:50 -0000 I think that routines kthread_suspend(), kthread_resume() and kthread_suspend_check() are not adeguately SMP protected. That is because, in particular, the critical path doesn't protect, together, TDF_KTH_SUSP and sleeping activity. The right pattern should be to use the thread lock spinlock as an interlock and use msleep. Such bugs have not been revealed probabilly because there has been a lack of testing of such primitives and there are not, currently, consumers within our stock kernel. Additively, kthread_suspend_check() seems to require to always pass curthread, which is silly (as we don't have to conform to any particular KPI), thus I think it is appropriate for the prototype to change. The following patch should fix the issue: http://www.freebsd.org/~attilio/kthread.diff That patch would have benefit from the priority argument (for PDROP) for msleep_spin(9), and I don't understand why we don't support it (the log message doesn't seem much clearer). If nobody objects, after this patch goes in I would add the priority argument to msleep_spin() too. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Mon Jan 11 07:01:12 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06AB0106566C; Mon, 11 Jan 2010 07:01:12 +0000 (UTC) (envelope-from Michael.Grant@students.wits.ac.za) Received: from viruswall.wits.ac.za (viruswall.wits.ac.za [146.141.15.4]) by mx1.freebsd.org (Postfix) with ESMTP id 829208FC08; Mon, 11 Jan 2010 07:01:11 +0000 (UTC) Received: from [146.141.15.4] (helo=imss.wits.ac.za) by viruswall.wits.ac.za with esmtp (Exim 4.42) id 1NUDho-0005oB-Ry; Mon, 11 Jan 2010 08:24:48 +0200 Received: from imss.wits.ac.za (localhost [127.0.0.1]) by localhost.wits.ac.za (Postfix) with ESMTP id 9C6A0211CB; Mon, 11 Jan 2010 08:24:48 +0200 (SAST) Received: from spmta01.inet.wits.ac.za (studentmail.wits.ac.za [146.141.13.132]) by imss.wits.ac.za (Postfix) with ESMTP id 77D36211B1; Mon, 11 Jan 2010 08:24:48 +0200 (SAST) MIME-version: 1.0 Content-transfer-encoding: 7bit Content-disposition: inline Content-type: text/plain; charset="us-ascii" Received: from students.wits.ac.za ([10.10.142.20]) by spmta01.inet.wits.ac.za (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) with ESMTP id <0KW200JM5KHDHU40@spmta01.inet.wits.ac.za>; Mon, 11 Jan 2010 08:24:49 +0200 (SAST) Received: from [10.10.101.20] by spmailstr01.inet.wits.ac.za (mshttpd); Mon, 11 Jan 2010 08:40:26 +0200 From: Michael Grant To: freebsd-bugs@freebsd.org, freebsd-arch@freebsd.org Message-id: Date: Mon, 11 Jan 2010 08:40:26 +0200 X-Mailer: Sun Java(tm) System Messenger Express 6.3-6.03 (built Mar 14 2008; 32bit) Content-language: en X-Accept-Language: en Priority: normal Cc: Subject: New USB ums driver returns incorrect value on probe X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2010 07:01:12 -0000 Good day, I am trying to port an excellent driver for a HID peripheral to the new USB stack on 8.0 and noticed some strange behaviour: The device, by default, operates as a mouse and only once a sensible driver communicates to it will the device enter an extended mode. The interesting thing is that this device is never given to the driver to be probed. I notice in the DEVICE_PROBE(9) man page: "If a success code of zero is returned, the driver can assume that it will be the one attached". Now looking in /usr/src/sys/dev/usb/input/ums.c if ((uaa->info.bInterfaceSubClass == UISUBCLASS_BOOT) && (uaa->info.bInterfaceProtocol == UIPROTO_MOUSE)) return (0); I think the return value should be BUS_PROBE_DEFAULT and not 0, after all the ums driver is generic. I have not submitted a PR, should I do that as well or is a post to this list sufficient? Kind regards, Mike Grant.

This communication is intended for the addressee only. It is confidential. If you have received this communication in error, please notify us immediately and destroy the original message. You may not copy or disseminate this communication without the permission of the University. Only authorized signatories are competent to enter into agreements on behalf of the University and recipients are thus advised that the content of this message may not be legally binding on the University and may contain the personal views and opinions of the author, which are not necessarily the views and opinions of The University of the Witwatersrand, Johannesburg. All agreements between the University and outsiders are subject to South African Law unless the University agrees in writing to the contrary.

From owner-freebsd-arch@FreeBSD.ORG Mon Jan 11 08:10:15 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E043106566C; Mon, 11 Jan 2010 08:10:15 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe10.tele2.se [212.247.155.33]) by mx1.freebsd.org (Postfix) with ESMTP id 948CA8FC14; Mon, 11 Jan 2010 08:10:14 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.0 c=1 a=7b-hCywwsQoA:10 a=MnI1ikcADjEx7bvsp0jZvQ==:17 a=kZdMSgTYvZJcta6hoWwA:9 a=KWypLC0h7pELgbFwhtCYdqhLRjsA:4 Received: from [188.126.201.140] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe10.swip.net (CommuniGate Pro SMTP 5.2.16) with ESMTPA id 1161889717; Mon, 11 Jan 2010 09:10:11 +0100 From: Hans Petter Selasky To: freebsd-arch@freebsd.org Date: Mon, 11 Jan 2010 09:08:49 +0100 User-Agent: KMail/1.12.4 (FreeBSD/8.0-STABLE; KDE/4.3.4; amd64; ; ) References: In-Reply-To: X-Face: +~\`s("[*|O,="7?X@L.elg*F"OA\I/3%^p8g?ab%RN'(; _IjlA: hGE..Ew, XAQ*o#\/M~SC=S1-f9{EzRfT'|Hhll5Q]ha5Bt-s|oTlKMusi:1e[wJl}kd}GR Z0adGx-x_0zGbZj'e(Y[(UNle~)8CQWXW@:DX+9)_YlB[tIccCPN$7/L' MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201001110908.49441.hselasky@c2i.net> Cc: Michael Grant , freebsd-bugs@freebsd.org Subject: Re: New USB ums driver returns incorrect value on probe X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2010 08:10:15 -0000 On Monday 11 January 2010 07:40:26 Michael Grant wrote: > Good day, > > I am trying to port an excellent driver for a HID peripheral to the new USB > stack on 8.0 and noticed some strange behaviour: The device, by default, > operates as a mouse and only once a sensible driver communicates to it > will the device enter an extended mode. The interesting thing is that > this device is never given to the driver to be probed. > > I notice in the DEVICE_PROBE(9) man page: "If a success code of zero is > returned, the driver can assume that it will be the one attached". > > Now looking in /usr/src/sys/dev/usb/input/ums.c > > if ((uaa->info.bInterfaceSubClass == UISUBCLASS_BOOT) && > (uaa->info.bInterfaceProtocol == UIPROTO_MOUSE)) > return (0); > > I think the return value should be BUS_PROBE_DEFAULT and not 0, after all > the ums driver is generic. > > I have not submitted a PR, should I do that as well or is a post to this > list sufficient? > if ((uaa->info.bInterfaceSubClass == UISUBCLASS_BOOT) && (uaa->info.bInterfaceProtocol == UIPROTO_MOUSE)) return (BUS_PROBE_GENERIC); This has been fixed in newer source code, 8+9 I believe. --HPS From owner-freebsd-arch@FreeBSD.ORG Mon Jan 11 11:06:53 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 245981065697 for ; Mon, 11 Jan 2010 11:06:53 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id EED9B8FC24 for ; Mon, 11 Jan 2010 11:06:52 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o0BB6qjP034589 for ; Mon, 11 Jan 2010 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o0BB6qhh034587 for freebsd-arch@FreeBSD.org; Mon, 11 Jan 2010 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 11 Jan 2010 11:06:52 GMT Message-Id: <201001111106.o0BB6qhh034587@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jan 2010 11:06:53 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 00:15:21 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9312A106568B for ; Wed, 13 Jan 2010 00:15:21 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id 07CB18FC17 for ; Wed, 13 Jan 2010 00:15:20 +0000 (UTC) Received: (qmail 24428 invoked by uid 399); 13 Jan 2010 00:15:20 -0000 Received: from localhost (HELO ?192.168.0.110?) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 13 Jan 2010 00:15:20 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4B4D109A.5060500@FreeBSD.org> Date: Tue, 12 Jan 2010 16:15:22 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0 MIME-Version: 1.0 To: "M. Warner Losh" References: <201001101744.o0AHiM2d095302@svn.freebsd.org> <20100110.210204.787670930858346133.imp@bsdimp.com> In-Reply-To: <20100110.210204.787670930858346133.imp@bsdimp.com> X-Enigmail-Version: 1.0 OpenPGP: id=D5B2F0FB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 00:15:21 -0000 On 1/10/2010 8:02 PM, M. Warner Losh wrote: > In message: > Doug Barton writes: > : On Sun, 10 Jan 2010, Warner Losh wrote: > : > : > Author: imp > : > Date: Sun Jan 10 17:44:22 2010 > : > New Revision: 202019 > : > URL: http://svn.freebsd.org/changeset/base/202019 > : > > : > Log: > : > Add INCLUDE_CONFIG_FILE in GENERIC on all non-embedded platforms. > : > : Thanks for doing this, however the comment about how to include the > : whole file (including the comments) were not included. Do I need to do > : this part of it myself? No problem if so, I just want to be sure to > : get it done in time to MFC it before the freeze for 7.3-release. > > In general, we don't put big comments like that in the config files, > preferring to leave them to NOTES. I was just following that > convention... Understood, however given that there is plenty of room for confusion on this point because the default is NOT to include the comments I think that some explanation is justified. My original text was: # Store the plain version of the configuration file in the kernel itself. # To store the entire file, including comments, put this in /etc/src.conf: # CONFIGARGS= -C # See config(8) for more details. I'm open to suggestions on shrinking it, but I do think some sort of explanation is warranted. Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 00:49:58 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40B6F106566C; Wed, 13 Jan 2010 00:49:58 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id F343A8FC0A; Wed, 13 Jan 2010 00:49:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o0D0gVUc081359; Tue, 12 Jan 2010 17:42:31 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Tue, 12 Jan 2010 17:43:26 -0700 (MST) Message-Id: <20100112.174326.337739863389869251.imp@bsdimp.com> To: dougb@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <4B4D109A.5060500@FreeBSD.org> References: <20100110.210204.787670930858346133.imp@bsdimp.com> <4B4D109A.5060500@FreeBSD.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 00:49:58 -0000 In message: <4B4D109A.5060500@FreeBSD.org> Doug Barton writes: : On 1/10/2010 8:02 PM, M. Warner Losh wrote: : > In message: : > Doug Barton writes: : > : On Sun, 10 Jan 2010, Warner Losh wrote: : > : : > : > Author: imp : > : > Date: Sun Jan 10 17:44:22 2010 : > : > New Revision: 202019 : > : > URL: http://svn.freebsd.org/changeset/base/202019 : > : > : > : > Log: : > : > Add INCLUDE_CONFIG_FILE in GENERIC on all non-embedded platforms. : > : : > : Thanks for doing this, however the comment about how to include the : > : whole file (including the comments) were not included. Do I need to do : > : this part of it myself? No problem if so, I just want to be sure to : > : get it done in time to MFC it before the freeze for 7.3-release. : > : > In general, we don't put big comments like that in the config files, : > preferring to leave them to NOTES. I was just following that : > convention... : : Understood, however given that there is plenty of room for confusion on : this point because the default is NOT to include the comments I think : that some explanation is justified. My original text was: : : # Store the plain version of the configuration file in the kernel itself. : # To store the entire file, including comments, put this in /etc/src.conf: : # CONFIGARGS= -C : # See config(8) for more details. : : I'm open to suggestions on shrinking it, but I do think some sort of : explanation is warranted. I'm not sure I see where there's confusion possible here, let alone plenty of room for it. Do you think you can describe what confusion is possible here? Warner From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 16:05:35 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D7AC106568B; Wed, 13 Jan 2010 16:05:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 59B958FC1C; Wed, 13 Jan 2010 16:05:35 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id DAD3746B39; Wed, 13 Jan 2010 11:05:34 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 2E9A98A01F; Wed, 13 Jan 2010 11:05:22 -0500 (EST) From: John Baldwin To: Doug Barton Date: Wed, 13 Jan 2010 08:44:21 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20091231; KDE/4.3.1; amd64; ; ) References: <201001101744.o0AHiM2d095302@svn.freebsd.org> <20100110.210204.787670930858346133.imp@bsdimp.com> <4B4D109A.5060500@FreeBSD.org> In-Reply-To: <4B4D109A.5060500@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201001130844.21241.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 13 Jan 2010 11:05:22 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 16:05:35 -0000 On Tuesday 12 January 2010 7:15:22 pm Doug Barton wrote: > On 1/10/2010 8:02 PM, M. Warner Losh wrote: > > In message: > > Doug Barton writes: > > : On Sun, 10 Jan 2010, Warner Losh wrote: > > : > > : > Author: imp > > : > Date: Sun Jan 10 17:44:22 2010 > > : > New Revision: 202019 > > : > URL: http://svn.freebsd.org/changeset/base/202019 > > : > > > : > Log: > > : > Add INCLUDE_CONFIG_FILE in GENERIC on all non-embedded platforms. > > : > > : Thanks for doing this, however the comment about how to include the > > : whole file (including the comments) were not included. Do I need to do > > : this part of it myself? No problem if so, I just want to be sure to > > : get it done in time to MFC it before the freeze for 7.3-release. > > > > In general, we don't put big comments like that in the config files, > > preferring to leave them to NOTES. I was just following that > > convention... > > Understood, however given that there is plenty of room for confusion on > this point because the default is NOT to include the comments I think > that some explanation is justified. My original text was: > > # Store the plain version of the configuration file in the kernel itself. > # To store the entire file, including comments, put this in /etc/src.conf: > # CONFIGARGS= -C > # See config(8) for more details. Just put this in sys/conf/NOTES. Users already know to look in NOTES for further details about kernel options. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 16:24:06 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 732561065692; Wed, 13 Jan 2010 16:24:06 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 21D868FC1E; Wed, 13 Jan 2010 16:24:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o0DGIHht092123; Wed, 13 Jan 2010 09:18:17 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Wed, 13 Jan 2010 09:19:13 -0700 (MST) Message-Id: <20100113.091913.1023414010414027273.imp@bsdimp.com> To: jhb@freebsd.org From: "M. Warner Losh" In-Reply-To: <201001130844.21241.jhb@freebsd.org> References: <20100110.210204.787670930858346133.imp@bsdimp.com> <4B4D109A.5060500@FreeBSD.org> <201001130844.21241.jhb@freebsd.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, dougb@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 16:24:06 -0000 In message: <201001130844.21241.jhb@freebsd.org> John Baldwin writes: : On Tuesday 12 January 2010 7:15:22 pm Doug Barton wrote: : > On 1/10/2010 8:02 PM, M. Warner Losh wrote: : > > In message: : > > Doug Barton writes: : > > : On Sun, 10 Jan 2010, Warner Losh wrote: : > > : : > > : > Author: imp : > > : > Date: Sun Jan 10 17:44:22 2010 : > > : > New Revision: 202019 : > > : > URL: http://svn.freebsd.org/changeset/base/202019 : > > : > : > > : > Log: : > > : > Add INCLUDE_CONFIG_FILE in GENERIC on all non-embedded platforms. : > > : : > > : Thanks for doing this, however the comment about how to include the : > > : whole file (including the comments) were not included. Do I need to do : > > : this part of it myself? No problem if so, I just want to be sure to : > > : get it done in time to MFC it before the freeze for 7.3-release. : > > : > > In general, we don't put big comments like that in the config files, : > > preferring to leave them to NOTES. I was just following that : > > convention... : > : > Understood, however given that there is plenty of room for confusion on : > this point because the default is NOT to include the comments I think : > that some explanation is justified. My original text was: : > : > # Store the plain version of the configuration file in the kernel itself. : > # To store the entire file, including comments, put this in /etc/src.conf: : > # CONFIGARGS= -C : > # See config(8) for more details. : : Just put this in sys/conf/NOTES. Users already know to look in NOTES for : further details about kernel options. This exact comment already is in NOTES. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 18:49:05 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B9DE1065693 for ; Wed, 13 Jan 2010 18:49:05 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id B6E968FC25 for ; Wed, 13 Jan 2010 18:49:04 +0000 (UTC) Received: (qmail 14521 invoked by uid 399); 13 Jan 2010 18:49:03 -0000 Received: from localhost (HELO foreign.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 13 Jan 2010 18:49:03 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4B4E1586.7090102@FreeBSD.org> Date: Wed, 13 Jan 2010 10:48:38 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.5) Gecko/20100112 Thunderbird/3.0 MIME-Version: 1.0 To: "M. Warner Losh" References: <20100110.210204.787670930858346133.imp@bsdimp.com> <4B4D109A.5060500@FreeBSD.org> <20100112.174326.337739863389869251.imp@bsdimp.com> In-Reply-To: <20100112.174326.337739863389869251.imp@bsdimp.com> X-Enigmail-Version: 1.0 OpenPGP: id=D5B2F0FB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 18:49:05 -0000 On 01/12/10 16:43, M. Warner Losh wrote: > In message: <4B4D109A.5060500@FreeBSD.org> > Doug Barton writes: > : On 1/10/2010 8:02 PM, M. Warner Losh wrote: > : > In message: > : > Doug Barton writes: > : > : On Sun, 10 Jan 2010, Warner Losh wrote: > : > : > : > : > Author: imp > : > : > Date: Sun Jan 10 17:44:22 2010 > : > : > New Revision: 202019 > : > : > URL: http://svn.freebsd.org/changeset/base/202019 > : > : > > : > : > Log: > : > : > Add INCLUDE_CONFIG_FILE in GENERIC on all non-embedded platforms. > : > : > : > : Thanks for doing this, however the comment about how to include the > : > : whole file (including the comments) were not included. Do I need to do > : > : this part of it myself? No problem if so, I just want to be sure to > : > : get it done in time to MFC it before the freeze for 7.3-release. > : > > : > In general, we don't put big comments like that in the config files, > : > preferring to leave them to NOTES. I was just following that > : > convention... > : > : Understood, however given that there is plenty of room for confusion on > : this point because the default is NOT to include the comments I think > : that some explanation is justified. My original text was: > : > : # Store the plain version of the configuration file in the kernel itself. > : # To store the entire file, including comments, put this in /etc/src.conf: > : # CONFIGARGS= -C > : # See config(8) for more details. > : > : I'm open to suggestions on shrinking it, but I do think some sort of > : explanation is warranted. > > I'm not sure I see where there's confusion possible here, let alone > plenty of room for it. Do you think you can describe what confusion > is possible here? I think that most users would expect that the actual config file is included, with the comments; as opposed to the stripped down version with just the actual lines of configuration information that is actually stored by default. Not only do I think it's obvious that this is what users would think, this exact issue came up in the discussion on -current in December. To address the other responses, Tom, sorry, your suggested text doesn't address my concern. John, I don't think that users would somehow magically know to look in NOTES for more information about an option that is already in GENERIC. In the interests of bringing this to a close: # Store the plain version of the configuration file in the kernel itself. # For information on extraction, and storing the comments also, see config(8). There are plenty of comments in GENERIC that are longer/more substantial than that, and let's be serious for a minute, IT DOESN'T MATTER ANYWAY. I'm sorry if adding a comment that is slightly larger than usual to a kernel config file defiles someone's view of the purity of all things kernel, but let's try to take a step back and realize that NOT making things so cryptic might actually benefit the users. And yes, if you can't already tell, my patience is at an end for this. I "get" why it needs to move to GENERIC instead of DEFAULTS, especially since that's where I wanted to put it in the first place. But specifically to Warner, if you had in mind to do something other than to just move what I did to GENERIC you should have said that, and we could have avoided this whole stupid discussion. And generally, let's forget for a second that y'all are annoying the crap out of me, how many other people are you discouraging from participation because an issue as simple as this one is generating such an overwhelmingly out of proportion response? Unless someone objects to the TEXT of the comment I proposed in the next 24 hours I'll be committing it after that. Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 20:21:29 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F644106566B; Wed, 13 Jan 2010 20:21:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1E7468FC16; Wed, 13 Jan 2010 20:21:29 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id C21D246B17; Wed, 13 Jan 2010 15:21:28 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id A2A4D8A01F; Wed, 13 Jan 2010 15:21:25 -0500 (EST) From: John Baldwin To: Doug Barton Date: Wed, 13 Jan 2010 15:15:08 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20091231; KDE/4.3.1; amd64; ; ) References: <20100112.174326.337739863389869251.imp@bsdimp.com> <4B4E1586.7090102@FreeBSD.org> In-Reply-To: <4B4E1586.7090102@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201001131515.08602.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 13 Jan 2010 15:21:25 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 20:21:29 -0000 On Wednesday 13 January 2010 1:48:38 pm Doug Barton wrote: > To address the other responses, Tom, sorry, your suggested text doesn't > address my concern. John, I don't think that users would somehow > magically know to look in NOTES for more information about an option > that is already in GENERIC. You really think users do not already know to look in manpages or NOTES to find out more details about kernel options? Why bother having those documents if our users aren't going to look at them? Seriously, have you actually looked at GENERIC and NOTES and compared their contents. There are _many_ options and devices in GENERIC that have expanded descriptions in NOTES. This is the norm for documenting kernel options and has been for well over a decade. > In the interests of bringing this to a close: > # Store the plain version of the configuration file in the kernel itself. > # For information on extraction, and storing the comments also, see > config(8). > > There are plenty of comments in GENERIC that are longer/more substantial > than that, and let's be serious for a minute, IT DOESN'T MATTER ANYWAY. > I'm sorry if adding a comment that is slightly larger than usual to a > kernel config file defiles someone's view of the purity of all things > kernel, but let's try to take a step back and realize that NOT making > things so cryptic might actually benefit the users. So why not add a 3-line comment to GENERIC for every other kernel option? Put another way, what makes 'INCLUDE_CONFIG_FILE' sufficiently special that it deserves special treatment relative to other kernel options? -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 20:36:26 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15F391065692 for ; Wed, 13 Jan 2010 20:36:26 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id B08998FC20 for ; Wed, 13 Jan 2010 20:36:25 +0000 (UTC) Received: (qmail 15519 invoked by uid 399); 13 Jan 2010 20:36:24 -0000 Received: from localhost (HELO ?192.168.0.110?) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 13 Jan 2010 20:36:24 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4B4E2ECA.90905@FreeBSD.org> Date: Wed, 13 Jan 2010 12:36:26 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0 MIME-Version: 1.0 To: John Baldwin References: <20100112.174326.337739863389869251.imp@bsdimp.com> <4B4E1586.7090102@FreeBSD.org> <201001131515.08602.jhb@freebsd.org> In-Reply-To: <201001131515.08602.jhb@freebsd.org> X-Enigmail-Version: 1.0 OpenPGP: id=D5B2F0FB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 20:36:26 -0000 On 1/13/2010 12:15 PM, John Baldwin wrote: > On Wednesday 13 January 2010 1:48:38 pm Doug Barton wrote: >> To address the other responses, Tom, sorry, your suggested text doesn't >> address my concern. John, I don't think that users would somehow >> magically know to look in NOTES for more information about an option >> that is already in GENERIC. > > You really think users do not already know to look in manpages or NOTES to > find out more details about kernel options? That's not what I said. > Put > another way, what makes 'INCLUDE_CONFIG_FILE' sufficiently special that it > deserves special treatment relative to other kernel options? Because the default behavior (not including the actual file) for the option is contrary to user' reasonable expectation of how the option should work .... and now I'm repeating myself. Seriously, don't you have anything better to do than argue against including a comment in a config file? I know I do. What is the overwhelming horror that will arise here if there are more comments GENERIC than you deem to be absolutely necessary? And yes, I read the part of your message that I snipped about "why do we have these documents if users don't read them?" The answer is, that's why I'm suggesting a comment that tells users what man page to read. Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 21:35:51 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15D341065676; Wed, 13 Jan 2010 21:35:51 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D905D8FC08; Wed, 13 Jan 2010 21:35:50 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 6FD5746B2C; Wed, 13 Jan 2010 16:35:50 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id B92A08A01D; Wed, 13 Jan 2010 16:35:49 -0500 (EST) From: John Baldwin To: Doug Barton Date: Wed, 13 Jan 2010 16:33:09 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20091231; KDE/4.3.1; amd64; ; ) References: <201001131515.08602.jhb@freebsd.org> <4B4E2ECA.90905@FreeBSD.org> In-Reply-To: <4B4E2ECA.90905@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201001131633.09669.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 13 Jan 2010 16:35:49 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 21:35:51 -0000 On Wednesday 13 January 2010 3:36:26 pm Doug Barton wrote: > On 1/13/2010 12:15 PM, John Baldwin wrote: > > On Wednesday 13 January 2010 1:48:38 pm Doug Barton wrote: > >> To address the other responses, Tom, sorry, your suggested text doesn't > >> address my concern. John, I don't think that users would somehow > >> magically know to look in NOTES for more information about an option > >> that is already in GENERIC. > > > > You really think users do not already know to look in manpages or NOTES to > > find out more details about kernel options? > > That's not what I said. I don't think that users would [..] know to look in NOTES for more information about an option that is [...] in GENERIC. That seems really straight forward to me, or my English isn't good. I do think users "would know to look in NOTES for more information about an option that is in GENERIC". > > Put > > another way, what makes 'INCLUDE_CONFIG_FILE' sufficiently special that it > > deserves special treatment relative to other kernel options? > > Because the default behavior (not including the actual file) for the > option is contrary to user' reasonable expectation of how the option > should work .... and now I'm repeating myself. I think a better change would be to just change the default behavior of config(8) to do the reasonable thing. > Seriously, don't you have anything better to do than argue against > including a comment in a config file? I know I do. What is the > overwhelming horror that will arise here if there are more comments > GENERIC than you deem to be absolutely necessary? What is the overwhelming horror about keeping a file readable and allowing users to find extended documentation for INCLUDE_CONFIG_FILE in the same place that they find extended documentation about every other kernel option? > And yes, I read the part of your message that I snipped about "why do we > have these documents if users don't read them?" The answer is, that's > why I'm suggesting a comment that tells users what man page to read. I think adding comments that merely redirect the users to further documentation only serves to obfuscate. Left unchecked this approach will render files such as GENERIC with a very low signal-to-noise ratio making it harder to parse in a "big picture" way. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 21:37:24 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A922A1065670; Wed, 13 Jan 2010 21:37:24 +0000 (UTC) (envelope-from julian@elischer.org) Received: from utility-0.aerioconnect.net (utility-0.aerioconnect.net [216.240.32.11]) by mx1.freebsd.org (Postfix) with ESMTP id 6AAF98FC13; Wed, 13 Jan 2010 21:37:24 +0000 (UTC) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by utility-0.aerioconnect.net (8.13.1/8.13.1) with ESMTP id o0DKSXBV011267; Wed, 13 Jan 2010 12:28:33 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 64EF72D6018; Wed, 13 Jan 2010 12:28:32 -0800 (PST) Message-ID: <4B4E2CEF.5030709@elischer.org> Date: Wed, 13 Jan 2010 12:28:31 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: John Baldwin References: <20100112.174326.337739863389869251.imp@bsdimp.com> <4B4E1586.7090102@FreeBSD.org> <201001131515.08602.jhb@freebsd.org> In-Reply-To: <201001131515.08602.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Doug Barton , svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 21:37:24 -0000 John Baldwin wrote: > On Wednesday 13 January 2010 1:48:38 pm Doug Barton wrote: >> To address the other responses, Tom, sorry, your suggested text doesn't >> address my concern. John, I don't think that users would somehow >> magically know to look in NOTES for more information about an option >> that is already in GENERIC. > > You really think users do not already know to look in manpages or NOTES to > find out more details about kernel options? how about a one line comment in GENERIC suggesting that people look at NOTES for more info. From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 21:55:10 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9913A106568D; Wed, 13 Jan 2010 21:55:10 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 6781C8FC0C; Wed, 13 Jan 2010 21:55:10 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 10B5A46B51; Wed, 13 Jan 2010 16:55:10 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 525DD8A01D; Wed, 13 Jan 2010 16:55:09 -0500 (EST) From: John Baldwin To: Julian Elischer Date: Wed, 13 Jan 2010 16:55:08 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20091231; KDE/4.3.1; amd64; ; ) References: <201001131515.08602.jhb@freebsd.org> <4B4E2CEF.5030709@elischer.org> In-Reply-To: <4B4E2CEF.5030709@elischer.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201001131655.08465.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 13 Jan 2010 16:55:09 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Doug Barton , svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 21:55:10 -0000 On Wednesday 13 January 2010 3:28:31 pm Julian Elischer wrote: > John Baldwin wrote: > > On Wednesday 13 January 2010 1:48:38 pm Doug Barton wrote: > >> To address the other responses, Tom, sorry, your suggested text doesn't > >> address my concern. John, I don't think that users would somehow > >> magically know to look in NOTES for more information about an option > >> that is already in GENERIC. > > > > You really think users do not already know to look in manpages or NOTES to > > find out more details about kernel options? > > how about a one line comment in GENERIC suggesting that people look at > NOTES for more info. There is already a large block comment at the top of GENERIC that gives people several other places to look for more detailed info on kernel options as a general rule: # # GENERIC -- Generic kernel configuration file for FreeBSD/amd64 # # For more information on this file, please read the config(5) manual page, # and/or the handbook section on Kernel Configuration Files: # # http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html # # The handbook is also available locally in /usr/share/doc/handbook # if you've installed the doc distribution, otherwise always see the # FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the # latest information. # # An exhaustive list of options and more detailed explanations of the # device lines is also present in the ../../conf/NOTES and NOTES files. # If you are in doubt as to the purpose or necessity of a line, check first # in NOTES. # # $FreeBSD: head/sys/amd64/conf/GENERIC 202019 2010-01-10 17:44:22Z imp $ I think the existing paragraph about NOTES in particular is sufficient without requiring additional detailed documentation for each option. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Jan 13 22:10:29 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82C4B1065676; Wed, 13 Jan 2010 22:10:29 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 34A088FC14; Wed, 13 Jan 2010 22:10:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o0DM3dAA096341; Wed, 13 Jan 2010 15:03:39 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Wed, 13 Jan 2010 15:04:35 -0700 (MST) Message-Id: <20100113.150435.650865766805848595.imp@bsdimp.com> To: jhb@freebsd.org From: "M. Warner Losh" In-Reply-To: <201001131633.09669.jhb@freebsd.org> References: <201001131515.08602.jhb@freebsd.org> <4B4E2ECA.90905@FreeBSD.org> <201001131633.09669.jhb@freebsd.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, dougb@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2010 22:10:29 -0000 In message: <201001131633.09669.jhb@freebsd.org> John Baldwin writes: : On Wednesday 13 January 2010 3:36:26 pm Doug Barton wrote: : > On 1/13/2010 12:15 PM, John Baldwin wrote: : > > On Wednesday 13 January 2010 1:48:38 pm Doug Barton wrote: : > >> To address the other responses, Tom, sorry, your suggested text doesn't : > >> address my concern. John, I don't think that users would somehow : > >> magically know to look in NOTES for more information about an option : > >> that is already in GENERIC. : > > : > > You really think users do not already know to look in manpages or NOTES to : > > find out more details about kernel options? : > : > That's not what I said. : : : I don't think that users would [..] know to look in NOTES for more information : about an option that is [...] in GENERIC. : : : That seems really straight forward to me, or my English isn't good. I do : think users "would know to look in NOTES for more information about an option : that is in GENERIC". Agreed. That's why I did what I did: I conformed to the usual practice. : > > Put : > > another way, what makes 'INCLUDE_CONFIG_FILE' sufficiently special that it : > > deserves special treatment relative to other kernel options? : > : > Because the default behavior (not including the actual file) for the : > option is contrary to user' reasonable expectation of how the option : > should work .... and now I'm repeating myself. : : I think a better change would be to just change the default behavior of : config(8) to do the reasonable thing. -C should be the default, and we should invent a new '--smaller-saved-config' option. : > Seriously, don't you have anything better to do than argue against : > including a comment in a config file? I know I do. What is the : > overwhelming horror that will arise here if there are more comments : > GENERIC than you deem to be absolutely necessary? : : What is the overwhelming horror about keeping a file readable and allowing : users to find extended documentation for INCLUDE_CONFIG_FILE in the same place : that they find extended documentation about every other kernel option? Yes. That's why I did what I did: to keep things readable. : > And yes, I read the part of your message that I snipped about "why do we : > have these documents if users don't read them?" The answer is, that's : > why I'm suggesting a comment that tells users what man page to read. : : I think adding comments that merely redirect the users to further : documentation only serves to obfuscate. Left unchecked this approach will : render files such as GENERIC with a very low signal-to-noise ratio making it : harder to parse in a "big picture" way. Yes. Basically, I'm annoyed too: Our users aren't idiots, and we shouldn't be treating them as such at every turn. If there are surprises with how INCLUDE_CONFIG_FILE behaves, we should work to make it better, not paper over it with a comment. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Jan 14 07:32:25 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B10C1106568B; Thu, 14 Jan 2010 07:32:24 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-fx0-f227.google.com (mail-fx0-f227.google.com [209.85.220.227]) by mx1.freebsd.org (Postfix) with ESMTP id E6BF68FC15; Thu, 14 Jan 2010 07:32:22 +0000 (UTC) Received: by fxm27 with SMTP id 27so525861fxm.3 for ; Wed, 13 Jan 2010 23:32:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=K6iODhB5ZuLwETp20LHwGnw35ZM9QS3R91jRnUXXmgk=; b=epsBugqNL+Sk/ijC7DGSa1gfKv0mPv/AdBkFrzd17zOtsgtisyNOp47125Niuxj0vY 4p8yLGx6/f/4X6d6c7vzacGe1rh38pU/n0drpp5gma9TIR2Bif/18pFZ68OHyPwh1TsC LeiKECV+8FqveyzoZjjhe+F8ZRBxWl+DmC1CY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=ipfNN8OV8QVuShDA+a43UeYkAPRUmKyvpeU0V50GUEn70JsAHR2sQCQIJc2S0FUB7i jJFXm7RvUI03uzfEDABfEssGKbWKs6WYsogij2vPlOYIe6EFZONkYD4q9/aKR3yQu9cg y/EVA7wx4d2cqWzOYIYe/nDi3qPHucHWFsdIw= Received: by 10.223.76.65 with SMTP id b1mr450497fak.5.1263452935066; Wed, 13 Jan 2010 23:08:55 -0800 (PST) Received: from mbp-gige.totalterror.net (93-152-151-19.ddns.onlinedirect.bg [93.152.151.19]) by mx.google.com with ESMTPS id 18sm457098fks.4.2010.01.13.23.08.52 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 13 Jan 2010 23:08:53 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Nikolay Denev In-Reply-To: <20100113.150435.650865766805848595.imp@bsdimp.com> Date: Thu, 14 Jan 2010 09:08:48 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201001131515.08602.jhb@freebsd.org> <4B4E2ECA.90905@FreeBSD.org> <201001131633.09669.jhb@freebsd.org> <20100113.150435.650865766805848595.imp@bsdimp.com> To: M. Warner Losh X-Mailer: Apple Mail (2.1077) Cc: dougb@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2010 07:32:25 -0000 On 14 Jan, 2010, at 24:04 , M. Warner Losh wrote: > In message: <201001131633.09669.jhb@freebsd.org> > John Baldwin writes: > : On Wednesday 13 January 2010 3:36:26 pm Doug Barton wrote: > : > On 1/13/2010 12:15 PM, John Baldwin wrote: > : > > On Wednesday 13 January 2010 1:48:38 pm Doug Barton wrote: > : > >> To address the other responses, Tom, sorry, your suggested text = doesn't > : > >> address my concern. John, I don't think that users would = somehow > : > >> magically know to look in NOTES for more information about an = option > : > >> that is already in GENERIC. > : > >=20 > : > > You really think users do not already know to look in manpages = or NOTES to=20 > : > > find out more details about kernel options?=20 > : >=20 > : > That's not what I said. > :=20 > : > : I don't think that users would [..] know to look in NOTES for more = information=20 > : about an option that is [...] in GENERIC. > : > :=20 > : That seems really straight forward to me, or my English isn't good. = I do=20 > : think users "would know to look in NOTES for more information about = an option=20 > : that is in GENERIC". >=20 > Agreed. That's why I did what I did: I conformed to the usual = practice. >=20 > : > > Put=20 > : > > another way, what makes 'INCLUDE_CONFIG_FILE' sufficiently = special that it=20 > : > > deserves special treatment relative to other kernel options? > : >=20 > : > Because the default behavior (not including the actual file) for = the > : > option is contrary to user' reasonable expectation of how the = option > : > should work .... and now I'm repeating myself. > :=20 > : I think a better change would be to just change the default behavior = of=20 > : config(8) to do the reasonable thing. >=20 > -C should be the default, and we should invent a new > '--smaller-saved-config' option. >=20 > : > Seriously, don't you have anything better to do than argue against > : > including a comment in a config file? I know I do. What is the > : > overwhelming horror that will arise here if there are more = comments > : > GENERIC than you deem to be absolutely necessary? > :=20 > : What is the overwhelming horror about keeping a file readable and = allowing=20 > : users to find extended documentation for INCLUDE_CONFIG_FILE in the = same place=20 > : that they find extended documentation about every other kernel = option? >=20 > Yes. That's why I did what I did: to keep things readable. >=20 > : > And yes, I read the part of your message that I snipped about "why = do we > : > have these documents if users don't read them?" The answer is, = that's > : > why I'm suggesting a comment that tells users what man page to = read. > :=20 > : I think adding comments that merely redirect the users to further=20 > : documentation only serves to obfuscate. Left unchecked this = approach will=20 > : render files such as GENERIC with a very low signal-to-noise ratio = making it=20 > : harder to parse in a "big picture" way. >=20 > Yes. >=20 > Basically, I'm annoyed too: Our users aren't idiots, and we shouldn't > be treating them as such at every turn. If there are surprises with > how INCLUDE_CONFIG_FILE behaves, we should work to make it better, not > paper over it with a comment. >=20 > Warner Hello, I just want to add a user's point of view : To me INCLUDE_CONFIG_FILE sounds like the whole config file will be included, not just the output after preprocessing. So I was thinking about something like two different options, one "INCLUDE_CONFIG_FILE" which includes the whole file with comments, and the other to be just "INCLUDE_CONFIG". I think these would be pretty self-explanatory. Yes, it adds another kernel option, but having options to kernel options looks even more cryptic :) -- Regards, Niki= From owner-freebsd-arch@FreeBSD.ORG Thu Jan 14 11:47:34 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98329106566C; Thu, 14 Jan 2010 11:47:34 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail10.syd.optusnet.com.au (mail10.syd.optusnet.com.au [211.29.132.191]) by mx1.freebsd.org (Postfix) with ESMTP id 0D0788FC0C; Thu, 14 Jan 2010 11:47:33 +0000 (UTC) Received: from c122-106-161-214.carlnfd1.nsw.optusnet.com.au (c122-106-161-214.carlnfd1.nsw.optusnet.com.au [122.106.161.214]) by mail10.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o0EBlP0U001407 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 14 Jan 2010 22:47:27 +1100 Date: Thu, 14 Jan 2010 22:47:25 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Nikolay Denev In-Reply-To: Message-ID: <20100114222154.M62659@delplex.bde.org> References: <201001131515.08602.jhb@freebsd.org> <4B4E2ECA.90905@FreeBSD.org> <201001131633.09669.jhb@freebsd.org> <20100113.150435.650865766805848595.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: dougb@FreeBSD.org, jhb@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, freebsd-arch@FreeBSD.org, svn-src-head@FreeBSD.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2010 11:47:34 -0000 On Thu, 14 Jan 2010, Nikolay Denev wrote: > I just want to add a user's point of view : > To me INCLUDE_CONFIG_FILE sounds like the > whole config file will be included, > not just the output after preprocessing. It used to mean to actually include the config file. This became a broken meaning when the include directive was added in 2001, if this directory wass actually used, so that a typical config file looked like: include MOST options extra nooptions noextra This was fixed in 2007, but at the same time, for some reason stripping the comments became the default. Of course GENERIC shouldn't have verbose comments (except meta ones). Please remove any others :-). There are only 2 really annoying ones in the i386 GENERIC: - one for puc - a set of 3 separate 1-line extra ones for bpf, the first line of which essentially just repeats the expansion of the acronym `bpf' but does it with different capitalization. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Jan 14 15:30:13 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A75A1065672 for ; Thu, 14 Jan 2010 15:30:13 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 457028FC12 for ; Thu, 14 Jan 2010 15:30:13 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id EB37B46B0D for ; Thu, 14 Jan 2010 10:30:12 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id C6B528A01D for ; Thu, 14 Jan 2010 10:30:11 -0500 (EST) From: John Baldwin To: arch@freebsd.org Date: Thu, 14 Jan 2010 10:30:11 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20091231; KDE/4.3.1; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201001141030.11301.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Thu, 14 Jan 2010 10:30:12 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Subject: Removing example 'hints' and 'env' directives from GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2010 15:30:13 -0000 This just moves the commented out examples of using GENERIC.hints or GENERIC.env to sys/conf/NOTES and out of various kernel config files. After these changes there is one other commented out 'hints' line in the tree: > grep '#hints' */conf/* arm/conf/BWCT:#hints "hints.at91rm9200" I am leaving it for now, but that file doesn't exist in the tree currently. Index: conf/NOTES =================================================================== --- conf/NOTES (revision 202187) +++ conf/NOTES (working copy) @@ -50,7 +50,16 @@ # maxusers 10 +# To statically compile in device wiring instead of /boot/device.hints +#hints "LINT.hints" # Default places to look for devices. + +# Use the following to compile in values accessible to the kernel +# through getenv() (or kenv(1) in userland). The format of the file +# is 'variable=value', see kenv(1) # +#env "LINT.env" + +# # The `makeoptions' parameter allows variables to be passed to the # generated Makefile in the build area. # Index: amd64/conf/XENHVM =================================================================== --- amd64/conf/XENHVM (revision 202286) +++ amd64/conf/XENHVM (working copy) @@ -21,15 +21,6 @@ cpu HAMMER ident XENHVM -# To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" # Default places to look for devices. - -# Use the following to compile in values accessible to the kernel -# through getenv() (or kenv(1) in userland). The format of the file -# is 'variable=value', see kenv(1) -# -# env "GENERIC.env" - makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols makeoptions MODULES_OVERRIDE="" Index: amd64/conf/GENERIC =================================================================== --- amd64/conf/GENERIC (revision 202187) +++ amd64/conf/GENERIC (working copy) @@ -21,15 +21,6 @@ cpu HAMMER ident GENERIC -# To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" # Default places to look for devices. - -# Use the following to compile in values accessible to the kernel -# through getenv() (or kenv(1) in userland). The format of the file -# is 'variable=value', see kenv(1) -# -# env "GENERIC.env" - makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols options SCHED_ULE # ULE scheduler Index: arm/conf/SIMICS =================================================================== --- arm/conf/SIMICS (revision 202187) +++ arm/conf/SIMICS (working copy) @@ -23,8 +23,6 @@ options KERNVIRTADDR=0xc0000000 options PHYSADDR=0xc0000000 include "../sa11x0/std.sa11x0" -#To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" #Default places to look for devices. makeoptions MODULES_OVERRIDE="" makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Index: arm/conf/GUMSTIX =================================================================== --- arm/conf/GUMSTIX (revision 202187) +++ arm/conf/GUMSTIX (working copy) @@ -32,8 +32,6 @@ options STARTUP_PAGETABLE_ADDR=0xa0000000 include "../xscale/pxa/std.pxa" -#To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" #Default places to look for devices. makeoptions MODULES_OVERRIDE="" makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Index: arm/conf/CRB =================================================================== --- arm/conf/CRB (revision 202187) +++ arm/conf/CRB (working copy) @@ -26,8 +26,6 @@ options COUNTS_PER_SEC=400000000 options STARTUP_PAGETABLE_ADDR=0x00000000 include "../xscale/i8134x/std.crb" -#To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" #Default places to look for devices. makeoptions MODULES_OVERRIDE="" #makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Index: arm/conf/IQ31244 =================================================================== --- arm/conf/IQ31244 (revision 202187) +++ arm/conf/IQ31244 (working copy) @@ -27,8 +27,6 @@ options STARTUP_PAGETABLE_ADDR=0xa0000000 include "../xscale/i80321/std.iq31244" -#To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" #Default places to look for devices. makeoptions MODULES_OVERRIDE="" #makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Index: arm/conf/EP80219 =================================================================== --- arm/conf/EP80219 (revision 202187) +++ arm/conf/EP80219 (working copy) @@ -26,8 +26,6 @@ options STARTUP_PAGETABLE_ADDR=0xa0000000 #options ARM32_NEW_VM_LAYOUT include "../xscale/i80321/std.ep80219" -#To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" #Default places to look for devices. makeoptions MODULES_OVERRIDE="" makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Index: arm/conf/SKYEYE =================================================================== --- arm/conf/SKYEYE (revision 202187) +++ arm/conf/SKYEYE (working copy) @@ -24,8 +24,6 @@ options KERNVIRTADDR=0xc0000000 options PHYSADDR=0xc0000000 include "../at91/std.kb920x" -#To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" #Default places to look for devices. makeoptions MODULES_OVERRIDE="" makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Index: i386/conf/GENERIC =================================================================== --- i386/conf/GENERIC (revision 202187) +++ i386/conf/GENERIC (working copy) @@ -23,15 +23,6 @@ cpu I686_CPU ident GENERIC -# To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" # Default places to look for devices. - -# Use the following to compile in values accessible to the kernel -# through getenv() (or kenv(1) in userland). The format of the file -# is 'variable=value', see kenv(1) -# -# env "GENERIC.env" - makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols options SCHED_ULE # ULE scheduler Index: mips/conf/QEMU =================================================================== --- mips/conf/QEMU (revision 202187) +++ mips/conf/QEMU (working copy) @@ -30,8 +30,6 @@ options KERNVIRTADDR=0x80100000 include "../adm5120/std.adm5120" -#hints "GENERIC.hints" #Default places to look for devices. - makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols options DDB Index: pc98/conf/GENERIC =================================================================== --- pc98/conf/GENERIC (revision 202187) +++ pc98/conf/GENERIC (working copy) @@ -23,15 +23,6 @@ cpu I686_CPU ident GENERIC -# To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" # Default places to look for devices. - -# Use the following to compile in values accessible to the kernel -# through getenv() (or kenv(1) in userland). The format of the file -# is 'variable=value', see kenv(1) -# -# env "GENERIC.env" - makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols options SCHED_4BSD # 4BSD scheduler Index: powerpc/conf/GENERIC =================================================================== --- powerpc/conf/GENERIC (revision 202187) +++ powerpc/conf/GENERIC (working copy) @@ -21,9 +21,6 @@ cpu AIM ident GENERIC -#To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" - makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols # Platform support Index: sparc64/conf/GENERIC =================================================================== --- sparc64/conf/GENERIC (revision 202187) +++ sparc64/conf/GENERIC (working copy) @@ -21,15 +21,6 @@ cpu SUN4U ident GENERIC -# To statically compile in device wiring instead of /boot/device.hints -#hints "GENERIC.hints" # Default places to look for devices. - -# Use the following to compile in values accessible to the kernel -# through getenv() (or kenv(1) in userland). The format of the file -# is 'variable=value', see kenv(1) -# -# env "GENERIC.env" - makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols # Platforms supported -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Jan 14 16:53:46 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E5341065670; Thu, 14 Jan 2010 16:53:46 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D90238FC0C; Thu, 14 Jan 2010 16:53:45 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 761D946B29; Thu, 14 Jan 2010 11:53:45 -0500 (EST) Date: Thu, 14 Jan 2010 16:53:45 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Doug Barton In-Reply-To: <4B4E1586.7090102@FreeBSD.org> Message-ID: References: <20100110.210204.787670930858346133.imp@bsdimp.com> <4B4D109A.5060500@FreeBSD.org> <20100112.174326.337739863389869251.imp@bsdimp.com> <4B4E1586.7090102@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2010 16:53:46 -0000 > In the interests of bringing this to a close: > # Store the plain version of the configuration file in the kernel itself. > # For information on extraction, and storing the comments also, see > config(8). Am I right in thinking that even with this change, you still end up with a single giant config file whereas it may have been made up of several files in the original and assembled using the include directive? This means some caution (and a caveat of some sort) are still required. I agree entirely that we should be including the comments by default. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Thu Jan 14 17:30:49 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44E041065694; Thu, 14 Jan 2010 17:30:49 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id DAA628FC1B; Thu, 14 Jan 2010 17:30:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o0EHKjMi012163; Thu, 14 Jan 2010 10:20:45 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Thu, 14 Jan 2010 10:21:42 -0700 (MST) Message-Id: <20100114.102142.328914705071816274.imp@bsdimp.com> To: rwatson@freebsd.org From: "M. Warner Losh" In-Reply-To: References: <20100112.174326.337739863389869251.imp@bsdimp.com> <4B4E1586.7090102@FreeBSD.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, dougb@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2010 17:30:49 -0000 In message: Robert Watson writes: : : > In the interests of bringing this to a close: : > # Store the plain version of the configuration file in the kernel : > # itself. : > # For information on extraction, and storing the comments also, see : > config(8). : : Am I right in thinking that even with this change, you still end up : with a single giant config file whereas it may have been made up of : several files in the original and assembled using the include : directive? No. Without -C you get something that looks like: options CONFIG_AUTOGENERATED\n\ ident GENERIC\n\ machine amd64\n\ cpu HAMMER\n\ makeoptions DEBUG=-g\n\ options AH_SUPPORT_AR5416\n\ options IEEE80211_SUPPORT_MESH\n\ options IEEE80211_AMPDU_AGE\n\ which is the totally pre-processed config, with stuff sorted in a weird order (I think backwards in the file). This is the fully-processed config, including include files, option/nooption cancellation, etc. With -C you get: #\n\ # GENERIC -- Generic kernel configuration file for FreeBSD/amd64\n\ #\n\ # For more information on this file, please read the config(5) manual page,\n\ # and/or the handbook section on Kernel Configuration Files:\n\ #\n\ eg, an verbatim copy of the config file used, but do not get any files that were included such as DEFAULTS. Personally, I'd rather see us have two different options here, as suggested by others, so that it is clear and self-contained. I'd vote for INCLUDE_CONFIG for today's behavior without -C, and INCLUDE_CONFIG_FILE for config -C. -C then becomes a nop and the man page becomes correct. We can then put a single line: options INCLUDE_CONFIG_FILE # include this config file in kernel and be done with it in head. For MFC, having the extra comment lines isn't so horrible as an explicitly granted exception to the rule of not doing that so others won't be tempted to do it too. : This means some caution (and a caveat of some sort) are : still required. I agree entirely that we should be including the : comments by default. I disagree, for the reasons above. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Jan 14 18:05:53 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 513E01065670; Thu, 14 Jan 2010 18:05:53 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id E408C8FC0C; Thu, 14 Jan 2010 18:05:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o0EHtODR012670; Thu, 14 Jan 2010 10:55:24 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Thu, 14 Jan 2010 10:56:22 -0700 (MST) Message-Id: <20100114.105622.457034909117828677.imp@bsdimp.com> To: rwatson@freebsd.org From: "M. Warner Losh" In-Reply-To: <20100114.102142.328914705071816274.imp@bsdimp.com> References: <4B4E1586.7090102@FreeBSD.org> <20100114.102142.328914705071816274.imp@bsdimp.com> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Multipart/Mixed; boundary="--Next_Part(Thu_Jan_14_10_56_22_2010_136)--" Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, dougb@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2010 18:05:53 -0000 ----Next_Part(Thu_Jan_14_10_56_22_2010_136)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit In message: <20100114.102142.328914705071816274.imp@bsdimp.com> "M. Warner Losh" writes: : Personally, I'd rather see us have two different options here, as : suggested by others, so that it is clear and self-contained. I'd vote : for INCLUDE_CONFIG for today's behavior without -C, and : INCLUDE_CONFIG_FILE for config -C. -C then becomes a nop and the man : page becomes correct. We can then put a single line: : : options INCLUDE_CONFIG_FILE # include this config file in kernel : : and be done with it in head. Consider the following patches, and the following lines in GENERIC: options INCLUDE_CONFIG_FILE # include this config file in kernel #options INCLUDE_CONFIG # processed config, without comments : For MFC, having the extra comment lines isn't so horrible as an : explicitly granted exception to the rule of not doing that so others : won't be tempted to do it too. That was worded badly. "I'm OK with merging the extra comments, so long as it is clear to everybody that this is done as a special case exception to the general rule that we don't do this." Finally, I'd like to appologize to Doug for hijacking his simple change and using it as an engine to make things better for the project. Warner P.S. I've also placed a copy of the diff at http://people.freebsd.org/~imp/config.diff ----Next_Part(Thu_Jan_14_10_56_22_2010_136)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="config.diff" Index: sys/kern/kern_mib.c =================================================================== --- sys/kern/kern_mib.c (revision 202102) +++ sys/kern/kern_mib.c (working copy) @@ -351,7 +351,7 @@ CTLTYPE_INT|CTLFLAG_RW|CTLFLAG_PRISON, 0, 0, sysctl_kern_securelvl, "I", "Current secure level"); -#ifdef INCLUDE_CONFIG_FILE +#if defined(INCLUDE_CONFIG_FILE) || defined(INCLUDE_CONFIG) /* Actual kernel configuration options. */ extern char kernconfstring[]; Index: usr.sbin/config/kernconf.tmpl =================================================================== --- usr.sbin/config/kernconf.tmpl (revision 202102) +++ usr.sbin/config/kernconf.tmpl (working copy) @@ -5,13 +5,19 @@ * $FreeBSD$ */ #include "opt_config.h" -#ifdef INCLUDE_CONFIG_FILE /* - * For !INCLUDE_CONFIG_FILE case, you should look at kern_mib.c. This is - * where kernconfstring is defined then. + * If INCLUDE_CONFIG_FILE is defined, then we include the config file + * verbatim (and that's the only config file we include). Otherwise, if + * INCLUDE_CONFIG is defined, we include it. Otherwise, we include nothing + * at all. */ +#ifdef INCLUDE_CONFIG_FILE const char kernconfstring[] __attribute__ ((section("kern_conf"))) = "%%KERNCONFFILE%%"; - +#else /* INCLUDE_CONFIG_FILE */ +#ifdef INCLUDE_CONFIG +const char kernconfstring[] __attribute__ ((section("kern_conf"))) = +"%%KERNCONF%%"; +#endif /* INCLUDE_CONFIG */ #endif /* INCLUDE_CONFIG_FILE */ Index: usr.sbin/config/main.c =================================================================== --- usr.sbin/config/main.c (revision 202102) +++ usr.sbin/config/main.c (working copy) @@ -79,12 +79,6 @@ int found_defaults; int incignore; -/* - * Preserve old behaviour in INCLUDE_CONFIG_FILE handling (files are included - * literally). - */ -int filebased = 0; - static void configfile(void); static void get_srcdir(void); static void usage(void); @@ -114,7 +108,6 @@ while ((ch = getopt(argc, argv, "Cd:gpVx:")) != -1) switch (ch) { case 'C': - filebased = 1; break; case 'd': if (*destdir == '\0') @@ -489,12 +482,18 @@ } } +static int +matches(const char *s1, const char *s2) +{ + return (strncmp(s1, s2, strlen(s2)) == 0); +} + static void configfile(void) { FILE *fo; - struct sbuf *sb; - char *p; + struct sbuf *sb1, *sb2; + char *p, *walker; /* Add main configuration file to the list of files to be included */ cfgfile_add(PREFIX); @@ -502,30 +501,30 @@ fo = fopen(p, "w"); if (!fo) err(2, "%s", p); - sb = sbuf_new(NULL, NULL, 2048, SBUF_AUTOEXTEND); - assert(sb != NULL); - sbuf_clear(sb); - if (filebased) { - /* Is needed, can be used for backward compatibility. */ - configfile_filebased(sb); - } else { - configfile_dynamic(sb); + sb1 = sbuf_new(NULL, NULL, 2048, SBUF_AUTOEXTEND); + assert(sb1 != NULL); + sb2 = sbuf_new(NULL, NULL, 2048, SBUF_AUTOEXTEND); + assert(sb2 != NULL); + sbuf_clear(sb1); + configfile_filebased(sb1); + sbuf_finish(sb1); + sbuf_clear(sb2); + configfile_dynamic(sb2); + sbuf_finish(sb2); + walker = kernconfstr; + while (*walker) { + if (matches(walker, KERNCONFFILETAG)) { + fprintf(fo, "%s", sbuf_data(sb1)); + walker += strlen(KERNCONFFILETAG); + } else if (matches(walker, KERNCONFTAG)) { + fprintf(fo, "%s", sbuf_data(sb2)); + walker += strlen(KERNCONFTAG); + } else { + fputc(*walker++, fo); + } } - sbuf_finish(sb); - /* - * We print first part of the tamplate, replace our tag with - * configuration files content and later continue writing our - * template. - */ - p = strstr(kernconfstr, KERNCONFTAG); - if (p == NULL) - errx(EXIT_FAILURE, "Something went terribly wrong!"); - *p = '\0'; - fprintf(fo, "%s", kernconfstr); - fprintf(fo, "%s", sbuf_data(sb)); - p += strlen(KERNCONFTAG); - fprintf(fo, "%s", p); - sbuf_delete(sb); + sbuf_delete(sb1); + sbuf_delete(sb2); fclose(fo); moveifchanged(path("config.c.new"), path("config.c")); cfgfile_removeall(); Index: usr.sbin/config/config.8 =================================================================== --- usr.sbin/config/config.8 (revision 202102) +++ usr.sbin/config/config.8 (working copy) @@ -65,9 +65,6 @@ .Nm version number. .It Fl C -If the INCLUDE_CONFIG_FILE is present in a configuration file, -kernel image will contain full configuration files included -literally (preserving comments). This flag is kept for backward compatibility. .It Fl d Ar destdir Use @@ -83,7 +80,9 @@ .It Fl x Ar kernel Print kernel configuration file embedded into a kernel file. -This option makes sense only if +This option makes sense only if either +.Cd "options INCLUDE_CONFIG" +or .Cd "options INCLUDE_CONFIG_FILE" entry was present in your configuration file. .It Fl p @@ -150,6 +149,17 @@ should be run again. Attempts to compile a system that had configuration errors are likely to fail. +.Pp +The kernel config file can be embedded into the kernel binary in one of +two ways. +First, a verbatim copy of the kernel config text file, without any of the +included config files, can be included using +.Cd "options INCLUDE_CONFIG_FILE" +in the config file. +Second, a copy of the fully process kernel config file, without any comments +at all, can be included using +.Cd "options INCLUDE_CONFIG" +in the config file. .Sh DEBUG KERNELS Traditional .Bx Index: usr.sbin/config/config.h =================================================================== --- usr.sbin/config/config.h (revision 202102) +++ usr.sbin/config/config.h (working copy) @@ -142,10 +142,11 @@ STAILQ_HEAD(hint_head, hint) hints; /* - * Tag present in the kernelconf.tmlp template file. It's mandatory for those - * two strings to be the same. Otherwise you'll get into trouble. + * Tags expanded in the kernconf.tmpl file. KERNCONFTAG is for the processed + * kernel file, while KERNCONFFILE is for the verbatim file. */ -#define KERNCONFTAG "%%KERNCONFFILE%%" +#define KERNCONFTAG "%%KERNCONF%%" +#define KERNCONFFILETAG "%%KERNCONFFILE%%" /* * Faked option to note, that the configuration file has been taken from the ----Next_Part(Thu_Jan_14_10_56_22_2010_136)---- From owner-freebsd-arch@FreeBSD.ORG Thu Jan 14 20:01:17 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E22F01065692 for ; Thu, 14 Jan 2010 20:01:17 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id 35F468FC22 for ; Thu, 14 Jan 2010 20:01:16 +0000 (UTC) Received: (qmail 20898 invoked by uid 399); 14 Jan 2010 20:01:16 -0000 Received: from localhost (HELO ?192.168.0.110?) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 14 Jan 2010 20:01:16 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4B4F7810.2080003@FreeBSD.org> Date: Thu, 14 Jan 2010 12:01:20 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0 MIME-Version: 1.0 To: "M. Warner Losh" References: <4B4E1586.7090102@FreeBSD.org> <20100114.102142.328914705071816274.imp@bsdimp.com> <20100114.105622.457034909117828677.imp@bsdimp.com> In-Reply-To: <20100114.105622.457034909117828677.imp@bsdimp.com> X-Enigmail-Version: 1.0 OpenPGP: id=D5B2F0FB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2010 20:01:18 -0000 On 1/14/2010 9:56 AM, M. Warner Losh wrote: + * If INCLUDE_CONFIG_FILE is defined, then we include the config file + * verbatim (and that's the only config file we include). Otherwise, if + * INCLUDE_CONFIG is defined, we include it. Otherwise, we include nothing + * at all. FWIW, I actually think this makes it worse, not better. The INCLUDE_CONFIG_FILE option should include everything needed to recreate the kernel. If it doesn't, it's worse than worthless, it leads to a false sense of security which makes it dangerous. I wasn't actually aware that if you do the config -C trick that you'll get ONLY the one file, not everything. Frankly I'm flabbergasted that we could do something so stupid. And on that note, I officially give up. I have better things to do with my time than argue about this any more, and it's pretty clear that my perspective on this is not shared. Maybe it's not even reasonable, who knows? Good luck, Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso From owner-freebsd-arch@FreeBSD.ORG Thu Jan 14 20:12:28 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F5E41065670; Thu, 14 Jan 2010 20:12:28 +0000 (UTC) (envelope-from rwatson@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 55EEF8FC1A; Thu, 14 Jan 2010 20:12:28 +0000 (UTC) Received: from [192.168.2.102] (host86-178-254-187.range86-178.btcentralplus.com [86.178.254.187]) by cyrus.watson.org (Postfix) with ESMTPSA id EE0DE46B0D; Thu, 14 Jan 2010 15:12:26 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <4B4F7810.2080003@FreeBSD.org> Date: Thu, 14 Jan 2010 20:12:24 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> References: <4B4E1586.7090102@FreeBSD.org> <20100114.102142.328914705071816274.imp@bsdimp.com> <20100114.105622.457034909117828677.imp@bsdimp.com> <4B4F7810.2080003@FreeBSD.org> To: Doug Barton X-Mailer: Apple Mail (2.1077) Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2010 20:12:28 -0000 On 14 Jan 2010, at 20:01, Doug Barton wrote: > FWIW, I actually think this makes it worse, not better. The > INCLUDE_CONFIG_FILE option should include everything needed to = recreate > the kernel. If it doesn't, it's worse than worthless, it leads to a > false sense of security which makes it dangerous. I wasn't actually > aware that if you do the config -C trick that you'll get ONLY the one > file, not everything. (...more irritated words stripped like the comments in a config file...) I agree. I see two kinds of users: - Desktop/server users who want their system to work without any special = tuning or magic, and likely feel the comments they put in configuration = files are important - Users who care a about a few hundred bytes of comments due to the = nature of their special embedded hardware target environment As such, I see a reasonable "default" -- i.e., i386/amd64 GENERIC -- be = to fully preserve the configuration and its comments. For the embedded = and appliance crowds, I definitely want them to be able to fully exclude = the configuration (by not including the option). I don't see a lot of = merit to supporting stripping the comments, but if someone wants that = special case, I'm OK with supporting it, but I think it's an unuseful = default for the desktop/server platforms. Robert= From owner-freebsd-arch@FreeBSD.ORG Thu Jan 14 21:06:55 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F396106566C; Thu, 14 Jan 2010 21:06:55 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id CF2218FC14; Thu, 14 Jan 2010 21:06:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o0EKwWth014817; Thu, 14 Jan 2010 13:58:32 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Thu, 14 Jan 2010 13:59:30 -0700 (MST) Message-Id: <20100114.135930.80200584442733547.imp@bsdimp.com> To: rwatson@freebsd.org From: "M. Warner Losh" In-Reply-To: <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> References: <20100114.105622.457034909117828677.imp@bsdimp.com> <4B4F7810.2080003@FreeBSD.org> <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, dougb@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jan 2010 21:06:55 -0000 In message: <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> "Robert N. M. Watson" writes: : : On 14 Jan 2010, at 20:01, Doug Barton wrote: : : > FWIW, I actually think this makes it worse, not better. The : > INCLUDE_CONFIG_FILE option should include everything needed to recreate : > the kernel. If it doesn't, it's worse than worthless, it leads to a : > false sense of security which makes it dangerous. I wasn't actually : > aware that if you do the config -C trick that you'll get ONLY the one : > file, not everything. : : (...more irritated words stripped like the comments in a config file...) : : I agree. I see two kinds of users: : : - Desktop/server users who want their system to work without any : special tuning or magic, and likely feel the comments they put in : configuration files are important : - Users who care a about a few hundred bytes of comments due to the : nature of their special embedded hardware target environment : : As such, I see a reasonable "default" -- i.e., i386/amd64 GENERIC -- : be to fully preserve the configuration and its comments. For the : embedded and appliance crowds, I definitely want them to be able to : fully exclude the configuration (by not including the option). I : don't see a lot of merit to supporting stripping the comments, but : if someone wants that special case, I'm OK with supporting it, but I : think it's an unuseful default for the desktop/server platforms. I think you are confusing two different things. The embedded folks omit this entirely. That's not the issue at all here. Space is not the concern. We're putting it into GENERIC. Why are there two ways? Because there are two points in the process one can squirt the data out of config. Let me explain (sorry for the length, but it is necessary to clear up the confusion): The first way is "preserve the FILE the user used to create the kernel" on the assumption that the FILE is what is important. This will preserve the comments, but assumes that every single included file from that file is recoverable in a trivial manner (eg, from cvs, svn, p4, release ISOs, etc). This model works well for the casual user. In this model, we're therefore able to preserve the kernel config by copying the file, verbatim, into the kernel. I think this is the model that best fits most user's needs, since they EITHER take GENERIC and hack on it (in which case we preserve all that), OR they include GENERIC and opt in/out of things based on that default. Either way, this is the best way for users of releases to preserve the data they need to recreate the kernel without losing data that's important to the user, but not config (comments, spacing, order, etc). The other way of preserving the config file is to say "I want EVERYTHING applied, and I want a copy of that!" To get that, you have to walk through config's internal data structures. By the time we get to *THAT* point in the processing, comments are but a distant memory. Those are discarded early, along with the variations in spacing, blank lines. By this point in the process, even the original ordering is long gone, as is the origin of each of the remaining items in config's internal data structures. However, the resulting config file that we generate here is GUARANTEED to recreate the same kernel from the same sources modulo any silly time/date/path stuff we encode. There's a class of user that would find this comforting, or essential, since they may have multiple include files that they control, etc. In addition, since the config is presented as a sysctl, users could query it to see if the kernel had certain things compiled in or not. For some applications, this is important to know, even though other methods usually work better. So it isn't that there's an option to just strip comments or not strip comments, these two different options save data at two radically different points in the process. Both have their pros and cons. If space isn't an issue, we could save BOTH. That would be a bigger patch, since we'd have to alter config to extract both kinds of data. All I did was to move the -C option from an obscure src.conf variable to be a full-fledged kernel option, leaving the rest of the infrastructure intact. The advantage is that we cover both bases and could export both views as a sysctl (right now we overload the two different views with one sysctl). Heck, we could save the whole src/sys tree as a tarball in a separate non-loadable ELF section if people that that was useful. This could be implemented entirely in the *.mk files, however, without the need to modify config(8). Warner From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 01:51:05 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A8ECB1065670; Fri, 15 Jan 2010 01:51:05 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail03.syd.optusnet.com.au (mail03.syd.optusnet.com.au [211.29.132.184]) by mx1.freebsd.org (Postfix) with ESMTP id 271048FC13; Fri, 15 Jan 2010 01:51:04 +0000 (UTC) Received: from c220-239-227-214.carlnfd1.nsw.optusnet.com.au (c220-239-227-214.carlnfd1.nsw.optusnet.com.au [220.239.227.214]) by mail03.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o0F1otJu012208 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 15 Jan 2010 12:50:58 +1100 Date: Fri, 15 Jan 2010 12:50:55 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: "M. Warner Losh" In-Reply-To: <20100114.135930.80200584442733547.imp@bsdimp.com> Message-ID: <20100115114822.O63406@delplex.bde.org> References: <20100114.105622.457034909117828677.imp@bsdimp.com> <4B4F7810.2080003@FreeBSD.org> <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> <20100114.135930.80200584442733547.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: dougb@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, rwatson@FreeBSD.org, freebsd-arch@FreeBSD.org, svn-src-head@FreeBSD.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 01:51:05 -0000 On Thu, 14 Jan 2010, M. Warner Losh wrote: > In message: <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> > "Robert N. M. Watson" writes: > : I agree. I see two kinds of users: > : > : - Desktop/server users who want their system to work without any > : special tuning or magic, and likely feel the comments they put in > : configuration files are important > : - Users who care a about a few hundred bytes of comments due to the > : nature of their special embedded hardware target environment "Stripping" comments has nothing to do with saving space. It is because it is technically difficult, and not implemented, to not strip them, except in the old limited code that just preserves the unprocessed top-level-only config file. > : As such, I see a reasonable "default" -- i.e., i386/amd64 GENERIC -- > : be to fully preserve the configuration and its comments. For the Your code to implement this is welcome :-). Even GENERIC is not quite complete, since it is missing the implicit include of DEFAULTS. OTOH, completing the config by merging with DEFAULTS, as I think the processed version must do, may give an unusable config by repeating things in DEFAULTS. The merge should probably have no processing at all, with include files concatenated and not replacing include directives, and DEFAULTS not included. > I think you are confusing two different things. > > The embedded folks omit this entirely. That's not the issue at all > here. Space is not the concern. We're putting it into GENERIC. > > Why are there two ways? Because there are two points in the process > one can squirt the data out of config. Let me explain (sorry for the > length, but it is necessary to clear up the confusion): > > The first way is "preserve the FILE the user used to create the > kernel" on the assumption that the FILE is what is important. No, it is on the non-assumption that the include directive didn't exist when the INCLUDE_CONFIG_FILE option was implemented. > This > will preserve the comments, but assumes that every single included > file from that file is recoverable in a trivial manner (eg, from cvs, > svn, p4, release ISOs, etc). This assumption is false, so this model became just broken when the include directive was implemented. History: INCLUDE_CONFIG_FILE option: 1996 include directive: 2001 processed output and -C: 2007 The -C option just preserves the breakage at its 2001-2007 level. > This model works well for the casual > user. Casual users presumably didn't notice the problem (until comments were lost in 2007) since they use GENERIC as is or with minor editing, and GENERIC doesn't use the include directive. Non-casual users don't notice since they know what is in their config files and don't use the INCLUDE_CONFIG_FILE option. (I use a combination of 3 levels of include files and 2 levels of symlinks to handle about 6 configurations times 2 arches times 4 FreeBSD versions.) > In this model, we're therefore able to preserve the kernel > config by copying the file, verbatim, into the kernel. I think this > is the model that best fits most user's needs, since they EITHER take > GENERIC and hack on it (in which case we preserve all that), OR they > include GENERIC and opt in/out of things based on that default. > Either way, this is the best way for users of releases to preserve the > data they need to recreate the kernel without losing data that's > important to the user, but not config (comments, spacing, order, etc). This may be enough, but it is a historical accident that config "works" this way. > The other way of preserving the config file is to say "I want > EVERYTHING applied, and I want a copy of that!" To get that, you have > to walk through config's internal data structures. By the time we get > to *THAT* point in the processing, comments are but a distant memory. Getting output from cpp has the same problems: - you can get fully preprocessed output fairly easily - cpp normally discards comments early, so you can't get them easily. I think gnu cpp now has an option to not discard them. - so you usually get only the fully preprocessed output - but sometimes you don't want so much preprocessing. None might be best. I don't know of any option in gnu cpp to get none. - the problem is harder in cpp than in config due to ifdefed out C code normally being completely removed, while config doesn't even have ifdef. Teaching cpp to preserve (only some) ifdefed out code, and presenting the results nicely (need to preserve the original ifdef structure) would be difficult. Except it might not be too difficult for either cpp or config to just concatenate all the files used. > Those are discarded early, along with the variations in spacing, blank > lines. By this point in the process, even the original ordering is > long gone, as is the origin of each of the remaining items in config's > internal data structures. However, the resulting config file that we > generate here is GUARANTEED to recreate the same kernel from the same > sources modulo any silly time/date/path stuff we encode. There's a I checked what happens with repeating DEFAULTS. Repeating it using "include" results in many warnings but the same generated files. > If space isn't an issue, we could save BOTH. That would be a bigger > patch, since we'd have to alter config to extract both kinds of data. > All I did was to move the -C option from an obscure src.conf variable > to be a full-fledged kernel option, leaving the rest of the > infrastructure intact. The advantage is that we cover both bases and > could export both views as a sysctl (right now we overload the two > different views with one sysctl). ISTR a long discussion about the -C option when it was implemented. Why didn't you complain back then? :-). I looked at my old mail about this. I wasn't involved with the initial discussion, but imp was :-). My mail was after the initial commit. I reported the following problems: - DEFAULTS wasn't processed - several ordering problems. Ones that reordered directives of the form: options FOO=bar nooptions FOO options FOO=baz made the generated file unusable. - many formatting problems in the generated file, making it a bad example. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 08:59:04 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32D6F106566B; Fri, 15 Jan 2010 08:59:04 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail12.syd.optusnet.com.au (mail12.syd.optusnet.com.au [211.29.132.193]) by mx1.freebsd.org (Postfix) with ESMTP id A65D48FC12; Fri, 15 Jan 2010 08:59:03 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c122-106-232-83.belrs3.nsw.optusnet.com.au [122.106.232.83]) by mail12.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o0F8x0DF007625 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 15 Jan 2010 19:59:01 +1100 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id o0F8wvg3002662; Fri, 15 Jan 2010 19:58:57 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id o0F8wvsT002661; Fri, 15 Jan 2010 19:58:57 +1100 (EST) (envelope-from peter) Date: Fri, 15 Jan 2010 19:58:57 +1100 From: Peter Jeremy To: "Robert N. M. Watson" Message-ID: <20100115085856.GA2556@server.vk2pj.dyndns.org> References: <4B4E1586.7090102@FreeBSD.org> <20100114.102142.328914705071816274.imp@bsdimp.com> <20100114.105622.457034909117828677.imp@bsdimp.com> <4B4F7810.2080003@FreeBSD.org> <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="X1bOJ3K7DJ5YkBrT" Content-Disposition: inline In-Reply-To: <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.20 (2009-06-14) Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 08:59:04 -0000 --X1bOJ3K7DJ5YkBrT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2010-Jan-14 20:12:24 +0000, "Robert N. M. Watson" = wrote: >- Desktop/server users who want their system to work without any > special tuning or magic, and likely feel the comments they put in > configuration files are important As far as I'm concerned, the most critical bit of my kernel config file is the $Header...$ comment - which lets me extract the remainder of the file from my CVS repository. I don't currently use includes (because most of my config files have roots pre-dating the include directive). I find it a PITA that INCLUDE_CONFIG_FILE _doesn't_ include comments (or at least my $Header$ line) by default. IMO, it would be useful to have an "include this literal string in the kernel" config directive. This would allow config file version control information to be embedded without needing the comments. And that would resolve the issue of embedding fully expanded details of all included files without the hassle of keeping the comments around. --=20 Peter Jeremy --X1bOJ3K7DJ5YkBrT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAktQLlAACgkQ/opHv/APuIdnTACbBsmd1V4Uc0RiXeRtoShXcNj+ NNkAoKNtcWOc91kvt/28hBQRiuaaVO9T =8ot2 -----END PGP SIGNATURE----- --X1bOJ3K7DJ5YkBrT-- From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 15:06:14 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF7F01065679; Fri, 15 Jan 2010 15:06:14 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.26]) by mx1.freebsd.org (Postfix) with ESMTP id 57FAC8FC0C; Fri, 15 Jan 2010 15:06:14 +0000 (UTC) Received: by qw-out-2122.google.com with SMTP id 5so39963qwd.7 for ; Fri, 15 Jan 2010 07:06:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=oLtabJzhVmQzneyS+NEzPe0L71tzjT0ZmfhCIyS65M0=; b=FIX0rBe61HBGoB0wF88WJhnIR3oU/Hti1rP1wGNe2EigmKA7FTS4AuvCK0nW19mOAo 2sckCmbgPSCPJNKBIJnnxonSjNSqbqkMp77GPrpdIMoT2kCUwY3KZ0K4wMA1dJI5ZST8 njwRf41+TApZfn2mSpo1Ahjs7ROPTWdwhXWdY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=KyzNv22/zC+7W+XseaFp65ZmocPbTkKhf3dTxTi73KJNwRnJ57gLKPMtHkmYcvh24W TZHj3vi19Bj7c1h5BvlJs1auZ78ezF1K1OrHfd71ZCQOgLkaANaUBO4BgePMHJ3lPxAy uTsvpLxNLbUYjlVujR9nkhOct0YQpu/61HPCs= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.102.17.28 with SMTP id 28mr1119720muq.120.1263567965020; Fri, 15 Jan 2010 07:06:05 -0800 (PST) In-Reply-To: <200911301305.30572.jhb@freebsd.org> References: <3bbf2fe10911271542h2b179874qa0d9a4a7224dcb2f@mail.gmail.com> <200911301305.30572.jhb@freebsd.org> Date: Fri, 15 Jan 2010 10:06:04 -0500 X-Google-Sender-Auth: b938bcab5359cbd5 Message-ID: <3bbf2fe11001150706y765159a2jbd37c7ae4cf378f0@mail.gmail.com> From: Attilio Rao To: John Baldwin Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Arch , Ed Maste Subject: Re: [PATCH] Statclock aliasing by LAPIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 15:06:15 -0000 2009/11/30 John Baldwin : > On Friday 27 November 2009 6:42:50 pm Attilio Rao wrote: >> Handling all the three clocks (hardclock, softclock, profclock) within >> the LAPIC can lead to aliasing for the softclock and profclock because >> hz is sized to fit mainly hardclock. The fashion to handle all of them >> within the LAPIC was introduced in 2005 and before than the softclock >> and profclock were supposed to be handled in the rtc. Right now, too, >> there is the necessary support to handle profclock and statclock in >> atrtc which gets enabled if the LAPIC signals it can't take in charge >> the three clocks. >> The proposed patch reverts the situation preferring the atrtc to >> handle the statclock and profclock (then a different source from the >> LAPIC) and then avoid the aliasing problem: >> > http://www.freebsd.org/~attilio/Sandvine/STABLE_8/statclock_aliasing/stat= clock_aliasing3.diff >> >> In this patch, lapic_setup_clock() has been changed in order to return >> a three-states variable which identified if the LAPIC got in charge >> all the three clocks, just the hardclock or none of them (the current >> situation is just none/all) and the rtc handling runs subsequently. >> A tunable as been added to force LAPI to get in charge all the three >> clocks if needed. >> In ia32 atrtc compiling is linked to atpic compiling, so a compile >> time flag has been added to check if atpic is not present and in case >> force LAPIC to take in charge all the three clocks (which is still >> better than the 'safe belt values' still present in the rtc code). >> >> Please note that statclock and profclock are widely used in our kernel >> (rusage is, for example, statclock driven) and fixing this would >> result in specific improvements (as a several-reported wrong CPU usage >> statistic in top). >> This bug has been found by Sandvine Incorporated. >> >> Reviews, comments and testing are welcome. > > Presumably in the RTC case lapic_timer_hz should always be hz and not som= e > multiple of hz. =C2=A0Also, did you check to make sure all the lock is pr= esent? =C2=A0I > think at one point I changed the locking for the RTC and/or ISA timer to = just > use critical_enter/exit so that UP machines running an SMP kernel wouldn'= t pay > the locking overhead since the code was only used on UP machines. =C2=A0I= t may very > well be that I only changed that in a development branch though and not i= n > HEAD. I still see clock_lock in place (and no particular critical section code in that paths) or you meant to say that the clock_lock doesn't still provide enough protection alone? BTW, you were right about the lapic_timer_hz (I forgot to revert to hz). There is an updated patch: http://www.freebsd.org/~attilio/Sandvine/STABLE_8/statclock_aliasing/statcl= ock_aliasing4.diff Thanks, Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 15:25:32 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 352B91065676; Fri, 15 Jan 2010 15:25:32 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id E3A4C8FC14; Fri, 15 Jan 2010 15:25:31 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 7A0CE46B29; Fri, 15 Jan 2010 10:25:31 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 935268A024; Fri, 15 Jan 2010 10:25:30 -0500 (EST) From: John Baldwin To: Attilio Rao Date: Fri, 15 Jan 2010 10:25:06 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20091231; KDE/4.3.1; amd64; ; ) References: <3bbf2fe10911271542h2b179874qa0d9a4a7224dcb2f@mail.gmail.com> <200911301305.30572.jhb@freebsd.org> <3bbf2fe11001150706y765159a2jbd37c7ae4cf378f0@mail.gmail.com> In-Reply-To: <3bbf2fe11001150706y765159a2jbd37c7ae4cf378f0@mail.gmail.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201001151025.06251.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 15 Jan 2010 10:25:30 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: FreeBSD Arch , Ed Maste Subject: Re: [PATCH] Statclock aliasing by LAPIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 15:25:32 -0000 On Friday 15 January 2010 10:06:04 am Attilio Rao wrote: > 2009/11/30 John Baldwin : > > On Friday 27 November 2009 6:42:50 pm Attilio Rao wrote: > >> Handling all the three clocks (hardclock, softclock, profclock) within > >> the LAPIC can lead to aliasing for the softclock and profclock because > >> hz is sized to fit mainly hardclock. The fashion to handle all of them > >> within the LAPIC was introduced in 2005 and before than the softclock > >> and profclock were supposed to be handled in the rtc. Right now, too, > >> there is the necessary support to handle profclock and statclock in > >> atrtc which gets enabled if the LAPIC signals it can't take in charge > >> the three clocks. > >> The proposed patch reverts the situation preferring the atrtc to > >> handle the statclock and profclock (then a different source from the > >> LAPIC) and then avoid the aliasing problem: > >> > > http://www.freebsd.org/~attilio/Sandvine/STABLE_8/statclock_aliasing/statclock_aliasing3.diff > >> > >> In this patch, lapic_setup_clock() has been changed in order to return > >> a three-states variable which identified if the LAPIC got in charge > >> all the three clocks, just the hardclock or none of them (the current > >> situation is just none/all) and the rtc handling runs subsequently. > >> A tunable as been added to force LAPI to get in charge all the three > >> clocks if needed. > >> In ia32 atrtc compiling is linked to atpic compiling, so a compile > >> time flag has been added to check if atpic is not present and in case > >> force LAPIC to take in charge all the three clocks (which is still > >> better than the 'safe belt values' still present in the rtc code). > >> > >> Please note that statclock and profclock are widely used in our kernel > >> (rusage is, for example, statclock driven) and fixing this would > >> result in specific improvements (as a several-reported wrong CPU usage > >> statistic in top). > >> This bug has been found by Sandvine Incorporated. > >> > >> Reviews, comments and testing are welcome. > > > > Presumably in the RTC case lapic_timer_hz should always be hz and not some > > multiple of hz. Also, did you check to make sure all the lock is present? I > > think at one point I changed the locking for the RTC and/or ISA timer to just > > use critical_enter/exit so that UP machines running an SMP kernel wouldn't pay > > the locking overhead since the code was only used on UP machines. It may very > > well be that I only changed that in a development branch though and not in > > HEAD. > > I still see clock_lock in place (and no particular critical section > code in that paths) or you meant to say that the clock_lock doesn't > still provide enough protection alone? I think I misremembered. I have had local patches (possibly committed?) to make the atpic driver just use critical_enter/exit, but I don't think I changed the clock drivers. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 17:41:35 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0058C106566B; Fri, 15 Jan 2010 17:41:34 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout025.mac.com (asmtpout025.mac.com [17.148.16.100]) by mx1.freebsd.org (Postfix) with ESMTP id D30D78FC13; Fri, 15 Jan 2010 17:41:34 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=us-ascii Received: from macbook-pro.jnpr.net (natint3.juniper.net [66.129.224.36]) by asmtp025.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KWA00BP5UH6GK30@asmtp025.mac.com>; Fri, 15 Jan 2010 09:41:34 -0800 (PST) From: Marcel Moolenaar In-reply-to: <20100115174711.GF1990@FreeBSD.org> Date: Fri, 15 Jan 2010 09:41:30 -0800 Message-id: References: <20100114.105622.457034909117828677.imp@bsdimp.com> <4B4F7810.2080003@FreeBSD.org> <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> <20100114.135930.80200584442733547.imp@bsdimp.com> <20100115114822.O63406@delplex.bde.org> <20100115174711.GF1990@FreeBSD.org> To: "Wojciech A. Koszek" X-Mailer: Apple Mail (2.1077) Cc: dougb@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, rwatson@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 17:41:35 -0000 On Jan 15, 2010, at 9:47 AM, Wojciech A. Koszek wrote: > On Fri, Jan 15, 2010 at 12:50:55PM +1100, Bruce Evans wrote: >> On Thu, 14 Jan 2010, M. Warner Losh wrote: >> >>> In message: <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> >>> "Robert N. M. Watson" writes: >>> : I agree. I see two kinds of users: >>> : > > [..] > > Reasoning behind -C was that in 2007, Robert and Peter mentioned about a need > of having comments in the config file. I divided users into two groups: Why don't we stop dividing people in groups and classes and instead just add directives for what to include. Such as: embed options embed devices embed comments embed includes embed "$FreeBSD$" Combine this with syntactic sugaring and you can have something like: embed everything noembed comments Is this really so hard? -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 17:57:07 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E936C106566B; Fri, 15 Jan 2010 17:57:07 +0000 (UTC) (envelope-from wkoszek@freebsd.czest.pl) Received: from freebsd.czest.pl (l95h.icis.pcz.pl [212.87.224.105]) by mx1.freebsd.org (Postfix) with ESMTP id 8A02C8FC13; Fri, 15 Jan 2010 17:57:07 +0000 (UTC) Received: from freebsd.czest.pl (l95h.icis.pcz.pl [212.87.224.105]) by freebsd.czest.pl (8.14.2/8.14.2) with ESMTP id o0FHlCmp050844; Fri, 15 Jan 2010 18:47:12 +0100 (CET) (envelope-from wkoszek@freebsd.czest.pl) Received: (from wkoszek@localhost) by freebsd.czest.pl (8.14.2/8.14.2/Submit) id o0FHlBc2050843; Fri, 15 Jan 2010 18:47:11 +0100 (CET) (envelope-from wkoszek) Date: Fri, 15 Jan 2010 18:47:11 +0100 From: "Wojciech A. Koszek" To: Bruce Evans Message-ID: <20100115174711.GF1990@FreeBSD.org> Mail-Followup-To: Bruce Evans , "M. Warner Losh" , dougb@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, rwatson@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org References: <20100114.105622.457034909117828677.imp@bsdimp.com> <4B4F7810.2080003@FreeBSD.org> <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> <20100114.135930.80200584442733547.imp@bsdimp.com> <20100115114822.O63406@delplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline In-Reply-To: <20100115114822.O63406@delplex.bde.org> User-Agent: Mutt/1.5.17 (2007-11-01) X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-3.0 (freebsd.czest.pl [212.87.224.105]); Fri, 15 Jan 2010 18:47:12 +0100 (CET) Cc: dougb@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, rwatson@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 17:57:08 -0000 On Fri, Jan 15, 2010 at 12:50:55PM +1100, Bruce Evans wrote: > On Thu, 14 Jan 2010, M. Warner Losh wrote: > >> In message: <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> >> "Robert N. M. Watson" writes: >> : I agree. I see two kinds of users: >> : [..] Reasoning behind -C was that in 2007, Robert and Peter mentioned about a need of having comments in the config file. I divided users into two groups: - those who don't care because they can't reconfigure the FreeBSD kernel anyway no matter what - with or without comments, they're baked anyway, because "make xconfig" doesn't work. - those who don't care in which format config file is, since they just glance at the output from config -x, use Copy's and Paste's method and get everything fixed; preprocessing of configuration files was done for them and for system administrators woken up at 3am with a task of recreating exact machine configuration. I don't want to argue on my view, but I glanced at Warner's patch and if people want it, I believe it can make our world a bit better. > "Stripping" comments has nothing to do with saving space. It is because > it is technically difficult, and not implemented, to not strip them, > except in the old limited code that just preserves the unprocessed > top-level-only config file. This is exactly why comments aren't there. Below in this thread, Peter Jeremy gives us good suggestion on bringing additional option like "comment", which would include string literal. I think this is a good proposal. >> : As such, I see a reasonable "default" -- i.e., i386/amd64 GENERIC -- >> : be to fully preserve the configuration and its comments. For the > > Your code to implement this is welcome :-). Even GENERIC is not quite > complete, since it is missing the implicit include of DEFAULTS. OTOH, > completing the config by merging with DEFAULTS, as I think the processed > version must do, may give an unusable config by repeating things in > DEFAULTS. The merge should probably have no processing at all, with > include files concatenated and not replacing include directives, and > DEFAULTS not included. My opinion is that DEFAULTS should be included in every GENERIC explicitly by include directive, and that the code dealing with DEFAULTS which is right now present in config(8) should be removed. >> This >> will preserve the comments, but assumes that every single included >> file from that file is recoverable in a trivial manner (eg, from cvs, >> svn, p4, release ISOs, etc). > > This assumption is false, so this model became just broken when the > include directive was implemented. History: > INCLUDE_CONFIG_FILE option: 1996 > include directive: 2001 > processed output and -C: 2007 > The -C option just preserves the breakage at its 2001-2007 level. I just tried to make is "suck less", so -C right now takes all "include " directives and brings in configuration file. >> If space isn't an issue, we could save BOTH. That would be a bigger >> patch, since we'd have to alter config to extract both kinds of data. >> All I did was to move the -C option from an obscure src.conf variable >> to be a full-fledged kernel option, leaving the rest of the >> infrastructure intact. The advantage is that we cover both bases and >> could export both views as a sysctl (right now we overload the two >> different views with one sysctl). > > ISTR a long discussion about the -C option when it was implemented. Why > didn't you complain back then? :-). I looked at my old mail about this. > I wasn't involved with the initial discussion, but imp was :-). My mail > was after the initial commit. I reported the following problems: Warner was within a very small group of people that actually did care about config(8) when I touched it, and helped me to fix world/config(8) breakage when people on DevSummit (BSDCan?) couldn't get their kernel to configure properly. > - DEFAULTS wasn't processed I don't seem to remember this one... With -C it's not included, but I propose a solution above. Is this an issue you're referring here? > - several ordering problems. Ones that reordered directives of the form: > options FOO=bar > nooptions FOO > options FOO=baz > made the generated file unusable. This was found and fixed thanks to your devil's configuration tarball. Anyway... Each time I see our kernel modules being built I cry and think of this page: http://wiki.freebsd.org/NewConfig This is a list of my random ideas related with solving config(8) problems in long run. Without promising anything I can say that narrowing requirements down would help me to figure out, how big the problem is. Maybe I could help somehow with the configuration file mess in the future.. Mail me your suggestions privately, and I'll bring them to the Wiki. -- Wojciech A. Koszek wkoszek@FreeBSD.org http://FreeBSD.czest.pl/~wkoszek/ From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 18:06:14 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 679ED1065670; Fri, 15 Jan 2010 18:06:14 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 26ACB8FC0A; Fri, 15 Jan 2010 18:06:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o0FI4iCQ034251; Fri, 15 Jan 2010 11:04:44 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Fri, 15 Jan 2010 11:05:28 -0700 (MST) Message-Id: <20100115.110528.849557997928257031.imp@bsdimp.com> To: xcllnt@mac.com From: "M. Warner Losh" In-Reply-To: References: <20100115114822.O63406@delplex.bde.org> <20100115174711.GF1990@FreeBSD.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: dougb@freebsd.org, svn-src-all@freebsd.org, wkoszek@freebsd.org, src-committers@freebsd.org, rwatson@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 18:06:14 -0000 In message: Marcel Moolenaar writes: : : On Jan 15, 2010, at 9:47 AM, Wojciech A. Koszek wrote: : : > On Fri, Jan 15, 2010 at 12:50:55PM +1100, Bruce Evans wrote: : >> On Thu, 14 Jan 2010, M. Warner Losh wrote: : >> : >>> In message: <86625798-F339-4863-8F97-63B5232A6CF7@freebsd.org> : >>> "Robert N. M. Watson" writes: : >>> : I agree. I see two kinds of users: : >>> : : > : > [..] : > : > Reasoning behind -C was that in 2007, Robert and Peter mentioned about a need : > of having comments in the config file. I divided users into two groups: : : Why don't we stop dividing people in groups and classes and : instead just add directives for what to include. Such as: : : embed options : embed devices : embed comments : embed includes : embed "$FreeBSD$" : : Combine this with syntactic sugaring and you can have something : like: : embed everything : noembed comments : : Is this really so hard? Yes. Didn't you read the message I posted about why this is hard? There would be a ton of lexer and parser work to make this happen, as well as a lot of work to internal data structures to keep the file as parsed, rather than as convenient to do config's job. It is a big pita. Right now there's two places where you can tap into the config file information. The first one is where we know the filename of the file we're configuring. The second is after all the lexing has happened and we know the options, but their order has been lost as well as any formatting and text. The latter was done to address concerns over the 'include' directive, but it has its own set of problems and isn't perfect either. I think the way forward isn't as you suggest. It would be a ton of work to fix the lexer to carry forward the comments. A comments directive is just plain silly. The embedding stuff is too fine-grained control. It is a bandaide on a bandaide over a kludge. What is needed really is a fresher approach to solving the problem correctly. I've been thinking hard about this. The better approach is actually very simple to implement and do, and is very extensible. Stay with me here for a moment. First, we change the section in the ELF file that contains the config file a little. This section will define a couple of symbols, with weak symbols backing them in kern/kern_mib.c (more on why that particular file later). This new section would be just a normal text file. The mib that exports the current config file would export this text file. The config -x command that looks for the config would still print out the same section of the file. So far, there's no change from what we have today, which is good. The reporting side can remain exactly as it is. However, we'd have config create a new file in the kernel directory. It would contain a list of all the files that were included (directly or indirectly) as part of the config parsing. This would also include not only files brought in by the 'config' directive, but also files brought in by the env or hints directives. This change is very small, and about a dozen lines of code to implement. I could produce a patch easily enough. The build process would change a little. Rather than config creating config.c, the build process would create it based on a dependency on this file and all the files in this file. The build scripts would then create a simple shar file of all the files listed in this file (included, env, hints, etc). This would be embedded in the kernel. This would solve the 'I forgot to save this included file' problem completely. It would solve the ordering problems we have without -C. It would solve the comments going away. Heck, we could have two options: one to include the entire src/sys tree (modulo binaries) and one to just include the config files. If there are space concerns, it can be made into a gzip -9'd tarball and no longer text for all I care :). But the src/sys option would be a little slow and really bloat the kernel. Now, the reason that kern/kern_mib.c was singled out is that's where we export this blob to userland. My idea is to keep the text format that we have now for compatibility, and I'd rather not have the mib do the decompression. But if we break with compat, then we can easily have the binary blob that's preserved there which can trivially be extracted either by config -x, or by the sysctl on the running kernel. Frankly, that sort of thing is the right way to fix the include problem. There's no space reason for including just a subset: it was purely an easy of implementation choice that drove the two ways. If we really make it include everything, then -C can go away, the weird pseudo thing we have can go away, and we know get everything. And it is easy to implement... Comments? Warner From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 18:59:42 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DAAD106568F; Fri, 15 Jan 2010 18:59:42 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout029.mac.com (asmtpout029.mac.com [17.148.16.104]) by mx1.freebsd.org (Postfix) with ESMTP id 205958FC16; Fri, 15 Jan 2010 18:59:41 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=us-ascii Received: from macbook-pro.jnpr.net (natint3.juniper.net [66.129.224.36]) by asmtp029.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KWA00DXOY2OK690@asmtp029.mac.com>; Fri, 15 Jan 2010 10:59:30 -0800 (PST) From: Marcel Moolenaar In-reply-to: <20100115.110528.849557997928257031.imp@bsdimp.com> Date: Fri, 15 Jan 2010 10:59:11 -0800 Message-id: References: <20100115114822.O63406@delplex.bde.org> <20100115174711.GF1990@FreeBSD.org> <20100115.110528.849557997928257031.imp@bsdimp.com> To: "M. Warner Losh" X-Mailer: Apple Mail (2.1077) Cc: dougb@freebsd.org, svn-src-all@freebsd.org, wkoszek@freebsd.org, src-committers@freebsd.org, rwatson@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 18:59:42 -0000 On Jan 15, 2010, at 10:05 AM, M. Warner Losh wrote: > : Is this really so hard? > > Yes. Didn't you read the message I posted about why this is hard? No, I didn't. > There would be a ton of lexer and parser work to make this happen, as > well as a lot of work to internal data structures to keep the file as > parsed, rather than as convenient to do config's job. It is a big > pita. PITA != hard. If we're not willing to put in the effort to fix something, I don't think we should call it hard to do. We should call it like it is: non-trivial, involved or significant. Heck, we can even call it a major undertaking. But hard? No, I don't think it's hard at all. > I think the way forward isn't as you suggest. Fine. Just stop trying to classify people as a basis for what behaviour we should implement. It never works... > If > we really make it include everything, then -C can go away, the weird > pseudo thing we have can go away, and we know get everything. And it > is easy to implement... > > Comments? How does this address the "I don't want everything, I just want my CVS keyword" example? How does it handle the delicate balance between space vs. functionality that exists on embedded and/or low-end platforms. An all inclusive implementation seems not to take that into account that well. Sure, you can compress but then you add a runtime overhead to uncompress. In any case: I personally don't use the option so I really should not get involved. If I were to implement something from scratch though, I would treat it as a C file: the config file is the source and you "compile" it into a binary form suitable for inclusion into the kernel and you have compile-time options that control the binary output. No comments will be included in that case and there will be no option for it. But that's just me... -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 19:54:18 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A567810656FD; Fri, 15 Jan 2010 19:54:18 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 52E528FC36; Fri, 15 Jan 2010 19:54:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o0FJlPCH035384; Fri, 15 Jan 2010 12:47:25 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Fri, 15 Jan 2010 12:48:09 -0700 (MST) Message-Id: <20100115.124809.21010533849792633.imp@bsdimp.com> To: xcllnt@mac.com From: "M. Warner Losh" In-Reply-To: References: <20100115.110528.849557997928257031.imp@bsdimp.com> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: dougb@freebsd.org, svn-src-all@freebsd.org, wkoszek@freebsd.org, src-committers@freebsd.org, rwatson@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 19:54:18 -0000 In message: Marcel Moolenaar writes: : : On Jan 15, 2010, at 10:05 AM, M. Warner Losh wrote: : > : Is this really so hard? : > : > Yes. Didn't you read the message I posted about why this is hard? : : No, I didn't. : : > There would be a ton of lexer and parser work to make this happen, as : > well as a lot of work to internal data structures to keep the file as : > parsed, rather than as convenient to do config's job. It is a big : > pita. : : PITA != hard. If we're not willing to put in the effort to fix : something, I don't think we should call it hard to do. We should : call it like it is: non-trivial, involved or significant. Heck, : we can even call it a major undertaking. But hard? No, I don't : think it's hard at all. Agreed. But having retrofitted grammars in the past, coupled with the fact that config doesn't create a proper parse tree means we'd be rewriting huge portions of config, almost a complete rewrite from scratch, I'd say, would be necessary. And then you've sunk a huge amount of time into solving a tiny problem. Rewriting config should produce more benefits than just this one problem. That's why I called it hard. : > I think the way forward isn't as you suggest. : : Fine. Just stop trying to classify people as a basis for what : behaviour we should implement. It never works... Actually, engineering is all about identifying classes of users, and showing that solutions map well onto those classes of users. In showing which classes of users the solutions works well for, we can also find holes in the current setup. : > If : > we really make it include everything, then -C can go away, the weird : > pseudo thing we have can go away, and we know get everything. And it : > is easy to implement... : > : > Comments? : : How does this address the "I don't want everything, I just want : my CVS keyword" example? How does it handle the delicate balance : between space vs. functionality that exists on embedded and/or : low-end platforms. An all inclusive implementation seems not to : take that into account that well. Sure, you can compress but : then you add a runtime overhead to uncompress. I don't understand the "I just want my CVS keywords expanded" example. I think that's an orthogonal problem entirely (and maybe a small bug-fix to config to include an ident keyword if people really want the config file saved). The CVS keywords for the rest of the kernel are controlled by sys/cdefs, which creates .ident lines which the toolchain is responsible for keeping or ditching. Embedded platforms wouldn't embed the whole config file at all, so I don't see what we do for platforms that aren't resource sensitive would matter here. But all the includes is only 14k on amd64 and i386 (our biggest config file platforms). These particular config files compress down to about 6k. Embedded config files tend to be even smaller, and would likely compress down to about 1.5k-2k (AR71XX is 1.3k without the extra included files). Since the kernel is itself compressed, we wouldn't have to worry too much about it. So even in embedded, the resources consumed are small. : In any case: I personally don't use the option so I really should : not get involved. If I were to implement something from scratch : though, I would treat it as a C file: the config file is the source : and you "compile" it into a binary form suitable for inclusion into : the kernel and you have compile-time options that control the binary : output. No comments will be included in that case and there will be : no option for it. But that's just me... That's basically what's implemented to day: we're but a whisper away from a pure binary for (no -C)... Warner From owner-freebsd-arch@FreeBSD.ORG Fri Jan 15 20:37:37 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 971F7106566B; Fri, 15 Jan 2010 20:37:37 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout023.mac.com (asmtpout023.mac.com [17.148.16.98]) by mx1.freebsd.org (Postfix) with ESMTP id 79C158FC18; Fri, 15 Jan 2010 20:37:37 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=us-ascii Received: from macbook-pro.jnpr.net (natint3.juniper.net [66.129.224.36]) by asmtp023.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KWB006JR2MM3440@asmtp023.mac.com>; Fri, 15 Jan 2010 12:37:35 -0800 (PST) From: Marcel Moolenaar In-reply-to: <20100115.124809.21010533849792633.imp@bsdimp.com> Date: Fri, 15 Jan 2010 12:37:34 -0800 Message-id: <86DE85B0-2556-48A3-8AB0-C4969823A93E@mac.com> References: <20100115.110528.849557997928257031.imp@bsdimp.com> <20100115.124809.21010533849792633.imp@bsdimp.com> To: "M. Warner Losh" X-Mailer: Apple Mail (2.1077) Cc: dougb@freebsd.org, svn-src-all@freebsd.org, wkoszek@freebsd.org, src-committers@freebsd.org, rwatson@freebsd.org, freebsd-arch@freebsd.org, svn-src-head@freebsd.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 20:37:37 -0000 On Jan 15, 2010, at 11:48 AM, M. Warner Losh wrote: > : PITA != hard. If we're not willing to put in the effort to fix > : something, I don't think we should call it hard to do. We should > : call it like it is: non-trivial, involved or significant. Heck, > : we can even call it a major undertaking. But hard? No, I don't > : think it's hard at all. > > Agreed. But having retrofitted grammars in the past, coupled with the > fact that config doesn't create a proper parse tree means we'd be > rewriting huge portions of config, almost a complete rewrite from > scratch, I'd say, would be necessary. And then you've sunk a huge > amount of time into solving a tiny problem. Rewriting config should > produce more benefits than just this one problem. That's why I called > it hard. Yes. I do believe that it's a "big job" and it's fair to ask whether it's worth. > : How does this address the "I don't want everything, I just want > : my CVS keyword" example? *snip* > I don't understand the "I just want my CVS keywords expanded" > example. The inspiration came from Peter Jeremy's email in this thread. -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Sat Jan 16 02:51:33 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1044E106566B; Sat, 16 Jan 2010 02:51:33 +0000 (UTC) (envelope-from gad@FreeBSD.org) Received: from smtp6.server.rpi.edu (smtp6.server.rpi.edu [128.113.2.226]) by mx1.freebsd.org (Postfix) with ESMTP id A40798FC08; Sat, 16 Jan 2010 02:51:32 +0000 (UTC) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp6.server.rpi.edu (8.13.1/8.13.1) with ESMTP id o0G1awAu013377; Fri, 15 Jan 2010 20:36:59 -0500 Mime-Version: 1.0 Message-Id: In-Reply-To: <86DE85B0-2556-48A3-8AB0-C4969823A93E@mac.com> References: <20100115.110528.849557997928257031.imp@bsdimp.com> <20100115.124809.21010533849792633.imp@bsdimp.com> <86DE85B0-2556-48A3-8AB0-C4969823A93E@mac.com> Date: Fri, 15 Jan 2010 20:36:57 -0500 To: Marcel Moolenaar , "M. Warner Losh" From: Garance A Drosehn Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Bayes-Prob: 0.0001 (Score 0) X-RPI-SA-Score: 0.70 () [Hold at 20.00] COMBINED_FROM,J_CHICKENPOX_72 X-CanItPRO-Stream: outgoing X-Canit-Stats-ID: Bayes signature not available X-Scanned-By: CanIt (www . roaringpenguin . com) on 128.113.2.226 Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, rwatson@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: INCLUDE_CONFIG_FILE in GENERIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2010 02:51:33 -0000 At the risk of sounding overly simplistic, how hard is it to just go ahead and include both versions? In my current /boot/kernel/kernel , it looks like there's 2675 bytes of CONFIG_AUTOGENERATED lines. My original config file is 13540 bytes. I don't see that there's all that much pain in including both versions, maybe with an extra separator-line or two added in. And personally, I feel that both versions are useful to have. Then have OPTIONs for those who only wanted the original config, or only wanted the autogen'ed version, or who don't want either version. -- Garance Alistair Drosehn = drosehn@rpi.edu Senior Systems Programmer or gad@FreeBSD.org Rensselaer Polytechnic Institute; Troy, NY; USA From owner-freebsd-arch@FreeBSD.ORG Sat Jan 16 10:36:32 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F00A91065679; Sat, 16 Jan 2010 10:36:32 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail02.syd.optusnet.com.au (mail02.syd.optusnet.com.au [211.29.132.183]) by mx1.freebsd.org (Postfix) with ESMTP id 7CC158FC14; Sat, 16 Jan 2010 10:36:32 +0000 (UTC) Received: from c220-239-227-214.carlnfd1.nsw.optusnet.com.au (c220-239-227-214.carlnfd1.nsw.optusnet.com.au [220.239.227.214]) by mail02.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o0GAaSZ9021775 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 16 Jan 2010 21:36:30 +1100 Date: Sat, 16 Jan 2010 21:36:28 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Attilio Rao In-Reply-To: <3bbf2fe11001150706y765159a2jbd37c7ae4cf378f0@mail.gmail.com> Message-ID: <20100116205752.J64514@delplex.bde.org> References: <3bbf2fe10911271542h2b179874qa0d9a4a7224dcb2f@mail.gmail.com> <200911301305.30572.jhb@freebsd.org> <3bbf2fe11001150706y765159a2jbd37c7ae4cf378f0@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Arch , Ed Maste Subject: Re: [PATCH] Statclock aliasing by LAPIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2010 10:36:33 -0000 On Fri, 15 Jan 2010, Attilio Rao wrote: > I still see clock_lock in place (and no particular critical section > code in that paths) or you meant to say that the clock_lock doesn't > still provide enough protection alone? > BTW, you were right about the lapic_timer_hz (I forgot to revert to > hz). There is an updated patch: > http://www.freebsd.org/~attilio/Sandvine/STABLE_8/statclock_aliasing/statclock_aliasing4.diff It seems to have the same fundamental bugs as the previous version. The atrtc interrupt is too slow to use for anything, so it should never be used if there is something better like the lapic timer available (even the i8254 is better), and using it here doesn't even fix the problem (malicious applications can very easily hide from statclock by default since the default hz is much larger than the default stathz, and malicious applications can not so easily hide from statclock irrespective of the misconfiguration of hz, since statclock is not random). See my previous reply and ftp://ftp.ee.lbl.gov/papers/statclk-usenix93.ps.Z for more details. I checked that the simple exploit in that old paper is too simple to exploit FreeBSD-~5.2 with SCHED_4BSD, hz = 100 and no lapic timer. Under FreeBSD-8 with defaults (ref8-i386), it is too simple to exploit even its own equal share of the cycles (it tries to hide from stathz but assumes stathz = 100, and my simple attempts to improve this didn't work, so it apparently loses due to misprediction -- a naive CPU hog got about 20% more cycles than this simplistic hog). I tested only with 1 naive CPU hog and 2 simplistic hogs (from the fork) per CPU. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Jan 16 12:09:48 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40D6D1065697; Sat, 16 Jan 2010 12:09:48 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-fx0-f227.google.com (mail-fx0-f227.google.com [209.85.220.227]) by mx1.freebsd.org (Postfix) with ESMTP id 7899B8FC17; Sat, 16 Jan 2010 12:09:46 +0000 (UTC) Received: by fxm27 with SMTP id 27so907982fxm.3 for ; Sat, 16 Jan 2010 04:09:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=SdCB0Yyh65L5inMqZkNZ1eFChZ3OfQQn02UnA5vsGy4=; b=xlm7oCu0/qzL2FXNJnw5Z+AvftNIa/vdPjMtspn5pAfMGgkz/WWDn6M46qVwesPslj JppiV6w8PWpOdGYR0HwwBVTfXy+pdMdeFf+jAEWx4WPXdyEBCPIEjr5bmpZy0/jbx6Ld ocOt2KvMova6KeZIu4l+Sya1GLMmlETHwUtg8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=tC/J4v9AjhGOXWv9IZR91f3qnmg6RA45eivrtzltaSfz6ALFcAp1IqTeVDzeOqCoSH Ito0rG33EVhIR9IwwaOXJZn8wXR1DACUUnV1Tynxc/ITvc+K5STn//Wu+Y0ayVxCazJa LMBitmeC1mJkGDDa+dvA4hQsZbSvpZXIqKb6Q= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.76.142 with SMTP id c14mr4020206fak.92.1263643778940; Sat, 16 Jan 2010 04:09:38 -0800 (PST) In-Reply-To: <20100116205752.J64514@delplex.bde.org> References: <3bbf2fe10911271542h2b179874qa0d9a4a7224dcb2f@mail.gmail.com> <200911301305.30572.jhb@freebsd.org> <3bbf2fe11001150706y765159a2jbd37c7ae4cf378f0@mail.gmail.com> <20100116205752.J64514@delplex.bde.org> Date: Sat, 16 Jan 2010 13:09:38 +0100 X-Google-Sender-Auth: 05ef76d70944c239 Message-ID: <3bbf2fe11001160409w1dfdbb9j36458c52d596c92a@mail.gmail.com> From: Attilio Rao To: Bruce Evans Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Arch , Ed Maste Subject: Re: [PATCH] Statclock aliasing by LAPIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2010 12:09:48 -0000 2010/1/16 Bruce Evans : > On Fri, 15 Jan 2010, Attilio Rao wrote: > >> I still see clock_lock in place (and no particular critical section >> code in that paths) or you meant to say that the clock_lock doesn't >> still provide enough protection alone? >> BTW, you were right about the lapic_timer_hz (I forgot to revert to >> hz). There is an updated patch: >> >> http://www.freebsd.org/~attilio/Sandvine/STABLE_8/statclock_aliasing/sta= tclock_aliasing4.diff > > It seems to have the same fundamental bugs as the previous version. > The atrtc interrupt is too slow to use for anything, so it should never > be used if there is something better like the lapic timer available > (even the i8254 is better), and using it here doesn't even fix the > problem (malicious applications can very easily hide from statclock > by default since the default hz is much larger than the default stathz, > and malicious applications can not so easily hide from statclock > irrespective > of the misconfiguration of hz, since statclock is not random). =C2=A0See = my > previous reply and ftp://ftp.ee.lbl.gov/papers/statclk-usenix93.ps.Z for > more details. Well, the primary things I wanted to fix is not the hiding of malicious programs but the clock aliasing created when handling all the clocks by the same source. About the slowness -- I'm fine with whatever additional source to LAPIC we would eventually use thus would you feel better if i8254 is used replacing atrtc? Also note that atrtc is the default if LAPIC cannot be used. I don't understand why another source, even simpler (eg. i8254) would have been used in that specific case by the 'old' code. What I mean, then is: I see your points, I'm not arguing that at all, but the old code has other problems that gets fixed with this patch (having different sources make the whole system more flexible) while the new things it does introduce are secondarilly (but still: I'm fine with whatever second source is picked up for statclock, profclock) if you really see a concern wrt atrtc slowness. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sat Jan 16 12:54:54 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 210F21065698; Sat, 16 Jan 2010 12:54:54 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id D8DD98FC23; Sat, 16 Jan 2010 12:54:53 +0000 (UTC) Received: from toad.stack.nl (toad.stack.nl [IPv6:2001:610:1108:5010::135]) by mx1.stack.nl (Postfix) with ESMTP id E3EA735A825; Sat, 16 Jan 2010 13:54:51 +0100 (CET) Received: by toad.stack.nl (Postfix, from userid 1677) id CDC7073F9D; Sat, 16 Jan 2010 13:54:51 +0100 (CET) Date: Sat, 16 Jan 2010 13:54:51 +0100 From: Jilles Tjoelker To: Attilio Rao Message-ID: <20100116125451.GA99364@stack.nl> References: <3bbf2fe11001091709t228c48cft1c17af686e9e9c46@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3bbf2fe11001091709t228c48cft1c17af686e9e9c46@mail.gmail.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-arch@freebsd.org Subject: Re: [PATCH] kthread_{suspend, resume, suspend_check} locking bugs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2010 12:54:54 -0000 On Sun, Jan 10, 2010 at 02:09:43AM +0100, Attilio Rao wrote: > I think that routines kthread_suspend(), kthread_resume() and > kthread_suspend_check() are not adeguately SMP protected. > That is because, in particular, the critical path doesn't protect, > together, TDF_KTH_SUSP and sleeping activity. The right pattern should > be to use the thread lock spinlock as an interlock and use msleep. > Such bugs have not been revealed probabilly because there has been a > lack of testing of such primitives and there are not, currently, > consumers within our stock kernel. > Additively, kthread_suspend_check() seems to require to always pass > curthread, which is silly (as we don't have to conform to any > particular KPI), thus I think it is appropriate for the prototype to > change. > The following patch should fix the issue: > http://www.freebsd.org/~attilio/kthread.diff The analysis and patch look sensible. > That patch would have benefit from the priority argument (for PDROP) > for msleep_spin(9), and I don't understand why we don't support it > (the log message doesn't seem much clearer). > If nobody objects, after this patch goes in I would add the priority > argument to msleep_spin() too. No objection here. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Sat Jan 16 13:18:40 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D559106566C for ; Sat, 16 Jan 2010 13:18:40 +0000 (UTC) (envelope-from bf1783@googlemail.com) Received: from mail-ew0-f226.google.com (mail-ew0-f226.google.com [209.85.219.226]) by mx1.freebsd.org (Postfix) with ESMTP id 181C68FC17 for ; Sat, 16 Jan 2010 13:18:39 +0000 (UTC) Received: by ewy26 with SMTP id 26so1790556ewy.3 for ; Sat, 16 Jan 2010 05:18:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=57jiny1X1IVcb6j8QPjGRuC9m4TkEInSBycs4uvD+/w=; b=LhmNZIjUa6V8WZarB5P01vHO95xHJk9eWJmT0jydfosoh6ZBRlBtdHvJYw6G9mbaHg yhcUfvKdAlD9LBgnXCPpi/Y0TNNq7a6pUwBo3lgnwTxBg9pKeP8RPkRQyiJDILYUMDB5 1/QfWCtuzcweDt9/A6eF7u1a841ZQxWfxGVAY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=Jb5iVq+r1xlNsml0vojETB5thzusHKbrnFO3esila4z2QoTjm5M8gPmtru+XTRpYJB mvRNANM2gFaHccSUh/pNZnEbWBvf/ODjMOQZBGDPlTqpUBBXYVLqI9I1VKJeluuI7iw/ tqhPNHZ8svLKAcOBZZ2tviQDiP/h2p1ExEwqo= MIME-Version: 1.0 Received: by 10.216.89.131 with SMTP id c3mr1286227wef.197.1263646558309; Sat, 16 Jan 2010 04:55:58 -0800 (PST) Date: Sat, 16 Jan 2010 07:55:58 -0500 Message-ID: From: "b. f." To: freebsd-arch@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Re: [PATCH] Statclock aliasing by LAPIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2010 13:18:40 -0000 >About the slowness -- I'm fine with whatever additional source to >LAPIC we would eventually use thus would you feel better if i8254 is >used replacing atrtc? ... >the new things it does introduce are secondarilly (but still: I'm fine >with whatever second source is picked up for statclock, profclock) if >you really see a concern wrt atrtc slowness. Would it be possible to to split kern.timecounter.hardware into several tunables, and make the choice of hardware (and perhaps frequency) configurable, at least to an extent, for each of the clocks? b. From owner-freebsd-arch@FreeBSD.ORG Sat Jan 16 14:01:55 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A896106566B for ; Sat, 16 Jan 2010 14:01:55 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-fx0-f227.google.com (mail-fx0-f227.google.com [209.85.220.227]) by mx1.freebsd.org (Postfix) with ESMTP id 924208FC0C for ; Sat, 16 Jan 2010 14:01:54 +0000 (UTC) Received: by fxm27 with SMTP id 27so955562fxm.3 for ; Sat, 16 Jan 2010 06:01:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=R1VfMQdYPHn+6QE4nxMhgWkVY0Hs58qzfUHJvzRISqI=; b=haoMQ/kxwt6n3S12/omYaNK3UbrXoM4NuiVUc70IWAR9iQLuR5uc73GkCrQJEbnCbj CBpqROxJZ7SbsZyriuh3K0MVralqxBse2PuRWpkOPYrSZ5eSOorkO5r67Fc6710d9SNU VZhpME4RFdoRfBawZK+h5zDY3Fhw1RSJ5546w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=K8o+i+BHG48z5WQCLeh2iBg7Rc3Kxk3kHbF0mSC2n5Lx0BVKwv+zen4xNs99T5cKA+ F90xxtqVihwkBF49M5XK6G4yT/OWgauCceIiHqBJZLpZzn7Gxc91gF3qBfTnnMvETPkF KIXoKDJDwefgQnTdtxOVQ7TLmx4bYe325Ef9w= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.144.208 with SMTP id a16mr764667fav.62.1263650509223; Sat, 16 Jan 2010 06:01:49 -0800 (PST) In-Reply-To: References: Date: Sat, 16 Jan 2010 15:01:49 +0100 X-Google-Sender-Auth: 0d0d40dfcea9c9aa Message-ID: <3bbf2fe11001160601t1f488da3ud848cad7100b75d4@mail.gmail.com> From: Attilio Rao To: "b. f." Content-Type: text/plain; charset=UTF-8 Cc: freebsd-arch@freebsd.org Subject: Re: [PATCH] Statclock aliasing by LAPIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2010 14:01:55 -0000 2010/1/16 b. f. : >>About the slowness -- I'm fine with whatever additional source to >>LAPIC we would eventually use thus would you feel better if i8254 is >>used replacing atrtc? > ... >>the new things it does introduce are secondarilly (but still: I'm fine >>with whatever second source is picked up for statclock, profclock) if >>you really see a concern wrt atrtc slowness. > > Would it be possible to to split kern.timecounter.hardware into > several tunables, and make the choice of hardware (and perhaps > frequency) configurable, at least to an extent, for each of the > clocks? I think that several things can be made. What this patch just does is to increase the flexibility of the clocks tuning and the 'loose end' is choosen by what was already happening in the old kernel. I don't think the concept is wrong per-se (we could get it being even more flexible honestly) maybe the choice of clocks is not ideal though as Bruce pointed out with his quality tests, but still falls in 'what was used before' category. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sat Jan 16 14:44:13 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E47E106566C; Sat, 16 Jan 2010 14:44:13 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail07.syd.optusnet.com.au (mail07.syd.optusnet.com.au [211.29.132.188]) by mx1.freebsd.org (Postfix) with ESMTP id A39F98FC14; Sat, 16 Jan 2010 14:44:12 +0000 (UTC) Received: from c220-239-227-214.carlnfd1.nsw.optusnet.com.au (c220-239-227-214.carlnfd1.nsw.optusnet.com.au [220.239.227.214]) by mail07.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o0GEi935002504 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 17 Jan 2010 01:44:10 +1100 Date: Sun, 17 Jan 2010 01:44:09 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Attilio Rao In-Reply-To: <3bbf2fe11001160409w1dfdbb9j36458c52d596c92a@mail.gmail.com> Message-ID: <20100116235558.E64689@delplex.bde.org> References: <3bbf2fe10911271542h2b179874qa0d9a4a7224dcb2f@mail.gmail.com> <200911301305.30572.jhb@freebsd.org> <3bbf2fe11001150706y765159a2jbd37c7ae4cf378f0@mail.gmail.com> <20100116205752.J64514@delplex.bde.org> <3bbf2fe11001160409w1dfdbb9j36458c52d596c92a@mail.gmail.com> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-201474786-1263653049=:64689" Cc: FreeBSD Arch , Ed Maste Subject: Re: [PATCH] Statclock aliasing by LAPIC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jan 2010 14:44:13 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-201474786-1263653049=:64689 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Sat, 16 Jan 2010, Attilio Rao wrote: > 2010/1/16 Bruce Evans : >> On Fri, 15 Jan 2010, Attilio Rao wrote: >>> ... There is an updated patch: >>> >>> http://www.freebsd.org/~attilio/Sandvine/STABLE_8/statclock_aliasing/st= atclock_aliasing4.diff >> >> It seems to have the same fundamental bugs as the previous version. >> The atrtc interrupt is too slow to use for anything, so it should never >> be used if there is something better like the lapic timer available >> (even the i8254 is better), and using it here doesn't even fix the >> problem (malicious applications can very easily hide from statclock >> by default since the default hz is much larger than the default stathz, >> and malicious applications can not so easily hide from statclock >> irrespective >> of the misconfiguration of hz, since statclock is not random). =C2=A0See= my >> previous reply and ftp://ftp.ee.lbl.gov/papers/statclk-usenix93.ps.Z for >> more details. > > Well, the primary things I wanted to fix is not the hiding of > malicious programs but the clock aliasing created when handling all > the clocks by the same source. Those could easily be a scheduler bug, or at least fixable there. > About the slowness -- I'm fine with whatever additional source to > LAPIC we would eventually use thus would you feel better if i8254 is > used replacing atrtc? You can just probably just use the LAPIC with programmed pseudo-not-very- randomness (to delivery, not to the LAPIC interrupts which probably need to remain periodic). > Also note that atrtc is the default if LAPIC cannot be used. I don't > understand why another source, even simpler (eg. i8254) would have > been used in that specific case by the 'old' code. The i8254 restarts itself automatically. It only needs an EOI, which is fairly efficient if it is on APIC. Thus interrupting at say 2 KHz with i8254 hopefully has less overhead than interrupting at 128 Hz with atrtc, provided the i8254 interrupt handler does no more than the lapic_timer one. The i8254 is also programmable, so you can change its period easily though not efficiently to program randomness with a resolution of a few microseconds. > What I mean, then is: I see your points, I'm not arguing that at all, > but the old code has other problems that gets fixed with this patch > (having different sources make the whole system more flexible) while > the new things it does introduce are secondarilly (but still: I'm fine > with whatever second source is picked up for statclock, profclock) if > you really see a concern wrt atrtc slowness. Did you see the points in my more detailed review? We want to remove support for the old clock sources eventually, not have more code to select them and more non-default configuration to avoid them again. I think I understand the actual bug now. It is in lapic_handle_timer(). Statclock interrupts should never be delivered on the same lapic timer interrupt as a hardclock interrupt (this is possible, at least with default hz's, since hardclock interrupts are delivered every second lapic timer interrupt), but they are. This at best results in every second statclock interrupt being in perfect sync with some hardclock interrupt (I think it actually gives bunches of about lapic_timer_hz/stathz= /2 (default 7 or 8) in or out of perfect sync. Maybe the bunches are what makes the problem serious). To fix this, statclock interrupts should be delayed until the next lapic timer interrupt if a hardclock interrupt was just delivered, or done early if the next delay would be more than half a lapic timer period. This delay/advance also gives some free pseudo-randomness. See my more detailed review about statistics utilities not liking any randomness. Too bad for them. The non-divisibility of lapic_timer_hz by stathz with defaults already gives large delays. You can even reduce the maximum jitter using delay/advance instead of the current method (from almost 1 lapic timer period (always late) to +- 1/2 of 1 lapic timer period (early or late)). I don't see any reason to keep using stathz =3D 133 or even a non-multiple or non-divisor or hz (but it needs to remain nearly 128 for historical reasons until other layers are changed). The non-multiple is to ensure that independent clocks don't stay in sync for long. Programmed non-sync ensures this better. Malicious programs just have different problems predicting the 2 types of pseudo-not-very-randomness. With statclock ticks occurring exactly half-way between hardclock ticks, malicious programs can a bit too easily wake up on a hardclock tick and run for a half less epsilon of a hardclock tick without getting accounted. Oops, non-malicious programs can also do this a bit too easily -- you can have a thundering herd wake up and all finish before the accounting. More pseudo-randomness seems to be needed. I don't see a good way to handle the thundering herd case. For that you actually want the statclock tick immediately after the hardclock tick (but not in sync) quite often. The following might work except for inefficiency (time/power): make lapic_timer_hz say 10 times larger and distribute statclock delivery randomly about hardclock delivery in the 39 slots 0, +-lapic_timer_period, ... +-19*lapic_timer_period, instead of only in the 2 +-lapic_timer_period slots. First try using the 0 slot with just these 2. Perhaps similarly for profclock, but when it is fixed it should be much larger than hz (10-10000 kHz according to machine speed and/or sysctl), so lapic_timer_hz would have to be enormous to give a small relative jitter for profclock. Bruce --0-201474786-1263653049=:64689--