From owner-freebsd-arch@FreeBSD.ORG Sun Jan 16 00:12:46 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0FF7106566B for ; Sun, 16 Jan 2011 00:12:46 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8CB908FC13 for ; Sun, 16 Jan 2011 00:12:46 +0000 (UTC) Received: by iwn39 with SMTP id 39so3624873iwn.13 for ; Sat, 15 Jan 2011 16:12:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:date:x-google-sender-auth :message-id:subject:from:to:content-type; bh=oVL5OOIXiSM6OTJkE1+U0j7omGPBHAKlwPIFOa3UTeQ=; b=dOCsCwtu4cGa4gBwR5TTuHvgJqZqa3c2vMuObrGLk43jBZnWsVGVFL2tRpQFSDnZRI on+c8elM/M03ydYAMVY8EGHHgXGeqx8YP81obd4oiw1wEoU6OjH/701ZjKLQpXwmJw2P Z2kq7DEA1qBNlx48xsfJy5Le4+x4xaDhgGsr8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=slh0Bf8d579z5uph/RSLRawe7ACipQXH8OxG5lNkrwKenTZF4hBi21UT0TFlwF5u7s oX2hwWZ5jplQinBoa1mIvKP3LRUm7+Cdvdect5GhNVLfwfPzoMB6KcsVEV+nC7tNHvmj s/ucOXunikJ2+ulpWMST+Rw+yhnoHd5sZWjME= MIME-Version: 1.0 Received: by 10.231.35.141 with SMTP id p13mr2430037ibd.79.1295136765957; Sat, 15 Jan 2011 16:12:45 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.231.160.147 with HTTP; Sat, 15 Jan 2011 16:12:45 -0800 (PST) Date: Sat, 15 Jan 2011 16:12:45 -0800 X-Google-Sender-Auth: QkULITlZ2OZ2JPqRN1AYFQYNAsk Message-ID: From: mdf@FreeBSD.org To: FreeBSD Arch Content-Type: text/plain; charset=ISO-8859-1 Subject: Automagic SYSCTLs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2011 00:12:46 -0000 I started out with a plan to import a patch from Isilon that adds type-safety to the existing SYSCTLs for scalar variables in the kernel. After the discussion on the svn mailing list here (http://lists.freebsd.org/pipermail/svn-src-head/2011-January/024097.html) I now have a prototype patch for SYSCTL_ADD_SCALAR that could replace most uses of SYSCTL_ADD_{INT, UINT, LONG, ULONG, QUAD} and is suitable for the SYSCTL_ADD_FOO that are used on 16 and 8 bit members (without type checking) today. The gist is that the handler knows the sizeof the variable in the kernel and uses this to copy out. For the case of a long, there's some goop for SCTL_MASK32. For the case of 8 and 16 bit variables, they are still copied in and out as 32-bit quantities. Let me know if this seems like the right or wrong direction in which to move. I haven't tackled the static sysctls as the code I have does some run-time evaluation because the code is easier to write that way. One possibility is to change the sysctl_oid struct or at least add a SIGNED flag; this would also have the advantage of making it possible to clean up the CTLTYPE_[U]INT issue where there is no real difference between the two CTLTYPEs. http://people.freebsd.org/~mdf/bsd-sysctl-scalar.diff Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Sun Jan 16 01:10:33 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5BA19106564A; Sun, 16 Jan 2011 01:10:33 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id BD2A78FC08; Sun, 16 Jan 2011 01:10:32 +0000 (UTC) Received: by wwf26 with SMTP id 26so4195901wwf.31 for ; Sat, 15 Jan 2011 17:10:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=2ee/h2P9NdKAwxIVUvaI7SS1i879NYuVjUI8lhAdwbY=; b=GnhbcbNNFEq4OsCR2F1z9WztzaoaiNHQUXJBFxoYRpKHftgNqoXtk8hI9T21gjpE9j Ji0tK/7U3DgARdnkKzatUA2RZN6x8E8Rsiqky0YVTwOX6vJqZ9X7sDXzqdaYQcWH/vSe aCE9WLwSaYI0RheZy5z5ryYCgGwW4oCgmeQH4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=fhiBjd+gh9Un2ceN3G4GUhPRtsOyFR+3J3TExyUZLv7+QusCJWGgEFtIvi/lnkw6HV u+yAENmc5WlehVTq+XeZ/cZN7qYa2daFZTszzD6+U5XhJ2Vq7AghHbrJkqGueKGtArc3 HYtBVinPiHQXHB0b1jdlnmPnzMx5Mn55MfyNE= MIME-Version: 1.0 Received: by 10.216.30.137 with SMTP id k9mr2026912wea.31.1295140231581; Sat, 15 Jan 2011 17:10:31 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Sat, 15 Jan 2011 17:10:31 -0800 (PST) In-Reply-To: References: Date: Sat, 15 Jan 2011 17:10:31 -0800 X-Google-Sender-Auth: HJ5AnRqLuxC40OuBzhyEwPk7WM8 Message-ID: From: Garrett Cooper To: mdf@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Arch Subject: Re: Automagic SYSCTLs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2011 01:10:33 -0000 On Sat, Jan 15, 2011 at 4:12 PM, wrote: > I started out with a plan to import a patch from Isilon that adds > type-safety to the existing SYSCTLs for scalar variables in the > kernel. =A0After the discussion on the svn mailing list here > (http://lists.freebsd.org/pipermail/svn-src-head/2011-January/024097.html= ) > I now have a prototype patch for SYSCTL_ADD_SCALAR that could replace > most uses of SYSCTL_ADD_{INT, UINT, LONG, ULONG, QUAD} and is suitable > for the SYSCTL_ADD_FOO that are used on 16 and 8 bit members (without > type checking) today. > > The gist is that the handler knows the sizeof the variable in the > kernel and uses this to copy out. =A0For the case of a long, there's > some goop for SCTL_MASK32. =A0For the case of 8 and 16 bit variables, > they are still copied in and out as 32-bit quantities. > > Let me know if this seems like the right or wrong direction in which > to move. =A0I haven't tackled the static sysctls as the code I have does > some run-time evaluation because the code is easier to write that way. > =A0One possibility is to change the sysctl_oid struct or at least add a > SIGNED flag; this would also have the advantage of making it possible > to clean up the CTLTYPE_[U]INT issue where there is no real difference > between the two CTLTYPEs. > > http://people.freebsd.org/~mdf/bsd-sysctl-scalar.diff Looks interesting, but what are the performance implications of the new log= ic? Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Sun Jan 16 05:35:54 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 496641065672; Sun, 16 Jan 2011 05:35:54 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id F2D718FC08; Sun, 16 Jan 2011 05:35:53 +0000 (UTC) Received: by iyb26 with SMTP id 26so3807055iyb.13 for ; Sat, 15 Jan 2011 21:35:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=cSbf6Pk9uBZpWF26YXUAvapuiLgsaTIYlppU7eK9U0Q=; b=O4jx+A3PQLo/nn+/sbCo+VNLSu31cii8NbXWumJVYQstL/xhlKC0a9hHQ40A0nw6cV xSLNPGz+t86pil8JFHD1Y9VI4iVXOitlwreOQv1mvO/idAJsmCbZS3IEpakZnlIGW75O X8N2CbP4L2BBh0XPJY288IHwOG8N6LLJK83ZQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=xVJuzttOYKfE+VV4qU+0AO+zw5eenQ3WLy02kamdHzl5flHsQCdV418U5fSSTom0Vx ql10AckjBWJVjXspfeH5mslma3TlNZRe05gpsNci8myvK6ZRSJ6CpMnOWJugk8eJhvJu RfhPo1SvIsJ7IYjx37KqxCMIpqj9PRtqrsh4k= MIME-Version: 1.0 Received: by 10.231.173.67 with SMTP id o3mr2661561ibz.29.1295156153493; Sat, 15 Jan 2011 21:35:53 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.231.160.147 with HTTP; Sat, 15 Jan 2011 21:35:53 -0800 (PST) In-Reply-To: References: Date: Sat, 15 Jan 2011 21:35:53 -0800 X-Google-Sender-Auth: 6WFjP7d0PXKl6vrYMOL3dAnYF9U Message-ID: From: mdf@FreeBSD.org To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Arch Subject: Re: Automagic SYSCTLs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2011 05:35:54 -0000 On Sat, Jan 15, 2011 at 5:10 PM, Garrett Cooper wrote= : > On Sat, Jan 15, 2011 at 4:12 PM, =A0 wrote: >> I started out with a plan to import a patch from Isilon that adds >> type-safety to the existing SYSCTLs for scalar variables in the >> kernel. =A0After the discussion on the svn mailing list here >> (http://lists.freebsd.org/pipermail/svn-src-head/2011-January/024097.htm= l) >> I now have a prototype patch for SYSCTL_ADD_SCALAR that could replace >> most uses of SYSCTL_ADD_{INT, UINT, LONG, ULONG, QUAD} and is suitable >> for the SYSCTL_ADD_FOO that are used on 16 and 8 bit members (without >> type checking) today. >> >> The gist is that the handler knows the sizeof the variable in the >> kernel and uses this to copy out. =A0For the case of a long, there's >> some goop for SCTL_MASK32. =A0For the case of 8 and 16 bit variables, >> they are still copied in and out as 32-bit quantities. >> >> Let me know if this seems like the right or wrong direction in which >> to move. =A0I haven't tackled the static sysctls as the code I have does >> some run-time evaluation because the code is easier to write that way. >> =A0One possibility is to change the sysctl_oid struct or at least add a >> SIGNED flag; this would also have the advantage of making it possible >> to clean up the CTLTYPE_[U]INT issue where there is no real difference >> between the two CTLTYPEs. >> >> http://people.freebsd.org/~mdf/bsd-sysctl-scalar.diff > > Looks interesting, but what are the performance implications of the new l= ogic? I guess I'm not quite sure how to answer this. The add should be about the same, and it's only done once at kernel or driver init. The handler time is essentially meaningless since even reading the value using sysctl(8) will take the SYSCTL lock several times, and doing a write will take it quite a few more. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Sun Jan 16 09:53:49 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E0F9106564A; Sun, 16 Jan 2011 09:53:49 +0000 (UTC) (envelope-from gljennjohn@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5A9978FC15; Sun, 16 Jan 2011 09:53:47 +0000 (UTC) Received: by fxm16 with SMTP id 16so4818062fxm.13 for ; Sun, 16 Jan 2011 01:53:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:x-mailer:mime-version:content-type :content-transfer-encoding; bh=SDdosm+6R6MCULql9byVUVQFstOqbSLHKNJUQgZDr8k=; b=b9HZjb61GqxM+987LTltEObo9CNvJ/jbwn3IYUFV1dTQykO6olndBlJDsdZu8JhvK6 rA0FI5fn/CXABptxTOGPS4HBVQdO8OwgsdSZ/c+/6YcM44A+E/TZ0h0gMGgEukFxzGkO s/AB+/M3okhmOmRbAcwB5dXKNc7JT5utvGbSw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; b=L1bUyV2iNk9crigOD3mzAngkuVfzDzZqLqD7lbCGKs9/RwfRK868EzE9dVBwk6Bz/9 OILBG4mlCvOVzUuFwVAD6nGpmwOlDP+pwTIY2q/4x/a23Q9gWt6eF4mykRbBOg1wUSAj VQtWG0g9OfLArklRxaXj+jw+NP+NK12oPcndU= Received: by 10.223.118.136 with SMTP id v8mr3172569faq.90.1295170102382; Sun, 16 Jan 2011 01:28:22 -0800 (PST) Received: from ernst.jennejohn.org (p578E3E8E.dip.t-dialin.net [87.142.62.142]) by mx.google.com with ESMTPS id n2sm1157020fam.28.2011.01.16.01.28.19 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 16 Jan 2011 01:28:21 -0800 (PST) Date: Sun, 16 Jan 2011 10:28:18 +0100 From: Gary Jennejohn To: Nathan Whitehorn Message-ID: <20110116102818.0667dd49@ernst.jennejohn.org> In-Reply-To: <4D309563.1000404@freebsd.org> References: <4D309563.1000404@freebsd.org> X-Mailer: Claws Mail 3.7.8 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, freebsd-arch@FreeBSD.org Subject: Re: BSDInstall: merging to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gljennjohn@googlemail.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2011 09:53:49 -0000 On Fri, 14 Jan 2011 12:26:43 -0600 Nathan Whitehorn wrote: > As those of you who have been reading freebsd-sysinstall and > freebsd-arch know, I have been working for a few weeks on a lightweight > new installer named 'bsdinstall'. This is designed to replace sysinstall > for the 9.0 release. > > After two weeks of testing and bug fixes on the sysinstall list, I > believe this now has all required functionality and is ready to be > merged into the main source tree. I would like to do this on Tuesday, 18 > January. Switching this to be the default installer would happen a few > weeks after that, pending discussion on release formats with the release > engineering team. This should provide a sufficient testing period before > 9.0 and allow a maximal number of bugs to be discovered and solved > before the release is shipped. > > Demo ISO for i386: > http://people.freebsd.org/~nwhitehorn/bsdinstall-i386-20110114.iso.bz2 > SVN repository: svn://svn.freebsd.org/base/user/nwhitehorn/bsdinstall > Wiki page: http://wiki.freebsd.org/BSDInstall > I installed this under VirtualBox yesterday. The only porblem I noticed was that adding a user didn't actually work, although it appeared to do so (it asked at the end of the process whether all data for the new user were correct and then claimed to have added the user). Looking at /etc/passwd and in /home after booting the new installation showed that the user was never added. Otherwise it was a smooth install, although I didn't try anything fancy and just used the quick install and the entire disk for simpicity. -- Gary Jennejohn From owner-freebsd-arch@FreeBSD.ORG Sun Jan 16 11:35:02 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F159F106564A; Sun, 16 Jan 2011 11:35:01 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 10D648FC08; Sun, 16 Jan 2011 11:35:00 +0000 (UTC) Received: by wwf26 with SMTP id 26so4412217wwf.31 for ; Sun, 16 Jan 2011 03:35:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=eA/SUMdFKUUpXpjmSyGSA7cYUhCYDewUq6bo3fpC3AI=; b=MrnO7jRBOmRX5bltAMYCE5cqlarx//gEjU1G571CwlJG8ZFNXDb0MzwRe8t9715YTr 9fhBR3j+oGdwdbrBu66ddZRcRKJmqR7RWUg6X9QI/UKmN8i+HFnIGDquoV5KecdLWkhY L+FVOpUBLNaFCg7xbIYQNBgHAYay8WE6nHtOc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=gCtaolOPsJVwZVOlNcze8keTV++YFnTM8T7xy58p0RFA+Ubcke7IIUEjuGPXpuOJfn LZjUhCJ+wb47O0iguhUqsujnKp0mYZCoF3qCL2qcpKeSQ/0U43zPBbtAtxvosTGus4YY WBmhZW6hwMpXGdm6NsFz65QTDyC/p1xsShDMQ= MIME-Version: 1.0 Received: by 10.216.51.135 with SMTP id b7mr1311984wec.29.1295176011430; Sun, 16 Jan 2011 03:06:51 -0800 (PST) Received: by 10.216.229.73 with HTTP; Sun, 16 Jan 2011 03:06:51 -0800 (PST) In-Reply-To: <4D309563.1000404@freebsd.org> References: <4D309563.1000404@freebsd.org> Date: Sun, 16 Jan 2011 11:06:51 +0000 Message-ID: From: krad To: Nathan Whitehorn Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, freebsd-arch@freebsd.org Subject: Re: BSDInstall: merging to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2011 11:35:02 -0000 On 14 January 2011 18:26, Nathan Whitehorn wrote: > > As those of you who have been reading freebsd-sysinstall and freebsd-arch= know, I have been working for a few weeks on a lightweight new installer n= amed 'bsdinstall'. This is designed to replace sysinstall for the 9.0 relea= se. > > After two weeks of testing and bug fixes on the sysinstall list, I believ= e this now has all required functionality and is ready to be merged into th= e main source tree. I would like to do this on Tuesday, 18 January. Switchi= ng this to be the default installer would happen a few weeks after that, pe= nding discussion on release formats with the release engineering team. This= should provide a sufficient testing period before 9.0 and allow a maximal = number of bugs to be discovered and solved before the release is shipped. > > Demo ISO for i386: http://people.freebsd.org/~nwhitehorn/bsdinstall-i386-= 20110114.iso.bz2 > SVN repository: svn://svn.freebsd.org/base/user/nwhitehorn/bsdinstall > Wiki page: http://wiki.freebsd.org/BSDInstall > > Goals > ----- > The primary goal of BSDInstall is to provide an easily extensible install= er without the limitations of sysinstall, in order to allow more modern ins= tallations of FreeBSD. This means that it should have additional features t= o support modern setups, but simultaneously frees us to remove complicating= features of sysinstall like making sure everything fits in floppy disk-siz= ed chunks. > > New Features: > - Allows installation onto GPT disks on x86 systems > - Can do installations spanning multiple disks > - Allows installation into jails > - Eases PXE installation > - Virtualization friendly: can install from a live system onto disk > =A0images > - Works on PowerPC > - Streamlined system installation > - More flexible scripting > - Easily tweakable > - All install CDs are live CDs > > Architecture > ------------ > BSDInstall is a set of tools that are called in sequence by a master scri= pt. These tools are, for example, the partition editor, the thing that fetc= hes the distributions from the network, the thing that untars them, etc. Si= nce these are just called in sequence from a shell script, a scripted insta= llation can easily replace them with other things, (e.g. hard-coded gpart c= ommands), leave steps out, add new ones, or interleave additional system mo= difications. > > Status > ------ > This provides functionality most similar to the existing sysinstall 'Expr= ess' track. It installs working, bootable systems you can ssh into immediat= ely after reboot on i386, amd64, sparc64, powerpc, and powerpc64. There is = untested support for pc98. The final architecture on which we use sysinstal= l, ia64, is currently unsupported, because I don't know how to set up booti= ng on those systems -- patches to solve this are very much welcome. > > There are still some missing features that I would like to see in the rel= ease, but these do not significantly impact the functionality of the instal= ler. Some will be addressed before merging to HEAD, in particular the lack = of a man page for bsdinstall. Others, like configuration of wireless networ= king and ZFS installation, can happen between merge and release. The test I= SOs are also lacking a ports tree at the moment, which is a statement about= the slow upload speed of my DSL line and not about the final layout of rel= eases. > > Please send any questions, comments, or patches you may have, and please = be aware when replying that this email has been cross-posted to three lists= . Technical discussion (bug reports, for instance) should be directed to th= e freebsd-sysinstall list only. Most other discussion belongs on -sysinstal= l and -current. > -Nathan > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org= " I dont follow the=A0freebsd-sysinstall and freebsd-arc list so sorry if this has already been discussed. From what I have seen pc-sysinstall already does all these things, and can install freebsd. Therefore why are we reinventing the wheel? I don't mean this as any disrespect to the work you have done. From owner-freebsd-arch@FreeBSD.ORG Sun Jan 16 11:49:02 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E6BD106566B; Sun, 16 Jan 2011 11:49:02 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 9C1188FC12; Sun, 16 Jan 2011 11:49:01 +0000 (UTC) Received: by wwf26 with SMTP id 26so4418079wwf.31 for ; Sun, 16 Jan 2011 03:49:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=cEf0bTj8Ljehw9ACGJkS/b50dXuzm90sRrQ0KVbOQuo=; b=mgN39mRzvZQ1e8zA/cagWyJ6cZhgNXJwbCX6DKPLg3GN0CwDexNgZ/4xnUc2wEbVU7 DzoOdlg2Y6qxYlV0c6cRWDt8iqZwSZuQiM9vQ6sBElMasiefxIRoay6wT+87qRhquDOx I9fMh5fXtYWMKFWqSUZhumvWuqILi9JoDlCpM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=JOTAINbPPFMsmmAHo6TL+LcwjBnB94badCUDdY+Hr51K+lHm2SRNgE0swtc10i8aC5 cnZdVHjv3dgn31JdzBbgtNblUDarHI5FKv8I6lmHR9DSOurHJ7j6IBNKM6uehGY0HU3h RZf0JA0i3fPc9KFQwc/zBujns54rQoZFTWauM= MIME-Version: 1.0 Received: by 10.216.49.15 with SMTP id w15mr1300302web.1.1295178539705; Sun, 16 Jan 2011 03:48:59 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Sun, 16 Jan 2011 03:48:59 -0800 (PST) In-Reply-To: References: <4D309563.1000404@freebsd.org> Date: Sun, 16 Jan 2011 03:48:59 -0800 X-Google-Sender-Auth: K5j7ZCijhGPOENgtGX9uSLdLu3A Message-ID: From: Garrett Cooper To: krad Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org, freebsd-current Current , Nathan Whitehorn , freebsd-sysinstall@freebsd.org Subject: Re: BSDInstall: merging to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2011 11:49:02 -0000 On Sun, Jan 16, 2011 at 3:06 AM, krad wrote: > On 14 January 2011 18:26, Nathan Whitehorn wrote= : >> >> As those of you who have been reading freebsd-sysinstall and freebsd-arc= h know, I have been working for a few weeks on a lightweight new installer = named 'bsdinstall'. This is designed to replace sysinstall for the 9.0 rele= ase. >> >> After two weeks of testing and bug fixes on the sysinstall list, I belie= ve this now has all required functionality and is ready to be merged into t= he main source tree. I would like to do this on Tuesday, 18 January. Switch= ing this to be the default installer would happen a few weeks after that, p= ending discussion on release formats with the release engineering team. Thi= s should provide a sufficient testing period before 9.0 and allow a maximal= number of bugs to be discovered and solved before the release is shipped. >> >> Demo ISO for i386: http://people.freebsd.org/~nwhitehorn/bsdinstall-i386= -20110114.iso.bz2 >> SVN repository: svn://svn.freebsd.org/base/user/nwhitehorn/bsdinstall >> Wiki page: http://wiki.freebsd.org/BSDInstall >> >> Goals >> ----- >> The primary goal of BSDInstall is to provide an easily extensible instal= ler without the limitations of sysinstall, in order to allow more modern in= stallations of FreeBSD. This means that it should have additional features = to support modern setups, but simultaneously frees us to remove complicatin= g features of sysinstall like making sure everything fits in floppy disk-si= zed chunks. >> >> New Features: >> - Allows installation onto GPT disks on x86 systems >> - Can do installations spanning multiple disks >> - Allows installation into jails >> - Eases PXE installation >> - Virtualization friendly: can install from a live system onto disk >> =A0images >> - Works on PowerPC >> - Streamlined system installation >> - More flexible scripting >> - Easily tweakable >> - All install CDs are live CDs >> >> Architecture >> ------------ >> BSDInstall is a set of tools that are called in sequence by a master scr= ipt. These tools are, for example, the partition editor, the thing that fet= ches the distributions from the network, the thing that untars them, etc. S= ince these are just called in sequence from a shell script, a scripted inst= allation can easily replace them with other things, (e.g. hard-coded gpart = commands), leave steps out, add new ones, or interleave additional system m= odifications. >> >> Status >> ------ >> This provides functionality most similar to the existing sysinstall 'Exp= ress' track. It installs working, bootable systems you can ssh into immedia= tely after reboot on i386, amd64, sparc64, powerpc, and powerpc64. There is= untested support for pc98. The final architecture on which we use sysinsta= ll, ia64, is currently unsupported, because I don't know how to set up boot= ing on those systems -- patches to solve this are very much welcome. >> >> There are still some missing features that I would like to see in the re= lease, but these do not significantly impact the functionality of the insta= ller. Some will be addressed before merging to HEAD, in particular the lack= of a man page for bsdinstall. Others, like configuration of wireless netwo= rking and ZFS installation, can happen between merge and release. The test = ISOs are also lacking a ports tree at the moment, which is a statement abou= t the slow upload speed of my DSL line and not about the final layout of re= leases. >> >> Please send any questions, comments, or patches you may have, and please= be aware when replying that this email has been cross-posted to three list= s. Technical discussion (bug reports, for instance) should be directed to t= he freebsd-sysinstall list only. Most other discussion belongs on -sysinsta= ll and -current. > > I dont follow the=A0freebsd-sysinstall and freebsd-arc list so sorry if > this has already been discussed. From what I have seen pc-sysinstall > already does all these things, and can install freebsd. Therefore why > are we reinventing the wheel? > > I don't mean this as any disrespect to the work you have done. Hi Krad, I asked this two weeks ago and in summary: - pc-sysinstall is x86-centric and porting to powerpc is non-trivial, and sysinstall is incomplete on powerpc. Nate sought to get a working powerpc port with minimal effort. Please read other replies in the archives on freebsd-arch / freebsd-sysinstall to get more info as to why things have been done the way they have been done. Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Sun Jan 16 15:46:06 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84CA9106566C; Sun, 16 Jan 2011 15:46:06 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from mail.icecube.wisc.edu (trout.icecube.wisc.edu [128.104.255.119]) by mx1.freebsd.org (Postfix) with ESMTP id 446128FC1A; Sun, 16 Jan 2011 15:46:05 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.icecube.wisc.edu (Postfix) with ESMTP id 56B2D5811D; Sun, 16 Jan 2011 09:46:05 -0600 (CST) X-Virus-Scanned: amavisd-new at icecube.wisc.edu Received: from mail.icecube.wisc.edu ([127.0.0.1]) by localhost (trout.icecube.wisc.edu [127.0.0.1]) (amavisd-new, port 10030) with ESMTP id z4Eh6-9RFUoT; Sun, 16 Jan 2011 09:46:05 -0600 (CST) Received: from comporellon.tachypleus.net (adsl-76-208-68-88.dsl.mdsnwi.sbcglobal.net [76.208.68.88]) by mail.icecube.wisc.edu (Postfix) with ESMTP id 6C6535811A; Sun, 16 Jan 2011 09:46:04 -0600 (CST) Message-ID: <4D3312BB.30203@freebsd.org> Date: Sun, 16 Jan 2011 09:46:03 -0600 From: Nathan Whitehorn User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101214 Thunderbird/3.1.7 MIME-Version: 1.0 To: krad References: <4D309563.1000404@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, freebsd-arch@freebsd.org Subject: Re: BSDInstall: merging to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2011 15:46:06 -0000 On 01/16/11 05:48, Garrett Cooper wrote: > On Sun, Jan 16, 2011 at 3:06 AM, krad wrote: >> On 14 January 2011 18:26, Nathan Whitehorn wrote: >>> As those of you who have been reading freebsd-sysinstall and freebsd-arch know, I have been working for a few weeks on a lightweight new installer named 'bsdinstall'. This is designed to replace sysinstall for the 9.0 release. >>> >>> After two weeks of testing and bug fixes on the sysinstall list, I believe this now has all required functionality and is ready to be merged into the main source tree. I would like to do this on Tuesday, 18 January. Switching this to be the default installer would happen a few weeks after that, pending discussion on release formats with the release engineering team. This should provide a sufficient testing period before 9.0 and allow a maximal number of bugs to be discovered and solved before the release is shipped. >>> >>> Demo ISO for i386: http://people.freebsd.org/~nwhitehorn/bsdinstall-i386-20110114.iso.bz2 >>> SVN repository: svn://svn.freebsd.org/base/user/nwhitehorn/bsdinstall >>> Wiki page: http://wiki.freebsd.org/BSDInstall >>> >>> Goals >>> ----- >>> The primary goal of BSDInstall is to provide an easily extensible installer without the limitations of sysinstall, in order to allow more modern installations of FreeBSD. This means that it should have additional features to support modern setups, but simultaneously frees us to remove complicating features of sysinstall like making sure everything fits in floppy disk-sized chunks. >>> >>> New Features: >>> - Allows installation onto GPT disks on x86 systems >>> - Can do installations spanning multiple disks >>> - Allows installation into jails >>> - Eases PXE installation >>> - Virtualization friendly: can install from a live system onto disk >>> images >>> - Works on PowerPC >>> - Streamlined system installation >>> - More flexible scripting >>> - Easily tweakable >>> - All install CDs are live CDs >>> >>> Architecture >>> ------------ >>> BSDInstall is a set of tools that are called in sequence by a master script. These tools are, for example, the partition editor, the thing that fetches the distributions from the network, the thing that untars them, etc. Since these are just called in sequence from a shell script, a scripted installation can easily replace them with other things, (e.g. hard-coded gpart commands), leave steps out, add new ones, or interleave additional system modifications. >>> >>> Status >>> ------ >>> This provides functionality most similar to the existing sysinstall 'Express' track. It installs working, bootable systems you can ssh into immediately after reboot on i386, amd64, sparc64, powerpc, and powerpc64. There is untested support for pc98. The final architecture on which we use sysinstall, ia64, is currently unsupported, because I don't know how to set up booting on those systems -- patches to solve this are very much welcome. >>> >>> There are still some missing features that I would like to see in the release, but these do not significantly impact the functionality of the installer. Some will be addressed before merging to HEAD, in particular the lack of a man page for bsdinstall. Others, like configuration of wireless networking and ZFS installation, can happen between merge and release. The test ISOs are also lacking a ports tree at the moment, which is a statement about the slow upload speed of my DSL line and not about the final layout of releases. >>> >>> Please send any questions, comments, or patches you may have, and please be aware when replying that this email has been cross-posted to three lists. Technical discussion (bug reports, for instance) should be directed to the freebsd-sysinstall list only. Most other discussion belongs on -sysinstall and -current. >> I dont follow the freebsd-sysinstall and freebsd-arc list so sorry if >> this has already been discussed. From what I have seen pc-sysinstall >> already does all these things, and can install freebsd. Therefore why >> are we reinventing the wheel? >> >> I don't mean this as any disrespect to the work you have done. > Hi Krad, > I asked this two weeks ago and in summary: > > - pc-sysinstall is x86-centric and porting to powerpc is non-trivial, > and sysinstall is incomplete on powerpc. Nate sought to get a working > powerpc port with minimal effort. > > Please read other replies in the archives on freebsd-arch / > freebsd-sysinstall to get more info as to why things have been done > the way they have been done. Here's the summary of why this doesn't use pc-sysinstall. PC-sysinstall is the backend of the PC-BSD installer, and was imported into FreeBSD in June 2010 with the goal of replacing sysinstall. It is much more full-featured that either bsdinstall or sysinstall, providing support for encrypted disks, ZFS, and mirroring. These are all good things. However, 9.0 is coming up quite soon, and pc-sysinstall still does not have a usable front-end (the code in the tree is just a script interpreter) nor support for non-x86 platforms. Adding these things appeared to be quite difficult and, in the second case, to require major rearchitecting of the installer backend. Since the original goal that Garrett alluded to was to provide an installer on PowerPC, this was obviously a non-starter. What BSDinstall is is a stopgap, designed to provide a subset of pc-sysinstall's features but to be available quickly, in time for 9.0, and to have all of the features it does provide actually hooked up and usable. At the same time, I have architectured bsdinstall in such a way that (a) I would be happy enough to be stuck with it permanently (i.e. it is not a quick hack and was designed to be easily maintainable and extendable) and (b) it could be later adapted into the missing front-end for pc-sysinstall without an enormous amount of trouble. Many of the issues we need to face now to integrate bsdinstall (making bootable live CDs or the integration of a newer libdialog, for instance) are common to any later effort to import a pc-sysinstall front-end. Having these things in use and in the tree will only help pc-sysinstall whenever it is finished. -Nathan From owner-freebsd-arch@FreeBSD.ORG Sun Jan 16 16:56:57 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9D9C106564A; Sun, 16 Jan 2011 16:56:57 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4894B8FC16; Sun, 16 Jan 2011 16:56:56 +0000 (UTC) Received: by qwj9 with SMTP id 9so3887493qwj.13 for ; Sun, 16 Jan 2011 08:56:56 -0800 (PST) Received: by 10.229.227.15 with SMTP id iy15mr2799966qcb.51.1295195690135; Sun, 16 Jan 2011 08:34:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.229.91.14 with HTTP; Sun, 16 Jan 2011 08:34:29 -0800 (PST) In-Reply-To: <4D309563.1000404@freebsd.org> References: <4D309563.1000404@freebsd.org> From: =?UTF-8?Q?Marius_N=C3=BCnnerich?= Date: Sun, 16 Jan 2011 17:34:29 +0100 Message-ID: To: Nathan Whitehorn Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, freebsd-arch@freebsd.org Subject: Re: BSDInstall: merging to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2011 16:56:58 -0000 On Fri, Jan 14, 2011 at 19:26, Nathan Whitehorn wr= ote: > As those of you who have been reading freebsd-sysinstall and freebsd-arch > know, I have been working for a few weeks on a lightweight new installer > named 'bsdinstall'. This is designed to replace sysinstall for the 9.0 > release. > > After two weeks of testing and bug fixes on the sysinstall list, I believ= e > this now has all required functionality and is ready to be merged into th= e > main source tree. I would like to do this on Tuesday, 18 January. Switchi= ng > this to be the default installer would happen a few weeks after that, > pending discussion on release formats with the release engineering team. > This should provide a sufficient testing period before 9.0 and allow a > maximal number of bugs to be discovered and solved before the release is > shipped. > > Demo ISO for i386: > http://people.freebsd.org/~nwhitehorn/bsdinstall-i386-20110114.iso.bz2 > SVN repository: svn://svn.freebsd.org/base/user/nwhitehorn/bsdinstall > Wiki page: http://wiki.freebsd.org/BSDInstall > > Goals > ----- > The primary goal of BSDInstall is to provide an easily extensible install= er > without the limitations of sysinstall, in order to allow more modern > installations of FreeBSD. This means that it should have additional featu= res > to support modern setups, but simultaneously frees us to remove complicat= ing > features of sysinstall like making sure everything fits in floppy disk-si= zed > chunks. > > New Features: > - Allows installation onto GPT disks on x86 systems > - Can do installations spanning multiple disks > - Allows installation into jails > - Eases PXE installation > - Virtualization friendly: can install from a live system onto disk > =C2=A0images > - Works on PowerPC > - Streamlined system installation > - More flexible scripting > - Easily tweakable > - All install CDs are live CDs > > Architecture > ------------ > BSDInstall is a set of tools that are called in sequence by a master scri= pt. > These tools are, for example, the partition editor, the thing that fetche= s > the distributions from the network, the thing that untars them, etc. Sinc= e > these are just called in sequence from a shell script, a scripted > installation can easily replace them with other things, (e.g. hard-coded > gpart commands), leave steps out, add new ones, or interleave additional > system modifications. > > Status > ------ > This provides functionality most similar to the existing sysinstall > 'Express' track. It installs working, bootable systems you can ssh into > immediately after reboot on i386, amd64, sparc64, powerpc, and powerpc64. > There is untested support for pc98. The final architecture on which we us= e > sysinstall, ia64, is currently unsupported, because I don't know how to s= et > up booting on those systems -- patches to solve this are very much welcom= e. > > There are still some missing features that I would like to see in the > release, but these do not significantly impact the functionality of the > installer. Some will be addressed before merging to HEAD, in particular t= he > lack of a man page for bsdinstall. Others, like configuration of wireless > networking and ZFS installation, can happen between merge and release. Th= e > test ISOs are also lacking a ports tree at the moment, which is a stateme= nt > about the slow upload speed of my DSL line and not about the final layout= of > releases. > > Please send any questions, comments, or patches you may have, and please = be > aware when replying that this email has been cross-posted to three lists. > Technical discussion (bug reports, for instance) should be directed to th= e > freebsd-sysinstall list only. Most other discussion belongs on -sysinstal= l > and -current. > -Nathan Clean new virtualbox on FreeBSD host. Install -> German ISO-8859-1 -> "vbox" -> Guided -> ad0 -> Partition -> "You have canceled an installation step" Actually I didn't cancel anything :) After using the entire disk and installing some distributions it hangs waiting for the root password, it won't continue when I just press enter. The screen output looks garbled by a LOR. The screen waiting for the root pw is garbled too. Seems like it's not doing a carriage return, just line-feeds. I tried this again a second time and everything worked normally. From owner-freebsd-arch@FreeBSD.ORG Mon Jan 17 15:53:30 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3773106566B for ; Mon, 17 Jan 2011 15:53:30 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 59AB08FC14 for ; Mon, 17 Jan 2011 15:53:30 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id p0HFnO6b088318 for ; Mon, 17 Jan 2011 08:49:25 -0700 (MST) (envelope-from imp@bsdimp.com) Message-ID: <4D346504.5050700@bsdimp.com> Date: Mon, 17 Jan 2011 08:49:24 -0700 From: Warner Losh User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101211 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <4D309563.1000404@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: BSDInstall: merging to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jan 2011 15:53:30 -0000 On 01/16/2011 04:48, Garrett Cooper wrote: > > - pc-sysinstall is x86-centric and porting to powerpc is non-trivial, > and sysinstall is incomplete on powerpc. Nate sought to get a working > powerpc port with minimal effort. Actually, that's not the case. Porting to powerpc for pc-sysinstall is easy. I think that it is unfortunate, but we're likely also going to have txt-sysinstall in the tree soon because nathan doesn't want to work with us. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Jan 17 16:05:39 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 252B91065670 for ; Mon, 17 Jan 2011 16:05:39 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id BEECF8FC08 for ; Mon, 17 Jan 2011 16:05:38 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id p0HFrj0J088340 for ; Mon, 17 Jan 2011 08:53:45 -0700 (MST) (envelope-from imp@bsdimp.com) Message-ID: <4D346609.2070608@bsdimp.com> Date: Mon, 17 Jan 2011 08:53:45 -0700 From: Warner Losh User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101211 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <4D309563.1000404@freebsd.org> <4D3312BB.30203@freebsd.org> In-Reply-To: <4D3312BB.30203@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: BSDInstall: merging to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jan 2011 16:05:39 -0000 On 01/16/2011 08:46, Nathan Whitehorn wrote: > On 01/16/11 05:48, Garrett Cooper wrote: >> On Sun, Jan 16, 2011 at 3:06 AM, krad wrote: >>> On 14 January 2011 18:26, Nathan Whitehorn >>> wrote: >>>> As those of you who have been reading freebsd-sysinstall and >>>> freebsd-arch know, I have been working for a few weeks on a >>>> lightweight new installer named 'bsdinstall'. This is designed to >>>> replace sysinstall for the 9.0 release. >>>> >>>> After two weeks of testing and bug fixes on the sysinstall list, I >>>> believe this now has all required functionality and is ready to be >>>> merged into the main source tree. I would like to do this on >>>> Tuesday, 18 January. Switching this to be the default installer >>>> would happen a few weeks after that, pending discussion on release >>>> formats with the release engineering team. This should provide a >>>> sufficient testing period before 9.0 and allow a maximal number of >>>> bugs to be discovered and solved before the release is shipped. >>>> >>>> Demo ISO for i386: >>>> http://people.freebsd.org/~nwhitehorn/bsdinstall-i386-20110114.iso.bz2 >>>> SVN repository: svn://svn.freebsd.org/base/user/nwhitehorn/bsdinstall >>>> Wiki page: http://wiki.freebsd.org/BSDInstall >>>> >>>> Goals >>>> ----- >>>> The primary goal of BSDInstall is to provide an easily extensible >>>> installer without the limitations of sysinstall, in order to allow >>>> more modern installations of FreeBSD. This means that it should >>>> have additional features to support modern setups, but >>>> simultaneously frees us to remove complicating features of >>>> sysinstall like making sure everything fits in floppy disk-sized >>>> chunks. >>>> >>>> New Features: >>>> - Allows installation onto GPT disks on x86 systems >>>> - Can do installations spanning multiple disks >>>> - Allows installation into jails >>>> - Eases PXE installation >>>> - Virtualization friendly: can install from a live system onto disk >>>> images >>>> - Works on PowerPC >>>> - Streamlined system installation >>>> - More flexible scripting >>>> - Easily tweakable >>>> - All install CDs are live CDs >>>> >>>> Architecture >>>> ------------ >>>> BSDInstall is a set of tools that are called in sequence by a >>>> master script. These tools are, for example, the partition editor, >>>> the thing that fetches the distributions from the network, the >>>> thing that untars them, etc. Since these are just called in >>>> sequence from a shell script, a scripted installation can easily >>>> replace them with other things, (e.g. hard-coded gpart commands), >>>> leave steps out, add new ones, or interleave additional system >>>> modifications. >>>> >>>> Status >>>> ------ >>>> This provides functionality most similar to the existing sysinstall >>>> 'Express' track. It installs working, bootable systems you can ssh >>>> into immediately after reboot on i386, amd64, sparc64, powerpc, and >>>> powerpc64. There is untested support for pc98. The final >>>> architecture on which we use sysinstall, ia64, is currently >>>> unsupported, because I don't know how to set up booting on those >>>> systems -- patches to solve this are very much welcome. >>>> >>>> There are still some missing features that I would like to see in >>>> the release, but these do not significantly impact the >>>> functionality of the installer. Some will be addressed before >>>> merging to HEAD, in particular the lack of a man page for >>>> bsdinstall. Others, like configuration of wireless networking and >>>> ZFS installation, can happen between merge and release. The test >>>> ISOs are also lacking a ports tree at the moment, which is a >>>> statement about the slow upload speed of my DSL line and not about >>>> the final layout of releases. >>>> >>>> Please send any questions, comments, or patches you may have, and >>>> please be aware when replying that this email has been cross-posted >>>> to three lists. Technical discussion (bug reports, for instance) >>>> should be directed to the freebsd-sysinstall list only. Most other >>>> discussion belongs on -sysinstall and -current. >>> I dont follow the freebsd-sysinstall and freebsd-arc list so sorry if >>> this has already been discussed. From what I have seen pc-sysinstall >>> already does all these things, and can install freebsd. Therefore why >>> are we reinventing the wheel? >>> >>> I don't mean this as any disrespect to the work you have done. >> Hi Krad, >> I asked this two weeks ago and in summary: >> >> - pc-sysinstall is x86-centric and porting to powerpc is non-trivial, >> and sysinstall is incomplete on powerpc. Nate sought to get a working >> powerpc port with minimal effort. >> >> Please read other replies in the archives on freebsd-arch / >> freebsd-sysinstall to get more info as to why things have been done >> the way they have been done. > > Here's the summary of why this doesn't use pc-sysinstall. > PC-sysinstall is the backend of the PC-BSD installer, and was imported > into FreeBSD in June 2010 with the goal of replacing sysinstall. It is > much more full-featured that either bsdinstall or sysinstall, > providing support for encrypted disks, ZFS, and mirroring. These are > all good things. However, 9.0 is coming up quite soon, and > pc-sysinstall still does not have a usable front-end (the code in the > tree is just a script interpreter) nor support for non-x86 platforms. > Adding these things appeared to be quite difficult and, in the second > case, to require major rearchitecting of the installer backend. Since > the original goal that Garrett alluded to was to provide an installer > on PowerPC, this was obviously a non-starter. so we should screw up our installer story so people with ancient Apple hardware can install FreeBSD on their old G4's? That doesn't sound very wise to me. > What BSDinstall is is a stopgap, designed to provide a subset of > pc-sysinstall's features but to be available quickly, in time for 9.0, > and to have all of the features it does provide actually hooked up and > usable. At the same time, I have architectured bsdinstall in such a > way that (a) I would be happy enough to be stuck with it permanently > (i.e. it is not a quick hack and was designed to be easily > maintainable and extendable) and (b) it could be later adapted into > the missing front-end for pc-sysinstall without an enormous amount of > trouble. Many of the issues we need to face now to integrate > bsdinstall (making bootable live CDs or the integration of a newer > libdialog, for instance) are common to any later effort to import a > pc-sysinstall front-end. Having these things in use and in the tree > will only help pc-sysinstall whenever it is finished. Do we really need another half-assed solution in the tree? that's how we got into the pickle we're in with sysinstall in the first place. As such, we're moving forward with our plans to import txt-sysinstall. We'll be doing it today or tomorrow. We had wanted to work with bsdinstaller, but so far all we've received is lip service and then this push to put bsdinstaller into the tree and make it the default in an insanely short period of time. That's crazy. Since talks have broken down, we're also going to push txt-sysinstall into the tree so it can mature and get more exposure. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Jan 17 17:00:26 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88893106564A for ; Mon, 17 Jan 2011 17:00:26 +0000 (UTC) (envelope-from jamesbrandongooch@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1C4D68FC12 for ; Mon, 17 Jan 2011 17:00:25 +0000 (UTC) Received: by wyf19 with SMTP id 19so5445221wyf.13 for ; Mon, 17 Jan 2011 09:00:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=vwrico1v9zi/gDMvinv4IYz49UJ4fy5cImL+or5SWxo=; b=dt6JqYK0/01BKGmNQw9sHXO9vRDnTOggGyyd3SiOYvp7hUVyx9YLYBpcOH1TIRhH2n 0PMa5Wt8QRV91BxWkCqvsn1iM7HO3KQwPcPQVPuU31EbJq/t8uiiTfRqwnilh7Oo4XSK G9fpimTtP4ZLxv0GO8xmI4EEqmSykxaPrqjrQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=N9i8UG3Jf4Dnk9BpwvYNuI0cPMugXzYm5AAqnTWg1HLKZ5ST1epXNwXaXJuII2sfb6 8pISIJPpP05OI6eu2vTZWdoIyZgI73VkZ5pHQ1KbjJ3ztMr/Wq5iXnV1JdsZbvitXMVq +/AD3no5L3rLdkmcz+4YGKebf59/qMBYmuT2Y= MIME-Version: 1.0 Received: by 10.216.73.203 with SMTP id v53mr3769292wed.48.1295281791626; Mon, 17 Jan 2011 08:29:51 -0800 (PST) Received: by 10.216.36.71 with HTTP; Mon, 17 Jan 2011 08:29:51 -0800 (PST) In-Reply-To: <4D346609.2070608@bsdimp.com> References: <4D309563.1000404@freebsd.org> <4D3312BB.30203@freebsd.org> <4D346609.2070608@bsdimp.com> Date: Mon, 17 Jan 2011 10:29:51 -0600 Message-ID: From: Brandon Gooch To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: BSDInstall: merging to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jan 2011 17:00:26 -0000 On Mon, Jan 17, 2011 at 9:53 AM, Warner Losh wrote: > On 01/16/2011 08:46, Nathan Whitehorn wrote: [SNIP FOR GREAT JUSTICE] > As such, we're moving forward with our plans to import txt-sysinstall. > =A0We'll be doing it today or tomorrow. =A0We had wanted to work with > bsdinstaller, but so far all we've received is lip service and then this > push to put bsdinstaller into the tree and make it the default in an > insanely short period of time. =A0That's crazy. =A0Since talks have broke= n down, > we're also going to push txt-sysinstall into the tree so it can mature an= d > get more exposure. Warner, Is there a test image, similar to the one prepared by Nathan, available for testing txt-sysintall? -Brandon From owner-freebsd-arch@FreeBSD.ORG Tue Jan 18 18:38:07 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15A2710656B1; Tue, 18 Jan 2011 18:38:07 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from agogare.doit.wisc.edu (agogare.doit.wisc.edu [144.92.197.211]) by mx1.freebsd.org (Postfix) with ESMTP id DEFB68FC12; Tue, 18 Jan 2011 18:38:06 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth2.wiscmail.wisc.edu by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LF800K00BNH4Y00@smtpauth2.wiscmail.wisc.edu>; Tue, 18 Jan 2011 11:38:05 -0600 (CST) Received: from anacreon.physics.wisc.edu (anacreon.physics.wisc.edu [128.104.160.176]) by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LF800GE1BNGOQ10@smtpauth2.wiscmail.wisc.edu>; Tue, 18 Jan 2011 11:38:04 -0600 (CST) Date: Tue, 18 Jan 2011 11:38:03 -0600 From: Nathan Whitehorn To: freebsd-arch@freebsd.org, freebsd-current@freebsd.org, freebsd-sysinstall@freebsd.org Message-id: <4D35CFFB.3010302@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=128.104.160.176 X-Spam-PmxInfo: Server=avs-9, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.1.18.173017, SenderIP=128.104.160.176 User-Agent: Mozilla/5.0 (X11; U; FreeBSD powerpc; en-US; rv:1.9.2.13) Gecko/20110104 Thunderbird/3.1.7 Cc: Subject: FreeBSD Installer Roadmap X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 18:38:07 -0000 After some discussion with M. Warner Losh and Josh Paetzel of iX Systems, we've come up with the following roadmap for an installer for 9.0. Over the next month, we intend to try to adapt bsdinstall as the front-end for the more featureful, but lacking a terminal-compatible user interface, pc-sysinstall. This implies that the user interface and installation flow for the hybrid installer will be extremely similar to what is currently available in bsdinstall, so please continue sending feedback and bug reports on it. What will be different is the backend code, which will allow use of additional features not currently present in bsdinstall, such as ZFS installation. At the end of that month period, we'll see how far we've gotten, and plan to merge either a successful hybridization or to merge bsdinstall with its own backend, which I will continue to maintain in the interim. At this point, we plan to integrate whichever installer is merged with the release infrastructure so that it becomes the default installation environment presented on snapshot ISOs. If we have have not completed the hybrid installer at this point, work on hybridization will still continue after this. Since the interface presented to user will be extremely similar, a bsdinstall -> pc-bsdinstall transition can happen with a minimum of user astonishment, or even awareness, at any point in the future, either before or after the 9.0 release. This plan ensures that we have a minimum of three months of testing of the new installer on snapshot media before the 9.0 release, which should ensure a minimum of bugs. I would also like to point out that there are no roads in this map that end up with us having sysinstall as the default installer past the 18th of February. After 15 years of sysinstall being "greatly in need of death", it will finally be time to retire it. Thanks to Warner Losh and Josh Paetzel for excellent discussions. Please continue to send any comments on this plan, bug reports or feature requests for bsdinstall or pc-sysinstall, and suggestions for the installation process. -Nathan From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 20:28:20 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF658106564A for ; Wed, 19 Jan 2011 20:28:20 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7A90B8FC1C for ; Wed, 19 Jan 2011 20:28:20 +0000 (UTC) Received: by iyb26 with SMTP id 26so1159426iyb.13 for ; Wed, 19 Jan 2011 12:28:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:date:x-google-sender-auth :message-id:subject:from:to:content-type; bh=6BIKT3hwTdGSsnCKr1WZh5xj/p7OSinUde7dShOq11E=; b=BSAroQnveoks+PFcg6ijVRmXf5Ek4AuNoyJRf1P+FuF9IgU4AavLv38l/oBIN60uuj 8/wp2gs78NL2bBlxlo2BcerD+clchzHfsneuDzD0wa6umGWZno+8/B7U2KYyirnO1LO2 WPaNc1MkyERMsllqYn8V3quttuIx1ToZkMHQc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=iHmhLcwOaSezCfn4bDVx6ln2gsoYvbvDop+oLLR8lCeso1UIFRJAOdZW/qAAxTieYW rWGFXqOu/Em7u9gMAFnOxSnU01mwfTDGfMth5h22WuSQILfDvifNhcXrA2qsUAN3u/yu SKl+286I+nmJCArC5iw+nKW5y09U5yrhlzo4o= MIME-Version: 1.0 Received: by 10.231.40.3 with SMTP id i3mr1420963ibe.129.1295468899892; Wed, 19 Jan 2011 12:28:19 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.231.160.147 with HTTP; Wed, 19 Jan 2011 12:28:19 -0800 (PST) Date: Wed, 19 Jan 2011 12:28:19 -0800 X-Google-Sender-Auth: 4XuV6sCSatEQTazPyzZpbr-gFGA Message-ID: From: mdf@FreeBSD.org To: FreeBSD Arch Content-Type: text/plain; charset=ISO-8859-1 Subject: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 20:28:20 -0000 As bde@ mentioned, there's very little actual use of the sysctl format strings. I recently changed it so the use is even smaller, but I've got a quandary as to how to finish the job. I agree with Bruce that the formatted structs can just be removed. This leaves just the "IK" format, which shows up in just a few files: sys/dev/acpica/acpi_thermal.c: sys/dev/amdtemp/amdtemp.c sys/dev/acpi_support/atk0110.c sys/dev/coretemp/coretemp.c sys/dev/iicbus/max6690.c sys/dev/iicbus/ds1775.c I see two solutions to "IK". The first is to preserve the interface as experienced by sysctl(8) users, and add some functions to push a string buffer back to userspace, and parse a string in the kernel. The second is to preserve the binary interface as experienced by sysctl(3) users, and either have the values be dumped in the slightly obscure 10ths of Kelvin values, or add a new CTLTYPE_KELVIN so sysctl(8) can also keep showing things as it does today. Given how infrequent the use is CTLTYPE_KELVIN seems a non-starter. So who is the worse client to break: those who use sysctl(8) to look at temperatures, or those who have a utility to manipulate these values using sysctl(3)? Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 20:59:48 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C927106566C; Wed, 19 Jan 2011 20:59:48 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D601B8FC17; Wed, 19 Jan 2011 20:59:47 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 920D646B89; Wed, 19 Jan 2011 15:59:47 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A97DC8A01D; Wed, 19 Jan 2011 15:59:46 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Date: Wed, 19 Jan 2011 15:59:07 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101191559.07713.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Wed, 19 Jan 2011 15:59:46 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: mdf@freebsd.org Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 20:59:48 -0000 On Wednesday, January 19, 2011 3:28:19 pm mdf@freebsd.org wrote: > As bde@ mentioned, there's very little actual use of the sysctl format > strings. I recently changed it so the use is even smaller, but I've > got a quandary as to how to finish the job. > > I agree with Bruce that the formatted structs can just be removed. Hmm, I've actually used 'sysctl kern.clockrate' in the past. > This leaves just the "IK" format, which shows up in just a few files: > > sys/dev/acpica/acpi_thermal.c: > sys/dev/amdtemp/amdtemp.c > sys/dev/acpi_support/atk0110.c > sys/dev/coretemp/coretemp.c > sys/dev/iicbus/max6690.c > sys/dev/iicbus/ds1775.c > > I see two solutions to "IK". The first is to preserve the interface > as experienced by sysctl(8) users, and add some functions to push a > string buffer back to userspace, and parse a string in the kernel. > The second is to preserve the binary interface as experienced by > sysctl(3) users, and either have the values be dumped in the slightly > obscure 10ths of Kelvin values, or add a new CTLTYPE_KELVIN so > sysctl(8) can also keep showing things as it does today. > > Given how infrequent the use is CTLTYPE_KELVIN seems a non-starter. > So who is the worse client to break: those who use sysctl(8) to look > at temperatures, or those who have a utility to manipulate these > values using sysctl(3)? So what is sufficiently broken elsewhere that requires us to break the existing sysctls? Many people use the output of sysctl(8) for coretemp, and the ACPI thermal zones and breaking that gratuitously would be unfortunate. I've no idea if there is existing code reading these sysctls in userland for things like gkrellm, but I wouldn't be surprised if there were. Even CTLTYPE_KELVIN would require changes to user apps, but I also would rather present an actual number to userland rather than a string so userland doesn't have to parse it just to get back to the original number. I guess I fail to see why sysctl giving a hint about the semantic type of the data ("temperature", "struct foo") which is separate from the "physical" type (e.g. 32-bit or 64-bit) is such a bad thing. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 21:13:45 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6102E1065670; Wed, 19 Jan 2011 21:13:45 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 3E4088FC19; Wed, 19 Jan 2011 21:13:45 +0000 (UTC) Received: by elvis.mu.org (Postfix, from userid 1192) id EBB5B1A3C49; Wed, 19 Jan 2011 12:54:45 -0800 (PST) Date: Wed, 19 Jan 2011 12:54:45 -0800 From: Alfred Perlstein To: mdf@FreeBSD.org Message-ID: <20110119205445.GX21872@elvis.mu.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Arch Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 21:13:45 -0000 * mdf@FreeBSD.org [110119 12:28] wrote: > As bde@ mentioned, there's very little actual use of the sysctl format > strings. I recently changed it so the use is even smaller, but I've > got a quandary as to how to finish the job. > > I agree with Bruce that the formatted structs can just be removed. > This leaves just the "IK" format, which shows up in just a few files: > > sys/dev/acpica/acpi_thermal.c: > sys/dev/amdtemp/amdtemp.c > sys/dev/acpi_support/atk0110.c > sys/dev/coretemp/coretemp.c > sys/dev/iicbus/max6690.c > sys/dev/iicbus/ds1775.c > > I see two solutions to "IK". The first is to preserve the interface > as experienced by sysctl(8) users, and add some functions to push a > string buffer back to userspace, and parse a string in the kernel. > The second is to preserve the binary interface as experienced by > sysctl(3) users, and either have the values be dumped in the slightly > obscure 10ths of Kelvin values, or add a new CTLTYPE_KELVIN so > sysctl(8) can also keep showing things as it does today. > > Given how infrequent the use is CTLTYPE_KELVIN seems a non-starter. > So who is the worse client to break: those who use sysctl(8) to look > at temperatures, or those who have a utility to manipulate these > values using sysctl(3)? I'd say that it's not great to break either system. The reason is that either the syscall or cmd line utility can be the basis of numerous system monitoring tools. By breaking either interface we discourage people from using it. I apologize for coming in so late, but why is CTLTYPE_KELVIN such an awful thing? Also, not to digress, but it sounds like there's a rototilling of the KPI going on here as well that might break 3rd parties who do their own monitoring software. Overall it's not a big thing, but when you consider all the changes like this that go on, it can add up to a discouraging target to track. Honestly I don't have a strong opinion on this and feel free to use your best judgement and as well as what other people bring to the table, I just wanted to bring the software churn issue up and leave the question, "is there a way to do this with minimal 3rd party breakage". thank you, -- - Alfred Perlstein .- VMOA #5191, 03 vmax, 92 gs500, 85 ch250, 07 zx10 .- FreeBSD committer From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 21:36:24 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DE9D1065673 for ; Wed, 19 Jan 2011 21:36:24 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id C0DDD8FC28 for ; Wed, 19 Jan 2011 21:36:23 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id p0JLVv73024879 for ; Wed, 19 Jan 2011 14:31:57 -0700 (MST) (envelope-from imp@bsdimp.com) Message-ID: <4D37584D.6060305@bsdimp.com> Date: Wed, 19 Jan 2011 14:31:57 -0700 From: Warner Losh User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101211 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <201101191559.07713.jhb@freebsd.org> In-Reply-To: <201101191559.07713.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 21:36:24 -0000 On 01/19/2011 13:59, John Baldwin wrote: > I guess I fail to see why sysctl giving a hint about the semantic type of > the data ("temperature", "struct foo") which is separate from the "physical" > type (e.g. 32-bit or 64-bit) is such a bad thing. Yes. Especially since the data could be degrees C, degrees F or some other weirdness. You can't divorce the semantic interpretation of the data from the data type. Both are important to interpret this data correctly. I'm in the "IK isn't icky enough to rototill" camp. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 21:46:24 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 266A6106564A for ; Wed, 19 Jan 2011 21:46:24 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from out1.smtp.messagingengine.com (out1.smtp.messagingengine.com [66.111.4.25]) by mx1.freebsd.org (Postfix) with ESMTP id E9F418FC19 for ; Wed, 19 Jan 2011 21:46:23 +0000 (UTC) Received: from compute1.internal (compute1.nyi.mail.srv.osa [10.202.2.41]) by gateway1.messagingengine.com (Postfix) with ESMTP id 0411A205F9 for ; Wed, 19 Jan 2011 16:26:54 -0500 (EST) Received: from frontend1.messagingengine.com ([10.202.2.160]) by compute1.internal (MEProxy); Wed, 19 Jan 2011 16:26:54 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=messagingengine.com; h=from:to:subject:date:mime-version:content-type:content-transfer-encoding:message-id; s=smtpout; bh=tM+43zfA2KiQWbb+j0rk+dHso48=; b=KWx7oTRsadcfP+phP0wWVembKeIApe7m3QDlYmxUFKlPBx8v7ecw1wh/A6VrBam8nUPJTF9Csypp9V6CjbaV0MROWSbswdLfbT1HONjxj3GBVLdX4QeKyN+hYf9d1A93ikqU749dadz+kTv9EXnlDSErPkLgEtFcgffE26GLdo0= X-Sasl-enc: ZmW1W/eV7qtd8coU8Ef7mozzPUK1AafWr+RRdWZYSMVa 1295472413 Received: from tcbug.ixsystems.com (74-34-19-98.dr01.rsmt.mn.frontiernet.net [74.34.19.98]) by mail.messagingengine.com (Postfix) with ESMTPSA id AF31A404B11 for ; Wed, 19 Jan 2011 16:26:53 -0500 (EST) From: Josh Paetzel To: arch@freebsd.org Date: Wed, 19 Jan 2011 15:26:46 -0600 User-Agent: KMail/1.13.5 (FreeBSD/8.1-RELEASE; KDE/4.4.5; amd64; ; ) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart12406381.IJJgndZekE"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201101191526.51723.josh@tcbug.org> Cc: Subject: Re: FreeBSD Installer Roadmap X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 21:46:24 -0000 --nextPart12406381.IJJgndZekE Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Quoted text from Nathan's email: After some discussion with M. Warner Losh and Josh Paetzel of iX=20 Systems This isn't a real reply, I discovered I wasn't subscribed to arch@, so it's= a=20 copy/paste. If your mail client hates it, it's my fault. I'm looking forward to finally moving on with life, and installers, seeing= =20 sysinstall laid to rest, and the merging of efforts from everyone involved. I'm also very committed to making this process as transparent as possible. = If=20 you have questions feel free to ask. The question and it's answer will=20 probably appear on wiki.freebsd.org =20 =2D-=20 Thanks, Josh Paetzel --nextPart12406381.IJJgndZekE Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) iQEcBAABAgAGBQJNN1cbAAoJEKFq1/n1feG2s5wH/3bU05MCeQruqo7JDorWAs6C mQe2BroPC4ipYAot901q5tipyywhSRuq86MUifwx4maDfkTzJFJAbFqqp23i7zzr yw3aK+3MRwrSVZA5LOKuP2GckbSuN2EXORC1BKpFzn5wcMZ0Vnsn7CoO/wYOtxya bdxKwLCKrzEMleH8RH8HGq4L9aFcFrMrPbT8rxgvk+EBll2V5emF1tqY3CK6LrLp JIY2YVgm3vk2vbEMxLkvjUP+zhKyz7uYD5b+5oQVpu3X2iM0G5IYArOLq6iDqdYt F4oKyc0SOFZ5MHhGdu+x4xipTz8AILVsEb8QHKJohv7rqjHbMX/6MTsop7B5PDw= =vpqR -----END PGP SIGNATURE----- --nextPart12406381.IJJgndZekE-- From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 21:56:46 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF4A7106566B; Wed, 19 Jan 2011 21:56:46 +0000 (UTC) (envelope-from jmg@h2.funkthat.com) Received: from h2.funkthat.com (gate.funkthat.com [70.36.235.232]) by mx1.freebsd.org (Postfix) with ESMTP id A66C88FC13; Wed, 19 Jan 2011 21:56:46 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id p0JLhd6p026419 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 19 Jan 2011 13:43:39 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id p0JLhd4Y026418; Wed, 19 Jan 2011 13:43:39 -0800 (PST) (envelope-from jmg) Date: Wed, 19 Jan 2011 13:43:39 -0800 From: John-Mark Gurney To: mdf@freebsd.org Message-ID: <20110119214339.GH66284@funkthat.com> Mail-Followup-To: mdf@freebsd.org, FreeBSD Arch References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 19 Jan 2011 13:43:40 -0800 (PST) Cc: FreeBSD Arch Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 21:56:46 -0000 mdf@freebsd.org wrote this message on Wed, Jan 19, 2011 at 12:28 -0800: > As bde@ mentioned, there's very little actual use of the sysctl format > strings. I recently changed it so the use is even smaller, but I've > got a quandary as to how to finish the job. > > I agree with Bruce that the formatted structs can just be removed. > This leaves just the "IK" format, which shows up in just a few files: > > sys/dev/acpica/acpi_thermal.c: > sys/dev/amdtemp/amdtemp.c > sys/dev/acpi_support/atk0110.c > sys/dev/coretemp/coretemp.c > sys/dev/iicbus/max6690.c > sys/dev/iicbus/ds1775.c > > I see two solutions to "IK". The first is to preserve the interface > as experienced by sysctl(8) users, and add some functions to push a > string buffer back to userspace, and parse a string in the kernel. > The second is to preserve the binary interface as experienced by > sysctl(3) users, and either have the values be dumped in the slightly > obscure 10ths of Kelvin values, or add a new CTLTYPE_KELVIN so > sysctl(8) can also keep showing things as it does today. > > Given how infrequent the use is CTLTYPE_KELVIN seems a non-starter. > So who is the worse client to break: those who use sysctl(8) to look > at temperatures, or those who have a utility to manipulate these > values using sysctl(3)? I've been watching this discussion pretty closely as I recently wrote a bsnmp module to export sysctl values and ran into the problem of parsing data exactly how sysctl(8) does it for presentation. What about creating a new SYSCTL_KELVIN that is SYSCTL_PROC that converts that variable in kelvin into 10ths of Kelvin, or as a floating point value? It would break people who were directly parsing sysctl(3) output, but they were also never quering the type via the undocumented API to get the correct format and type information in the first place and depending upon the undocumented type behavior. The man pages for acpi_thermal(4), amdtemp(4) and coretemp(4) only say that the temp is supplied by those sysctl nodes, not the format nor that they are in the magic 10ths of Kelvin formation. So only preserving sysctl(8) output would be my vote. (Preserving sysctl(3) users that properly understands format and/or type would also be best). I was about to write a patch to add a function to libc to be able to get both format string and type, but obviously this has changed a bit in the last week, and at least the format string part will not be necessary. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 22:37:31 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FF40106564A for ; Wed, 19 Jan 2011 22:37:31 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A5F128FC19 for ; Wed, 19 Jan 2011 22:37:30 +0000 (UTC) Received: by ewy24 with SMTP id 24so979128ewy.13 for ; Wed, 19 Jan 2011 14:37:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=uyj7phMRtO7YTZ4VwVJ3233QiGYDpV+1IGHNcwQwfTM=; b=TuwUZqAK3dAefbY0EA9NodHT922dRFELnrXjMtp3LPfKeLVytNUDC2PzlSWX+KvVZS 5UMzvv330TUQv3J5FjQ45F/X2crQlnHY7AIA9VRuIjHrP+u7uslNulVLcqHjea2kEKO4 ST11t2lGVuERI9ycG7bO0H71XnVhfWz69b1FM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=kZjnRDO58pjPYYtYZmrwu+Wpw8G3IgdTjYBHJSNc9ZYNjA/D+gXajSe4Uid57GkJrt cZVcFKiO8NS15K9WPcEwMvEHIlSHaWZV8V2xmHV+kig44f2OtOzce47Cul6mS26w8LfW g/0sPgMiNs/9JlpqBOnZ1xGwfT6AG0TSJ6V9c= MIME-Version: 1.0 Received: by 10.227.183.198 with SMTP id ch6mr1488527wbb.227.1295476649153; Wed, 19 Jan 2011 14:37:29 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Wed, 19 Jan 2011 14:37:29 -0800 (PST) In-Reply-To: <201101191526.51723.josh@tcbug.org> References: <201101191526.51723.josh@tcbug.org> Date: Wed, 19 Jan 2011 14:37:29 -0800 X-Google-Sender-Auth: N5B2Tfa-Y4lpgDYfRevShRgMx5g Message-ID: From: Garrett Cooper To: Josh Paetzel Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org Subject: Re: FreeBSD Installer Roadmap X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 22:37:31 -0000 On Wed, Jan 19, 2011 at 1:26 PM, Josh Paetzel wrote: > Quoted text from Nathan's email: > > After some discussion with M. Warner Losh and Josh Paetzel of iX > Systems > > > > This isn't a real reply, I discovered I wasn't subscribed to arch@, so it= 's a > copy/paste. =A0If your mail client hates it, it's my fault. > > > I'm looking forward to finally moving on with life, and installers, seein= g > sysinstall laid to rest, and the merging of efforts from everyone involve= d. > > I'm also very committed to making this process as transparent as possible= . =A0If > you have questions feel free to ask. =A0The question and it's answer will > probably appear on wiki.freebsd.org The one question I'm curious about right now is txt-sysinstall going to use dialog or libdialog? Thanks! -Garrett From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 23:18:10 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8BB8106566B; Wed, 19 Jan 2011 23:18:10 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 810118FC1B; Wed, 19 Jan 2011 23:18:10 +0000 (UTC) Received: by iwn39 with SMTP id 39so1379780iwn.13 for ; Wed, 19 Jan 2011 15:18:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=B5wv9Yf5vnZM7aqRufisMxUSlxXpPLlrZrh1fVCoLlM=; b=yGqx1JAP8CMc1PBP6EZqRW+TxTxxN6LFlbV72HM9ot9pWa7K/NnRBIEE8SJp2o6Ahf dCul3aIMjE0CPLcL9cMAbx/G+88axVV8/zJpbWhof1K1JHudjiF2x3qkTqTZ98b2fdkt TB9tJ0ueDcH6J4ivb3nLDuImIaqyboftytsQg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=Hli6O3HwIvN5vE20nY08b2K0gl/5yJWs7rLIGkrprcQe0w5I/PC91xmxPjK7X0G+lm 7LAFk96JRJYORkfdu99w9phoLggtppa4udraTVQkiBh3EmS9XmH3I+TtxE2rKmjUluqk vFD1Aiju0D+H+oR+ZitoXzJSKwR5QHMGiz6pg= MIME-Version: 1.0 Received: by 10.42.171.70 with SMTP id i6mr1683018icz.322.1295479089896; Wed, 19 Jan 2011 15:18:09 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.231.160.147 with HTTP; Wed, 19 Jan 2011 15:18:09 -0800 (PST) In-Reply-To: <201101191559.07713.jhb@freebsd.org> References: <201101191559.07713.jhb@freebsd.org> Date: Wed, 19 Jan 2011 15:18:09 -0800 X-Google-Sender-Auth: WEpXy2oVOpgrTQXMk9cq4ff8v1E Message-ID: From: mdf@FreeBSD.org To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 23:18:11 -0000 On Wed, Jan 19, 2011 at 12:59 PM, John Baldwin wrote: > On Wednesday, January 19, 2011 3:28:19 pm mdf@freebsd.org wrote: >> As bde@ mentioned, there's very little actual use of the sysctl format >> strings. =A0I recently changed it so the use is even smaller, but I've >> got a quandary as to how to finish the job. >> >> I agree with Bruce that the formatted structs can just be removed. > > Hmm, I've actually used 'sysctl kern.clockrate' in the past. > >> This leaves just the "IK" format, which shows up in just a few files: >> >> sys/dev/acpica/acpi_thermal.c: >> sys/dev/amdtemp/amdtemp.c >> sys/dev/acpi_support/atk0110.c >> sys/dev/coretemp/coretemp.c >> sys/dev/iicbus/max6690.c >> sys/dev/iicbus/ds1775.c >> >> I see two solutions to "IK". =A0The first is to preserve the interface >> as experienced by sysctl(8) users, and add some functions to push a >> string buffer back to userspace, and parse a string in the kernel. >> The second is to preserve the binary interface as experienced by >> sysctl(3) users, and either have the values be dumped in the slightly >> obscure 10ths of Kelvin values, or add a new CTLTYPE_KELVIN so >> sysctl(8) can also keep showing things as it does today. >> >> Given how infrequent the use is CTLTYPE_KELVIN seems a non-starter. >> So who is the worse client to break: those who use sysctl(8) to look >> at temperatures, or those who have a utility to manipulate these >> values using sysctl(3)? > > So what is sufficiently broken elsewhere that requires us to break the > existing sysctls? =A0Many people use the output of sysctl(8) for coretemp= , and > the ACPI thermal zones and breaking that gratuitously would be unfortunat= e. > I've no idea if there is existing code reading these sysctls in userland = for > things like gkrellm, but I wouldn't be surprised if there were. The usage of the format string is (was) limited to sysctl(8) (as is the ctltype), and more importantly it was being set inconsistently. There is no way to check at compile-time if the format string is correct. > Even CTLTYPE_KELVIN would require changes to user apps, but I also would > rather present an actual number to userland rather than a string so userl= and > doesn't have to parse it just to get back to the original number. At the moment, only sysctl(8) would require a change since no other code knows about the ctltype. > I guess I fail to see why sysctl giving a hint about the semantic type of > the data ("temperature", "struct foo") which is separate from the "physic= al" > type (e.g. 32-bit or 64-bit) is such a bad thing. Except in a few circumstances, it's unused. There's a lot of "S,foo" that is being ignored. Bruce I hope will chime in with his reasons. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 23:22:10 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6073106566B for ; Wed, 19 Jan 2011 23:22:10 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5BADE8FC12 for ; Wed, 19 Jan 2011 23:22:10 +0000 (UTC) Received: by iwn39 with SMTP id 39so1382937iwn.13 for ; Wed, 19 Jan 2011 15:22:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=3n2zuS4XlH8Xu0FRWM95pISI2bWlXzKL0AxcEqpvT/o=; b=YnCc0VC/cWpbsWQKgp4KJkzKte/botKC0VbSUkIYsWLo49w6WR9xETwNC2cEnNHCkP /jOdmhpwVhH0p1WcoOM/29yFvmMl/J0hXb/FPuYqzfnvqXu9hk2PqKmo5CACzvyPQfSC cKk5uI3zlT597nKkbpQp4kBVjyzbmL3wzjUCA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=oZ05VZadtxuviAfAPNl+WNZPWRMJZv+KV4ten3J/KZCVyoyPpvxUK9wh7A0Mm2lWLM OG8TqYJXbST2E2xfiXC63k/6ZGUWZHGid1P2wuuchyLsTIfU6sx8ZUYhvjKN/KBzQpfv aXbzo/12ADaj+q6I1lXWRxYXZ7w6XBg7TVCEo= MIME-Version: 1.0 Received: by 10.231.173.67 with SMTP id o3mr1667962ibz.29.1295479259223; Wed, 19 Jan 2011 15:20:59 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.231.160.147 with HTTP; Wed, 19 Jan 2011 15:20:59 -0800 (PST) In-Reply-To: <20110119214339.GH66284@funkthat.com> References: <20110119214339.GH66284@funkthat.com> Date: Wed, 19 Jan 2011 15:20:59 -0800 X-Google-Sender-Auth: OBSLMuuP_8zsp7TIYgylNZdQLsY Message-ID: From: mdf@FreeBSD.org To: jmg@funkthat.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Arch Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 23:22:10 -0000 On Wed, Jan 19, 2011 at 1:43 PM, John-Mark Gurney wrote: > mdf@freebsd.org wrote this message on Wed, Jan 19, 2011 at 12:28 -0800: >> As bde@ mentioned, there's very little actual use of the sysctl format >> strings. =A0I recently changed it so the use is even smaller, but I've >> got a quandary as to how to finish the job. >> >> I agree with Bruce that the formatted structs can just be removed. >> This leaves just the "IK" format, which shows up in just a few files: >> >> sys/dev/acpica/acpi_thermal.c: >> sys/dev/amdtemp/amdtemp.c >> sys/dev/acpi_support/atk0110.c >> sys/dev/coretemp/coretemp.c >> sys/dev/iicbus/max6690.c >> sys/dev/iicbus/ds1775.c >> >> I see two solutions to "IK". =A0The first is to preserve the interface >> as experienced by sysctl(8) users, and add some functions to push a >> string buffer back to userspace, and parse a string in the kernel. >> The second is to preserve the binary interface as experienced by >> sysctl(3) users, and either have the values be dumped in the slightly >> obscure 10ths of Kelvin values, or add a new CTLTYPE_KELVIN so >> sysctl(8) can also keep showing things as it does today. >> >> Given how infrequent the use is CTLTYPE_KELVIN seems a non-starter. >> So who is the worse client to break: those who use sysctl(8) to look >> at temperatures, or those who have a utility to manipulate these >> values using sysctl(3)? > > I've been watching this discussion pretty closely as I recently wrote > a bsnmp module to export sysctl values and ran into the problem of > parsing data exactly how sysctl(8) does it for presentation. > > What about creating a new SYSCTL_KELVIN that is SYSCTL_PROC that > converts that variable in kelvin into 10ths of Kelvin, or as a > floating point value? > > It would break people who were directly parsing sysctl(3) output, but > they were also never quering the type via the undocumented API to get > the correct format and type information in the first place and depending > upon the undocumented type behavior. > > The man pages for acpi_thermal(4), amdtemp(4) and coretemp(4) only > say that the temp is supplied by those sysctl nodes, not the format > nor that they are in the magic 10ths of Kelvin formation. =A0So only > preserving sysctl(8) output would be my vote. =A0(Preserving sysctl(3) > users that properly understands format and/or type would also be best). > > I was about to write a patch to add a function to libc to be able to > get both format string and type, but obviously this has changed a bit > in the last week, and at least the format string part will not be > necessary. IMO it would be nice to have a real API to fetch the type. The sysctl documentation claims it's being thought about, but I think it's said that for several years. :-) The format string was mostly unused and I'd prefer it to die than be used more. Except for the case of "IK" and 4 specially formatted structs, the string provides no new information over the ctltype. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 23:26:35 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21F4D106564A; Wed, 19 Jan 2011 23:26:35 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id B0FC18FC14; Wed, 19 Jan 2011 23:26:34 +0000 (UTC) Received: by iyb26 with SMTP id 26so1318068iyb.13 for ; Wed, 19 Jan 2011 15:26:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Q03U5IjoPeHBVTxe6gsGc5WhwXbejqmDw3DF5zeOTIk=; b=icfNYyBSj0nKU/If/5y0bsldo9bBGwiD/TyTeFOf+iEUoFRASoAobeySgptlCpWiBq pSSCDOmKU3B/BauGH2+YTFoefHMfkDyRSkR+2jJm36FDzsFIEcg3DyB877Edl5gI5kNZ SNM0uf+YjVSf4kxWzJspc8eUxh7ukUEuH/IrA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=ixjpWllihVXOLUczaUuTr9yiWfhAJWA9TfeErIwO5+AwqE4WeXcOlsKaaLOUZcgnUe T3J1jjTOTaHkJ6oUFUHDAIin/L9kRDyEHso+SGStFypMXMtCAt6oPB3r6nT1kfApy7EL tW9hSmQ4q1ney8943FpLL9L7WnaIaYLliI0+4= MIME-Version: 1.0 Received: by 10.231.35.141 with SMTP id p13mr1652793ibd.79.1295479594333; Wed, 19 Jan 2011 15:26:34 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.231.160.147 with HTTP; Wed, 19 Jan 2011 15:26:34 -0800 (PST) In-Reply-To: <20110119205445.GX21872@elvis.mu.org> References: <20110119205445.GX21872@elvis.mu.org> Date: Wed, 19 Jan 2011 15:26:34 -0800 X-Google-Sender-Auth: 9ROba8rG1nb_7-uusjnJYKphDyA Message-ID: From: mdf@FreeBSD.org To: Alfred Perlstein Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Arch Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 23:26:35 -0000 On Wed, Jan 19, 2011 at 12:54 PM, Alfred Perlstein wro= te: > * mdf@FreeBSD.org [110119 12:28] wrote: >> As bde@ mentioned, there's very little actual use of the sysctl format >> strings. =A0I recently changed it so the use is even smaller, but I've >> got a quandary as to how to finish the job. >> >> I agree with Bruce that the formatted structs can just be removed. >> This leaves just the "IK" format, which shows up in just a few files: >> >> sys/dev/acpica/acpi_thermal.c: >> sys/dev/amdtemp/amdtemp.c >> sys/dev/acpi_support/atk0110.c >> sys/dev/coretemp/coretemp.c >> sys/dev/iicbus/max6690.c >> sys/dev/iicbus/ds1775.c >> >> I see two solutions to "IK". =A0The first is to preserve the interface >> as experienced by sysctl(8) users, and add some functions to push a >> string buffer back to userspace, and parse a string in the kernel. >> The second is to preserve the binary interface as experienced by >> sysctl(3) users, and either have the values be dumped in the slightly >> obscure 10ths of Kelvin values, or add a new CTLTYPE_KELVIN so >> sysctl(8) can also keep showing things as it does today. >> >> Given how infrequent the use is CTLTYPE_KELVIN seems a non-starter. >> So who is the worse client to break: those who use sysctl(8) to look >> at temperatures, or those who have a utility to manipulate these >> values using sysctl(3)? > > I'd say that it's not great to break either system. > > The reason is that either the syscall or cmd line utility can be > the basis of numerous system monitoring tools. > > By breaking either interface we discourage people from using it. > > I apologize for coming in so late, but why is CTLTYPE_KELVIN such > an awful thing? Mostly because it's relatively unused, and except for three acpi_thermal instances it's read-only. I wasn't able to find any tools in the tree that knew what format these are in, and as someone else pointed out the units aren't part of any documentation either. > Also, not to digress, but it sounds like there's a rototilling of > the KPI going on here as well that might break 3rd parties who do > their own monitoring software. So far I don't believe I have changed any KPI except for if someone had a custom handler for a QUAD. I would like to do so at some point as 99% of SYSCTL set-up uses OID_AUTO, and it should be 100% but for some legacy code. > Overall it's not a big thing, but when you consider all the changes > like this that go on, it can add up to a discouraging target to > track. > > Honestly I don't have a strong opinion on this and feel free to use > your best judgement and as well as what other people bring to the > table, I just wanted to bring the software churn issue up and > leave the question, "is there a way to do this with minimal 3rd party > breakage". Existing uses of the standard SYSCTL_[ADD_]FOO can be made backwards compatible for a release or so, with minimal effort on my part. It makes the tree a little uglier for a while, though. I was involved with a FreeBSD merge at $WORK from 6.1 to stable/7 and there was a decent bit of change in there. For my part, I preferred a nice clean compile-break to something that still compiled but wasn't the right way to do things going forward. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Wed Jan 19 23:43:44 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA929106564A; Wed, 19 Jan 2011 23:43:44 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from agogare.doit.wisc.edu (agogare.doit.wisc.edu [144.92.197.211]) by mx1.freebsd.org (Postfix) with ESMTP id 9D6CF8FC14; Wed, 19 Jan 2011 23:43:44 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth2.wiscmail.wisc.edu by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LFA0070MKGVR200@smtpauth2.wiscmail.wisc.edu>; Wed, 19 Jan 2011 16:43:43 -0600 (CST) Received: from anacreon.physics.wisc.edu (anacreon.physics.wisc.edu [128.104.160.176]) by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LFA00JF7KGO1Q40@smtpauth2.wiscmail.wisc.edu>; Wed, 19 Jan 2011 16:43:36 -0600 (CST) Date: Wed, 19 Jan 2011 16:43:36 -0600 From: Nathan Whitehorn In-reply-to: To: Garrett Cooper Message-id: <4D376918.9070208@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=128.104.160.176 X-Spam-PmxInfo: Server=avs-9, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.1.19.223620, SenderIP=128.104.160.176 References: <201101191526.51723.josh@tcbug.org> User-Agent: Mozilla/5.0 (X11; U; FreeBSD powerpc; en-US; rv:1.9.2.13) Gecko/20110104 Thunderbird/3.1.7 Cc: Josh Paetzel , arch@freebsd.org Subject: Re: FreeBSD Installer Roadmap X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 23:43:44 -0000 On 01/19/11 16:37, Garrett Cooper wrote: > On Wed, Jan 19, 2011 at 1:26 PM, Josh Paetzel wrote: >> Quoted text from Nathan's email: >> >> After some discussion with M. Warner Losh and Josh Paetzel of iX >> Systems >> >> >> >> This isn't a real reply, I discovered I wasn't subscribed to arch@, so it's a >> copy/paste. If your mail client hates it, it's my fault. >> >> >> I'm looking forward to finally moving on with life, and installers, seeing >> sysinstall laid to rest, and the merging of efforts from everyone involved. >> >> I'm also very committed to making this process as transparent as possible. If >> you have questions feel free to ask. The question and it's answer will >> probably appear on wiki.freebsd.org > The one question I'm curious about right now is txt-sysinstall > going to use dialog or libdialog? > Thanks! Since we will borrow the UI from bsdinstall, it will use the new libdialog. -Nathan From owner-freebsd-arch@FreeBSD.ORG Thu Jan 20 08:38:38 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 595CF106566C; Thu, 20 Jan 2011 08:38:38 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail08.syd.optusnet.com.au (mail08.syd.optusnet.com.au [211.29.132.189]) by mx1.freebsd.org (Postfix) with ESMTP id D922D8FC0A; Thu, 20 Jan 2011 08:38:37 +0000 (UTC) Received: from c122-106-165-206.carlnfd1.nsw.optusnet.com.au (c122-106-165-206.carlnfd1.nsw.optusnet.com.au [122.106.165.206]) by mail08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p0K8cXPn030172 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 20 Jan 2011 19:38:35 +1100 Date: Thu, 20 Jan 2011 19:38:33 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: John Baldwin In-Reply-To: <201101191559.07713.jhb@freebsd.org> Message-ID: <20110120190703.H11630@besplex.bde.org> References: <201101191559.07713.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: mdf@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 08:38:38 -0000 On Wed, 19 Jan 2011, John Baldwin wrote: > On Wednesday, January 19, 2011 3:28:19 pm mdf@freebsd.org wrote: >> As bde@ mentioned, there's very little actual use of the sysctl format >> strings. I recently changed it so the use is even smaller, but I've >> got a quandary as to how to finish the job. >> >> I agree with Bruce that the formatted structs can just be removed. > > Hmm, I've actually used 'sysctl kern.clockrate' in the past. But you would have found it easier to use if its output were kern.clockrate.hz: 100 kern.clockrate.tick: 10000 kern.clockrate.profhz: 1024 kern.clockrate.stathz: 128 instead of kern.clockrate: { hz = 100, tick = 10000, profhz = 1024, stathz = 128 } , no? Even more so for use in a script, say hz=$(sysctl -n kern.clockrate.hz) # can't do this now. The old clockrate sysctl would be retained for binary compatibility, but need not have special support in sysctl(8). The new sysctl would have to have a different name, but `clockrate' isn't such a good name anyway (`tick' isn't a rate, but the inverse of a rate, i.e., a time. However, it is closely associated with `hz' so it should be in the same sysctl tree, perhaps kern.clock.) >> This leaves just the "IK" format, which shows up in just a few files: >> >> sys/dev/acpica/acpi_thermal.c: >> sys/dev/amdtemp/amdtemp.c >> sys/dev/acpi_support/atk0110.c >> sys/dev/coretemp/coretemp.c >> sys/dev/iicbus/max6690.c >> sys/dev/iicbus/ds1775.c >> >> I see two solutions to "IK". The first is to preserve the interface >> as experienced by sysctl(8) users, and add some functions to push a >> string buffer back to userspace, and parse a string in the kernel. >> The second is to preserve the binary interface as experienced by >> sysctl(3) users, and either have the values be dumped in the slightly >> obscure 10ths of Kelvin values, or add a new CTLTYPE_KELVIN so >> sysctl(8) can also keep showing things as it does today. >> >> Given how infrequent the use is CTLTYPE_KELVIN seems a non-starter. >> So who is the worse client to break: those who use sysctl(8) to look >> at temperatures, or those who have a utility to manipulate these >> values using sysctl(3)? Special formatting for these in sysctl(8) was a mistake, but I think have a special CTLTYPE to keep indicating that these are temperatures is very easy, especially since it is used infrequently. Also, it's not Kelvin, but deci-Kelvin :-). > So what is sufficiently broken elsewhere that requires us to break the > existing sysctls? Many people use the output of sysctl(8) for coretemp, and > the ACPI thermal zones and breaking that gratuitously would be unfortunate. I only want to remove the format strings. This saves checking that they are consistent with the data types, and some space. > I've no idea if there is existing code reading these sysctls in userland for > things like gkrellm, but I wouldn't be surprised if there were. Hopefully thes is none using the format string except sysctl(8). sysctl(8) has to use intentionally still (?) undocumented sysctls even to read the format string. Other applications just need to know what sysctls returning degrees in deci-Kelvin actually are. They should have no difficulty knowing that the format is deci-Kelvin. > Even CTLTYPE_KELVIN would require changes to user apps, but I also would > rather present an actual number to userland rather than a string so userland > doesn't have to parse it just to get back to the original number. > > I guess I fail to see why sysctl giving a hint about the semantic type of > the data ("temperature", "struct foo") which is separate from the "physical" > type (e.g. 32-bit or 64-bit) is such a bad thing. Note that this is not done for newer sysctls like kern.cp_time and kern.cptimes. The latter is especially cryptic since it doesn't even have delimiters for each CPU. Please keep them that way :-). (I have occasionally used kern.cptime output and had to stare at it to make sense of it. Dividing it by stathz would have been only a small help. I needed differences of it and ran a filter to get them.) Converting deci-Kelvin to Kelvin is relatively trivial. So is converting to Centigrade. But many would prefer Fahrenheit, or a GUI representation... The fancy presentation belongs in specialized apps. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Jan 20 10:59:35 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 910D91065673; Thu, 20 Jan 2011 10:59:35 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id 12DB48FC1B; Thu, 20 Jan 2011 10:59:34 +0000 (UTC) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.3/8.14.3) with ESMTP id p0KAxHUM053964; Thu, 20 Jan 2011 11:59:33 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.3/8.14.3/Submit) id p0KAxHgn053963; Thu, 20 Jan 2011 11:59:17 +0100 (CET) (envelope-from olli) Date: Thu, 20 Jan 2011 11:59:17 +0100 (CET) Message-Id: <201101201059.p0KAxHgn053963@lurza.secnetix.de> From: Oliver Fromme To: freebsd-arch@FreeBSD.ORG, mdf@FreeBSD.ORG In-Reply-To: X-Newsgroups: list.freebsd-arch User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.4-PRERELEASE-20080904 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.3.5 (lurza.secnetix.de [127.0.0.1]); Thu, 20 Jan 2011 11:59:33 +0100 (CET) Cc: Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 10:59:35 -0000 mdf@freebsd.org wrote: > As bde@ mentioned, there's very little actual use of the sysctl format > strings. I recently changed it so the use is even smaller, but I've > got a quandary as to how to finish the job. > > I agree with Bruce that the formatted structs can just be removed. Will that break scripts that use sysctl(8) for monitoring, logging and similar tasks (vm.loadavg for example)? I've installed such scripts at a few customers' sites over the past years. It would be somewhat unfortunate if they break when the admins at those sites decide to update the OS. Don't get me wrong, I'm not trying to convince you to keep the formatted output for script compatibility. I understand the reasons why they should be removed. I'm just trying to evaluate the consequences. Best regards Oliver PS: Personally I like the format very much, because it's easy to use in shell scripts. For example, when you write set $(sysctl -n vm.loadavg) then you have the three values in $2, $3 and $4. Another phrase I've used in scripts quite often is this: X=$(sysctl -n kern.boottime) echo ${X#*\}} There are other ways to get that piece of information, but they're more complicated and/or less efficient. -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Unix gives you just enough rope to hang yourself -- and then a couple of more feet, just to be sure." -- Eric Allman From owner-freebsd-arch@FreeBSD.ORG Thu Jan 20 12:50:09 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79642106564A; Thu, 20 Jan 2011 12:50:09 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 316108FC1F; Thu, 20 Jan 2011 12:50:09 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 9DAC046B09; Thu, 20 Jan 2011 07:50:08 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 8BCE58A01D; Thu, 20 Jan 2011 07:50:07 -0500 (EST) From: John Baldwin To: mdf@freebsd.org Date: Thu, 20 Jan 2011 07:48:49 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <201101191559.07713.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101200748.49931.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Thu, 20 Jan 2011 07:50:07 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: freebsd-arch@freebsd.org Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 12:50:09 -0000 On Wednesday, January 19, 2011 6:18:09 pm mdf@freebsd.org wrote: > On Wed, Jan 19, 2011 at 12:59 PM, John Baldwin wrote: > > On Wednesday, January 19, 2011 3:28:19 pm mdf@freebsd.org wrote: > >> As bde@ mentioned, there's very little actual use of the sysctl format > >> strings. I recently changed it so the use is even smaller, but I've > >> got a quandary as to how to finish the job. > >> > >> I agree with Bruce that the formatted structs can just be removed. > > > > Hmm, I've actually used 'sysctl kern.clockrate' in the past. > > > >> This leaves just the "IK" format, which shows up in just a few files: > >> > >> sys/dev/acpica/acpi_thermal.c: > >> sys/dev/amdtemp/amdtemp.c > >> sys/dev/acpi_support/atk0110.c > >> sys/dev/coretemp/coretemp.c > >> sys/dev/iicbus/max6690.c > >> sys/dev/iicbus/ds1775.c > >> > >> I see two solutions to "IK". The first is to preserve the interface > >> as experienced by sysctl(8) users, and add some functions to push a > >> string buffer back to userspace, and parse a string in the kernel. > >> The second is to preserve the binary interface as experienced by > >> sysctl(3) users, and either have the values be dumped in the slightly > >> obscure 10ths of Kelvin values, or add a new CTLTYPE_KELVIN so > >> sysctl(8) can also keep showing things as it does today. > >> > >> Given how infrequent the use is CTLTYPE_KELVIN seems a non-starter. > >> So who is the worse client to break: those who use sysctl(8) to look > >> at temperatures, or those who have a utility to manipulate these > >> values using sysctl(3)? > > > > So what is sufficiently broken elsewhere that requires us to break the > > existing sysctls? Many people use the output of sysctl(8) for coretemp, and > > the ACPI thermal zones and breaking that gratuitously would be unfortunate. > > I've no idea if there is existing code reading these sysctls in userland for > > things like gkrellm, but I wouldn't be surprised if there were. > > The usage of the format string is (was) limited to sysctl(8) (as is > the ctltype), and more importantly it was being set inconsistently. > There is no way to check at compile-time if the format string is > correct. The two cons I see to this are 1) many scripts rely on the output of sysctl(8), and 2) jmg@ was actually talking about exporting the metadata via libc routines so that applications could begin to use them. > > Even CTLTYPE_KELVIN would require changes to user apps, but I also would > > rather present an actual number to userland rather than a string so userland > > doesn't have to parse it just to get back to the original number. > > At the moment, only sysctl(8) would require a change since no other > code knows about the ctltype. > > > I guess I fail to see why sysctl giving a hint about the semantic type of > > the data ("temperature", "struct foo") which is separate from the "physical" > > type (e.g. 32-bit or 64-bit) is such a bad thing. > > Except in a few circumstances, it's unused. There's a lot of "S,foo" > that is being ignored. Well, currently applications hardcode what certain sysctls are because that information is not exposed to userland. We could stick with applications having to hardcode that, but I do think that 1) the output of sysctl(8) for the temperature nodes has to be preserved due to existing scripts, and that 2) the raw data exported via sysctl(3) should be an int, not a string. I'm not sure how you can do that without some sort of hint to tag certain nodes as having deci-Kelvin units. Note that you could change the format string to not include type information. For example, it could just contain "K", or "X" (for hex), but not include "I" or "L" or "U". This avoids duplicating the CTLTYPE_* flags that you can't add compile time assertions for while leaving the semantic tagging. I think it would be ok to break the existing format strings so they only hold that sort of data which would let you not have to worry about them being out of sync with the CTLTYPE_* flags. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Jan 20 16:58:36 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9308106564A; Thu, 20 Jan 2011 16:58:36 +0000 (UTC) (envelope-from mexas@bristol.ac.uk) Received: from dirg.bris.ac.uk (dirg.bris.ac.uk [137.222.10.102]) by mx1.freebsd.org (Postfix) with ESMTP id A1B408FC0C; Thu, 20 Jan 2011 16:58:36 +0000 (UTC) Received: from ncsc.bris.ac.uk ([137.222.10.41]) by dirg.bris.ac.uk with esmtp (Exim 4.72) (envelope-from ) id 1PfxbV-0001eP-Bb; Thu, 20 Jan 2011 16:43:21 +0000 Received: from mech-cluster241.men.bris.ac.uk ([137.222.187.241]) by ncsc.bris.ac.uk with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1PfxbV-00064k-3v; Thu, 20 Jan 2011 16:43:21 +0000 Received: from mech-cluster241.men.bris.ac.uk (localhost [127.0.0.1]) by mech-cluster241.men.bris.ac.uk (8.14.4/8.14.4) with ESMTP id p0KGhKNh018264; Thu, 20 Jan 2011 16:43:20 GMT (envelope-from mexas@bristol.ac.uk) Received: (from mexas@localhost) by mech-cluster241.men.bris.ac.uk (8.14.4/8.14.4/Submit) id p0KGhKNm018263; Thu, 20 Jan 2011 16:43:20 GMT (envelope-from mexas@bristol.ac.uk) X-Authentication-Warning: mech-cluster241.men.bris.ac.uk: mexas set sender to mexas@bristol.ac.uk using -f Date: Thu, 20 Jan 2011 16:43:20 +0000 From: Anton Shterenlikht To: Ade Lovett Message-ID: <20110120164320.GA17624@mech-cluster241.men.bris.ac.uk> Mail-Followup-To: Ade Lovett , Marcel Moolenaar , freebsd-current@freebsd.org, freebsd-arch@freebsd.org References: <4D309563.1000404@freebsd.org> <7029F1A3-87A2-4203-840C-48B712EA70B8@mac.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org, Marcel Moolenaar , freebsd-current@freebsd.org Subject: Re: BSDInstall: merging to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 16:58:37 -0000 On Sat, Jan 15, 2011 at 12:58:33AM -0600, Ade Lovett wrote: > > On Jan 14, 2011, at 19:31 , Marcel Moolenaar wrote: > > On Jan 14, 2011, at 10:26 AM, Nathan Whitehorn wrote: > > > >> The final architecture on which we use sysinstall, ia64, is currently unsupported, because I don't know how to set up booting on those systems -- patches to solve this are very much welcome. > > > > Don't let this stop you. I'll work with you on this after the dust > > has settled. > > Just out of random curiosity. Seriously. > > Exactly why, short of "of course it runs", in which case NetBSD is --> way, why are we even trying to handle ia64 as a platform, regardless of tier, when it is patently obvious that it is going absolutely _nowhere_ in terms of a viable platform? > I was waiting for a developer to reply, but anyway.. I don't know why you, the FreeBSD project, offer ia64 distribution, but as a user I'm very grateful to you for providing it. I think NetBSD is nowhere near FreeBSD on ia64 support. I've learnt things with FreeBSD/ia64 (and with FreeBSD/alpha before it) which I woudn't have leart otherwise. I've 3 FreeBSD/ia64 boxes and by and large I'm very happy with this platform. I can do all I need to do with it. The ~400 or so ports installed on my system is good enough. > At least I can pick up a box for <$50 from ebay and run /sparc64 on it. Say the same for /ia64? Didn't think so. > Well.. ia64 is more expensive, yes. But.. I think it can give you more. I'm quite tempted to get this server, for example: http://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&item=280618792191 But sparc64 is good as well. In fact, I use a FreeBSD/sparc64 box as an Xserver to view clients running on FreeBSD/ia64. many thanks for your excellent work anton -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 331 5944 Fax: +44 (0)117 929 4423 From owner-freebsd-arch@FreeBSD.ORG Thu Jan 20 18:30:18 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E7731065693 for ; Thu, 20 Jan 2011 18:30:18 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id B5A2E8FC14 for ; Thu, 20 Jan 2011 18:30:17 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.4/8.14.4) with ESMTP id p0KIUGOL067556; Thu, 20 Jan 2011 13:30:16 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.4/8.14.4/Submit) id p0KIUFx9067555; Thu, 20 Jan 2011 13:30:15 -0500 (EST) (envelope-from wollman) Date: Thu, 20 Jan 2011 13:30:15 -0500 (EST) From: Garrett Wollman Message-Id: <201101201830.p0KIUFx9067555@hergotha.csail.mit.edu> To: brde@optusnet.com.au X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: References: Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (hergotha.csail.mit.edu [127.0.0.1]); Thu, 20 Jan 2011 13:30:16 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on hergotha.csail.mit.edu Cc: arch@freebsd.org Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 18:30:18 -0000 In article , Bruce Evans writes: >Also, it's not Kelvin, but deci-Kelvin :-). If we're being pedantic, the name of the unit is the "kelvin", lowercase "k", and the derived unit is the "decikelvin", lowercase "k" and no hyphen. The unit symbol is "K" (uppercase) for the base unit, "dK" for the derived unit. All SI unit names are in lower case; only the symbols are (sometimes) capitalized if the unit is named after a proper noun. (Thus the symbol for the second is "s"; the symbol "S" is for the Siemens, the unit of conductivity formerly called the "mho".) -GAWollman From owner-freebsd-arch@FreeBSD.ORG Thu Jan 20 18:43:19 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C22A11065670 for ; Thu, 20 Jan 2011 18:43:19 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5279E8FC1B for ; Thu, 20 Jan 2011 18:43:19 +0000 (UTC) Received: by wyf19 with SMTP id 19so936116wyf.13 for ; Thu, 20 Jan 2011 10:43:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=w2r5JjVqLWUWQpJ8klQtxvgeh35ztzT/1vRYKOVmAmM=; b=X+JibJtoj2useZBzamnPw0PYueiLv+CwLzgRIeVGFeVAPw2LLC3CsZbVF+xCT5S7/O TDhf+znw8WPGLA4jyn58J0bJYEaVwiKPrx/uaNDyTfBQC8NR/Xp5h+BjYG8CE1mWFWb7 F0O4ZCgoaFz8DXZ2H8IwdyDbYyDyBlg8gITfs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=BEUspwPXozRfEQcBzN+yks2YdFpBvHedatm1acK/H5lnl4a9002TcacPoF4ACCHGYO ZDD5AWGdmXXvmrIZEbUIztFP9R8oXUTQL9UGpbaiNB5I8KQJPFOv0D11HIj4/ysxgLFm ylNRUgJnMQez3AiYB9L/MKlsyFk96wT0B8sJ0= MIME-Version: 1.0 Received: by 10.216.30.137 with SMTP id k9mr2342665wea.31.1295548998238; Thu, 20 Jan 2011 10:43:18 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Thu, 20 Jan 2011 10:43:14 -0800 (PST) In-Reply-To: <201101201830.p0KIUFx9067555@hergotha.csail.mit.edu> References: <201101201830.p0KIUFx9067555@hergotha.csail.mit.edu> Date: Thu, 20 Jan 2011 10:43:14 -0800 X-Google-Sender-Auth: FolvKVPmLn-cKCCwZjh-71J6yv0 Message-ID: From: Garrett Cooper To: Garrett Wollman Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 18:43:19 -0000 On Thu, Jan 20, 2011 at 10:30 AM, Garrett Wollman wrote: > In article , > Bruce Evans writes: > >>Also, it's not Kelvin, but deci-Kelvin :-). > > If we're being pedantic, the name of the unit is the "kelvin", > lowercase "k", and the derived unit is the "decikelvin", lowercase "k" > and no hyphen. =A0The unit symbol is "K" (uppercase) for the base unit, > "dK" for the derived unit. =A0All SI unit names are in lower case; only > the symbols are (sometimes) capitalized if the unit is named after a > proper noun. =A0(Thus the symbol for the second is "s"; the symbol "S" > is for the Siemens, the unit of conductivity formerly called the > "mho".) Tunables support mixed-case units incorrectly as well, i.e. 1. 'g' / 'G' -> 'giga' 2. 'm' / 'M' -> 'mega' 3. 'k' / 'K' -> 'kilo' Something that Bruce brought up in another email thread with me. Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Thu Jan 20 21:12:16 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72EB4106566C for ; Thu, 20 Jan 2011 21:12:16 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 2F28A8FC08 for ; Thu, 20 Jan 2011 21:12:16 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id p0KL9A35039448 for ; Thu, 20 Jan 2011 14:09:10 -0700 (MST) (envelope-from imp@bsdimp.com) Message-ID: <4D38A476.10709@bsdimp.com> Date: Thu, 20 Jan 2011 14:09:10 -0700 From: Warner Losh User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101211 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <201101201830.p0KIUFx9067555@hergotha.csail.mit.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 21:12:16 -0000 On 01/20/2011 11:43, Garrett Cooper wrote: > On Thu, Jan 20, 2011 at 10:30 AM, Garrett Wollman > wrote: >> In article, >> Bruce Evans writes: >> >>> Also, it's not Kelvin, but deci-Kelvin :-). >> If we're being pedantic, the name of the unit is the "kelvin", >> lowercase "k", and the derived unit is the "decikelvin", lowercase "k" >> and no hyphen. The unit symbol is "K" (uppercase) for the base unit, >> "dK" for the derived unit. All SI unit names are in lower case; only >> the symbols are (sometimes) capitalized if the unit is named after a >> proper noun. (Thus the symbol for the second is "s"; the symbol "S" >> is for the Siemens, the unit of conductivity formerly called the >> "mho".) > > Tunables support mixed-case units incorrectly as well, i.e. > 1. 'g' / 'G' -> 'giga' > 2. 'm' / 'M' -> 'mega' > 3. 'k' / 'K' -> 'kilo' > Something that Bruce brought up in another email thread with me. > No. 'k' is the proper SI prefix for kilo. And all the SI prefixes are powers of ten only, but that's another can of worms. http://physics.nist.gov/cuu/Units/prefixes.html Warner Warner From owner-freebsd-arch@FreeBSD.ORG Thu Jan 20 21:14:51 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D89561065674 for ; Thu, 20 Jan 2011 21:14:51 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 684818FC18 for ; Thu, 20 Jan 2011 21:14:51 +0000 (UTC) Received: by wyf19 with SMTP id 19so1087832wyf.13 for ; Thu, 20 Jan 2011 13:14:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=9fCfDQp6mSULxFARIADaJDnqVWbIqj8ejqRYpLoh/aY=; b=k6cRtQlzbF1e9pInctPRP/rBgskYlA5BKubvRQvSjOj7Lb+gYx/Cy/A+q2+yIYpcCj fonqGf0gtM2XIC+jzTxDB3RkdPgsB7zSfWYuf5x6+Ixgb1W39VgSu/bsCUt7jtv5WH0/ LqKRqsLR23I5EkTzJjptfQRX89gHar20UIv00= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=RnwOQs/x+DfzL6VLnlzIdMJ/TGak+m9p9ZwMNGOOi/0UHuZWkpNhwgOEm0kD+Mn1+s a7PNVuWmOGN7lV/8mCkkb/W2nyP/Upntyf5pIA67vJ7DSXIdV9MX4Ub9RNU4etJKIhDb N2I2V7K4ABIFOa0DWogfKOPSOKDrLzMXohn+Y= MIME-Version: 1.0 Received: by 10.216.30.137 with SMTP id k9mr2508724wea.31.1295558090227; Thu, 20 Jan 2011 13:14:50 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Thu, 20 Jan 2011 13:14:49 -0800 (PST) In-Reply-To: <4D38A476.10709@bsdimp.com> References: <201101201830.p0KIUFx9067555@hergotha.csail.mit.edu> <4D38A476.10709@bsdimp.com> Date: Thu, 20 Jan 2011 13:14:49 -0800 X-Google-Sender-Auth: sGTALlkt80y-nBa1-lVS8RFs0l0 Message-ID: From: Garrett Cooper To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 21:14:51 -0000 On Thu, Jan 20, 2011 at 1:09 PM, Warner Losh wrote: > On 01/20/2011 11:43, Garrett Cooper wrote: >> >> On Thu, Jan 20, 2011 at 10:30 AM, Garrett Wollman >> =A0wrote: >>> >>> In >>> article, >>> Bruce Evans writes: >>> >>>> Also, it's not Kelvin, but deci-Kelvin :-). >>> >>> If we're being pedantic, the name of the unit is the "kelvin", >>> lowercase "k", and the derived unit is the "decikelvin", lowercase "k" >>> and no hyphen. =A0The unit symbol is "K" (uppercase) for the base unit, >>> "dK" for the derived unit. =A0All SI unit names are in lower case; only >>> the symbols are (sometimes) capitalized if the unit is named after a >>> proper noun. =A0(Thus the symbol for the second is "s"; the symbol "S" >>> is for the Siemens, the unit of conductivity formerly called the >>> "mho".) >> >> >> Tunables support mixed-case units incorrectly as well, i.e. >> 1. 'g' / 'G' -> =A0'giga' >> 2. 'm' / 'M' -> =A0'mega' >> 3. 'k' / 'K' -> =A0'kilo' >> Something that Bruce brought up in another email thread with me. >> > > No. =A0'k' is the proper SI prefix for kilo. =A0And all the SI prefixes a= re > powers of ten only, but that's another can of worms. > > http://physics.nist.gov/cuu/Units/prefixes.html 1. 'g' doesn't map to anything. 2. 'm' is incorrect ('milli' vs 'mega'). 3. 'K' doesn't map to anything. That's what I meant. Thanks! -Garrett From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 00:24:45 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C940F106564A for ; Fri, 21 Jan 2011 00:24:45 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id 801AA8FC0C for ; Fri, 21 Jan 2011 00:24:45 +0000 (UTC) Received: from [192.168.221.2] (remotevpn [192.168.221.2]) by ns1.feral.com (8.14.4/8.14.3) with ESMTP id p0KNrJ5Q054182 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 20 Jan 2011 15:53:33 -0800 (PST) (envelope-from mj@feral.com) Message-ID: <4D38CAEE.2060705@feral.com> Date: Thu, 20 Jan 2011 15:53:18 -0800 From: Matthew Jacob Organization: Feral Software User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.6 (ns1.feral.com [192.168.221.1]); Thu, 20 Jan 2011 15:53:34 -0800 (PST) Cc: Subject: removing -frename-registers from sys/conf.mk X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 00:24:45 -0000 It helps substantially in trying to debug amd64 kernels to remove this flag. Has anyone heard of a good reason we need to keep this? Whom else should I ask? From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 00:34:51 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC4B81065672 for ; Fri, 21 Jan 2011 00:34:51 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id B93868FC19 for ; Fri, 21 Jan 2011 00:34:51 +0000 (UTC) Received: from [192.168.221.2] (remotevpn [192.168.221.2]) by ns1.feral.com (8.14.4/8.14.3) with ESMTP id p0L0YZ2e054534 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 20 Jan 2011 16:34:38 -0800 (PST) (envelope-from mj@feral.com) Message-ID: <4D38D49A.2010909@feral.com> Date: Thu, 20 Jan 2011 16:34:34 -0800 From: Matthew Jacob Organization: Feral Software User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <4D38CAEE.2060705@feral.com> In-Reply-To: <4D38CAEE.2060705@feral.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.6 (ns1.feral.com [192.168.221.1]); Thu, 20 Jan 2011 16:34:39 -0800 (PST) Subject: Re: removing -frename-registers from sys/conf.mk X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 00:34:52 -0000 Never mind this ill thought out request. On 1/20/2011 3:53 PM, Matthew Jacob wrote: > It helps substantially in trying to debug amd64 kernels to remove this > flag. > > Has anyone heard of a good reason we need to keep this? Whom else > should I ask? > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 05:14:49 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAA07106567A; Fri, 21 Jan 2011 05:14:49 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail02.syd.optusnet.com.au (mail02.syd.optusnet.com.au [211.29.132.183]) by mx1.freebsd.org (Postfix) with ESMTP id 617078FC14; Fri, 21 Jan 2011 05:14:49 +0000 (UTC) Received: from c122-106-165-206.carlnfd1.nsw.optusnet.com.au (c122-106-165-206.carlnfd1.nsw.optusnet.com.au [122.106.165.206]) by mail02.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p0L5EeVT027514 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 21 Jan 2011 16:14:42 +1100 Date: Fri, 21 Jan 2011 16:14:40 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Garrett Cooper In-Reply-To: Message-ID: <20110121153528.P2616@besplex.bde.org> References: <201101201830.p0KIUFx9067555@hergotha.csail.mit.edu> <4D38A476.10709@bsdimp.com> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-1396774578-1295586880=:2616" Cc: freebsd-arch@freebsd.org Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 05:14:50 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-1396774578-1295586880=:2616 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Thu, 20 Jan 2011, Garrett Cooper wrote: > On Thu, Jan 20, 2011 at 1:09 PM, Warner Losh wrote: >> On 01/20/2011 11:43, Garrett Cooper wrote: >>> >>> Tunables support mixed-case units incorrectly as well, i.e. >>> 1. 'g' / 'G' -> =A0'giga' >>> 2. 'm' / 'M' -> =A0'mega' >>> 3. 'k' / 'K' -> =A0'kilo' >>> Something that Bruce brought up in another email thread with me. >>> >> >> No. =A0'k' is the proper SI prefix for kilo. =A0And all the SI prefixes = are >> powers of ten only, but that's another can of worms. >> >> http://physics.nist.gov/cuu/Units/prefixes.html > > 1. 'g' doesn't map to anything. > 2. 'm' is incorrect ('milli' vs 'mega'). > 3. 'K' doesn't map to anything. > > That's what I meant. Same can. There is a significant shortage of letters, and significant confusion between prefixes for powers of 10 and powers of 2, and significant need for both, but scientificize^Wde^Whumanize_number() is of such quality that it wastes a letter for an alias for every supported "prefix". (I'm looking at the possibly-old man page on ref9-amd64, where "suffix" is still spelled "prefix" and the aliases are of course undocumented. Garrett started fixing this.) 'K' is the most broken in the man page. SI 'k' comflicts with computerspeak 'K' least among the prefixes (I think all others are capitals in both SI and computerspeak), but the man page says that the "prefix" for 1024 and 1000 is 'k' and doesn't mention its undocumented alias 'K'. OTOH, humanize_number() actually agrees with both SI and computerspeak according to df output, since the suffix is 'k' for powers of 10 format (df -H) and 'K' for powers of 2 format (df -h). Bugs noticed while testing this: - inode counts are printed in power of 10 even with df -h. This is actuall= y documented. You can see the difference if their suffix is k, but not if it is M or larger. - verbose names (mostly for nfs mounts) cause the output to not fit in 80 columns even with -h and without -i. Lots of columns are usually wasted in -h format. This despite df having lots of dynamic formatting. So k vs K more or less works as well as possible for scientificize_number()= , but is not documented to do so. parse^Wexpand_number() is considerably more primitive and broken. It only supports power of 2 format, and only documents k, but accepts K as an alias for k. So it converts 1000 printed as 1k by scientificize_number() to 1024. Bruce --0-1396774578-1295586880=:2616-- From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 06:15:19 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85BCF1065673 for ; Fri, 21 Jan 2011 06:15:19 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail08.syd.optusnet.com.au (mail08.syd.optusnet.com.au [211.29.132.189]) by mx1.freebsd.org (Postfix) with ESMTP id 0B0F38FC18 for ; Fri, 21 Jan 2011 06:15:18 +0000 (UTC) Received: from c122-106-165-206.carlnfd1.nsw.optusnet.com.au (c122-106-165-206.carlnfd1.nsw.optusnet.com.au [122.106.165.206]) by mail08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p0L6FBJM017016 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 21 Jan 2011 17:15:13 +1100 Date: Fri, 21 Jan 2011 17:15:11 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Matthew Jacob In-Reply-To: <4D38D49A.2010909@feral.com> Message-ID: <20110121162202.S2816@besplex.bde.org> References: <4D38CAEE.2060705@feral.com> <4D38D49A.2010909@feral.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: Re: removing -frename-registers from sys/conf.mk X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 06:15:19 -0000 On Thu, 20 Jan 2011, Matthew Jacob wrote: > Never mind this ill thought out request. > > On 1/20/2011 3:53 PM, Matthew Jacob wrote: >> It helps substantially in trying to debug amd64 kernels to remove this >> flag. >> >> Has anyone heard of a good reason we need to keep this? Whom else should I >> ask? What's ill-thought out about it? It may be bad for debugging, but can hardly be as bad as -O2 and a couple other optimizations now under -O, at least with ddb. gdb with full debugging info can sometimes follow when the code was reordered, but it has to show all the reordering when stepping. I didn't know about -frename-registers and just tried it on libm. It gives only pessimizations of up to 60% on (uncommitted) log functions in the float precision case. This turned out to be just a pessimization that I've seen before and had worked around, but the workaround stopped working with -frename-registers. gcc and clang in FreeBSD don't really understand SSE, so they like to try to generate partial register stalls, which cost 20-30 cycles on at least AthlonXP, Athlon64 and core2. Fortunately, the stall doesn't occur in most cases where it might happen. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 12:12:46 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 263D31065670 for ; Fri, 21 Jan 2011 12:12:46 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id D58DE8FC0C for ; Fri, 21 Jan 2011 12:12:45 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 385491FFC36; Fri, 21 Jan 2011 12:04:40 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 10F708457B; Fri, 21 Jan 2011 13:04:40 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: mdf@FreeBSD.org References: Date: Fri, 21 Jan 2011 13:04:39 +0100 In-Reply-To: (mdf@freebsd.org's message of "Sat, 15 Jan 2011 16:12:45 -0800") Message-ID: <86hbd2bgyw.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Arch Subject: Re: Automagic SYSCTLs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 12:12:46 -0000 mdf@FreeBSD.org writes: > The gist is that the handler knows the sizeof the variable in the > kernel and uses this to copy out. For the case of a long, there's > some goop for SCTL_MASK32. For the case of 8 and 16 bit variables, > they are still copied in and out as 32-bit quantities. The inevitable question: - does this break the KBI? (I assume it does, almost inevitably) - does this break the ABI? (I hope it does not) I'm in favor of the principle, what we have now is a mess. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 12:17:45 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71B4F1065672; Fri, 21 Jan 2011 12:17:45 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 3217C8FC0C; Fri, 21 Jan 2011 12:17:44 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id EDAD91FFC34; Fri, 21 Jan 2011 11:58:12 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id BC3CB8457B; Fri, 21 Jan 2011 12:58:12 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Nathan Whitehorn References: <4D20C8BF.701@freebsd.org> <20110102203548.000052e8@unknown> <4D20E28C.4090907@freebsd.org> Date: Fri, 21 Jan 2011 12:58:12 +0100 In-Reply-To: <4D20E28C.4090907@freebsd.org> (Nathan Whitehorn's message of "Sun, 02 Jan 2011 14:39:40 -0600") Message-ID: <86lj2ebh9n.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Bruce Cran , freebsd-sysinstall@freebsd.org, freebsd-arch@FreeBSD.org Subject: Re: BSDInstall: I want the bikeshed painted plaid X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 12:17:45 -0000 Nathan Whitehorn writes: > That's a good point. I was planning on using libfetch's restart > feature for this case. Do you think that would be unreliable? (old thread, but) libfetch does not have a restart feature, unless you mean the HTTP byte-range stuff. The restart / mirror etc. stuff is implemented in fetch(1). DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 14:04:17 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E72B4106564A; Fri, 21 Jan 2011 14:04:17 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from adsum.doit.wisc.edu (adsum.doit.wisc.edu [144.92.197.210]) by mx1.freebsd.org (Postfix) with ESMTP id B8C318FC0A; Fri, 21 Jan 2011 14:04:17 +0000 (UTC) MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Received: from avs-daemon.smtpauth1.wiscmail.wisc.edu by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LFD00C0MLR4IY00@smtpauth1.wiscmail.wisc.edu>; Fri, 21 Jan 2011 08:04:16 -0600 (CST) Received: from [10.0.2.97] (adsl-76-208-68-88.dsl.mdsnwi.sbcglobal.net [76.208.68.88]) by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LFD00KSALR0CB20@smtpauth1.wiscmail.wisc.edu>; Fri, 21 Jan 2011 08:04:14 -0600 (CST) Date: Fri, 21 Jan 2011 08:04:12 -0600 From: Nathan Whitehorn In-reply-to: <86lj2ebh9n.fsf@ds4.des.no> To: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= Message-id: X-Mailer: Apple Mail (2.936) Content-transfer-encoding: quoted-printable X-Spam-Report: AuthenticatedSender=yes, SenderIP=10.0.2.97 X-Spam-PmxInfo: Server=avs-14, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.1.21.135416, SenderIP=10.0.2.97 References: <4D20C8BF.701@freebsd.org> <20110102203548.000052e8@unknown> <4D20E28C.4090907@freebsd.org> <86lj2ebh9n.fsf@ds4.des.no> Cc: Bruce Cran , freebsd-sysinstall@freebsd.org, freebsd-arch@FreeBSD.org Subject: Re: BSDInstall: I want the bikeshed painted plaid X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 14:04:18 -0000 On Jan 21, 2011, at 5:58 AM, Dag-Erling Sm=F8rgrav wrote: > Nathan Whitehorn writes: >> That's a good point. I was planning on using libfetch's restart >> feature for this case. Do you think that would be unreliable? > > (old thread, but) > > libfetch does not have a restart feature, unless you mean the HTTP > byte-range stuff. The restart / mirror etc. stuff is implemented in > fetch(1). I did mean the byte range stuff (the URL offset argument). I took a =20 look at the fetch(1) source, and it seemed like its restart =20 implementation was just a very small amount of logic on top of this. -Nathan= From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 14:05:58 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4102C1065674; Fri, 21 Jan 2011 14:05:58 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail03.syd.optusnet.com.au (mail03.syd.optusnet.com.au [211.29.132.184]) by mx1.freebsd.org (Postfix) with ESMTP id B850C8FC0A; Fri, 21 Jan 2011 14:05:57 +0000 (UTC) Received: from c122-106-165-206.carlnfd1.nsw.optusnet.com.au (c122-106-165-206.carlnfd1.nsw.optusnet.com.au [122.106.165.206]) by mail03.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p0LE5shi021429 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 22 Jan 2011 01:05:55 +1100 Date: Sat, 22 Jan 2011 01:05:54 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Oliver Fromme In-Reply-To: <201101201059.p0KAxHgn053963@lurza.secnetix.de> Message-ID: <20110122005919.W14506@besplex.bde.org> References: <201101201059.p0KAxHgn053963@lurza.secnetix.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: mdf@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Weed-whacking sysctl(8) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 14:05:58 -0000 On Thu, 20 Jan 2011, Oliver Fromme wrote: > mdf@freebsd.org wrote: > > As bde@ mentioned, there's very little actual use of the sysctl format > > strings. I recently changed it so the use is even smaller, but I've > > got a quandary as to how to finish the job. > > > > I agree with Bruce that the formatted structs can just be removed. > > Will that break scripts that use sysctl(8) for monitoring, > logging and similar tasks (vm.loadavg for example)? > > I've installed such scripts at a few customers' sites over > the past years. It would be somewhat unfortunate if they > break when the admins at those sites decide to update the > OS. Yes, we shouldn't break the old formatting immediately. > PS: Personally I like the format very much, because it's > easy to use in shell scripts. For example, when you write > > set $(sysctl -n vm.loadavg) > > then you have the three values in $2, $3 and $4. > Another phrase I've used in scripts quite often is this: > > X=$(sysctl -n kern.boottime) > echo ${X#*\}} > > There are other ways to get that piece of information, but > they're more complicated and/or less efficient. You are a better shell programmer than most :-). I couldn't write the ${X# expansion without looking it up, and would normally use much larger awk code. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 15:40:33 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E4C51065675 for ; Fri, 21 Jan 2011 15:40:33 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id 5AAF98FC15 for ; Fri, 21 Jan 2011 15:40:33 +0000 (UTC) Received: from [192.168.1.100] (m206-63.dsl.tsoft.com [198.144.206.63]) by ns1.feral.com (8.14.4/8.14.3) with ESMTP id p0LFeVi0059371; Fri, 21 Jan 2011 07:40:32 -0800 (PST) (envelope-from mj@feral.com) Message-ID: <4D39A8FA.8060406@feral.com> Date: Fri, 21 Jan 2011 07:40:42 -0800 From: Matthew Jacob Organization: Feral Software User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Bruce Evans References: <4D38CAEE.2060705@feral.com> <4D38D49A.2010909@feral.com> <20110121162202.S2816@besplex.bde.org> In-Reply-To: <20110121162202.S2816@besplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.2.6 (ns1.feral.com [192.67.166.1]); Fri, 21 Jan 2011 07:40:32 -0800 (PST) Cc: freebsd-arch@freebsd.org Subject: Re: removing -frename-registers from sys/conf.mk X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 15:40:33 -0000 On 1/20/2011 10:15 PM, Bruce Evans wrote: > On Thu, 20 Jan 2011, Matthew Jacob wrote: > >> Never mind this ill thought out request. >> >> On 1/20/2011 3:53 PM, Matthew Jacob wrote: >>> It helps substantially in trying to debug amd64 kernels to remove >>> this flag. >>> >>> Has anyone heard of a good reason we need to keep this? Whom else >>> should I ask? > > What's ill-thought out about it? What was ill thought out about it was that somebody at Panasas had reported that removing just this option made kernel crash dumps way easier to follow. But it turns out that they had also removed all optimizations as well (and hadn't told me). That made this a non-starter. From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 17:04:30 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9957A1065672 for ; Fri, 21 Jan 2011 17:04:30 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5D8958FC0C for ; Fri, 21 Jan 2011 17:04:30 +0000 (UTC) Received: by iwn39 with SMTP id 39so1899673iwn.13 for ; Fri, 21 Jan 2011 09:04:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=MzNDMHu3CJ4PKJZulkpuMLe/NhIkGWHESZi4kjyBNRY=; b=gGriEQQUH4dpvEIix6RxNXu2tHjKHWRm2aVKrn+wxuHO1fXrRs0WFjZrj9gbPMSqrf sx279YThqOVQwu4K6EkdK/QY1guXLapPuiaDvZ2/TbgWVede3z2h6ditpkSYr81Wco7d yj47ZGF1WlL0Q94JoVCVuhQCY9pUUdg0KJSwY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=ijPp3ojW+AMHNXiEllEM4vrEkymQkUfZkRVf2gE92Xn//MMKdg05OPWrZIbCL4oNlV LkVz5K7D0Yi/MgKngKWPXV1zFBMZqhc91d2d/kh3xw6AMEUWuLYQKqzsyNdW+5xKVyih ix0X4sStNjzIPV9L7QgOXiakZeutDIvztvABg= MIME-Version: 1.0 Received: by 10.231.16.132 with SMTP id o4mr1024724iba.154.1295629468329; Fri, 21 Jan 2011 09:04:28 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.231.160.147 with HTTP; Fri, 21 Jan 2011 09:04:28 -0800 (PST) In-Reply-To: <86hbd2bgyw.fsf@ds4.des.no> References: <86hbd2bgyw.fsf@ds4.des.no> Date: Fri, 21 Jan 2011 09:04:28 -0800 X-Google-Sender-Auth: viwb_ixBUQLevAT83DS_zebfCjM Message-ID: From: mdf@FreeBSD.org To: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Arch Subject: Re: Automagic SYSCTLs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 17:04:30 -0000 2011/1/21 Dag-Erling Sm=F8rgrav : > mdf@FreeBSD.org writes: >> The gist is that the handler knows the sizeof the variable in the >> kernel and uses this to copy out. =A0For the case of a long, there's >> some goop for SCTL_MASK32. =A0For the case of 8 and 16 bit variables, >> they are still copied in and out as 32-bit quantities. > > The inevitable question: > > =A0- does this break the KBI? =A0(I assume it does, almost inevitably) It can be made gradual, such as #defining SYSCTL_ADD_INT to use the new interface (but this only works at the moment with OID_AUTO and I'd rather not add a nbr parameter to the macro). > =A0- does this break the ABI? =A0(I hope it does not) Mostly no, but the answer is slightly tricky. At the moment only LONG/ULONG has code to manage a read from 32-bit app on 64-bit kernel. There is no way to detect at compile time the difference between a long and an int64_t on a 64-bit build, since one is a typedef of the other. So the new handler has to assume any 8-byte quantity *may* be coming from a long in order to maintain the 32/64 compat code. Bruce suggested an alternate mechanism, which is to use the size the app passed in if the data will fit, and ENOMEM otherwise. This would in theory affect the ABI but in practice shouldn't do much, except that today sysctl_handle_long() will truncate the data for a 32-bit app and my intended new code would only SYSCTL_OUT to a 4-byte user-space variable when the data fits without loss of information. The answer is further complicated by the difference between sysctl(8) and sysctl(3). sysctl(8) can essentially be made well-behaved, but it's not clear what people may be doing with sysctl(3). However, it is true that, at the moment, it requires a lot of work to push out an array of longs that are 4/8 bytes on 64-bit kernel depending only on the bitness of the app, without a specialized handler in the kernel. This answer probably did not make anything clearer. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 17:44:36 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3A107106566C; Fri, 21 Jan 2011 17:44:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id E82168FC15; Fri, 21 Jan 2011 17:44:35 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 52F5D46B0C; Fri, 21 Jan 2011 12:44:35 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5C4638A009; Fri, 21 Jan 2011 12:44:32 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Date: Fri, 21 Jan 2011 12:40:24 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <86hbd2bgyw.fsf@ds4.des.no> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Message-Id: <201101211240.24571.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 21 Jan 2011 12:44:32 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: mdf@freebsd.org, Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?= Subject: Re: Automagic SYSCTLs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 17:44:36 -0000 On Friday, January 21, 2011 12:04:28 pm mdf@freebsd.org wrote: > 2011/1/21 Dag-Erling Sm=F8rgrav : > > mdf@FreeBSD.org writes: > >> The gist is that the handler knows the sizeof the variable in the > >> kernel and uses this to copy out. For the case of a long, there's > >> some goop for SCTL_MASK32. For the case of 8 and 16 bit variables, > >> they are still copied in and out as 32-bit quantities. > > > > The inevitable question: > > > > - does this break the KBI? (I assume it does, almost inevitably) >=20 > It can be made gradual, such as #defining SYSCTL_ADD_INT to use the > new interface (but this only works at the moment with OID_AUTO and I'd > rather not add a nbr parameter to the macro). >=20 > > - does this break the ABI? (I hope it does not) >=20 > Mostly no, but the answer is slightly tricky. At the moment only > LONG/ULONG has code to manage a read from 32-bit app on 64-bit kernel. > There is no way to detect at compile time the difference between a > long and an int64_t on a 64-bit build, since one is a typedef of the > other. So the new handler has to assume any 8-byte quantity *may* be > coming from a long in order to maintain the 32/64 compat code. >=20 > Bruce suggested an alternate mechanism, which is to use the size the > app passed in if the data will fit, and ENOMEM otherwise. This would > in theory affect the ABI but in practice shouldn't do much, except > that today sysctl_handle_long() will truncate the data for a 32-bit > app and my intended new code would only SYSCTL_OUT to a 4-byte > user-space variable when the data fits without loss of information. I think your approach is better since 64-bit stat counters may overflow and= =20 I'd rather get an overflowed counter in my 32-bit app in that case than no= =20 counter at all, esp. since the 32-bit app may know that the counter can wra= p=20 and handle wrapping internally. > The answer is further complicated by the difference between sysctl(8) > and sysctl(3). sysctl(8) can essentially be made well-behaved, but > it's not clear what people may be doing with sysctl(3). However, it > is true that, at the moment, it requires a lot of work to push out an > array of longs that are 4/8 bytes on 64-bit kernel depending only on > the bitness of the app, without a specialized handler in the kernel. The array of longs was my initial main worry, but sysctl_handle_long() does= n't=20 actually handle arrays, it only handles a single long. Anything that uses = an=20 array of longs (such as cp_time[]) has to use a SYSCTL_PROC() handler and t= hat=20 will just have to use the SCTL_32 flag (or whatever it is called) just like= it=20 does now to handle 32-bit compat. =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Jan 21 20:28:59 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0EAD71065674; Fri, 21 Jan 2011 20:28:59 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id C3B788FC1B; Fri, 21 Jan 2011 20:28:58 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id C9F911FFC34; Fri, 21 Jan 2011 20:28:57 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 9B19584548; Fri, 21 Jan 2011 21:28:57 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: mdf@FreeBSD.org References: <86hbd2bgyw.fsf@ds4.des.no> Date: Fri, 21 Jan 2011 21:28:56 +0100 In-Reply-To: (mdf@freebsd.org's message of "Fri, 21 Jan 2011 09:04:28 -0800") Message-ID: <867hdyvw53.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Arch Subject: Re: Automagic SYSCTLs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 20:28:59 -0000 mdf@FreeBSD.org writes: > Dag-Erling Sm=C3=B8rgrav writes: > > - does this break the KBI? (I assume it does, almost inevitably) > It can be made gradual, such as #defining SYSCTL_ADD_INT to use the > new interface (but this only works at the moment with OID_AUTO and I'd > rather not add a nbr parameter to the macro). I wrote KBI, not KPI, but if this is done before 9.0 and not backported, it shouldn't be an issue. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sat Jan 22 02:18:53 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D686106564A; Sat, 22 Jan 2011 02:18:53 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail08.syd.optusnet.com.au (mail08.syd.optusnet.com.au [211.29.132.189]) by mx1.freebsd.org (Postfix) with ESMTP id 9383A8FC08; Sat, 22 Jan 2011 02:18:52 +0000 (UTC) Received: from c122-106-165-206.carlnfd1.nsw.optusnet.com.au (c122-106-165-206.carlnfd1.nsw.optusnet.com.au [122.106.165.206]) by mail08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p0M2Ij4i028791 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 22 Jan 2011 13:18:46 +1100 Date: Sat, 22 Jan 2011 13:18:45 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: John Baldwin In-Reply-To: <201101211240.24571.jhb@freebsd.org> Message-ID: <20110122124549.N4261@besplex.bde.org> References: <86hbd2bgyw.fsf@ds4.des.no> <201101211240.24571.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: mdf@freebsd.org, Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?= , freebsd-arch@freebsd.org Subject: Re: Automagic SYSCTLs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jan 2011 02:18:53 -0000 On Fri, 21 Jan 2011, John Baldwin wrote: > On Friday, January 21, 2011 12:04:28 pm mdf@freebsd.org wrote: >> Bruce suggested an alternate mechanism, which is to use the size the >> app passed in if the data will fit, and ENOMEM otherwise. This would >> in theory affect the ABI but in practice shouldn't do much, except >> that today sysctl_handle_long() will truncate the data for a 32-bit >> app and my intended new code would only SYSCTL_OUT to a 4-byte >> user-space variable when the data fits without loss of information. > > I think your approach is better since 64-bit stat counters may overflow and > I'd rather get an overflowed counter in my 32-bit app in that case than no > counter at all, esp. since the 32-bit app may know that the counter can wrap > and handle wrapping internally. > >> The answer is further complicated by the difference between sysctl(8) >> and sysctl(3). sysctl(8) can essentially be made well-behaved, but >> it's not clear what people may be doing with sysctl(3). However, it >> is true that, at the moment, it requires a lot of work to push out an >> array of longs that are 4/8 bytes on 64-bit kernel depending only on >> the bitness of the app, without a specialized handler in the kernel. > > The array of longs was my initial main worry, but sysctl_handle_long() doesn't > actually handle arrays, it only handles a single long. Anything that uses an > array of longs (such as cp_time[]) has to use a SYSCTL_PROC() handler and that > will just have to use the SCTL_32 flag (or whatever it is called) just like it > does now to handle 32-bit compat. cp_time? This shows how messy things are. With stathz = 128, it overflows after only 194 days on 32-bit systems. Its type is bogusly long[], so overflows causes undefined behaviour. OTOH, the overflow is benign on 2's complement machines, and 32-bit applications always had to deal with it. They should pretend it is unsigned long and mostly take differences of it, so it mostly works even after it wraps on after only 388 days. But 388 days is is enough for someones, so I guess most applications ignore the possibilities of overflows and wraps. There are many other counters that stop working properly after some time, but some use honest u_int's so they only wrap and wrap at the same point for 64-bit systems and aren't so hard to export. The sysctl cp_time handlers have a messy specialized handler to convert from longs to unsigned ints (should be from longs to int32_t's, but there is no difference if longs are 2's complement of the uptime is less than 194 days). This conversion can be done in an unspecialized way, at least for small arrays where you can allocate buffers on the stack like the cp_time handlers do. Other cases might be even messier. Anyway, failing when a value in the array cannot be represented in 32 bits is not the right error handling. The kernel itself allows the values of most statistics counters to blindly overflow. When the kernel is fixed to fail on overflow, the sysctls should fail too :-). BTW, gmon(3) shuts down the whole profiling when a single counter overflows (after settting the overflowed value from 0 back to 0xFFFF, and my low-resolution kernel profiling does this too). After enough failures, someone might fix the counter sizes to work in 2011 as well as they did in 1981. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Jan 22 15:25:54 2011 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3A52F106566C for ; Sat, 22 Jan 2011 15:25:54 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id EF8578FC12 for ; Sat, 22 Jan 2011 15:25:53 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 5660646B0C; Sat, 22 Jan 2011 10:25:53 -0500 (EST) Date: Sat, 22 Jan 2011 15:25:53 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: arch@FreeBSD.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Cc: cl-capsicum-discuss@cl.cam.ac.uk Subject: Capsicum -- 9.x merge in sight X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jan 2011 15:25:54 -0000 Dear all: As many of you will now have heard, the Computer Laboratory at the University of Cambridge and Google have been collaborating for the last few years on a security research project called Capsicum. It consists of a set of extensions to the POSIX API adding a new "capability mode", "capabilities", "process descriptors", and several other additions required to implement a capability-oriented sandbox model in UNIX. These features are targeted at application compartmentalisation, in which applications are separated into mutually untrusting components in order to improve robustness. Such applications often span multiple security domains (such as web browsers), mapping a non-UNIX policy (such as the same origin policy) into local OS primitives (such as sandboxed processes). Jon Anderson, Ben Laurie, Kris Kennaway, and I implemented our research prototype on FreeBSD 9-CURRENT, with a backport to 8-STABLE, and first publicaly presented the work at the USENIX Security Symposium in 2010. Google also has an in-flight port to Linux underway, with a goal of demonstrating its use with ChromeOS and the Chromium web browser (which is able to use Capsicum to sandbox HTML rendering and Javascript execution on FreeBSD already); there's also discussion of adopting Capsicum in the NetBSD community. We've modified a number of base FreeBSD components to use Capsicum, including tcpdump, sshd, and dhclient -- sometimes reinforcing existing privilege separation, and sometimes adding it. There are also in-progress investigations of adding Capsicum sandboxing to third-party network applications such as BIND and Apache. Those attending FreeBSD developer summits in Ottawa/Cambridge will by now likely have seen a couple of different talks on Capsicum, and it was also featured in USENIX's most recent ;login magazine, as well as having been discussed on the mailing lists on and off for a while. It seems that in those venues, there's a strong consensus among attending developers that this is something that both developers and users of FreeBSD would like to see in the base system, and this e-mail is an attempt to make sure everyone knows before it turns up -- no surprises! :-) Jon and my current plan is to merge, over the next few months, various kernel features required to support Capscium sandboxing for FreeBSD 9.0: first capability mode support (this week), then capabilities themselves (which are a form of file descriptor in Capsicum), followed by process descriptors (a file descriptor alternative to process IDs that may be used by supporting applications). The current plan is *not* to merge libcapsicum, a userspace library used by certain applications to construct sandboxes, as we feel the API remains insufficiently mature at this point. However, the Capsicum system calls can still be used directly by applications, including Chromium. We would distribute libcapsicum as a package alongside 9.0, just not as a supported OS API for the time being. For those who want to learn more, you can read our USENIX Security paper, or watch the video of the USENIX Security talk, find reference material, information on our mailing list, etc, on the Capsicum web site at Cambridge: http://www.cl.cam.ac.uk/research/security/capsicum/ A number of organisations are contributing to continuing improvements in Capsicum and its applications, including Cambridge (supported by Google and DARPA), Google, and SRI (supported by DARPA). There also appear to be a number of folks inside and outside the FreeBSD community who are eager to get started -- once it's in the tree! Please feel free to join our mailing list, and get involved. Thanks, Robert N M Watson Computer Laboratory University of Cambridge