From owner-freebsd-arch@FreeBSD.ORG Sun Apr 20 09:59:13 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0ABC9106564A for ; Sun, 20 Apr 2008 09:59:13 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from palm.hoeg.nl (mx0.hoeg.nl [IPv6:2001:610:652::211]) by mx1.freebsd.org (Postfix) with ESMTP id BD9508FC17 for ; Sun, 20 Apr 2008 09:59:12 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: by palm.hoeg.nl (Postfix, from userid 1000) id BBE2D1CC6F; Sun, 20 Apr 2008 11:59:11 +0200 (CEST) Date: Sun, 20 Apr 2008 11:59:11 +0200 From: Ed Schouten To: Jeremie Le Hen Message-ID: <20080420095911.GT5934@hoeg.nl> References: <20080418132749.GB4840@obiwan.tataz.chchile.org> <200804181945.59189.max@love2party.net> <20080418204738.GE4840@obiwan.tataz.chchile.org> <20080419071400.GP73016@server.vk2pj.dyndns.org> <20080419074921.GI4840@obiwan.tataz.chchile.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2RPSo2VWgDbGU+zh" Content-Disposition: inline In-Reply-To: <20080419074921.GI4840@obiwan.tataz.chchile.org> User-Agent: Mutt/1.5.17 (2007-11-01) Cc: freebsd-arch@freebsd.org Subject: Re: Integration of ProPolice in FreeBSD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2008 09:59:13 -0000 --2RPSo2VWgDbGU+zh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable * Jeremie Le Hen wrote: > If you want to look at the patch, this is the last file. > http://tataz.chchile.org/~tataz/FreeBSD/SSP/fbsd8-ssp.diff Couldn't __stack_chk_init() be implemented like this: | static void | __stack_chk_init(void *dummy __unused) | { | arc4rand(__stack_chk_guard, sizeof(__stack_chk_guard), 0); | } --=20 Ed Schouten WWW: http://g-rave.nl/ --2RPSo2VWgDbGU+zh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (FreeBSD) iEYEARECAAYFAkgLE+8ACgkQ52SDGA2eCwXkDACdFmySFzfDRaaki55UW5I6FXWc /nEAnRchxiafCQyiuPx/XiyBKoxnk+c0 =sDjp -----END PGP SIGNATURE----- --2RPSo2VWgDbGU+zh-- From owner-freebsd-arch@FreeBSD.ORG Sun Apr 20 10:20:46 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 682E8106564A for ; Sun, 20 Apr 2008 10:20:46 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.178]) by mx1.freebsd.org (Postfix) with ESMTP id 1B27D8FC20 for ; Sun, 20 Apr 2008 10:20:45 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: by py-out-1112.google.com with SMTP id u52so1868726pyb.10 for ; Sun, 20 Apr 2008 03:20:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=UA6RX7kWa5mosyc1PMdylWbjtYhkXO20e6iikLbrRho=; b=RyXayo/F2hOfofUqgaLpgfd3I1ez+IujAyuebNqBCR7aqDEdaKUz2aB9/yvXmoioHfmgR8lX1KL9LCyAtbMrH9Uw3+paHx+6G+7c+k5ajEWf0wCCHU2CnNZk+t+9TkqO5ZMkz72BgqNkHNm1R2naQFQtYqNJFxsQhYLumUKz07U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=DXbf806jwDec32P5qjsbPXP1GrAmxAPhcUn0XUYiZ1PMy8SVwr6RiRrgDbDt43aeTO81Cp+m6DMad8KFU2kjH0lOJvAVDgQWxJ7W+XkvY+v6rhBA3+R1bnnLNXADVibv65Vh+ylwOlISF0qtYq/wMqOH8tIw3qhnMPRLAOQ/swg= Received: by 10.35.99.15 with SMTP id b15mr8387277pym.0.1208686834891; Sun, 20 Apr 2008 03:20:34 -0700 (PDT) Received: by 10.35.38.6 with HTTP; Sun, 20 Apr 2008 03:20:34 -0700 (PDT) Message-ID: Date: Sun, 20 Apr 2008 12:20:34 +0200 From: "Antoine Brodin" Sender: antoine.brodin.freebsd@gmail.com To: "Ed Schouten" In-Reply-To: <20080420095911.GT5934@hoeg.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20080418132749.GB4840@obiwan.tataz.chchile.org> <200804181945.59189.max@love2party.net> <20080418204738.GE4840@obiwan.tataz.chchile.org> <20080419071400.GP73016@server.vk2pj.dyndns.org> <20080419074921.GI4840@obiwan.tataz.chchile.org> <20080420095911.GT5934@hoeg.nl> X-Google-Sender-Auth: c3e79e68e6501718 Cc: Jeremie Le Hen , freebsd-arch@freebsd.org Subject: Re: Integration of ProPolice in FreeBSD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2008 10:20:46 -0000 On Sun, Apr 20, 2008 at 11:59 AM, Ed Schouten wrote: > * Jeremie Le Hen wrote: > > > If you want to look at the patch, this is the last file. > > http://tataz.chchile.org/~tataz/FreeBSD/SSP/fbsd8-ssp.diff > > Couldn't __stack_chk_init() be implemented like this: > > | static void > | __stack_chk_init(void *dummy __unused) > | { > | arc4rand(__stack_chk_guard, sizeof(__stack_chk_guard), 0); > | } Hi Ed, You can't do this because arc4rand will be protected and the guard won't be same when you return from arc4rand. Cheers, Antoine From owner-freebsd-arch@FreeBSD.ORG Mon Apr 21 11:06:45 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9BAD10656E6 for ; Mon, 21 Apr 2008 11:06:45 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AEF438FC14 for ; Mon, 21 Apr 2008 11:06:45 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m3LB6jaj095101 for ; Mon, 21 Apr 2008 11:06:45 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m3LB6j5X095097 for freebsd-arch@FreeBSD.org; Mon, 21 Apr 2008 11:06:45 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 21 Apr 2008 11:06:45 GMT Message-Id: <200804211106.m3LB6j5X095097@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2008 11:06:46 -0000 Current FreeBSD problem reports Critical problems Serious problems Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Mon Apr 21 18:23:27 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 46536106564A; Mon, 21 Apr 2008 18:23:27 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (cl-162.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:a1::2]) by mx1.freebsd.org (Postfix) with ESMTP id B85D38FC25; Mon, 21 Apr 2008 18:23:26 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.2/8.14.2) with ESMTP id m3LINcCZ020671; Mon, 21 Apr 2008 13:23:38 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.2/8.14.2/Submit) id m3LINcuP020670; Mon, 21 Apr 2008 13:23:38 -0500 (CDT) (envelope-from brooks) Date: Mon, 21 Apr 2008 13:23:38 -0500 From: Brooks Davis To: current@freebsd.org, arch@freebsd.org Message-ID: <20080421182338.GB20417@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uZ3hkaAS1mZxFaxD" Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Mon, 21 Apr 2008 13:23:38 -0500 (CDT) Cc: Subject: ddb scripts now load by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2008 18:23:27 -0000 --uZ3hkaAS1mZxFaxD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I've changed the default value of ddb_enable to YES so we now load ddb scripts from ddb.conf by default. Developers expecting to perform active deugging should add ddb_enable=3DNO or change the script to not perform at textdump and reboot the machine. This is another step toward hopefully shipping 8.0 with DDB built in and generating useful crash dumps. The security implications of doing so need to be carefully considered and in particular the GENERIC kernel will probably want to be shipped with at least the options DDB_UNATTENDED and SC_DISABLE_KDBKEY. The fact that the watchdog driver changes it's behavior based on DDB_UNATTENDED is probably a bug that should be fixed. -- Brooks ----- Forwarded message from Brooks Davis ----- =46rom: Brooks Davis Date: Mon, 21 Apr 2008 18:18:00 +0000 (UTC) To: brooks@FreeBSD.ORG Subject: [src] cvs commit: src/etc/defaults rc.conf brooks 2008-04-21 18:17:48 UTC FreeBSD src repository Modified files: etc/defaults rc.conf=20 Log: Change the default of ddb_enable to YES so we default to generating textd= umps on panic. This means you get a potentially useful dump even if your syst= em is running X when you panic. =20 X-MFC after: never =20 Revision Changes Path 1.332 +1 -1 src/etc/defaults/rc.conf _______________________________________________ cvs-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/cvs-all To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org" Index: src/etc/defaults/rc.conf diff -u src/etc/defaults/rc.conf:1.331 src/etc/defaults/rc.conf:1.332 --- src/etc/defaults/rc.conf:1.331 Sun Apr 20 20:37:20 2008 +++ src/etc/defaults/rc.conf Mon Apr 21 18:17:48 2008 @@ -33,7 +33,7 @@ apm_enable=3D"NO" # Set to YES to enable APM BIOS functions (or NO). apmd_enable=3D"NO" # Run apmd to handle APM event from userland. apmd_flags=3D"" # Flags to apmd (if enabled). -ddb_enable=3D"NO" # Set to YES to load ddb scripts at boot. +ddb_enable=3D"YES" # Load ddb scripts at boot. ddb_config=3D"/etc/ddb.conf" # ddb(8) config file. devd_enable=3D"YES" # Run devd, to trigger programs on device tree change= s. devd_flags=3D"" # Additional flags for devd(8). ----- End forwarded message ----- --uZ3hkaAS1mZxFaxD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (FreeBSD) iD8DBQFIDNupXY6L6fI4GtQRAtdjAJwPdHZUZqli7sELZAopBBfSKwc5kQCgh3UZ ovXYqK48cVWHGFMcjznnkh0= =uT6F -----END PGP SIGNATURE----- --uZ3hkaAS1mZxFaxD-- From owner-freebsd-arch@FreeBSD.ORG Mon Apr 21 18:55:58 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 491B5106564A for ; Mon, 21 Apr 2008 18:55:58 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id CB5038FC12 for ; Mon, 21 Apr 2008 18:55:57 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id A6ABF1A4D84; Mon, 21 Apr 2008 11:55:57 -0700 (PDT) Date: Mon, 21 Apr 2008 11:55:57 -0700 From: Alfred Perlstein To: Brooks Davis Message-ID: <20080421185557.GB95731@elvis.mu.org> References: <20080421182338.GB20417@lor.one-eyed-alien.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080421182338.GB20417@lor.one-eyed-alien.net> User-Agent: Mutt/1.4.2.3i Cc: arch@freebsd.org, current@freebsd.org Subject: Re: ddb scripts now load by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2008 18:55:58 -0000 I know this is a big step forward in a lot of ways, but it's also a big step backwards for people used to how things just work. Can we somehow put a "press a key to abort reboot" option in that script before acting on it? -Alfred (who thinks this is really cool, but wants to be able to debug kernels without having to learn and disable something new) * Brooks Davis [080421 11:24] wrote: > I've changed the default value of ddb_enable to YES so we now load ddb > scripts from ddb.conf by default. Developers expecting to perform > active deugging should add ddb_enable=NO or change the script to not > perform at textdump and reboot the machine. > > This is another step toward hopefully shipping 8.0 with DDB built > in and generating useful crash dumps. The security implications of > doing so need to be carefully considered and in particular the GENERIC > kernel will probably want to be shipped with at least the options > DDB_UNATTENDED and SC_DISABLE_KDBKEY. > > The fact that the watchdog driver changes it's behavior based on > DDB_UNATTENDED is probably a bug that should be fixed. > > -- Brooks > > ----- Forwarded message from Brooks Davis ----- > > From: Brooks Davis > Date: Mon, 21 Apr 2008 18:18:00 +0000 (UTC) > To: brooks@FreeBSD.ORG > Subject: [src] cvs commit: src/etc/defaults rc.conf > > brooks 2008-04-21 18:17:48 UTC > > FreeBSD src repository > > Modified files: > etc/defaults rc.conf > Log: > Change the default of ddb_enable to YES so we default to generating textdumps > on panic. This means you get a potentially useful dump even if your system > is running X when you panic. > > X-MFC after: never > > Revision Changes Path > 1.332 +1 -1 src/etc/defaults/rc.conf > _______________________________________________ > cvs-all@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/cvs-all > To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org" > > > Index: src/etc/defaults/rc.conf > diff -u src/etc/defaults/rc.conf:1.331 src/etc/defaults/rc.conf:1.332 > --- src/etc/defaults/rc.conf:1.331 Sun Apr 20 20:37:20 2008 > +++ src/etc/defaults/rc.conf Mon Apr 21 18:17:48 2008 > @@ -33,7 +33,7 @@ > apm_enable="NO" # Set to YES to enable APM BIOS functions (or NO). > apmd_enable="NO" # Run apmd to handle APM event from userland. > apmd_flags="" # Flags to apmd (if enabled). > -ddb_enable="NO" # Set to YES to load ddb scripts at boot. > +ddb_enable="YES" # Load ddb scripts at boot. > ddb_config="/etc/ddb.conf" # ddb(8) config file. > devd_enable="YES" # Run devd, to trigger programs on device tree changes. > devd_flags="" # Additional flags for devd(8). > > > ----- End forwarded message ----- -- - Alfred Perlstein From owner-freebsd-arch@FreeBSD.ORG Mon Apr 21 19:20:47 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5D57106566B; Mon, 21 Apr 2008 19:20:47 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (cl-162.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:a1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 832328FC12; Mon, 21 Apr 2008 19:20:47 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.2/8.14.2) with ESMTP id m3LJKxph021119; Mon, 21 Apr 2008 14:20:59 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.2/8.14.2/Submit) id m3LJKxxQ021118; Mon, 21 Apr 2008 14:20:59 -0500 (CDT) (envelope-from brooks) Date: Mon, 21 Apr 2008 14:20:59 -0500 From: Brooks Davis To: Alfred Perlstein Message-ID: <20080421192059.GA20748@lor.one-eyed-alien.net> References: <20080421182338.GB20417@lor.one-eyed-alien.net> <20080421185557.GB95731@elvis.mu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IS0zKkzwUGydFO0o" Content-Disposition: inline In-Reply-To: <20080421185557.GB95731@elvis.mu.org> User-Agent: Mutt/1.5.17 (2007-11-01) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Mon, 21 Apr 2008 14:20:59 -0500 (CDT) Cc: arch@freebsd.org, Brooks Davis , current@freebsd.org Subject: Re: ddb scripts now load by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2008 19:20:47 -0000 --IS0zKkzwUGydFO0o Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 21, 2008 at 11:55:57AM -0700, Alfred Perlstein wrote: > I know this is a big step forward in a lot of ways, but it's also > a big step backwards for people used to how things just work. >=20 > Can we somehow put a "press a key to abort reboot" option in that > script before acting on it? echo 'ddb_enable=3D"NO" >> /etc/rc.conf -- Brooks > -Alfred (who thinks this is really cool, but wants to be able to > debug kernels without having to learn and disable something > new) >=20 > * Brooks Davis [080421 11:24] wrote: > > I've changed the default value of ddb_enable to YES so we now load ddb > > scripts from ddb.conf by default. Developers expecting to perform > > active deugging should add ddb_enable=3DNO or change the script to not > > perform at textdump and reboot the machine. > >=20 > > This is another step toward hopefully shipping 8.0 with DDB built > > in and generating useful crash dumps. The security implications of > > doing so need to be carefully considered and in particular the GENERIC > > kernel will probably want to be shipped with at least the options > > DDB_UNATTENDED and SC_DISABLE_KDBKEY. > >=20 > > The fact that the watchdog driver changes it's behavior based on > > DDB_UNATTENDED is probably a bug that should be fixed. > >=20 > > -- Brooks > >=20 > > ----- Forwarded message from Brooks Davis ----- > >=20 > > From: Brooks Davis > > Date: Mon, 21 Apr 2008 18:18:00 +0000 (UTC) > > To: brooks@FreeBSD.ORG > > Subject: [src] cvs commit: src/etc/defaults rc.conf > >=20 > > brooks 2008-04-21 18:17:48 UTC > >=20 > > FreeBSD src repository > >=20 > > Modified files: > > etc/defaults rc.conf=20 > > Log: > > Change the default of ddb_enable to YES so we default to generating t= extdumps > > on panic. This means you get a potentially useful dump even if your = system > > is running X when you panic. > > =20 > > X-MFC after: never > > =20 > > Revision Changes Path > > 1.332 +1 -1 src/etc/defaults/rc.conf > > _______________________________________________ > > cvs-all@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/cvs-all > > To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org" > >=20 > >=20 > > Index: src/etc/defaults/rc.conf > > diff -u src/etc/defaults/rc.conf:1.331 src/etc/defaults/rc.conf:1.332 > > --- src/etc/defaults/rc.conf:1.331 Sun Apr 20 20:37:20 2008 > > +++ src/etc/defaults/rc.conf Mon Apr 21 18:17:48 2008 > > @@ -33,7 +33,7 @@ > > apm_enable=3D"NO" # Set to YES to enable APM BIOS functions (or NO). > > apmd_enable=3D"NO" # Run apmd to handle APM event from userland. > > apmd_flags=3D"" # Flags to apmd (if enabled). > > -ddb_enable=3D"NO" # Set to YES to load ddb scripts at boot. > > +ddb_enable=3D"YES" # Load ddb scripts at boot. > > ddb_config=3D"/etc/ddb.conf" # ddb(8) config file. > > devd_enable=3D"YES" # Run devd, to trigger programs on device tree ch= anges. > > devd_flags=3D"" # Additional flags for devd(8). > >=20 > >=20 > > ----- End forwarded message ----- >=20 >=20 >=20 > --=20 > - Alfred Perlstein > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >=20 --IS0zKkzwUGydFO0o Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (FreeBSD) iD8DBQFIDOkaXY6L6fI4GtQRAgtjAJ0S4jpn8Fq+yPBp6EgdOgxdjxydhwCfcBT2 h38dM64mdrbaM9ucAumHSF8= =Q8+F -----END PGP SIGNATURE----- --IS0zKkzwUGydFO0o-- From owner-freebsd-arch@FreeBSD.ORG Mon Apr 21 19:22:57 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6DFA01065686; Mon, 21 Apr 2008 19:22:57 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 4AE568FC1F; Mon, 21 Apr 2008 19:22:57 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 2BB451A4D86; Mon, 21 Apr 2008 12:22:57 -0700 (PDT) Date: Mon, 21 Apr 2008 12:22:57 -0700 From: Alfred Perlstein To: Brooks Davis Message-ID: <20080421192257.GF95731@elvis.mu.org> References: <20080421182338.GB20417@lor.one-eyed-alien.net> <20080421185557.GB95731@elvis.mu.org> <20080421192059.GA20748@lor.one-eyed-alien.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080421192059.GA20748@lor.one-eyed-alien.net> User-Agent: Mutt/1.4.2.3i Cc: arch@freebsd.org, current@freebsd.org Subject: Re: ddb scripts now load by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2008 19:22:57 -0000 * Brooks Davis [080421 12:20] wrote: > On Mon, Apr 21, 2008 at 11:55:57AM -0700, Alfred Perlstein wrote: > > I know this is a big step forward in a lot of ways, but it's also > > a big step backwards for people used to how things just work. > > > > Can we somehow put a "press a key to abort reboot" option in that > > script before acting on it? > > echo 'ddb_enable="NO" >> /etc/rc.conf See, that's not really what I asked for. :) but it's OK. -Alfred > > -- Brooks > > > -Alfred (who thinks this is really cool, but wants to be able to > > debug kernels without having to learn and disable something > > new) > > > > * Brooks Davis [080421 11:24] wrote: > > > I've changed the default value of ddb_enable to YES so we now load ddb > > > scripts from ddb.conf by default. Developers expecting to perform > > > active deugging should add ddb_enable=NO or change the script to not > > > perform at textdump and reboot the machine. > > > > > > This is another step toward hopefully shipping 8.0 with DDB built > > > in and generating useful crash dumps. The security implications of > > > doing so need to be carefully considered and in particular the GENERIC > > > kernel will probably want to be shipped with at least the options > > > DDB_UNATTENDED and SC_DISABLE_KDBKEY. > > > > > > The fact that the watchdog driver changes it's behavior based on > > > DDB_UNATTENDED is probably a bug that should be fixed. > > > > > > -- Brooks > > > > > > ----- Forwarded message from Brooks Davis ----- > > > > > > From: Brooks Davis > > > Date: Mon, 21 Apr 2008 18:18:00 +0000 (UTC) > > > To: brooks@FreeBSD.ORG > > > Subject: [src] cvs commit: src/etc/defaults rc.conf > > > > > > brooks 2008-04-21 18:17:48 UTC > > > > > > FreeBSD src repository > > > > > > Modified files: > > > etc/defaults rc.conf > > > Log: > > > Change the default of ddb_enable to YES so we default to generating textdumps > > > on panic. This means you get a potentially useful dump even if your system > > > is running X when you panic. > > > > > > X-MFC after: never > > > > > > Revision Changes Path > > > 1.332 +1 -1 src/etc/defaults/rc.conf > > > _______________________________________________ > > > cvs-all@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/cvs-all > > > To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org" > > > > > > > > > Index: src/etc/defaults/rc.conf > > > diff -u src/etc/defaults/rc.conf:1.331 src/etc/defaults/rc.conf:1.332 > > > --- src/etc/defaults/rc.conf:1.331 Sun Apr 20 20:37:20 2008 > > > +++ src/etc/defaults/rc.conf Mon Apr 21 18:17:48 2008 > > > @@ -33,7 +33,7 @@ > > > apm_enable="NO" # Set to YES to enable APM BIOS functions (or NO). > > > apmd_enable="NO" # Run apmd to handle APM event from userland. > > > apmd_flags="" # Flags to apmd (if enabled). > > > -ddb_enable="NO" # Set to YES to load ddb scripts at boot. > > > +ddb_enable="YES" # Load ddb scripts at boot. > > > ddb_config="/etc/ddb.conf" # ddb(8) config file. > > > devd_enable="YES" # Run devd, to trigger programs on device tree changes. > > > devd_flags="" # Additional flags for devd(8). > > > > > > > > > ----- End forwarded message ----- > > > > > > > > -- > > - Alfred Perlstein > > _______________________________________________ > > freebsd-arch@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > -- - Alfred Perlstein From owner-freebsd-arch@FreeBSD.ORG Mon Apr 21 19:48:20 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31F32106576D; Mon, 21 Apr 2008 19:48:20 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (cl-162.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:a1::2]) by mx1.freebsd.org (Postfix) with ESMTP id D12CA8FC24; Mon, 21 Apr 2008 19:48:19 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.2/8.14.2) with ESMTP id m3LJmVmU021390; Mon, 21 Apr 2008 14:48:31 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.2/8.14.2/Submit) id m3LJmVT1021389; Mon, 21 Apr 2008 14:48:31 -0500 (CDT) (envelope-from brooks) Date: Mon, 21 Apr 2008 14:48:31 -0500 From: Brooks Davis To: Alfred Perlstein Message-ID: <20080421194831.GB21205@lor.one-eyed-alien.net> References: <20080421182338.GB20417@lor.one-eyed-alien.net> <20080421185557.GB95731@elvis.mu.org> <20080421192059.GA20748@lor.one-eyed-alien.net> <20080421192257.GF95731@elvis.mu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="CUfgB8w4ZwR/yMy5" Content-Disposition: inline In-Reply-To: <20080421192257.GF95731@elvis.mu.org> User-Agent: Mutt/1.5.17 (2007-11-01) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Mon, 21 Apr 2008 14:48:32 -0500 (CDT) Cc: arch@freebsd.org, current@freebsd.org Subject: Re: ddb scripts now load by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2008 19:48:20 -0000 --CUfgB8w4ZwR/yMy5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 21, 2008 at 12:22:57PM -0700, Alfred Perlstein wrote: > * Brooks Davis [080421 12:20] wrote: > > On Mon, Apr 21, 2008 at 11:55:57AM -0700, Alfred Perlstein wrote: > > > I know this is a big step forward in a lot of ways, but it's also > > > a big step backwards for people used to how things just work. > > >=20 > > > Can we somehow put a "press a key to abort reboot" option in that > > > script before acting on it? > >=20 > > echo 'ddb_enable=3D"NO" >> /etc/rc.conf >=20 > See, that's not really what I asked for. :) >=20 > but it's OK. Actually, what we need to implement what you want is a new ddb command or possibly an optional countdown time for reset. I'm just working on the userland side at this point. :) My goal here is to hit the right default for most people and figure developers can adapt a bit, though hopefully not too much. Ideally we'd have some good example scripts in ddb.conf for developer use so new developers could switch to them with some comment changes. I definitely encourage those who do use ddb a lot to add useful entries. -- Brooks > -Alfred >=20 > >=20 > > -- Brooks > >=20 > > > -Alfred (who thinks this is really cool, but wants to be able to > > > debug kernels without having to learn and disable something > > > new) > > >=20 > > > * Brooks Davis [080421 11:24] wrote: > > > > I've changed the default value of ddb_enable to YES so we now load = ddb > > > > scripts from ddb.conf by default. Developers expecting to perform > > > > active deugging should add ddb_enable=3DNO or change the script to = not > > > > perform at textdump and reboot the machine. > > > >=20 > > > > This is another step toward hopefully shipping 8.0 with DDB built > > > > in and generating useful crash dumps. The security implications of > > > > doing so need to be carefully considered and in particular the GENE= RIC > > > > kernel will probably want to be shipped with at least the options > > > > DDB_UNATTENDED and SC_DISABLE_KDBKEY. > > > >=20 > > > > The fact that the watchdog driver changes it's behavior based on > > > > DDB_UNATTENDED is probably a bug that should be fixed. > > > >=20 > > > > -- Brooks > > > >=20 > > > > ----- Forwarded message from Brooks Davis ----- > > > >=20 > > > > From: Brooks Davis > > > > Date: Mon, 21 Apr 2008 18:18:00 +0000 (UTC) > > > > To: brooks@FreeBSD.ORG > > > > Subject: [src] cvs commit: src/etc/defaults rc.conf > > > >=20 > > > > brooks 2008-04-21 18:17:48 UTC > > > >=20 > > > > FreeBSD src repository > > > >=20 > > > > Modified files: > > > > etc/defaults rc.conf=20 > > > > Log: > > > > Change the default of ddb_enable to YES so we default to generati= ng textdumps > > > > on panic. This means you get a potentially useful dump even if y= our system > > > > is running X when you panic. > > > > =20 > > > > X-MFC after: never > > > > =20 > > > > Revision Changes Path > > > > 1.332 +1 -1 src/etc/defaults/rc.conf > > > > _______________________________________________ > > > > cvs-all@freebsd.org mailing list > > > > http://lists.freebsd.org/mailman/listinfo/cvs-all > > > > To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org" > > > >=20 > > > >=20 > > > > Index: src/etc/defaults/rc.conf > > > > diff -u src/etc/defaults/rc.conf:1.331 src/etc/defaults/rc.conf:1.3= 32 > > > > --- src/etc/defaults/rc.conf:1.331 Sun Apr 20 20:37:20 2008 > > > > +++ src/etc/defaults/rc.conf Mon Apr 21 18:17:48 2008 > > > > @@ -33,7 +33,7 @@ > > > > apm_enable=3D"NO" # Set to YES to enable APM BIOS functions (or N= O). > > > > apmd_enable=3D"NO" # Run apmd to handle APM event from userland. > > > > apmd_flags=3D"" # Flags to apmd (if enabled). > > > > -ddb_enable=3D"NO" # Set to YES to load ddb scripts at boot. > > > > +ddb_enable=3D"YES" # Load ddb scripts at boot. > > > > ddb_config=3D"/etc/ddb.conf" # ddb(8) config file. > > > > devd_enable=3D"YES" # Run devd, to trigger programs on device tre= e changes. > > > > devd_flags=3D"" # Additional flags for devd(8). > > > >=20 > > > >=20 > > > > ----- End forwarded message ----- > > >=20 > > >=20 > > >=20 > > > --=20 > > > - Alfred Perlstein > > > _______________________________________________ > > > freebsd-arch@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > > > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.or= g" > > >=20 >=20 >=20 >=20 > --=20 > - Alfred Perlstein >=20 --CUfgB8w4ZwR/yMy5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (FreeBSD) iD8DBQFIDO+PXY6L6fI4GtQRAgppAKCYHnKJXc9V3nsWt1fb34MsDFHPLgCgx/se eSa3d4MBE8Fz8M7H6vOJW9s= =O/01 -----END PGP SIGNATURE----- --CUfgB8w4ZwR/yMy5-- From owner-freebsd-arch@FreeBSD.ORG Mon Apr 21 20:11:12 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F405C1065673 for ; Mon, 21 Apr 2008 20:11:11 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id C0AA88FC38 for ; Mon, 21 Apr 2008 20:11:11 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper [152.3.145.30]) by duke.cs.duke.edu (8.14.2/8.14.2) with ESMTP id m3LKBAcK007901 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 21 Apr 2008 16:11:10 -0400 (EDT) X-DKIM: Sendmail DKIM Filter v2.5.3 duke.cs.duke.edu m3LKBAcK007901 Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id m3LKAgtw012102; Mon, 21 Apr 2008 16:10:42 -0400 (EDT) (envelope-from gallatin) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18444.62681.319881.638165@grasshopper.cs.duke.edu> Date: Mon, 21 Apr 2008 16:10:42 -0400 (EDT) To: Jeff Roberson In-Reply-To: <20080419004911.R942@desktop> References: <20080419004911.R942@desktop> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Cc: arch@freebsd.org Subject: Re: monitor/mwait support for idle X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2008 20:11:12 -0000 Jeff Roberson writes: > http://people.freebsd.org/~jeff/mwait.diff > > This patch implements support for the x86/amd64 monitor and mwait > instructions in the idle loop. This also implements idle loop selection > via a sysctl string. The following loops are supported, in > decreasing order of performance and power consumption: > > spin - Simply returns > mwait - Always use mwait to sleep. CPU enters C0 or C1 depending on > how busy it is. > mwait_hlt - Use mwait when busy but fall back to hlt/acpi when not. > hlt - pure hlt loop > acpi - uses acpi_cpu_idle if available and hlt if not. This is the > default. > Something which may be a bit confusing is that machines like recent Core2 Xeons will go into C1E when hlt is executed, depending on an MSR setting that most BIOSes enable (bit 25, MSR 0x1a0). I think C1E might be a deeper sleep than what is reached by mwait. I confess that I don't know much about C1E, other than it kills network io intensive performance, and I turn off this (mis)feature whenever I can. It will be interesting to see how much mwait helps. I'm sure it will save power over my current workaround of disabling hlt altogether :) Drew From owner-freebsd-arch@FreeBSD.ORG Mon Apr 21 22:43:13 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2077E106564A for ; Mon, 21 Apr 2008 22:43:13 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from rn-out-0910.google.com (rn-out-0910.google.com [64.233.170.189]) by mx1.freebsd.org (Postfix) with ESMTP id E02AF8FC16 for ; Mon, 21 Apr 2008 22:43:12 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by rn-out-0910.google.com with SMTP id j40so598610rnf.12 for ; Mon, 21 Apr 2008 15:43:12 -0700 (PDT) Received: by 10.142.11.2 with SMTP id 2mr1926157wfk.297.1208817791345; Mon, 21 Apr 2008 15:43:11 -0700 (PDT) Received: from ?10.0.1.199? ( [24.94.72.120]) by mx.google.com with ESMTPS id 24sm8435415wff.10.2008.04.21.15.43.05 (version=SSLv3 cipher=OTHER); Mon, 21 Apr 2008 15:43:06 -0700 (PDT) Date: Mon, 21 Apr 2008 12:43:42 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Andrew Gallatin In-Reply-To: <18444.62681.319881.638165@grasshopper.cs.duke.edu> Message-ID: <20080421124258.M942@desktop> References: <20080419004911.R942@desktop> <18444.62681.319881.638165@grasshopper.cs.duke.edu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org Subject: Re: monitor/mwait support for idle X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2008 22:43:13 -0000 On Mon, 21 Apr 2008, Andrew Gallatin wrote: > > Jeff Roberson writes: > > http://people.freebsd.org/~jeff/mwait.diff > > > > This patch implements support for the x86/amd64 monitor and mwait > > instructions in the idle loop. This also implements idle loop selection > > via a sysctl string. The following loops are supported, in > > decreasing order of performance and power consumption: > > > > spin - Simply returns > > mwait - Always use mwait to sleep. CPU enters C0 or C1 depending on > > how busy it is. > > mwait_hlt - Use mwait when busy but fall back to hlt/acpi when not. > > hlt - pure hlt loop > > acpi - uses acpi_cpu_idle if available and hlt if not. This is the > > default. > > > > Something which may be a bit confusing is that machines like recent > Core2 Xeons will go into C1E when hlt is executed, depending on an MSR > setting that most BIOSes enable (bit 25, MSR 0x1a0). I think C1E > might be a deeper sleep than what is reached by mwait. I confess that > I don't know much about C1E, other than it kills network io intensive > performance, and I turn off this (mis)feature whenever I can. > > It will be interesting to see how much mwait helps. I'm sure it will > save power over my current workaround of disabling hlt altogether :) I don't have any useful measurements for power. For performance I see as much as 10-15% in some workloads. This is due to both latency and cpu time reduction in wakeup heavy workloads that also idle a lot. Jeff > > Drew > From owner-freebsd-arch@FreeBSD.ORG Tue Apr 22 21:04:34 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B8B710656AE for ; Tue, 22 Apr 2008 21:04:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 17A7D8FC2E for ; Tue, 22 Apr 2008 21:04:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (unknown [208.65.91.234]) by elvis.mu.org (Postfix) with ESMTP id 8AAD41A4D8C for ; Tue, 22 Apr 2008 14:04:33 -0700 (PDT) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.2/8.14.2) with ESMTP id m3ML4D30018843 for ; Tue, 22 Apr 2008 17:04:22 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: arch@freebsd.org Date: Tue, 22 Apr 2008 17:04:05 -0400 User-Agent: KMail/1.9.7 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200804221704.05473.jhb@FreeBSD.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 22 Apr 2008 17:04:22 -0400 (EDT) X-Virus-Scanned: ClamAV 0.91.2/6883/Tue Apr 22 14:58:25 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Subject: Rethinking kernel module version dependencies.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2008 21:04:34 -0000 So a while back I added the automatic kernel module version dependencies to HEAD where a given kernel module has a dependency on certain versions of __FreeBSD_version so that you can't kldload a 6.x module on a 7.x kernel, etc. However, the more I have thought about it, the more I think the current module version stuff is rather backwards. Specifically, the list of "supported" versions is not in the module providing the interface, but in all the client modules and I think this is backwards. So first off, some specific observations of what we have now: * A given module (or interface) has a single version (the current version). * In theory, we can apparently have multiple versions of the same module (the module dependency code supports this, and as a result we get non-useful error messages (kernel not found vs. kernel version not supported) when a version doesn't match). In practice, though, all the symbols live in a single global namespace AFAICT, so having multiple versions of a module would probably not work very well. * A client module lists a min version, a max version, and a preferred version and the module code attempts to match it to the "optimal" kernel module. * As a consequence, the current automatic kernel dependency stuff requires clients to assume they will work ok with future versions of the kernel with no way to revoke that. What stands out to me is that the model appears to be assuming that one would implement compatability for older versions of an interface/module by having a separate module with the older version. However, in other places in our existing code we don't follow that model. Instead, the newer version of the module includes its own compatiblity shims for the older version of the interface and provides both the old and new versions of the interface from a single module (as it were). For example, 'struct cdevsw' has a d_version field that is always set to the current interface version when it is compiled, and the cdevsw handling code in kern_conf.c can choose to support older versions by checking the version. With symbol versioning my understanding is that a single library will include symbols for all supported versions and that the client just specifies a single desired version of a given symbol that it requires. I'd like to change the kernel module versions to have the client modules just specify a single version (i.e. what I'm compiled against ala D_VERSION for cdevsw) and allow the kernel module declaration to specify a min,max version range. One thing I really like about this is that the modules now have control to specify exactly which interface versions they support (w/o having to have 1 MODULE_VERSION() per version). Specifically, I'd like to change the MODULE_DEPEND() and MODULE_VERSION() macros from: module implementation: MODULE_VERSION(foo, ); module client: MODULE_DEPEND(foo, , , ); to: module_implementation: MODULE_VERSION(foo, , ); module client: MODULE_DEPEND(foo, ); So in terms of a specific example, assume that you have a kernel with a version of 800100 and you compile a kernel module using that set of headers. The kernel currently says that it only supports version 800100. The kernel module says that it would work fine with any kernel version from 800100 to 899999. If for some reason you have a warranted ABI breakage (security hole, etc.) at 800200 then the module will still kldload ok even though it won't work and we have no easy way to prevent that. However, if you let the kernel specify the range, then instead you end up with the kernel saying that it supports versions 800000 - 800100 and the kernel module just says it wants to use version 800100. Normally the 800200 kernel would support versions of 800000 - 800200. However, if you did have an ABI breakage, then you could make the 800200 kernel have a different min version for when the ABI changed (e.g. a range of 800150 - 800200) and then the module would correctly fail to kldload. I'm not advocating ABI breakage per se, but I think that the module implementation should be the one to set the policy about which versions of an interface are supported and that a client should just specify which interface version it uses and not have to have knowledge about other interface versions. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Apr 23 10:02:38 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F5FB106566C; Wed, 23 Apr 2008 10:02:38 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from out1.smtp.messagingengine.com (out1.smtp.messagingengine.com [66.111.4.25]) by mx1.freebsd.org (Postfix) with ESMTP id F17F38FC17; Wed, 23 Apr 2008 10:02:37 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from compute1.internal (compute1.internal [10.202.2.41]) by out1.messagingengine.com (Postfix) with ESMTP id 4148310238F; Wed, 23 Apr 2008 06:02:37 -0400 (EDT) Received: from web6.messagingengine.com ([10.202.2.215]) by compute1.internal (MEProxy); Wed, 23 Apr 2008 06:02:37 -0400 Received: by web6.messagingengine.com (Postfix, from userid 99) id 233117791D; Wed, 23 Apr 2008 06:02:37 -0400 (EDT) Message-Id: <1208944957.9641.1249417345@webmail.messagingengine.com> X-Sasl-Enc: npSVo2kzw/mdgxJ725U+lc8B/qYP648Vrehulv/gWScH 1208944957 From: "Darren Reed" To: "Robert Watson" Content-Disposition: inline Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="ISO-8859-1" MIME-Version: 1.0 X-Mailer: MessagingEngine.com Webmail Interface References: <20080317133029.GA19369@sub.vaned.net> <20080317134335.A3253@fledge.watson.org> <47FB586F.90606@freebsd.org> <20080408132058.U10870@fledge.watson.org> In-Reply-To: <20080408132058.U10870@fledge.watson.org> Date: Wed, 23 Apr 2008 12:02:37 +0200 Cc: arch@freebsd.org, freebsd-current@freebsd.org, "Christian S.J. Peron" Subject: Re: HEADS UP: zerocopy bpf commits impending X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: darrenr@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2008 10:02:38 -0000 On Tue, 8 Apr 2008 13:28:18 +0100 (BST), "Robert Watson" said: > > On Tue, 8 Apr 2008, Darren Reed wrote: > > > Is there a performance analysis of the copy vs zerocopy available? (I don't > > see one in the paper, just a "to do" item.) > > > > The numbers I'm interested in seeing are how many Mb/s you can capture > > before you start suffering packet loss. This needs to be done with > > sequenced packets so that you can observe gaps in the sequence captured. > > We've done some analysis, and a couple of companies have the zero-copy > BPF > code deployed. I hope to generate a more detailed analysis before the > developer summit so we can review it at BSDCan. The basic observation is > that > for quite a few types of network links, the win isn't in packet loss per > se, > but in reduced CPU use, freeing up CPU for other activities. There are a > number of sources of win: > > - Reduced system call overhead -- as load increases, # system calls goes > down, > especially if you get a two-CPU pipeline going. > > - Reduced memory access, especially for larger buffer sizes, avoids > filling > the cache twice (first in copyout, then again in using the buffer in > userspace). > > - Reduced lock contention, as only a single thread, the device driver > ithread, > is acquiring the bpf descriptor's lock, and it's no longer contending > with > the user thread. > > One interesting, and in retrospect reasonable, side effect is that user > CPU > time goes up in the SMP scenario, as cache misses on the BPF buffer move > from > the read() system call to userspace. And, as you observe, you have to > use > somewhat larger buffer sizes, as in the previous scenario there were > three > buffers: two kernel buffers and a user buffer, and now there are simply > two > kernel buffers shared directly with user space. > > The original committed version has a problem in that it allows only one > kernel > buffer to be "owned" by userspace at a time, which can lead to excess > calls to > select(); this has now been corrected, so if people have run performance > benchmarks, they should update to the new code and re-run them. > > I don't have numbers off-hand, but 5%-25% were numbers that appeared in > some > of the measurements, and I'd like to think that the recent fix will > further > improve that. Out of curiosity, were those numbers for single cpu/core systems or systems with more than one cpu/core active/available? I know the testing I did was all single threaded, so moving time from kernel to user couldn't be expected to make a large overall difference in a non-SMP kernel (NetBSD-something at the time.) Darren -- Darren Reed darrenr@fastmail.net From owner-freebsd-arch@FreeBSD.ORG Wed Apr 23 10:05:26 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9805106566B; Wed, 23 Apr 2008 10:05:26 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5D3E68FC24; Wed, 23 Apr 2008 10:05:26 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 072AB46B84; Wed, 23 Apr 2008 06:05:26 -0400 (EDT) Date: Wed, 23 Apr 2008 11:05:25 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Darren Reed In-Reply-To: <1208944957.9641.1249417345@webmail.messagingengine.com> Message-ID: <20080423110419.M35222@fledge.watson.org> References: <20080317133029.GA19369@sub.vaned.net> <20080317134335.A3253@fledge.watson.org> <47FB586F.90606@freebsd.org> <20080408132058.U10870@fledge.watson.org> <1208944957.9641.1249417345@webmail.messagingengine.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org, freebsd-current@freebsd.org, "Christian S.J. Peron" Subject: Re: HEADS UP: zerocopy bpf commits impending X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2008 10:05:26 -0000 On Wed, 23 Apr 2008, Darren Reed wrote: > Out of curiosity, were those numbers for single cpu/core systems or systems > with more than one cpu/core active/available? > > I know the testing I did was all single threaded, so moving time from kernel > to user couldn't be expected to make a large overall difference in a non-SMP > kernel (NetBSD-something at the time.) I believe all multi-core. BTW, if you are set up to do performance measurement on BPF, we'd really love to see further feedback relating to successful or unsuccessful measurement. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Wed Apr 23 13:19:46 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16BA01065674; Wed, 23 Apr 2008 13:19:46 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp5-g19.free.fr (smtp5-g19.free.fr [212.27.42.35]) by mx1.freebsd.org (Postfix) with ESMTP id 94AC18FC15; Wed, 23 Apr 2008 13:19:45 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp5-g19.free.fr (localhost.localdomain [127.0.0.1]) by smtp5-g19.free.fr (Postfix) with ESMTP id 748363F61A5; Wed, 23 Apr 2008 15:19:44 +0200 (CEST) Received: from tatooine.tataz.chchile.org (tataz.chchile.org [82.233.239.98]) by smtp5-g19.free.fr (Postfix) with ESMTP id 68CB53F6195; Wed, 23 Apr 2008 15:19:44 +0200 (CEST) Received: from obiwan.tataz.chchile.org (unknown [192.168.1.25]) by tatooine.tataz.chchile.org (Postfix) with ESMTP id 6DE129BF12; Wed, 23 Apr 2008 13:17:20 +0000 (UTC) Received: by obiwan.tataz.chchile.org (Postfix, from userid 1000) id 60A96405B; Wed, 23 Apr 2008 15:17:20 +0200 (CEST) Date: Wed, 23 Apr 2008 15:17:20 +0200 From: Jeremie Le Hen To: Antoine Brodin Message-ID: <20080423131720.GP92168@obiwan.tataz.chchile.org> References: <20080418132749.GB4840@obiwan.tataz.chchile.org> <200804181945.59189.max@love2party.net> <20080418204738.GE4840@obiwan.tataz.chchile.org> <20080419071400.GP73016@server.vk2pj.dyndns.org> <20080419074921.GI4840@obiwan.tataz.chchile.org> <20080420095911.GT5934@hoeg.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.15 (2007-04-06) Cc: freebsd-arch@freebsd.org Subject: Re: Integration of ProPolice in FreeBSD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2008 13:19:46 -0000 Hi Antoine, On Fri, Apr 18, 2008 at 04:37:06PM +0200, Antoine Brodin wrote: > Last time I looked at your patch, there was a problem when using > -fstack-protector-all instead of -fstack-protector: > when you compile lib/csu/*, gnu/lib/csu/*, or > src/lib/libc/sys/stack_protector.c with this flag, there is a kind of > chicken/egg problem and you end up with an unusable world. > That said, it would be great to be able to compile world with SSP when > an option is set in src.conf. You were right. I had a chance to test it this weekend. Thank you for pointing this out. On Sun, Apr 20, 2008 at 12:20:34PM +0200, Antoine Brodin wrote: > On Sun, Apr 20, 2008 at 11:59 AM, Ed Schouten wrote: > > Couldn't __stack_chk_init() be implemented like this: > > > > | static void > > | __stack_chk_init(void *dummy __unused) > > | { > > | arc4rand(__stack_chk_guard, sizeof(__stack_chk_guard), 0); > > | } > > You can't do this because arc4rand will be protected and the guard > won't be same when you return from arc4rand. This limitation also exists in the kernel. Currently, the kernel canary is initialized with: +/* SI_SUB_EVENTHANDLER is right after SI_SUB_LOCK, used by arc4rand() init. */ +SYSINIT(stack_chk, SI_SUB_EVENTHANDLER, SI_ORDER_ANY, __stack_chk_init, NULL); Luckily it seems that for now there is no function on the calling path to __stack_chk_init() that GCC deem useful to protect with stack-smashing protection. There is nothing that will prevent this to occur because of a careless change in the future though. So obviously, using -fstack-protector-all will break the kernel too. FWIW, it is easier to handle this in NetBSD as the canary is initialized in main(). Nonetheless I suppose it may arise if main() happens to return. I'm not sure what is the best way to handle this. Should I write special rules for those files with ${CFLAGS:S/^-fstack-protector-all$/-fstack-protector/g} or simply document that building the system with -fstack-protector-all is not supported? Thank you for your advices. Best regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org > From owner-freebsd-arch@FreeBSD.ORG Wed Apr 23 14:03:36 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A96E10656D2 for ; Wed, 23 Apr 2008 14:03:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5518E8FC1F for ; Wed, 23 Apr 2008 14:03:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from zion.baldwin.cx (unknown [208.65.91.234]) by elvis.mu.org (Postfix) with ESMTP id 240F21A4D7E; Wed, 23 Apr 2008 07:03:36 -0700 (PDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Wed, 23 Apr 2008 09:34:42 -0400 User-Agent: KMail/1.9.7 References: <20080418132749.GB4840@obiwan.tataz.chchile.org> <20080423131720.GP92168@obiwan.tataz.chchile.org> In-Reply-To: <20080423131720.GP92168@obiwan.tataz.chchile.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200804230934.42497.jhb@freebsd.org> Cc: Antoine Brodin , Jeremie Le Hen Subject: Re: Integration of ProPolice in FreeBSD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2008 14:03:36 -0000 On Wednesday 23 April 2008 09:17:20 am Jeremie Le Hen wrote: > This limitation also exists in the kernel. Currently, the kernel canary > is initialized with: > > +/* SI_SUB_EVENTHANDLER is right after SI_SUB_LOCK, used by arc4rand() > init. */ +SYSINIT(stack_chk, SI_SUB_EVENTHANDLER, SI_ORDER_ANY, > __stack_chk_init, NULL); > > Luckily it seems that for now there is no function on the calling path > to __stack_chk_init() that GCC deem useful to protect with > stack-smashing protection. There is nothing that will prevent this to > occur because of a careless change in the future though. > > So obviously, using -fstack-protector-all will break the kernel too. > > FWIW, it is easier to handle this in NetBSD as the canary is initialized > in main(). Nonetheless I suppose it may arise if main() happens to > return. mi_startup() is what runs the sysinit's and is the equivalent of main(). > I'm not sure what is the best way to handle this. Should I write special > rules for those files with > ${CFLAGS:S/^-fstack-protector-all$/-fstack-protector/g} > or simply document that building the system with -fstack-protector-all > is not supported? Does GCC provide an attribute that can be applied to a function to disable stack protection? We could explicitly disable it for the few functions (mi_startup(), initi386(), etc.) on the call path to mi_startup(). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Apr 23 14:36:44 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1F7D106566B; Wed, 23 Apr 2008 14:36:44 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp5-g19.free.fr (smtp5-g19.free.fr [212.27.42.35]) by mx1.freebsd.org (Postfix) with ESMTP id 333C48FC29; Wed, 23 Apr 2008 14:36:44 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp5-g19.free.fr (localhost.localdomain [127.0.0.1]) by smtp5-g19.free.fr (Postfix) with ESMTP id 36B843F61E5; Wed, 23 Apr 2008 16:36:43 +0200 (CEST) Received: from tatooine.tataz.chchile.org (tataz.chchile.org [82.233.239.98]) by smtp5-g19.free.fr (Postfix) with ESMTP id DC8EA3F6248; Wed, 23 Apr 2008 16:36:20 +0200 (CEST) Received: from obiwan.tataz.chchile.org (unknown [192.168.1.25]) by tatooine.tataz.chchile.org (Postfix) with ESMTP id 6DBFF9BF12; Wed, 23 Apr 2008 14:33:56 +0000 (UTC) Received: by obiwan.tataz.chchile.org (Postfix, from userid 1000) id 65A54405B; Wed, 23 Apr 2008 16:33:56 +0200 (CEST) Date: Wed, 23 Apr 2008 16:33:56 +0200 From: Jeremie Le Hen To: John Baldwin Message-ID: <20080423143356.GQ92168@obiwan.tataz.chchile.org> References: <20080418132749.GB4840@obiwan.tataz.chchile.org> <20080423131720.GP92168@obiwan.tataz.chchile.org> <200804230934.42497.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200804230934.42497.jhb@freebsd.org> User-Agent: Mutt/1.5.15 (2007-04-06) Cc: Antoine Brodin , freebsd-arch@freebsd.org Subject: Re: Integration of ProPolice in FreeBSD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2008 14:36:44 -0000 Hi John, On Wed, Apr 23, 2008 at 09:34:42AM -0400, John Baldwin wrote: > On Wednesday 23 April 2008 09:17:20 am Jeremie Le Hen wrote: > > This limitation also exists in the kernel. Currently, the kernel canary > > is initialized with: > > > > +/* SI_SUB_EVENTHANDLER is right after SI_SUB_LOCK, used by arc4rand() > > init. */ +SYSINIT(stack_chk, SI_SUB_EVENTHANDLER, SI_ORDER_ANY, > > __stack_chk_init, NULL); > > > > Luckily it seems that for now there is no function on the calling path > > to __stack_chk_init() that GCC deem useful to protect with > > stack-smashing protection. There is nothing that will prevent this to > > occur because of a careless change in the future though. > > > > So obviously, using -fstack-protector-all will break the kernel too. > > > > FWIW, it is easier to handle this in NetBSD as the canary is initialized > > in main(). Nonetheless I suppose it may arise if main() happens to > > return. > > mi_startup() is what runs the sysinit's and is the equivalent of main(). Ok thanks for the info. > > I'm not sure what is the best way to handle this. Should I write special > > rules for those files with > > ${CFLAGS:S/^-fstack-protector-all$/-fstack-protector/g} > > or simply document that building the system with -fstack-protector-all > > is not supported? > > Does GCC provide an attribute that can be applied to a function to disable > stack protection? We could explicitly disable it for the few functions > (mi_startup(), initi386(), etc.) on the call path to mi_startup(). Sorry, I should have mentionned that I've already skimmed over gcc info page and then asked on #gcc on FreeNode for such an atttribute, but there isn't: % 22:16 < Guilt> there are a lot of problems in enabling/disabling % fstack-protector in the mid of the program % 22:16 < Guilt> one is that specs for libssp are taken from the driver % program % 22:17 < Guilt> not the compiler (cc1) and it's not possible to % arbitrarily enable/disable those Ultimately those functions should be moved into separate compilation units. Maybe the current layout is sufficient, I don't know. Would you please give me some hint about the functions that must not be protected? Maybe all the MD stuff? Thank you very much. Best regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org > From owner-freebsd-arch@FreeBSD.ORG Wed Apr 23 19:54:30 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F234106566B; Wed, 23 Apr 2008 19:54:30 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 6C0AE8FC15; Wed, 23 Apr 2008 19:54:30 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (unknown [208.65.91.234]) by elvis.mu.org (Postfix) with ESMTP id 02D7E1A4D7E; Wed, 23 Apr 2008 12:54:29 -0700 (PDT) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.2/8.14.2) with ESMTP id m3NJs87S031322; Wed, 23 Apr 2008 15:54:08 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: Jeremie Le Hen Date: Wed, 23 Apr 2008 11:49:17 -0400 User-Agent: KMail/1.9.7 References: <20080418132749.GB4840@obiwan.tataz.chchile.org> <200804230934.42497.jhb@freebsd.org> <20080423143356.GQ92168@obiwan.tataz.chchile.org> In-Reply-To: <20080423143356.GQ92168@obiwan.tataz.chchile.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200804231149.17560.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Wed, 23 Apr 2008 15:54:09 -0400 (EDT) X-Virus-Scanned: ClamAV 0.91.2/6905/Wed Apr 23 11:20:18 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.2 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00, DATE_IN_PAST_03_06 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Antoine Brodin , freebsd-arch@freebsd.org Subject: Re: Integration of ProPolice in FreeBSD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2008 19:54:30 -0000 On Wednesday 23 April 2008 10:33:56 am Jeremie Le Hen wrote: > > Does GCC provide an attribute that can be applied to a function to disable > > stack protection? We could explicitly disable it for the few functions > > (mi_startup(), initi386(), etc.) on the call path to mi_startup(). > > Sorry, I should have mentionned that I've already skimmed over gcc info > page and then asked on #gcc on FreeNode for such an atttribute, but > there isn't: > > % 22:16 < Guilt> there are a lot of problems in enabling/disabling > % fstack-protector in the mid of the program > % 22:16 < Guilt> one is that specs for libssp are taken from the driver > % program > % 22:17 < Guilt> not the compiler (cc1) and it's not possible to > % arbitrarily enable/disable those > > Ultimately those functions should be moved into separate compilation > units. Maybe the current layout is sufficient, I don't know. Would you > please give me some hint about the functions that must not be protected? > Maybe all the MD stuff? Well, we never return from mi_startup() (the last SYSINIT() calls scheduler() where thread0 runs for the rest of its life). I'm not sure how the ssp stuff works, but if it happens on return from the function, then given that you are probably just fine as it is? -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Apr 23 20:25:20 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B556710656AF for ; Wed, 23 Apr 2008 20:25:20 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.180]) by mx1.freebsd.org (Postfix) with ESMTP id 4CC5D8FC27 for ; Wed, 23 Apr 2008 20:25:20 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: by py-out-1112.google.com with SMTP id u52so4470612pyb.10 for ; Wed, 23 Apr 2008 13:25:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=W0FgAttSgr7XggeerGvWmE+qLJoYama5shJkhWyeFFo=; b=Wdp1I2s0cms9ucOi/281KqUPz5w3GWwnJBkWpRes/Twr9yBnooE1qeVf5kS4DD+r7fpau6fKSkWSkAMYHeb1zN899HDV7RC3a7Nr3z/NSyVV+tkwNel2OuXRvEqUcsazf2tE56mDZ860L3tDP/Jrj8hnAqr1HOrDuS/pJYEk2DY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=HhtRi5TclYglakUaWC/Y5Oe02KcHg0oA6fAvvN5mcueO6CHxwqHeZuDmjqOg53RZYBlSRMIEA4/3baBcWSA4telxKwaOQE1EkR45ADt61FTQvE+V69ScrqoiSmTlpFY6eO8AeFLQiiZgUUps7f0c+ClZY4OEKwlY1ioB9BO9ynU= Received: by 10.35.77.3 with SMTP id e3mr3643911pyl.7.1208982319304; Wed, 23 Apr 2008 13:25:19 -0700 (PDT) Received: by 10.35.38.6 with HTTP; Wed, 23 Apr 2008 13:25:19 -0700 (PDT) Message-ID: Date: Wed, 23 Apr 2008 22:25:19 +0200 From: "Antoine Brodin" Sender: antoine.brodin.freebsd@gmail.com To: "Jeremie Le Hen" In-Reply-To: <20080423143356.GQ92168@obiwan.tataz.chchile.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20080418132749.GB4840@obiwan.tataz.chchile.org> <20080423131720.GP92168@obiwan.tataz.chchile.org> <200804230934.42497.jhb@freebsd.org> <20080423143356.GQ92168@obiwan.tataz.chchile.org> X-Google-Sender-Auth: 98bb57cd77be9247 Cc: freebsd-arch@freebsd.org Subject: Re: Integration of ProPolice in FreeBSD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2008 20:25:20 -0000 On Wed, Apr 23, 2008 at 4:33 PM, Jeremie Le Hen wrote: > Hi John, > On Wed, Apr 23, 2008 at 09:34:42AM -0400, John Baldwin wrote: > > On Wednesday 23 April 2008 09:17:20 am Jeremie Le Hen wrote: > > > This limitation also exists in the kernel. Currently, the kernel canary > > > is initialized with: > > > > > > +/* SI_SUB_EVENTHANDLER is right after SI_SUB_LOCK, used by arc4rand() > > > init. */ +SYSINIT(stack_chk, SI_SUB_EVENTHANDLER, SI_ORDER_ANY, > > > __stack_chk_init, NULL); > > > > > > Luckily it seems that for now there is no function on the calling path > > > to __stack_chk_init() that GCC deem useful to protect with > > > stack-smashing protection. There is nothing that will prevent this to > > > occur because of a careless change in the future though. > > > > > > So obviously, using -fstack-protector-all will break the kernel too. > > > > > > FWIW, it is easier to handle this in NetBSD as the canary is initialized > > > in main(). Nonetheless I suppose it may arise if main() happens to > > > return. > > > > mi_startup() is what runs the sysinit's and is the equivalent of main(). > > Ok thanks for the info. > > > > > I'm not sure what is the best way to handle this. Should I write special > > > rules for those files with > > > ${CFLAGS:S/^-fstack-protector-all$/-fstack-protector/g} > > > or simply document that building the system with -fstack-protector-all > > > is not supported? > > > > Does GCC provide an attribute that can be applied to a function to disable > > stack protection? We could explicitly disable it for the few functions > > (mi_startup(), initi386(), etc.) on the call path to mi_startup(). > > Sorry, I should have mentionned that I've already skimmed over gcc info > page and then asked on #gcc on FreeNode for such an atttribute, but > there isn't: > > % 22:16 < Guilt> there are a lot of problems in enabling/disabling > % fstack-protector in the mid of the program > % 22:16 < Guilt> one is that specs for libssp are taken from the driver > % program > % 22:17 < Guilt> not the compiler (cc1) and it's not possible to > % arbitrarily enable/disable those > > Ultimately those functions should be moved into separate compilation > units. Maybe the current layout is sufficient, I don't know. Would you > please give me some hint about the functions that must not be protected? > Maybe all the MD stuff? > > Thank you very much. Using the following patch, I can boot and run a kernel compiled with -fstack-protector-all (tested on i386, UP) Cheers, Antoine Index: sys/conf/files =================================================================== RCS file: /home/ncvs/src/sys/conf/files,v retrieving revision 1.1294 diff -u -r1.1294 files --- sys/conf/files 21 Apr 2008 10:09:53 -0000 1.1294 +++ sys/conf/files 23 Apr 2008 20:14:01 -0000 @@ -1499,6 +1499,8 @@ kern/sched_4bsd.c optional sched_4bsd kern/sched_ule.c optional sched_ule kern/serdev_if.m standard +kern/stack_protector.c standard \ + compile-with "${NORMAL_C:N-fstack-protector*}" kern/subr_acl_posix1e.c standard kern/subr_autoconf.c standard kern/subr_blist.c standard From owner-freebsd-arch@FreeBSD.ORG Thu Apr 24 01:32:03 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD5391065683 for ; Thu, 24 Apr 2008 01:32:03 +0000 (UTC) (envelope-from scf@FreeBSD.org) Received: from mail.farley.org (farley.org [67.64.95.201]) by mx1.freebsd.org (Postfix) with ESMTP id 93A7D8FC1D for ; Thu, 24 Apr 2008 01:32:03 +0000 (UTC) (envelope-from scf@FreeBSD.org) Received: from thor.farley.org (HPooka@thor.farley.org [192.168.1.5]) by mail.farley.org (8.14.2/8.14.2) with ESMTP id m3O1HQuw065687 for ; Wed, 23 Apr 2008 20:17:26 -0500 (CDT) (envelope-from scf@FreeBSD.org) Date: Wed, 23 Apr 2008 20:17:26 -0500 (CDT) From: "Sean C. Farley" To: freebsd-arch@FreeBSD.org Message-ID: User-Agent: Alpine 1.10 (BSF 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Spam-Status: No, score=-2.5 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, URIBL_AB_SURBL, URIBL_JP_SURBL, URIBL_SC_SURBL autolearn=no version=3.2.4 X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on mail.farley.org Cc: Subject: API type/include corrections X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2008 01:32:03 -0000 I am going through a list where I track items I wish to fix as I run across them. I just like to make sure they are actually broken before fixing them. :) The items in question: 1. Should the man page for mkdir(2) include sys/types.h? Open Group docs[1] do not have it, yet they do use it in the example. It is not needed to compile their example. 2. Should readpassphrase(3) include sys/types.h in the man page, or should it be added to readpassphrase.h? It is needed to compile even a simple program to pick up size_t. 3. Should chflags(2), lchflags(2) and fchflags(2) have the flags argument be of type fflags_t (uint32_t) instead of u_long? The man page says u_long while the type for st_flags in struct stat is fflags_t. Sean 1. http://www.opengroup.org/onlinepubs/000095399/functions/mkdir.html -- scf@FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Thu Apr 24 03:26:23 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78882106566B for ; Thu, 24 Apr 2008 03:26:23 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail15.syd.optusnet.com.au (mail15.syd.optusnet.com.au [211.29.132.196]) by mx1.freebsd.org (Postfix) with ESMTP id 2ACD98FC12 for ; Thu, 24 Apr 2008 03:26:22 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c220-239-252-11.carlnfd3.nsw.optusnet.com.au (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail15.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m3O3QHwl000815 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 24 Apr 2008 13:26:19 +1000 Date: Thu, 24 Apr 2008 13:26:17 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: "Sean C. Farley" In-Reply-To: Message-ID: <20080424121743.P70536@delplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@FreeBSD.org Subject: Re: API type/include corrections X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2008 03:26:23 -0000 On Wed, 23 Apr 2008, Sean C. Farley wrote: > I am going through a list where I track items I wish to fix as I run > across them. I just like to make sure they are actually broken before > fixing them. :) > > The items in question: > 1. Should the man page for mkdir(2) include sys/types.h? Open Group > docs[1] do not have it, yet they do use it in the example. It is not > needed to compile their example. No. POSIX required for most syscalls including mkdir() until 2001. 4.4 and earlier BSDs were very inconsistent about the necessary includes. They generally give a prototype and the include that declares the prototype, but are sloppy about the includes that a prerequisites for the primary include. I fixed many syscall manpages so that the example obtained by turning the include list and the declaration into a program compiles, but only searched for examples that didn't compile and tried not to add prerequisites like when they would not be needed according to future standards. OpenBSD fixed many syscall manpages so that the old POSIX prerequisite of is given explicitly, and FreeBSD imported this fix in a few manpages, in particular in mkdir.2 in 1998. This fix became a bug in 2001 or earlier when POSIX obsoleted everything having to include . I think it was a mistake even in 1998, since had enough namespace pollution (including all of and much more) to work without any explicit prequisites. However, it wasn't a bug in 1998, since was still required for portability. FreeBSD cleaned up most of the namespace pollution in and some other headers in 2001-2003, so the change to mkdir.2 in 1998 has been completely bogus for more than 5 years and the 15+ year old include list is correct again :-). > 2. Should readpassphrase(3) include sys/types.h in the man page, or > should it be added to readpassphrase.h? It is needed to compile even > a simple program to pick up size_t. Neither. readpassphrase.h should declare everything that it uses but no more. Since readpassphrase.h was born broken, the man page should have documented prerequisites that actuallyt work. > 3. Should chflags(2), lchflags(2) and fchflags(2) have the flags > argument be of type fflags_t (uint32_t) instead of u_long? The man > page says u_long while the type for st_flags in struct stat is > fflags_t. The APIs and ABIs for this are broken and depend on bugs to work, so fixing them risks disturbing the bugs. fflags_t is the result of incomplete fixes. Now the brokenness is as follows: - these syscalls never really took anything except an int arg, since syscalls.master has always said that the arg is an int and the kernel parts of the syscalls has always copied this arg to "int flags". - chflags(2) and fchflags(2) originally took a u_long arg. Now they take an fflags_t arg. Binary magic results in these incompatibilities having little effect. - lchflags(2) takes an int arg, because it was blindly imported from OtherBSD where this bug suite had apparently been fixed differently and completely by changing all the types to int. The others had already been changed to take an fflags_t arg in FreeBSD, but only at the library level. This works without so much binary magic. - struct stat now uses fflags_t st_flags. - vnode.h has always used u_long va_flags. So flags usually start as a an fflags_t which is usually a uint32_t, then are converted to an int which is usually an int32_t, then are converted to a u_long which is often larger than a uint32_t, then are converted to an fflags_t which is usually a uint32_t. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Apr 24 07:18:02 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 33C371065672 for ; Thu, 24 Apr 2008 07:18:02 +0000 (UTC) (envelope-from scf@FreeBSD.org) Received: from mail.farley.org (farley.org [67.64.95.201]) by mx1.freebsd.org (Postfix) with ESMTP id EFF868FC20 for ; Thu, 24 Apr 2008 07:18:01 +0000 (UTC) (envelope-from scf@FreeBSD.org) Received: from thor.farley.org (HPooka@thor.farley.org [192.168.1.5]) by mail.farley.org (8.14.2/8.14.2) with ESMTP id m3O7HxaS071560; Thu, 24 Apr 2008 02:17:59 -0500 (CDT) (envelope-from scf@FreeBSD.org) Date: Thu, 24 Apr 2008 02:17:59 -0500 (CDT) From: "Sean C. Farley" To: Bruce Evans In-Reply-To: <20080424121743.P70536@delplex.bde.org> Message-ID: References: <20080424121743.P70536@delplex.bde.org> User-Agent: Alpine 1.10 (BSF 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.4 X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on mail.farley.org Cc: freebsd-arch@FreeBSD.org Subject: Re: API type/include corrections X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2008 07:18:02 -0000 On Thu, 24 Apr 2008, Bruce Evans wrote: > On Wed, 23 Apr 2008, Sean C. Farley wrote: > >> I am going through a list where I track items I wish to fix as I run >> across them. I just like to make sure they are actually broken >> before fixing them. :) >> >> The items in question: >> 1. Should the man page for mkdir(2) include sys/types.h? Open Group >> docs[1] do not have it, yet they do use it in the example. It is >> not needed to compile their example. > > No. > > POSIX required for most syscalls including mkdir() until > 2001. 4.4 and earlier BSDs were very inconsistent about the necessary > includes. They generally give a prototype and the include that > declares the prototype, but are sloppy about the includes that a > prerequisites for the primary include. I fixed many syscall manpages > so that the example obtained by turning the include list and the > declaration into a program compiles, but only searched for examples > that didn't compile and tried not to add prerequisites like > when they would not be needed according to future > standards. OpenBSD fixed many syscall manpages so that the old POSIX > prerequisite of is given explicitly, and FreeBSD > imported this fix in a few manpages, in particular in mkdir.2 in 1998. > This fix became a bug in 2001 or earlier when POSIX obsoleted > everything having to include . I think it was a mistake > even in 1998, since had enough namespace pollution > (including all of and much more) to work without any > explicit prequisites. However, it wasn't a bug in 1998, since > was still required for portability. FreeBSD cleaned up > most of the namespace pollution in and some other headers > in 2001-2003, so the change to mkdir.2 in 1998 has been completely > bogus for more than 5 years and the 15+ year old include list is > correct again :-). Thank you; that explains a lot. I am beginning to grasp the convoluted history involved in these ancient API calls. :) I will fix the man page by removing that prerequisite. >> 2. Should readpassphrase(3) include sys/types.h in the man page, or >> should it be added to readpassphrase.h? It is needed to compile >> even a simple program to pick up size_t. > > Neither. readpassphrase.h should declare everything that it uses but no > more. Since readpassphrase.h was born broken, the man page should have > documented prerequisites that actuallyt work. Is the following appropriate, or should the man page be the one changed? That was basically my original question. From your comment, I see that the header should have included sys/_types.h instead of sys/types.h and defined size_t itself. Is that correct? ----------------------------------- --- readpassphrase.h 8 Mar 2002 20:52:52 -0000 1.2 +++ readpassphrase.h 24 Apr 2008 06:07:21 -0000 @@ -39,6 +39,12 @@ #define RPP_SEVENBIT 0x10 /* Strip the high bit from input. */ #include +#include + +#ifndef _SIZE_T_DECLARED +typedef __size_t size_t; +#define _SIZE_T_DECLARED +#endif __BEGIN_DECLS char * readpassphrase(const char *, char *, size_t, int); ----------------------------------- >> 3. Should chflags(2), lchflags(2) and fchflags(2) have the flags >> argument be of type fflags_t (uint32_t) instead of u_long? The >> man page says u_long while the type for st_flags in struct stat is >> fflags_t. > > The APIs and ABIs for this are broken and depend on bugs to work, so > fixing them risks disturbing the bugs. fflags_t is the result of > incomplete fixes. Now the brokenness is as follows: > - these syscalls never really took anything except an int arg, since > syscalls.master has always said that the arg is an int and the > kernel parts of the syscalls has always copied this arg to "int > flags". > - chflags(2) and fchflags(2) originally took a u_long arg. Now they > take an fflags_t arg. Binary magic results in these > incompatibilities having little effect. > - lchflags(2) takes an int arg, because it was blindly imported from > OtherBSD where this bug suite had apparently been fixed differently > and completely by changing all the types to int. The others had > already been changed to take an fflags_t arg in FreeBSD, but only at > the library level. This works without so much binary magic. > - struct stat now uses fflags_t st_flags. > - vnode.h has always used u_long va_flags. > So flags usually start as a an fflags_t which is usually a uint32_t, > then are converted to an int which is usually an int32_t, then are > converted to a u_long which is often larger than a uint32_t, then are > converted to an fflags_t which is usually a uint32_t. I feel converted. :) Since fflags_t is usually (or is it actually always) a uint32_t, then the following would need to be done: - Syscalls changed to accept fflags_t (uint32_t) - Symbol.map addition to reflect new version of ABI - sys/stat.h change to *flags() calls to accept fflags_t - sys/vnode.h change from u_long to fflags_t With a casual glance in the kernel, files needing changes or scrutiny with this change: sys/fs/cd9660/cd9660_vnops.c sys/fs/coda/coda.h sys/kern/vfs_vnops.c (The u_long to uint32_t conversion you mentioned? sb->st_flags = vap->va_flags) sys/kern/vfs_syscalls.c sys/ufs/ufs/ufs_vnops.c - I am probably forgetting with my limited experience in the kernel. - I almost forgot the change to the man page, which is where I noticed this first. /me smacks forehead :) Sean -- scf@FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Fri Apr 25 10:15:13 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B42A1065670 for ; Fri, 25 Apr 2008 10:15:13 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from proxy.meer.net (proxy.meer.net [64.13.141.13]) by mx1.freebsd.org (Postfix) with ESMTP id 132278FC1E for ; Fri, 25 Apr 2008 10:15:12 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from outbound0.mx.meer.net (outbound0.mx.meer.net [209.157.153.23]) by proxy.meer.net (8.14.2/8.14.2) with ESMTP id m3PAF1pZ039130 for ; Fri, 25 Apr 2008 03:15:12 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [209.157.152.14]) by outbound0.mx.meer.net (8.12.10/8.12.6) with ESMTP id m3PA42ij058397 for ; Fri, 25 Apr 2008 03:04:09 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id m3PA3wCP057139 for ; Fri, 25 Apr 2008 03:03:58 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from xxxu000142.ocv.ne.jp.neville-neil.com (xxxu000142.ocv.ne.jp [203.205.0.142]) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.1) with ESMTP id m3PA3vL6035503 for ; Fri, 25 Apr 2008 03:03:57 -0700 (PDT) (envelope-from gnn@neville-neil.com) Date: Fri, 25 Apr 2008 19:03:56 +0900 Message-ID: From: gnn@freebsd.org To: arch@freebsd.org User-Agent: Wanderlust/2.15.5 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.7 Emacs/22.1.50 (i386-apple-darwin8.11.1) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: multipart/mixed; boundary="Multipart_Fri_Apr_25_19:03:56_2008-1" X-Bayes-Prob: 0.5 (Score 0) X-Spam-Score: 0.70 () [Tag at 5.00] COMBINED_FROM,NO_REAL_NAME X-CanItPRO-Stream: default X-Canit-Stats-ID: 155478 - 6502905e55fe X-Scanned-By: CanIt (www . roaringpenguin . com) on 64.13.141.13 Cc: Subject: Accounting for mbufs and clusters assigned to a socket buffer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2008 10:15:13 -0000 --Multipart_Fri_Apr_25_19:03:56_2008-1 Content-Type: text/plain; charset=US-ASCII Howdy, The following patch updates the kernel (CURRENT as of 23 April or so) and netstat(1) to show not only the bytes in the receive and send queues but also the mbuf and cluster usage per socket buffer. I'd be interested in people's comments on this. I'd like to extend such counting to show more information, in particular how much of a cluster or mbuf is actually in use. Best, George --Multipart_Fri_Apr_25_19:03:56_2008-1 Content-Type: application/octet-stream; type=patch Content-Disposition: attachment; filename="mbuf_count.diff" Content-Transfer-Encoding: base64 SW5kZXg6IHN5cy9rZXJuL3VpcGNfc29ja2J1Zi5jCj09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KUkNTIGZpbGU6IC9ob21l L25jdnMvc3JjL3N5cy9rZXJuL3VpcGNfc29ja2J1Zi5jLHYKcmV0cmlldmluZyByZXZpc2lvbiAx LjE3NgpkaWZmIC11IC1yMS4xNzYgdWlwY19zb2NrYnVmLmMKLS0tIHN5cy9rZXJuL3VpcGNfc29j a2J1Zi5jCTQgRmViIDIwMDggMTI6MjU6MTMgLTAwMDAJMS4xNzYKKysrIHN5cy9rZXJuL3VpcGNf c29ja2J1Zi5jCTI0IEFwciAyMDA4IDA3OjIwOjMzIC0wMDAwCkBAIC0xMDI3LDYgKzEwMjcsOCBA QAogCXhzYi0+c2JfY2MgPSBzYi0+c2JfY2M7CiAJeHNiLT5zYl9oaXdhdCA9IHNiLT5zYl9oaXdh dDsKIAl4c2ItPnNiX21iY250ID0gc2ItPnNiX21iY250OworCXhzYi0+c2JfbWNudCA9IHNiLT5z Yl9tY250OwkKKwl4c2ItPnNiX2NjbnQgPSBzYi0+c2JfY2NudDsKIAl4c2ItPnNiX21ibWF4ID0g c2ItPnNiX21ibWF4OwogCXhzYi0+c2JfbG93YXQgPSBzYi0+c2JfbG93YXQ7CiAJeHNiLT5zYl9m bGFncyA9IHNiLT5zYl9mbGFnczsKSW5kZXg6IHN5cy9zeXMvc29ja2V0dmFyLmgKPT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PQpSQ1MgZmlsZTogL2hvbWUvbmN2cy9zcmMvc3lzL3N5cy9zb2NrZXR2YXIuaCx2CnJldHJpZXZp bmcgcmV2aXNpb24gMS4xNjIKZGlmZiAtdSAtcjEuMTYyIHNvY2tldHZhci5oCi0tLSBzeXMvc3lz L3NvY2tldHZhci5oCTQgRmViIDIwMDggMTI6MjU6MTMgLTAwMDAJMS4xNjIKKysrIHN5cy9zeXMv c29ja2V0dmFyLmgJMjQgQXByIDIwMDggMDc6MjA6NDQgLTAwMDAKQEAgLTExMCw2ICsxMTAsOCBA QAogCQl1X2ludAlzYl9jYzsJCS8qIChjL2QpIGFjdHVhbCBjaGFycyBpbiBidWZmZXIgKi8KIAkJ dV9pbnQJc2JfaGl3YXQ7CS8qIChjL2QpIG1heCBhY3R1YWwgY2hhciBjb3VudCAqLwogCQl1X2lu dAlzYl9tYmNudDsJLyogKGMvZCkgY2hhcnMgb2YgbWJ1ZnMgdXNlZCAqLworCQl1X2ludCAgIHNi X21jbnQ7ICAgICAgICAvKiBudW1iZXIgb2YgbWJ1ZnMgaW4gYnVmZmVyICovCisJCXVfaW50ICAg c2JfY2NudDsgICAgICAgIC8qIG51bWJlciBvZiBjbHVzdGVycyBpbiBidWZmZXIgKi8KIAkJdV9p bnQJc2JfbWJtYXg7CS8qIChjL2QpIG1heCBjaGFycyBvZiBtYnVmcyB0byB1c2UgKi8KIAkJdV9p bnQJc2JfY3RsOwkJLyogKGMvZCkgbm9uLWRhdGEgY2hhcnMgaW4gYnVmZmVyICovCiAJCWludAlz Yl9sb3dhdDsJLyogKGMvZCkgbG93IHdhdGVyIG1hcmsgKi8KQEAgLTI1OCw2ICsyNjAsOCBAQAog CQl1X2ludAlzYl9jYzsKIAkJdV9pbnQJc2JfaGl3YXQ7CiAJCXVfaW50CXNiX21iY250OworCQl1 X2ludCAgIHNiX21jbnQ7CisJCXVfaW50ICAgc2JfY2NudDsKIAkJdV9pbnQJc2JfbWJtYXg7CiAJ CWludAlzYl9sb3dhdDsKIAkJaW50CXNiX3RpbWVvOwpAQCAtMzE5LDggKzMyMywxMSBAQAogCWlm ICgobSktPm1fdHlwZSAhPSBNVF9EQVRBICYmIChtKS0+bV90eXBlICE9IE1UX09PQkRBVEEpIFwK IAkJKHNiKS0+c2JfY3RsICs9IChtKS0+bV9sZW47IFwKIAkoc2IpLT5zYl9tYmNudCArPSBNU0la RTsgXAotCWlmICgobSktPm1fZmxhZ3MgJiBNX0VYVCkgXAorCShzYiktPnNiX21jbnQgKz0gMTsg XAorCWlmICgobSktPm1fZmxhZ3MgJiBNX0VYVCkgeyBcCiAJCShzYiktPnNiX21iY250ICs9ICht KS0+bV9leHQuZXh0X3NpemU7IFwKKwkJKHNiKS0+c2JfY2NudCArPSAxOyBcCisJfSBcCiB9CiAK IC8qIGFkanVzdCBjb3VudGVycyBpbiBzYiByZWZsZWN0aW5nIGZyZWVpbmcgb2YgbSAqLwpAQCAt MzI5LDggKzMzNiwxMSBAQAogCWlmICgobSktPm1fdHlwZSAhPSBNVF9EQVRBICYmIChtKS0+bV90 eXBlICE9IE1UX09PQkRBVEEpIFwKIAkJKHNiKS0+c2JfY3RsIC09IChtKS0+bV9sZW47IFwKIAko c2IpLT5zYl9tYmNudCAtPSBNU0laRTsgXAotCWlmICgobSktPm1fZmxhZ3MgJiBNX0VYVCkgXAor CShzYiktPnNiX21jbnQgLT0gMTsgXAorCWlmICgobSktPm1fZmxhZ3MgJiBNX0VYVCkgeyBcCiAJ CShzYiktPnNiX21iY250IC09IChtKS0+bV9leHQuZXh0X3NpemU7IFwKKwkJKHNiKS0+c2JfY2Nu dCAtPSAxOyBcCisJfSBcCiAJaWYgKChzYiktPnNiX3NuZHB0ciA9PSAobSkpIHsgXAogCQkoc2Ip LT5zYl9zbmRwdHIgPSBOVUxMOyBcCiAJCShzYiktPnNiX3NuZHB0cm9mZiA9IDA7IFwKSW5kZXg6 IHVzci5iaW4vbmV0c3RhdC9pbmV0LmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQpSQ1MgZmlsZTogL2hvbWUvbmN2cy9z cmMvdXNyLmJpbi9uZXRzdGF0L2luZXQuYyx2CnJldHJpZXZpbmcgcmV2aXNpb24gMS44MgpkaWZm IC11IC1yMS44MiBpbmV0LmMKLS0tIHVzci5iaW4vbmV0c3RhdC9pbmV0LmMJNCBKYW4gMjAwOCAw MzowOToyOCAtMDAwMAkxLjgyCisrKyB1c3IuYmluL25ldHN0YXQvaW5ldC5jCTI0IEFwciAyMDA4 IDA3OjIwOjU3IC0wMDAwCkBAIC0xNDIsNiArMTQyLDggQEAKIAl4c2ItPnNiX2NjID0gc2ItPnNi X2NjOwogCXhzYi0+c2JfaGl3YXQgPSBzYi0+c2JfaGl3YXQ7CiAJeHNiLT5zYl9tYmNudCA9IHNi LT5zYl9tYmNudDsKKwl4c2ItPnNiX21jbnQgPSBzYi0+c2JfbWNudDsKKwl4c2ItPnNiX2NjbnQg PSBzYi0+c2JfY2NudDsKIAl4c2ItPnNiX21ibWF4ID0gc2ItPnNiX21ibWF4OwogCXhzYi0+c2Jf bG93YXQgPSBzYi0+c2JfbG93YXQ7CiAJeHNiLT5zYl9mbGFncyA9IHNiLT5zYl9mbGFnczsKQEAg LTQwNyw5ICs0MDksMTEgQEAKIAkJCQkgICAgIlByb3RvIiwgIkxpc3RlbiIsICJMb2NhbCBBZGRy ZXNzIik7CiAJCQllbHNlCiAJCQkJcHJpbnRmKChBZmxhZyAmJiAhV2ZsYWcpID8KLQkJIiUtNS41 cyAlLTYuNnMgJS02LjZzICAlLTE4LjE4cyAlLTE4LjE4cyAlc1xuIiA6Ci0JCSIlLTUuNXMgJS02 LjZzICUtNi42cyAgJS0yMi4yMnMgJS0yMi4yMnMgJXNcbiIsCisJCSIlLTUuNXMgJS02LjZzICUt Ni42cyAlLTYuNnMgJS02LjZzICUtNi42cyAlLTYuNnMgJS0xOC4xOHMgJS0xOC4xOHMgJXNcbiIg OgorCQkiJS01LjVzICUtNi42cyAlLTYuNnMgJS02LjZzICUtNi42cyAlLTYuNnMgJS02LjZzICAl LTIyLjIycyAlLTIyLjIycyAlc1xuIiwKIAkJCQkgICAgIlByb3RvIiwgIlJlY3YtUSIsICJTZW5k LVEiLAorCQkJCSAgICAiUmVjdi1NIiwgIlNlbmQtTSIsCisJCQkJICAgICJSZWN2LUMiLCAiU2Vu ZC1DIiwKIAkJCQkgICAgIkxvY2FsIEFkZHJlc3MiLCAiRm9yZWlnbiBBZGRyZXNzIiwKIAkJCQkg ICAgIihzdGF0ZSkiKTsKIAkJCWZpcnN0ID0gMDsKQEAgLTQzOCw3ICs0NDIsMTAgQEAKIAkJCSAg ICBzby0+c29faW5jcWxlbiwgc28tPnNvX3FsaW1pdCk7CiAJCQlwcmludGYoIiUtMTQuMTRzICIs IGJ1ZjEpOwogCQl9IGVsc2UgewotCQkJcHJpbnRmKCIlNnUgJTZ1ICAiLCBzby0+c29fcmN2LnNi X2NjLCBzby0+c29fc25kLnNiX2NjKTsKKwkJCXByaW50ZigiJTZ1ICU2dSAlNnUgJTZ1ICU2dSAl NnUgICIsIAorCQkJICAgICAgIHNvLT5zb19yY3Yuc2JfY2MsIHNvLT5zb19zbmQuc2JfY2MsIAor CQkJICAgICAgIHNvLT5zb19yY3Yuc2JfbWNudCwgc28tPnNvX3NuZC5zYl9tY250LAorCQkJICAg ICAgIHNvLT5zb19yY3Yuc2JfY2NudCwgc28tPnNvX3NuZC5zYl9jY250KTsKIAkJfQogCQlpZiAo bnVtZXJpY19wb3J0KSB7CiAJCQlpZiAoaW5wLT5pbnBfdmZsYWcgJiBJTlBfSVBWNCkgewo= --Multipart_Fri_Apr_25_19:03:56_2008-1-- From owner-freebsd-arch@FreeBSD.ORG Fri Apr 25 14:26:28 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 684A01065674 for ; Fri, 25 Apr 2008 14:26:28 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id B70198FC18 for ; Fri, 25 Apr 2008 14:26:27 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 45542 invoked from network); 25 Apr 2008 13:02:29 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 25 Apr 2008 13:02:29 -0000 Message-ID: <4811E384.5020604@freebsd.org> Date: Fri, 25 Apr 2008 15:58:28 +0200 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: gnn@freebsd.org References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re: Accounting for mbufs and clusters assigned to a socket buffer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2008 14:26:28 -0000 gnn@freebsd.org wrote: > Howdy, > > The following patch updates the kernel (CURRENT as of 23 April or so) > and netstat(1) to show not only the bytes in the receive and send > queues but also the mbuf and cluster usage per socket buffer. I'd be > interested in people's comments on this. I'd like to extend such > counting to show more information, in particular how much of a cluster > or mbuf is actually in use. The intent of tracking that information is good. However there are some problems with your approach: M_EXT does not mean the mbuf has a 2k cluster attached. It could by any external storage. That is a 2k (classic) cluster, a 4k (page size) cluster, a 9k cluster, a VM page (sendfile), and so on. The field sb_mbcnt already gives you the aggregated gross storage space in use for the socket. And sb_cc tells how much actual payload it contains. Just printing the already available sb_mbcnt in netstat is probably sufficient to get a good real memory usage picture. sb_mbcnt is already exported in xsb and doesn't require a KPI change. -- Andre From owner-freebsd-arch@FreeBSD.ORG Sat Apr 26 19:16:52 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F52C1065673 for ; Sat, 26 Apr 2008 19:16:52 +0000 (UTC) (envelope-from errors@mail6.reachmail.net) Received: from mail6.reachmail.net (mail6.reachmail.net [216.177.115.3]) by mx1.freebsd.org (Postfix) with ESMTP id 03A5C8FC1B for ; Sat, 26 Apr 2008 19:16:52 +0000 (UTC) (envelope-from errors@mail6.reachmail.net) From: "Veracity USA, Inc." To: "Interop 2008 Attendee" Date: Sat, 26 Apr 2008 15:03:26 -0400 Message-ID: <20080426-15032608-1498@rmmailer.colo.reachmail.com> X-BPS1: 237056 X-BPS2: 2494 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="--=1278ED0518AD471B8D03_C6A1_DEF8_5F2F" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Visit Veracity at INTEROP 2008 in Booth #2401 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "Veracity USA, Inc." List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2008 19:16:52 -0000 ----=1278ED0518AD471B8D03_C6A1_DEF8_5F2F Content-Type: text/plain;charset="iso-8859-1" Content-Transfer-Encoding: 8bit This email is being sent to freebsd-arch@freebsd.org. Use this link to be deleted or to update your email address http://go.reachmail.net/r.asp?l=64772&ee=2494!free&s=237056,237056 _________________________________________________________________ You can choose to not receive further mailings by clicking on the link above. If you have trouble with this link, simply forward this message to rem@reachmail.com with "#RM#237056,237056" in the subject line. ReachMail does not tolerate spam. Please notify us via email at abuse@reachmail.com regarding any spam issues. If you have trouble with any of these methods, you can reach us toll-free at 800-404-6885. This message was sent by Veracity USA, Inc. using ReachMail. Read our Privacy Policy: http://reachmail.net/privacy.htm ----=1278ED0518AD471B8D03_C6A1_DEF8_5F2F--