From owner-freebsd-arch@FreeBSD.ORG Wed Apr 7 02:41:39 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C2CC116A4CE for ; Wed, 7 Apr 2004 02:41:39 -0700 (PDT) Received: from mail.cs.tu-berlin.de (mail.cs.tu-berlin.de [130.149.17.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id B6AA343D54 for ; Wed, 7 Apr 2004 02:41:38 -0700 (PDT) (envelope-from harti@cs.tu-berlin.de) Received: from 130-149-145-115.dialup.cs.tu-berlin.de (130-149-145-115.dialup.cs.tu-berlin.de [130.149.145.115]) by mail.cs.tu-berlin.de (8.9.3p2/8.9.3) with ESMTP id LAA23789 for ; Wed, 7 Apr 2004 11:38:28 +0200 (MET DST) Date: Wed, 7 Apr 2004 11:38:45 +0200 (CEST) From: Harti Brandt X-X-Sender: harti@130-149-145-115.dialup.cs.tu-berlin.de To: arch@freebsd.org Message-ID: <20040407111613.W759@130-149-145-115.dialup.cs.tu-berlin.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: interface renaming of an running interface X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: harti@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Apr 2004 09:41:39 -0000 Hi all, I'm currently trying to teach bsnmp to correctly handle interface renaming. One problem that I encounter is that a process listening on the routing socket sees an interface departure and an interface arrival message. This cause interfaces that run stateful protocols like SNMP on ATM interfaces to drop all connections which isn't really all that nice. The SNMP daemon would also loose all interface state and would report the renamed interface as a new interface with a new ifindex. This directly violates the IF-MIB RFC, because the daemon is required to understand that this is the same interface (the ifindex doesn't really help here, because unloading/loading the driver gives the same behaviour). I would like to do one of the following two things: 1) disallow renaming an interface while it is up, or 2) instead of emiting a departure/arrival pair of routing messages, generate a rename message. Additionally I would like to create new sysctls: net.link.generic.ifdata..dname net.link.generic.ifdata..dunit to access the driver's name of an interface. Comments? harti From owner-freebsd-arch@FreeBSD.ORG Wed Apr 7 10:53:38 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8DB3B16A4CE; Wed, 7 Apr 2004 10:53:38 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0B56443D5A; Wed, 7 Apr 2004 10:53:38 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i37HqHPq030602; Wed, 7 Apr 2004 13:52:17 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i37HqHcF030597; Wed, 7 Apr 2004 13:52:17 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 7 Apr 2004 13:52:17 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: harti@freebsd.org In-Reply-To: <20040407111613.W759@130-149-145-115.dialup.cs.tu-berlin.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: interface renaming of an running interface X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Apr 2004 17:53:38 -0000 On Wed, 7 Apr 2004, Harti Brandt wrote: > I'm currently trying to teach bsnmp to correctly handle interface > renaming. One problem that I encounter is that a process listening on > the routing socket sees an interface departure and an interface arrival > message. This cause interfaces that run stateful protocols like SNMP on > ATM interfaces to drop all connections which isn't really all that nice. > The SNMP daemon would also loose all interface state and would report > the renamed interface as a new interface with a new ifindex. This > directly violates the IF-MIB RFC, because the daemon is required to > understand that this is the same interface (the ifindex doesn't really > help here, because unloading/loading the driver gives the same > behaviour). I would like to do one of the following two things: > > 1) disallow renaming an interface while it is up, or 2) instead of > emiting a departure/arrival pair of routing messages, generate a rename > message. > > Additionally I would like to create new sysctls: > > net.link.generic.ifdata..dname > net.link.generic.ifdata..dunit > > to access the driver's name of an interface. > > Comments? I was actually worried about this issue also -- FWIW, the issue already exists even without generic interface renaming because some interfaces (specifically, if_sl) already support renumbering on demand using ioctl(). I agree that what we need is a rename event -- the only real question is whether and how to handle compatibility for applications that don't know what to do with it. Presumably the only real question is for things like SNMP and routing daemons, although we've also talked about event monitoring for interfaces using devctl, in which case devd would need to learn about the "rename" primitive as well. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Wed Apr 7 11:02:50 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4986F16A4CE; Wed, 7 Apr 2004 11:02:50 -0700 (PDT) Received: from odin.ac.hmc.edu (Odin.AC.HMC.Edu [134.173.32.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2856443D49; Wed, 7 Apr 2004 11:02:50 -0700 (PDT) (envelope-from brdavis@odin.ac.hmc.edu) Received: from odin.ac.hmc.edu (IDENT:brdavis@localhost.localdomain [127.0.0.1]) by odin.ac.hmc.edu (8.12.10/8.12.3) with ESMTP id i37I19T7013529; Wed, 7 Apr 2004 11:01:09 -0700 Received: (from brdavis@localhost) by odin.ac.hmc.edu (8.12.10/8.12.3/Submit) id i37I18t7013528; Wed, 7 Apr 2004 11:01:08 -0700 Date: Wed, 7 Apr 2004 11:01:08 -0700 From: Brooks Davis To: harti@freebsd.org Message-ID: <20040407180107.GB20636@Odin.AC.HMC.Edu> References: <20040407111613.W759@130-149-145-115.dialup.cs.tu-berlin.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4bRzO86E/ozDv8r1" Content-Disposition: inline In-Reply-To: <20040407111613.W759@130-149-145-115.dialup.cs.tu-berlin.de> User-Agent: Mutt/1.5.4i X-Virus-Scanned: by amavisd-milter (http://amavis.org/) on odin.ac.hmc.edu cc: arch@freebsd.org Subject: Re: interface renaming of an running interface X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Apr 2004 18:02:50 -0000 --4bRzO86E/ozDv8r1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Apr 07, 2004 at 11:38:45AM +0200, Harti Brandt wrote: >=20 > I'm currently trying to teach bsnmp to correctly handle interface > renaming. One problem that I encounter is that a process listening on > the routing socket sees an interface departure and an interface arrival > message. This cause interfaces that run stateful protocols like SNMP on > ATM interfaces to drop all connections which isn't really all that nice. > The SNMP daemon would also loose all interface state and would report > the renamed interface as a new interface with a new ifindex. This directly > violates the IF-MIB RFC, because the daemon is required to understand that > this is the same interface (the ifindex doesn't really help here, because > unloading/loading the driver gives the same behaviour). I would like to do > one of the following two things: >=20 > 1) disallow renaming an interface while it is up, or > 2) instead of emiting a departure/arrival pair of routing messages, > generate a rename message. For the record, I wouldn't object to 1, but I prefer 2. My view is that the name is an ephemeral property of an interface intended to the convenience of the administrator, nothing more. I random though just hit me as I thought about this message. For a variety of good reasons, if_index's are densly packed so if you destroy an interface and create a new one, you will almost certaintly get the index of a recently used one. This leads to the problem that you can't tell the rename and create-destory operations apart with the current set of routing messages. One solution to this problem might be to add a generation number to the interface either using a global counter or just a timestamp. > Additionally I would like to create new sysctls: >=20 > net.link.generic.ifdata..dname > net.link.generic.ifdata..dunit >=20 > to access the driver's name of an interface. Fine with me. -- Brooks --=20 Any statement of the form "X is the one, true Y" is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 --4bRzO86E/ozDv8r1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQFAdEHjXY6L6fI4GtQRAqYJAJ9gIX888u95efnmfIUJn4PkX9+RlgCfU745 7mJZYD2viqchU1F7gBPo8dg= =CIDo -----END PGP SIGNATURE----- --4bRzO86E/ozDv8r1-- From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 01:30:52 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AFA5E16A4CF; Thu, 8 Apr 2004 01:30:52 -0700 (PDT) Received: from mx.nsu.ru (mx.nsu.ru [212.192.164.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2BD9043D39; Thu, 8 Apr 2004 01:30:52 -0700 (PDT) (envelope-from danfe@regency.nsu.ru) Received: from regency.nsu.ru ([193.124.210.26]) by mx.nsu.ru with esmtp (Exim 4.30) id 1BBUxm-0005br-8p; Thu, 08 Apr 2004 15:32:42 +0700 Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.12.10/8.12.10) with ESMTP id i388UlGt057222; Thu, 8 Apr 2004 15:30:47 +0700 (NOVST) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.12.10/8.12.10/Submit) id i388UkIJ057156; Thu, 8 Apr 2004 15:30:46 +0700 (NOVST) (envelope-from danfe) Date: Thu, 8 Apr 2004 15:30:46 +0700 From: Alexey Dokuchaev To: Brooks Davis Message-ID: <20040408083046.GA37831@regency.nsu.ru> References: <20040407111613.W759@130-149-145-115.dialup.cs.tu-berlin.de> <20040407180107.GB20636@Odin.AC.HMC.Edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040407180107.GB20636@Odin.AC.HMC.Edu> User-Agent: Mutt/1.4.1i cc: arch@freebsd.org cc: harti@freebsd.org Subject: Re: interface renaming of an running interface X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 08:30:52 -0000 On Wed, Apr 07, 2004 at 11:01:08AM -0700, Brooks Davis wrote: > On Wed, Apr 07, 2004 at 11:38:45AM +0200, Harti Brandt wrote: > > Additionally I would like to create new sysctls: > > > > net.link.generic.ifdata..dname > > net.link.generic.ifdata..dunit > > > > to access the driver's name of an interface. Why not simply net.link.generic.if.... ? ./danfe From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 02:25:03 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3527516A4D8 for ; Thu, 8 Apr 2004 02:25:03 -0700 (PDT) Received: from mxsf29.cluster1.charter.net (mxsf29.cluster1.charter.net [209.225.28.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id A637643D62 for ; Thu, 8 Apr 2004 02:25:02 -0700 (PDT) (envelope-from bgriffis@charter.net) Received: from [192.168.1.3] (24-205-164-232.wc-eres.charterpipeline.net [24.205.164.232])i389OS4h021862 for ; Thu, 8 Apr 2004 05:24:28 -0400 (EDT) User-Agent: Microsoft-Entourage/10.1.4.030702.0 Date: Thu, 08 Apr 2004 02:24:18 -0700 From: Beau Griffis To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: Question X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 09:25:03 -0000 I am running os x panther 10.1.3 it says its running freebsd 5 but there is no /stand alone dir nor is there a ports directory is there any way I could add this with doing some sort of network install.. I WAS SO HAPPY THEY MOVED FROM DARWIN TO FREEBSD MY TWO FAVIORTE OS COMBINED IN ONE ITS LIKE HEAVEN CAME TRUE HEH IF JKH LOGO PACHELL OR JET ARE AROUND TELL EM SPAZ SAYS HELLO I JUST GOT OUT OF PRISON SOLD MY OLD COMPNIE AND AM STARTING A NEW ONE TELL EM TO EMAIL ME AT SPAZ@SPAZ.TV ITHEY ARE SMART PEOPLE I COULD ALLWAYS USE SOME HELP SPAZ AKA BEAU FROM # FROM #DOEK ON UNDERNET WAY BAK IN THE DAYS AOL AND BBS AND ALL THAT STUFF HEH. Thank you BeEAU GRIFFIS SPAZ COMMUNICATIONS A DOUBLEDISCOUNT.COM COMAPNIE From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 04:55:06 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC65016A4CE for ; Thu, 8 Apr 2004 04:55:06 -0700 (PDT) Received: from mail.cs.tu-berlin.de (mail.cs.tu-berlin.de [130.149.17.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8BF6943D1D for ; Thu, 8 Apr 2004 04:55:05 -0700 (PDT) (envelope-from novo@cs.tu-berlin.de) Received: from 130-149-145-31.dialup.cs.tu-berlin.de (130-149-145-31.dialup.cs.tu-berlin.de [130.149.145.31]) by mail.cs.tu-berlin.de (8.9.3p2/8.9.3) with ESMTP id NAA09704; Thu, 8 Apr 2004 13:50:03 +0200 (MET DST) Date: Thu, 8 Apr 2004 13:50:23 +0200 (CEST) From: Harti Brandt X-X-Sender: novo@130-149-145-31.dialup.cs.tu-berlin.de To: Alexey Dokuchaev In-Reply-To: <20040408083046.GA37831@regency.nsu.ru> Message-ID: <20040408134912.B724@130-149-145-31.dialup.cs.tu-berlin.de> References: <20040407111613.W759@130-149-145-115.dialup.cs.tu-berlin.de> <20040408083046.GA37831@regency.nsu.ru> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.ORG Subject: Re: interface renaming of an running interface X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: harti@FreeBSD.ORG List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 11:55:07 -0000 On Thu, 8 Apr 2004, Alexey Dokuchaev wrote: > On Wed, Apr 07, 2004 at 11:01:08AM -0700, Brooks Davis wrote: > > On Wed, Apr 07, 2004 at 11:38:45AM +0200, Harti Brandt wrote: > > > Additionally I would like to create new sysctls: > > > > > > net.link.generic.ifdata..dname > > > net.link.generic.ifdata..dunit > > > > > > to access the driver's name of an interface. > > Why not simply net.link.generic.if.... ? Because it fits into the already existing net.link.generic.ifdata..general net.link.generic.ifdata..linkspecific harti From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 05:18:31 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 72CAE16A4CE; Thu, 8 Apr 2004 05:18:31 -0700 (PDT) Received: from mail.cs.tu-berlin.de (mail.cs.tu-berlin.de [130.149.17.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E53543D60; Thu, 8 Apr 2004 05:18:30 -0700 (PDT) (envelope-from novo@cs.tu-berlin.de) Received: from 130-149-145-102.dialup.cs.tu-berlin.de (130-149-145-102.dialup.cs.tu-berlin.de [130.149.145.102]) by mail.cs.tu-berlin.de (8.9.3p2/8.9.3) with ESMTP id OAA13304; Thu, 8 Apr 2004 14:13:19 +0200 (MET DST) Date: Thu, 8 Apr 2004 14:13:38 +0200 (CEST) From: Harti Brandt X-X-Sender: novo@130-149-145-102.dialup.cs.tu-berlin.de To: Robert Watson In-Reply-To: Message-ID: <20040408140343.O785@130-149-145-99.dialup.cs.tu-berlin.de> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.ORG Subject: Re: interface renaming of an running interface X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: harti@FreeBSD.ORG List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 12:18:31 -0000 On Wed, 7 Apr 2004, Robert Watson wrote: > On Wed, 7 Apr 2004, Harti Brandt wrote: > > > I'm currently trying to teach bsnmp to correctly handle interface > > renaming. One problem that I encounter is that a process listening on > > the routing socket sees an interface departure and an interface arrival > > message. This cause interfaces that run stateful protocols like SNMP on > > ATM interfaces to drop all connections which isn't really all that nice. > > The SNMP daemon would also loose all interface state and would report > > the renamed interface as a new interface with a new ifindex. This > > directly violates the IF-MIB RFC, because the daemon is required to > > understand that this is the same interface (the ifindex doesn't really > > help here, because unloading/loading the driver gives the same > > behaviour). I would like to do one of the following two things: > > > > 1) disallow renaming an interface while it is up, or 2) instead of > > emiting a departure/arrival pair of routing messages, generate a rename > > message. > > > > Additionally I would like to create new sysctls: > > > > net.link.generic.ifdata..dname > > net.link.generic.ifdata..dunit > > > > to access the driver's name of an interface. > > > > Comments? > > I was actually worried about this issue also -- FWIW, the issue already > exists even without generic interface renaming because some interfaces > (specifically, if_sl) already support renumbering on demand using ioctl(). > I agree that what we need is a rename event -- the only real question is > whether and how to handle compatibility for applications that don't know > what to do with it. Presumably the only real question is for things like > SNMP and routing daemons, although we've also talked about event > monitoring for interfaces using devctl, in which case devd would need to > learn about the "rename" primitive as well. I don't know how to deal with devctl, but for the routing socket I'd introduce a new IFAN_RENAME for the ifan_what field of the RTM_IFANNOUNCE routing message. The interface name in the message would be the new one, so by tracking these message the daemon will understand what happens. I'll prepare a patch and try to get some testing on net@. harti From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 10:49:33 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CE9FA16A4CE for ; Thu, 8 Apr 2004 10:49:33 -0700 (PDT) Received: from mail3.speakeasy.net (mail3.speakeasy.net [216.254.0.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 97EB643D53 for ; Thu, 8 Apr 2004 10:49:33 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (qmail 23303 invoked from network); 8 Apr 2004 17:49:33 -0000 Received: from dsl017-045-168.spk4.dsl.speakeasy.net (HELO hydrogen.funkthat.com) ([69.17.45.168]) (envelope-sender ) by mail3.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 8 Apr 2004 17:49:33 -0000 Received: from hydrogen.funkthat.com (urahsr@localhost.funkthat.com [127.0.0.1])i38HnWOE078730; Thu, 8 Apr 2004 10:49:32 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.10/8.12.10/Submit) id i38HnVKj078729; Thu, 8 Apr 2004 10:49:31 -0700 (PDT) Date: Thu, 8 Apr 2004 10:49:31 -0700 From: John-Mark Gurney To: Beau Griffis Message-ID: <20040408174931.GH567@funkthat.com> Mail-Followup-To: Beau Griffis , freebsd-arch@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html cc: freebsd-arch@freebsd.org Subject: Re: Question X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 17:49:33 -0000 Beau Griffis wrote this message on Thu, Apr 08, 2004 at 02:24 -0700: > I am running os x panther 10.1.3 it says its running freebsd 5 but there is I assume you mean 10.3.x, not 10.1... Also, they did not say that they are running FreeBSD 5, but that they have updated parts to be more like FreeBSD 5... OSX is lacking a really big feature of FreeBSD 5, and that is sendfile... Since this is a FreeBSD list, and we have no affiliation with Apple, please redirect your comments to Apple... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 14:17:33 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AFB6016A4CE for ; Thu, 8 Apr 2004 14:17:33 -0700 (PDT) Received: from nhddns1.nippo-c.jp (nhddns1.nippo-c.jp [211.125.50.170]) by mx1.FreeBSD.org (Postfix) with ESMTP id 08AB543D48 for ; Thu, 8 Apr 2004 14:17:33 -0700 (PDT) (envelope-from root@nhdsmtp1.nippo-c.jp) Received: from nhdsmtp1.nippo-c.jp (nhdsmtp1 [192.168.168.11]) by nhddns1.nippo-c.jp (8.12.10/8.12.10) with ESMTP id i38LHVM0028086 for ; Fri, 9 Apr 2004 06:17:31 +0900 (JST) Received: from localhost (root@localhost) by nhdsmtp1.nippo-c.jp (8.12.10/8.12.10) with SMTP id i38LHLlg027272 for ; Fri, 9 Apr 2004 06:17:31 +0900 (JST) Date: Fri, 9 Apr 2004 06:17:21 +0900 (JST) Message-Id: <200404082117.i38LHLlg027272@nhdsmtp1.nippo-c.jp> From: root@nhdsmtp1.nippo-c.jp To: freebsd-arch@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Virus Alert X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 21:17:33 -0000 The mail message (file: report4.pif) you sent to sasaki_shouhei@nippo-c.jp contains a virus. (on nhdsmtp1) From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 14:36:48 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DDA3316A4CE for ; Thu, 8 Apr 2004 14:36:48 -0700 (PDT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8240243D1D for ; Thu, 8 Apr 2004 14:36:48 -0700 (PDT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id 0BA64ACAEB; Thu, 8 Apr 2004 23:36:47 +0200 (CEST) Date: Thu, 8 Apr 2004 23:36:47 +0200 From: Pawel Jakub Dawidek To: freebsd-arch@freebsd.org Message-ID: <20040408213647.GL661@darkness.comp.waw.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0+35XlDF45POFHfm" Content-Disposition: inline User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 Subject: mtx_lock_recurse/mtx_unlock_recurse functions (proof-of-concept). X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 21:36:49 -0000 --0+35XlDF45POFHfm Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. As was discussed, it will be helpful to have functions, that are able to acquire lock recursively, even if lock itself isn't recursable. Here is a patch, that should implement this functionality: http://people.freebsd.org/~pjd/patches/mtx_lock_recurse.patch I also added a KASSERT() to protect against mixed locking modes, e.g.: mtx_lock(&mtx); [...] mtx_lock_recurse(&mtx); [...] mtx_unlock(&mtx); [...] mtx_unlock_recurse(&mtx); This is not permitted, but this should work just fine: mtx_lock(&mtx); [...] mtx_lock_recurse(&mtx); [...] mtx_unlock_recurse(&mtx); [...] mtx_unlock(&mtx); --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --0+35XlDF45POFHfm Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFAdcXvForvXbEpPzQRAm4AAJ93RZtpHN4byfpwJUr/z3ElvyyW3wCfXkK5 weeNN/NPnvoRJnmk54Xf4PU= =WbaK -----END PGP SIGNATURE----- --0+35XlDF45POFHfm-- From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 14:40:08 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D62E16A4CF; Thu, 8 Apr 2004 14:40:08 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB7D643D66; Thu, 8 Apr 2004 14:40:07 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.11/8.12.11) with ESMTP id i38Le7NG029213; Thu, 8 Apr 2004 23:40:07 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Pawel Jakub Dawidek From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 08 Apr 2004 23:36:47 +0200." <20040408213647.GL661@darkness.comp.waw.pl> Date: Thu, 08 Apr 2004 23:40:07 +0200 Message-ID: <29212.1081460407@critter.freebsd.dk> cc: freebsd-arch@freebsd.org Subject: Re: mtx_lock_recurse/mtx_unlock_recurse functions (proof-of-concept). X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 21:40:08 -0000 In message <20040408213647.GL661@darkness.comp.waw.pl>, Pawel Jakub Dawidek wri tes: > >--0+35XlDF45POFHfm >Content-Type: text/plain; charset=iso-8859-2 >Content-Disposition: inline >Content-Transfer-Encoding: quoted-printable > >Hi. > >As was discussed, it will be helpful to have functions, that are able >to acquire lock recursively, even if lock itself isn't recursable. Sounds evil to me... Does your patch also make witness aware of this ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 14:59:52 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E0DA16A4CE for ; Thu, 8 Apr 2004 14:59:52 -0700 (PDT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id A6EB443D49 for ; Thu, 8 Apr 2004 14:59:51 -0700 (PDT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id C87F0ACAEB; Thu, 8 Apr 2004 23:59:48 +0200 (CEST) Date: Thu, 8 Apr 2004 23:59:48 +0200 From: Pawel Jakub Dawidek To: Poul-Henning Kamp Message-ID: <20040408215948.GM661@darkness.comp.waw.pl> References: <20040408213647.GL661@darkness.comp.waw.pl> <29212.1081460407@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="s8J8qjRpxLkp1Gun" Content-Disposition: inline In-Reply-To: <29212.1081460407@critter.freebsd.dk> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-arch@freebsd.org Subject: Re: mtx_lock_recurse/mtx_unlock_recurse functions (proof-of-concept). X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 21:59:52 -0000 --s8J8qjRpxLkp1Gun Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 08, 2004 at 11:40:07PM +0200, Poul-Henning Kamp wrote: +> >As was discussed, it will be helpful to have functions, that are able +> >to acquire lock recursively, even if lock itself isn't recursable. +>=20 +> Sounds evil to me... Why? I don't think that better is to just use MTX_RECURSE if we don't need it in 90% of cases. And sometimes it is really hard to pass information if lock is held or not, just like for that rwatson's kqueue thing. +> Does your patch also make witness aware of this ? Nope, it is just a proof-of-concept and as jhb@ pointed out, we want to keep our interface simple, so it probably will never reach the tree. --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --s8J8qjRpxLkp1Gun Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFAdctUForvXbEpPzQRArClAKCklcqQo6dXRn9oj0oes8DYHaKsrwCeL3fM COlYy2Qa3ZsI7+jWGksYYFc= =Ada8 -----END PGP SIGNATURE----- --s8J8qjRpxLkp1Gun-- From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 15:33:23 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D6E5116A4CE; Thu, 8 Apr 2004 15:33:23 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5242343D5A; Thu, 8 Apr 2004 15:33:23 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i38MX0Pq054206; Thu, 8 Apr 2004 18:33:00 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i38MX0MJ054203; Thu, 8 Apr 2004 18:33:00 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Thu, 8 Apr 2004 18:33:00 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Pawel Jakub Dawidek In-Reply-To: <20040408213647.GL661@darkness.comp.waw.pl> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-arch@FreeBSD.org Subject: Re: mtx_lock_recurse/mtx_unlock_recurse functions (proof-of-concept). X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 22:33:24 -0000 On Thu, 8 Apr 2004, Pawel Jakub Dawidek wrote: > As was discussed, it will be helpful to have functions, that are able to > acquire lock recursively, even if lock itself isn't recursable. > > Here is a patch, that should implement this functionality: > > http://people.freebsd.org/~pjd/patches/mtx_lock_recurse.patch > > I also added a KASSERT() to protect against mixed locking modes, e.g.: As you know, this is of interest to me because several situations have come up in the network stack locking code where we've worked around locking issues by conditionally acquiring a lock based on whether it's not already held. For example, in the rwatson_netperf branch, we conditionally acquire the socket buffer lock in the KQueue filter because we don't know if we're being called from a context that has already acquired the mutex: @@ -1875,8 +1915,11 @@ filt_sowrite(struct knote *kn, long hint) { struct socket *so = kn->kn_fp->f_data; - int result; + int needlock, result; + needlock = !SOCKBUF_OWNED(&so->so_snd); + if (needlock) + SOCKBUF_LOCK(&so->so_snd); kn->kn_data = sbspace(&so->so_snd); if (so->so_state & SS_CANTSENDMORE) { kn->kn_flags |= EV_EOF; @@ -1891,6 +1934,8 @@ result = (kn->kn_data >= kn->kn_sdata); else result = (kn->kn_data >= so->so_snd.sb_lowat); + if (needlock) + SOCKBUF_UNLOCK(&so->so_snd); return (result); } This stems directly from the fact that in general we wanted to avoid recursion on socket buffer mutexes, but that in one case due to existing implementation constraints, we don't know what state the lock will be in. In this case, that happens because invocation of filt_sowrite() sometimes occurs directly from the KQueue code to poll a filter, and sometimes from the socket code during a wakeup that invokes KNOTE(). There are arguably at least two ways in which the current code might be modified to try and address this: (0) Use recursive mutexes so that this recursion is permitted. (1) Use separate APIs to enter the per-object filters so that the origins of the invocation (and hence locking state) are more clear. (2) Avoid holding existing locks over calls into KNOTE(), which goes off and does a lot of stuff. Neither of these is 100% desirable, and I guess I'd prefer (1) over (2) since (2) conflicts with the use of locks as a form of reference counting. However, the easy answer (0) is also undesirable because the socket code generally doesn't need recursion of the socket buffer mutex. Given the ability to selectively say "Ok, recursion here is fine" we would change the above conditional locking into such a case. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 01:57:42 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 29BEC16A4CE; Fri, 9 Apr 2004 01:57:42 -0700 (PDT) Received: from sohgo.tanimura.dyndns.org (IP1A1429.kng.mesh.ad.jp [61.203.105.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0B6D243D54; Fri, 9 Apr 2004 01:57:40 -0700 (PDT) (envelope-from tanimura@tanimura.dyndns.org) Received: from sohgo.tanimura.dyndns.org (localhost [IPv6:::1]) ESMTP id i398vada038972 ; Fri, 9 Apr 2004 17:57:36 +0900 (JST) Received: (from uucp@localhost) (8.12.11/3.7W-submit-carrots-Tokyu-Meguro) with UUCP id i398vaCc038971 ; Fri, 9 Apr 2004 17:57:36 +0900 (JST) Received: from urban.nkth.tanimura.dyndns.org (localhost [IPv6:::1]) with ESMTP id i398rwAq029617 ; Fri, 9 Apr 2004 17:53:58 +0900 (JST) Message-Id: <200404090853.i398rwAq029617@urban> Date: Fri, 09 Apr 2004 17:53:58 +0900 From: Seigo Tanimura To: Robert Watson In-Reply-To: References: <20040408213647.GL661@darkness.comp.waw.pl> User-Agent: Wanderlust/2.10.1 (Watching The Wheels) SEMI/1.14.5 (Awara-Onsen) FLIM/1.14.5 (Demachiyanagi) APEL/10.6 MULE XEmacs/21.4 (patch 14) (Reasonable Discussion) (i386--freebsd) Organization: My Home MIME-Version: 1.0 (generated by SEMI 1.14.5 - "Awara-Onsen") Content-Type: text/plain; charset=US-ASCII cc: Seigo Tanimura cc: Pawel Jakub Dawidek cc: freebsd-arch@FreeBSD.org Subject: Re: mtx_lock_recurse/mtx_unlock_recurse functions (proof-of-concept). X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2004 08:57:42 -0000 On Thu, 8 Apr 2004 18:33:00 -0400 (EDT), Robert Watson said: rwatson> On Thu, 8 Apr 2004, Pawel Jakub Dawidek wrote: >> As was discussed, it will be helpful to have functions, that are able to >> acquire lock recursively, even if lock itself isn't recursable. >> >> Here is a patch, that should implement this functionality: >> >> http://people.freebsd.org/~pjd/patches/mtx_lock_recurse.patch >> >> I also added a KASSERT() to protect against mixed locking modes, e.g.: rwatson> As you know, this is of interest to me because several situations have rwatson> come up in the network stack locking code where we've worked around rwatson> locking issues by conditionally acquiring a lock based on whether it's not rwatson> already held. For example, in the rwatson_netperf branch, we rwatson> conditionally acquire the socket buffer lock in the KQueue filter because rwatson> we don't know if we're being called from a context that has already rwatson> acquired the mutex: rwatson> @@ -1875,8 +1915,11 @@ rwatson> filt_sowrite(struct knote *kn, long hint) rwatson> { rwatson> struct socket *so = kn->kn_fp->f_data; rwatson> - int result; rwatson> + int needlock, result; rwatson> + needlock = !SOCKBUF_OWNED(&so->so_snd); rwatson> + if (needlock) rwatson> + SOCKBUF_LOCK(&so->so_snd); rwatson> kn->kn_data = sbspace(&so->so_snd); rwatson> if (so->so_state & SS_CANTSENDMORE) { rwatson> kn->kn_flags |= EV_EOF; rwatson> @@ -1891,6 +1934,8 @@ rwatson> result = (kn->kn_data >= kn->kn_sdata); rwatson> else rwatson> result = (kn->kn_data >= so->so_snd.sb_lowat); rwatson> + if (needlock) rwatson> + SOCKBUF_UNLOCK(&so->so_snd); rwatson> return (result); rwatson> } rwatson> This stems directly from the fact that in general we wanted to avoid rwatson> recursion on socket buffer mutexes, but that in one case due to existing rwatson> implementation constraints, we don't know what state the lock will be in. rwatson> In this case, that happens because invocation of filt_sowrite() sometimes rwatson> occurs directly from the KQueue code to poll a filter, and sometimes from rwatson> the socket code during a wakeup that invokes KNOTE(). There are arguably rwatson> at least two ways in which the current code might be modified to try and rwatson> address this: rwatson> (0) Use recursive mutexes so that this recursion is permitted. rwatson> (1) Use separate APIs to enter the per-object filters so that the origins rwatson> of the invocation (and hence locking state) are more clear. rwatson> (2) Avoid holding existing locks over calls into KNOTE(), which goes off rwatson> and does a lot of stuff. rwatson> Neither of these is 100% desirable, and I guess I'd prefer (1) over (2) rwatson> since (2) conflicts with the use of locks as a form of reference counting. rwatson> However, the easy answer (0) is also undesirable because the socket code rwatson> generally doesn't need recursion of the socket buffer mutex. Given the rwatson> ability to selectively say "Ok, recursion here is fine" we would change rwatson> the above conditional locking into such a case. (3) Let a dedicated kernel thread call the methods of struct filterops. The headache of kqueue is that KNOTE() ultimately calls back the caller subsystem. Tty would suffer from the same pain as well. Assuming that a kqueue event need not be delivered synchronously upon the event, maybe we can attach an event queue to a knote list. KNOTE() appends an event to the queue and wakes up a dedicated kernel thread (kqueued). It then dequeues the event in a queue to call a filter *without* the lock of the event source. (socket, tty, etc.) -- Seigo Tanimura From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 07:39:18 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 55FFE16A4CE; Fri, 9 Apr 2004 07:39:18 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D5B2143D39; Fri, 9 Apr 2004 07:39:17 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i39EcrPq063893; Fri, 9 Apr 2004 10:38:54 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i39Ecrej063890; Fri, 9 Apr 2004 10:38:53 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 9 Apr 2004 10:38:53 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Seigo Tanimura In-Reply-To: <200404090853.i398rwAq029617@urban> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Pawel Jakub Dawidek cc: freebsd-arch@FreeBSD.org Subject: Re: mtx_lock_recurse/mtx_unlock_recurse functions (proof-of-concept). X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2004 14:39:18 -0000 On Fri, 9 Apr 2004, Seigo Tanimura wrote: > (3) Let a dedicated kernel thread call the methods of struct > filterops. > > The headache of kqueue is that KNOTE() ultimately calls back the caller > subsystem. Tty would suffer from the same pain as well. > > Assuming that a kqueue event need not be delivered synchronously upon > the event, maybe we can attach an event queue to a knote list. KNOTE() > appends an event to the queue and wakes up a dedicated kernel thread > (kqueued). It then dequeues the event in a queue to call a filter > *without* the lock of the event source. (socket, tty, etc.) Such an approach would be fine by me, but it would also have some complications. For example, we need to make sure we handle the case where a kqueue event thread wakes up to handle a socket event, but the socket has been free'd. It's not all that bad, it just involves a necessary amount of paperwork as well. It does seem that the fundamental problem here is that KQueue is one of the more incestuous of support subsystems in the kernel, and a little bit of redesign might well be called for, because it will simplify the locking, but probably also clean up other concerns. The MAC Framework is also an incestuous subsystem, but it does a lot less "calling back" than KQueue does. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 09:57:59 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E516F16A4CE for ; Fri, 9 Apr 2004 09:57:58 -0700 (PDT) Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id AD11B43D3F for ; Fri, 9 Apr 2004 09:57:58 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (qmail 8072 invoked from network); 9 Apr 2004 16:57:58 -0000 Received: from dsl017-045-168.spk4.dsl.speakeasy.net (HELO hydrogen.funkthat.com) ([69.17.45.168]) (envelope-sender ) by mail6.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 9 Apr 2004 16:57:58 -0000 Received: from hydrogen.funkthat.com (xinaxy@localhost.funkthat.com [127.0.0.1])i39GvvOE098737; Fri, 9 Apr 2004 09:57:57 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.10/8.12.10/Submit) id i39GvvvG098736; Fri, 9 Apr 2004 09:57:57 -0700 (PDT) Date: Fri, 9 Apr 2004 09:57:57 -0700 From: John-Mark Gurney To: Robert Watson Message-ID: <20040409165757.GL567@funkthat.com> Mail-Followup-To: Robert Watson , Seigo Tanimura , Pawel Jakub Dawidek , freebsd-arch@freebsd.org References: <200404090853.i398rwAq029617@urban> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html cc: Seigo Tanimura cc: Pawel Jakub Dawidek cc: freebsd-arch@freebsd.org Subject: Re: mtx_lock_recurse/mtx_unlock_recurse functions (proof-of-concept). X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2004 16:57:59 -0000 Robert Watson wrote this message on Fri, Apr 09, 2004 at 10:38 -0400: > > On Fri, 9 Apr 2004, Seigo Tanimura wrote: > > > (3) Let a dedicated kernel thread call the methods of struct > > filterops. > > > > The headache of kqueue is that KNOTE() ultimately calls back the caller > > subsystem. Tty would suffer from the same pain as well. > > > > Assuming that a kqueue event need not be delivered synchronously upon > > the event, maybe we can attach an event queue to a knote list. KNOTE() > > appends an event to the queue and wakes up a dedicated kernel thread > > (kqueued). It then dequeues the event in a queue to call a filter > > *without* the lock of the event source. (socket, tty, etc.) > > Such an approach would be fine by me, but it would also have some > complications. For example, we need to make sure we handle the case where > a kqueue event thread wakes up to handle a socket event, but the socket > has been free'd. It's not all that bad, it just involves a necessary > amount of paperwork as well. It does seem that the fundamental problem > here is that KQueue is one of the more incestuous of support subsystems in > the kernel, and a little bit of redesign might well be called for, because > it will simplify the locking, but probably also clean up other concerns. > The MAC Framework is also an incestuous subsystem, but it does a lot less > "calling back" than KQueue does. I haven't been thinking too hard of a redesign of kqueue since I came up with the locking method that I sent to you (rwatson) in an email. And your proposed method really isn't any different than the one the is currently done. You forget that there is also the case of where once the socket is closed, all KNOTES of that socket had better be destroyed, otherwise every module that is kqueue aware is going to have to add extra book keeping to make sure that you don't unload the filter event that is about to be called... It does make it easier in that you can tree all the KNOTE locks as leaf locks, but you still may run into the case of needing the object lock.. What happens in the case of a one shot event? How do you remove it from the object? If you treat the KNOTE lock as a leaf lock, that means you'll have to unlock the KNOTE lock, grab the object lock and then relock the KNOTE... Still a lot of work.. Is our kthread context switch times and wakeup times low enough that it won't add too much extra cpu and latency? I have devised a locking method that I believe will work with the current architecture of kqueue. And it doesn't involve adding a lock to any KNOTEs... It's handled solely by object and kqueue lock. If you want a copy, I can send it to you.. Or should I repost it here? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 11:01:11 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2D5B916A4CE for ; Fri, 9 Apr 2004 11:01:11 -0700 (PDT) Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id EA80B43D45 for ; Fri, 9 Apr 2004 11:01:10 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (qmail 3594 invoked from network); 9 Apr 2004 18:01:10 -0000 Received: from dsl017-045-168.spk4.dsl.speakeasy.net (HELO hydrogen.funkthat.com) ([69.17.45.168]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 9 Apr 2004 18:01:10 -0000 Received: from hydrogen.funkthat.com (bixsto@localhost.funkthat.com [127.0.0.1])i39I17OE000246; Fri, 9 Apr 2004 11:01:09 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.10/8.12.10/Submit) id i39I16Id000245; Fri, 9 Apr 2004 11:01:06 -0700 (PDT) Date: Fri, 9 Apr 2004 11:01:06 -0700 From: John-Mark Gurney To: Robert Watson , Seigo Tanimura , Pawel Jakub Dawidek , freebsd-arch@freebsd.org Message-ID: <20040409180106.GM567@funkthat.com> Mail-Followup-To: Robert Watson , Seigo Tanimura , Pawel Jakub Dawidek , freebsd-arch@freebsd.org References: <200404090853.i398rwAq029617@urban> <20040409165757.GL567@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040409165757.GL567@funkthat.com> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html Subject: Re: mtx_lock_recurse/mtx_unlock_recurse functions (proof-of-concept). X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2004 18:01:11 -0000 John-Mark Gurney wrote this message on Fri, Apr 09, 2004 at 09:57 -0700: > I have devised a locking method that I believe will work with the > current architecture of kqueue. And it doesn't involve adding a lock > to any KNOTEs... It's handled solely by object and kqueue lock. If you > want a copy, I can send it to you.. Or should I repost it here? By request, I am reposting the brief document that I wrote describing my idea to put kqueues under locks: Just to let others know about the work on this. I think I have a solution for the locking of kqueues, but it is compilicated by the fact that there are linked lists that belong to both the kqueue and the object that kqueue is watching, and each need to be protected by a seperate locks, yet both need to be obtained in different orders... so, I came up with an algorithm that is able to treat both locks as leaf locks, yet avoid deadlock... There might be a better implementation of this, so input would be helpful... Basicly, we have two different call paths for kqueue: a) adding an event from userland: create knote add knote to kqueue add knote to watch object b) marking an event as ready/available via object: add knote to kqueue's active list Now the problem is that there needs to be locks for both the kqueue and the object, and they need to be seperate locks, yet, they both need to be obtained in different orders... So, the solution I came up with is something similar to a read/write lock... Assumptions: 1) holding the kqueue lock means that any knotes on the kqueue are valid, and the objects referenced by the knotes are also valid.. 2) holding the object lock means that all knotes associated with the object are valid and that the kqueue of all knote's are also valid.. Now, the nice thing about 2 is that you can reference a kqueue w/o holding it's lock, but of course you can't maniuplate it.. and of course thats the same with 1, is that you can reference the object of all the knotes associated with the kqueue... The solution is to introduce a flag, locked by the kqueue lock, that needs to be obtained before you can manipulate the kqueue.. Nothing special needs to be done for the object... What now happens is that nothing special happens in case a above, the flag is obtained as if it's a normal lock (we man want to switch this for performance reasons, but this way makes it more generic)... but in case b, once we have obtained the object lock, we can now walk the list of KNOTEs that we need to activate.. We then attempt to grab the flag associated with the kqueue that we are trying to put the KNOTE on.. If we can't, before we release the lock that protects the flag, we unlock our object lock, and then msleep on the kqueue lock and using P_DROP (since the lock isn't guranteed to be valid to reaquire since we don't have our object lock anymore).. We then do a wakeup on the kqueue lock when ever we release the flag, and this will then give all the objects waiting for the kqueue to become available to restart processing of their KNOTE queue... There are possible optimizations, like walking the rest of the list of KNOTEs associated with the object in a failure case, and retry again, etc. But I want to make sure that the locking works and won't cause a problem... How does this sound to people? I have some code starting to implement this, but I haven't gotten very far with it yet... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."