From owner-freebsd-arch@FreeBSD.ORG Sun May 10 04:53:24 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F423106567E for ; Sun, 10 May 2009 04:53:24 +0000 (UTC) (envelope-from scholz@scriptolutions.com) Received: from dd17920.kasserver.com (dd17920.kasserver.com [85.13.138.236]) by mx1.freebsd.org (Postfix) with ESMTP id 5FA868FC1D for ; Sun, 10 May 2009 04:53:24 +0000 (UTC) (envelope-from scholz@scriptolutions.com) Received: from X64SSD (unknown [117.47.148.131]) by dd17920.kasserver.com (Postfix) with ESMTP id 1A62A181B39BE; Sun, 10 May 2009 06:52:18 +0200 (CEST) Date: Sun, 10 May 2009 06:49:24 +0200 From: Lothar Scholz X-Mailer: The Bat! (v3.62.08) Professional Organization: Scriptolutions X-Priority: 3 (Normal) Message-ID: <19461540.20090510064924@scriptolutions.com> To: Jilles Tjoelker In-Reply-To: <20090509200724.GA25714@stack.nl> References: <588815840.20090509203115@scriptolutions.com> <20090509200724.GA25714@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re[2]: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Lothar Scholz List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2009 04:53:25 -0000 Hello Jilles, Saturday, May 9, 2009, 10:07:24 PM, you wrote: JT> On Sat, May 09, 2009 at 08:31:15PM +0200, Lothar Scholz wrote: >> Thanks for solving the posix semaphore problem. But with shared memory >> there comes the next issue: >> int main() { >> int m; >> shm_unlink("/barfoo"); >> m = shm_open("/barfoo", O_RDWR|O_CREAT|O_EXCL, S_IRWXU); >> if (m == 1) perror("shm_open error"); >> } >> i always get permission denied error, and i tried many values >> for flags and mode? I can only get this working as root but not >> as a normal user. JT> shm_open/shm_unlink refer to the filesystem; they are fairly direct JT> wrappers around open and unlink. Question is where are they stored? In Linux it is "/dev/shm". In my case it looks like the directory for shm_open files has some wrong access rights so that a normal user can't generate it. JT> POSIX suggests making the pathname a configuration option; JT> alternatively, using a directory for temporary files such as /tmp could JT> work. I will try this hack soon if nobody comes up with a solution. -- Best regards, Lothar Scholz mailto:scholz@scriptolutions.com From owner-freebsd-arch@FreeBSD.ORG Sun May 10 05:13:04 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C49A1065670 for ; Sun, 10 May 2009 05:13:04 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (hergotha.csail.mit.edu [66.92.79.170]) by mx1.freebsd.org (Postfix) with ESMTP id C39268FC08 for ; Sun, 10 May 2009 05:13:03 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.2/8.14.2) with ESMTP id n4A50IwZ050729; Sun, 10 May 2009 01:00:18 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.2/8.13.8/Submit) id n4A50GOa050728; Sun, 10 May 2009 01:00:16 -0400 (EDT) (envelope-from wollman) Date: Sun, 10 May 2009 01:00:16 -0400 (EDT) From: Garrett Wollman Message-Id: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> To: scholz@scriptolutions.com X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: References: Organization: None X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (hergotha.csail.mit.edu [127.0.0.1]); Sun, 10 May 2009 01:00:18 -0400 (EDT) X-Spam-Status: No, score=-1.4 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on hergotha.csail.mit.edu Cc: arch@freebsd.org Subject: Re: Re[2]: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2009 05:13:04 -0000 In scholz@scriptolutions.com writes: >JT> shm_open/shm_unlink refer to the filesystem; they are fairly direct >JT> wrappers around open and unlink. > >Question is where are they stored? In the fileststem, in the path that you specify. They are just ordinary files. There was some thought that this was a bad (or at least not-like-Linux) way of implementing this feature, so I believe more-recent versions of FreeBSD do it differently. When I wrote this code, I could not see any reason for the "path" argument to be interpreted differently from any other path. -GAWollman From owner-freebsd-arch@FreeBSD.ORG Sun May 10 06:25:28 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66F291065675 for ; Sun, 10 May 2009 06:25:28 +0000 (UTC) (envelope-from scholz@scriptolutions.com) Received: from dd17920.kasserver.com (dd17920.kasserver.com [85.13.138.236]) by mx1.freebsd.org (Postfix) with ESMTP id 284FE8FC1B for ; Sun, 10 May 2009 06:25:28 +0000 (UTC) (envelope-from scholz@scriptolutions.com) Received: from X64SSD (unknown [117.47.148.131]) by dd17920.kasserver.com (Postfix) with ESMTP id 50215181B39BE; Sun, 10 May 2009 08:00:19 +0200 (CEST) Date: Sun, 10 May 2009 07:57:06 +0200 From: Lothar Scholz X-Mailer: The Bat! (v3.62.08) Professional Organization: Scriptolutions X-Priority: 3 (Normal) Message-ID: <7710650619.20090510075706@scriptolutions.com> To: Garrett Wollman In-Reply-To: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re[4]: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Lothar Scholz List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2009 06:25:28 -0000 Hello Garrett, Sunday, May 10, 2009, 7:00:16 AM, you wrote: GW> In GW> GW> scholz@scriptolutions.com writes: >>JT> shm_open/shm_unlink refer to the filesystem; they are fairly direct >>JT> wrappers around open and unlink. >> >>Question is where are they stored? GW> In the fileststem, in the path that you specify. They are just GW> ordinary files. GW> There was some thought that this was a bad (or at least GW> not-like-Linux) way of implementing this feature, so I believe GW> more-recent versions of FreeBSD do it differently. When I wrote this GW> code, I could not see any reason for the "path" argument to be GW> interpreted differently from any other path. Oh thats a very very bad idea. First of all you can't use '/' if you want stay portable. It is also just a maximum of 13 char long (says the FreeBSD 6.X man page) and usually you now pass names like "com.mycompany.myproduct.mypurpose" as names to prevent namespace collisons. The path has nothing to do with the filesystem, it's a separate namespace. Let alone that semaphores and shared memory already use the same namespace is something i didn't expect on Linux. Now it is clear where my problem is and i go to a mmap to a $HOME/. file. Not nice but if anybody gives a shit about compatibility (backward and to other systems before implementing stuff) it is the only way. -- Best regards, Lothar Scholz mailto:scholz@scriptolutions.com From owner-freebsd-arch@FreeBSD.ORG Sun May 10 09:01:33 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 116DD106566C for ; Sun, 10 May 2009 09:01:33 +0000 (UTC) (envelope-from kmsujit@gmail.com) Received: from ti-out-0910.google.com (ti-out-0910.google.com [209.85.142.190]) by mx1.freebsd.org (Postfix) with ESMTP id A20E68FC15 for ; Sun, 10 May 2009 09:01:32 +0000 (UTC) (envelope-from kmsujit@gmail.com) Received: by ti-out-0910.google.com with SMTP id u3so211350tia.3 for ; Sun, 10 May 2009 02:01:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=cN0DK46fUGdMOTLZ7DfXIahf3LrOjUMgus5Eje5RZQE=; b=SxBG9m3MRUlsYGoNK3QZGIZoq3w71kWBZ6jjO6EDa1EFd0+A9lY0fhHeTRao+2tlSy OsXYFkv6K0ZA8zlZikHocr8GKbSa364FG697ZHz8ScvRFl+HIZdFaBWa4Rz5Syy59Tjo KdOAfzhu9hcot/HbmeKLYr5okaR9NrVbGJ/fc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=LWgSJG1w6TstJJefEtLD8qNP27O5mrS/buBc9yLJ9kkvbQOKhRMiBJ7wyumudqrTfJ 3IKS+XlfZVW4yy3NhOmDzkGFgkKaAW+n08s0IESNRGE0C3u+ivQ0Gsmx7kxeeFbouw/E Lun/58ez1XtXXAPnwjRXxX5Lnjhk0VKqnAO9M= MIME-Version: 1.0 Received: by 10.110.3.15 with SMTP id 15mr379551tic.27.1241944541090; Sun, 10 May 2009 01:35:41 -0700 (PDT) In-Reply-To: <7710650619.20090510075706@scriptolutions.com> References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> <7710650619.20090510075706@scriptolutions.com> Date: Sun, 10 May 2009 14:05:41 +0530 Message-ID: <74fe56020905100135y7f44b5fapfee3ef2ae70a2a0b@mail.gmail.com> From: Sujit K M To: Lothar Scholz Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Garrett Wollman Subject: Re: Re[4]: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2009 09:01:33 -0000 > Now it is clear where my problem is and i go to a mmap to a $HOME/. > file. Not nice but if anybody gives a shit about compatibility > (backward and to other systems before implementing stuff) it is > the only way. i donot understand why this is an compatility issue. Just use /path/to/file. say /proc/shm/shm[0-9]+[a-z]+[0-9]+ From owner-freebsd-arch@FreeBSD.ORG Sun May 10 15:54:36 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7983E10656BC for ; Sun, 10 May 2009 15:54:36 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (hergotha.csail.mit.edu [66.92.79.170]) by mx1.freebsd.org (Postfix) with ESMTP id 254408FC20 for ; Sun, 10 May 2009 15:54:32 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.2/8.14.2) with ESMTP id n4AFsVVi054854; Sun, 10 May 2009 11:54:31 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.2/8.13.8/Submit) id n4AFsVM6054851; Sun, 10 May 2009 11:54:31 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18950.63671.323324.756287@hergotha.csail.mit.edu> Date: Sun, 10 May 2009 11:54:31 -0400 From: Garrett Wollman To: Lothar Scholz In-Reply-To: <7710650619.20090510075706@scriptolutions.com> References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> <7710650619.20090510075706@scriptolutions.com> X-Mailer: VM 7.17 under 21.4 (patch 21) "Educational Television" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (hergotha.csail.mit.edu [127.0.0.1]); Sun, 10 May 2009 11:54:31 -0400 (EDT) X-Spam-Status: No, score=-1.4 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on hergotha.csail.mit.edu Cc: arch@freebsd.org Subject: Re: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2009 15:54:39 -0000 < said: > First of all you can't use '/' if you want stay portable. The Standard says otherwise. > It is also just a maximum of 13 char long (says the FreeBSD 6.X man page) Not in the manual page I have, and the Standard says otherwise. > The path has nothing to do with the filesystem, it's a separate > namespace. Again, the Standard says otherwise (or rather, it says that this is an implementation option). To quote the 2001 edition of the standard (XSH6 page 1313): It is unspecified whether the name appears in the file system and is visible to other functions that take pathnames as arguments. The name argument conforms to the construction rules for a pathname. -GAWollman From owner-freebsd-arch@FreeBSD.ORG Sun May 10 20:02:46 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DEA21065674 for ; Sun, 10 May 2009 20:02:46 +0000 (UTC) (envelope-from ntarmos@cs.uoi.gr) Received: from gaia.cs.uoi.gr (gaia.cs.uoi.gr [195.130.121.201]) by mx1.freebsd.org (Postfix) with ESMTP id B127C8FC1B for ; Sun, 10 May 2009 20:02:45 +0000 (UTC) (envelope-from ntarmos@cs.uoi.gr) Received: from zeus.cs.uoi.gr (zeus.cs.uoi.gr [195.130.121.11]) by gaia.cs.uoi.gr (8.14.1/8.14.1) with ESMTP id n4AJfgv1078930 for ; Sun, 10 May 2009 22:41:47 +0300 (EEST) (envelope-from ntarmos@cs.uoi.gr) Received: from zeus.cs.uoi.gr (localhost [127.0.0.1]) by zeus.cs.uoi.gr (8.13.5/8.13.5) with ESMTP id n4AJfZ6F001863 for ; Sun, 10 May 2009 22:41:40 +0300 (EEST) Received: (from ntarmos@localhost) by zeus.cs.uoi.gr (8.13.5/8.13.5/Submit) id n4AJfZWr001860 for arch@freebsd.org; Sun, 10 May 2009 22:41:35 +0300 (EEST) X-Authentication-Warning: zeus.cs.uoi.gr: ntarmos set sender to ntarmos@cs.uoi.gr using -f Date: Sun, 10 May 2009 22:41:33 +0300 From: Nikos Ntarmos To: arch@freebsd.org Message-ID: <20090510194133.GG20749@ace.cs.uoi.gr> References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> Organization: Computer Science Dept., U. of Ioannina, Greece WWW-Homepage: http://ntarmos.dyndns.org/ X-PGP-Fingerprint: 9680 60A7 DE60 0298 B1F0 9B22 9BA2 7569 CF95 160A Office-Phone: +30-26510-98866 GPS-Info: 39.617660N, 20.838790E User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: ClamAV 0.91.2/9350/Sun May 10 07:13:25 2009 on gaia.cs.uoi.gr X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (gaia.cs.uoi.gr [195.130.121.201]); Sun, 10 May 2009 22:41:47 +0300 (EEST) Cc: Subject: Re: Re[2]: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2009 20:02:46 -0000 On Sun, May 10, 2009 at 01:00:16AM -0400, Garrett Wollman wrote: > In > scholz@scriptolutions.com writes: > > >JT> shm_open/shm_unlink refer to the filesystem; they are fairly direct > >JT> wrappers around open and unlink. > > > >Question is where are they stored? > > In the fileststem, in the path that you specify. They are just > ordinary files. > > There was some thought that this was a bad (or at least > not-like-Linux) way of implementing this feature, so I believe > more-recent versions of FreeBSD do it differently. When I wrote this > code, I could not see any reason for the "path" argument to be > interpreted differently from any other path. FWIW the test code in the original email still fails even if an absolute path is used as a sem name, ie: sem_t *s = sem_open("/path/to/foobar", O_CREAT | O_EXCL, S_IWUSR, 0); with /path/to/foobar pointing to a user writable directory, segfaults with "invalid system call". Note that the error is not printed by perror(3) but by the system itself. A backtrace of the resulting core shows that the problem is burried deep in ksem_open(): ntarmos@ace:~% ./ts zsh: invalid system call (core dumped) ./ts ntarmos@ace:~% gdb -q ./ts ts.core Core was generated by `ts'. Program terminated with signal 12, Bad system call. Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x280c214b in ksem_open () from /lib/libc.so.7 (gdb) bt #0 0x280c214b in ksem_open () from /lib/libc.so.7 #1 0x280b78fc in sem_open () from /lib/libc.so.7 #2 0x080484e5 in main () at test-sem.c:7 (gdb) This is on i386/7.2-RELEASE. Cheers. \n\n From owner-freebsd-arch@FreeBSD.ORG Mon May 11 09:29:54 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AA96106566C for ; Mon, 11 May 2009 09:29:54 +0000 (UTC) (envelope-from scholz@scriptolutions.com) Received: from dd17920.kasserver.com (dd17920.kasserver.com [85.13.138.236]) by mx1.freebsd.org (Postfix) with ESMTP id C26FF8FC08 for ; Mon, 11 May 2009 09:29:53 +0000 (UTC) (envelope-from scholz@scriptolutions.com) Received: from X64SSD (unknown [222.123.87.216]) by dd17920.kasserver.com (Postfix) with ESMTP id EBD03182DAC5B; Mon, 11 May 2009 11:29:49 +0200 (CEST) Date: Mon, 11 May 2009 11:25:37 +0200 From: Lothar Scholz X-Mailer: The Bat! (v3.62.08) Professional Organization: Scriptolutions X-Priority: 3 (Normal) Message-ID: <1393224851.20090511112537@scriptolutions.com> To: Garrett Wollman In-Reply-To: <18950.63671.323324.756287@hergotha.csail.mit.edu> References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> <7710650619.20090510075706@scriptolutions.com> <18950.63671.323324.756287@hergotha.csail.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re[2]: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Lothar Scholz List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 09:29:54 -0000 Hello Garrett, Sunday, May 10, 2009, 5:54:31 PM, you wrote: GW> < said: >> First of all you can't use '/' if you want stay portable. GW> The Standard says otherwise. It's not a standard think. Read about the real world programming hints. You see recommendations to only use a starting '/' >> It is also just a maximum of 13 char long (says the FreeBSD 6.X man page) GW> Not in the manual page I have, and the Standard says otherwise. This time you are right. It was about named semaphores and there the limit seems to be removed - it was ridiculous low anyway. GW> Again, the Standard says otherwise (or rather, it says that this is an GW> implementation option). To quote the 2001 edition of the standard GW> (XSH6 page 1313): GW> It is unspecified whether the name appears in the file system GW> and is visible to other functions that take pathnames as GW> arguments. The name argument conforms to the construction GW> rules for a pathname. Thats why the man page calls this parameter 'name' and not 'path'. Some idiots started to think about this as a file path. But it isn't and it shouldn't. Thats what this spec is saying in the typical commitee polite form when some members made a mistake but are to important to be blamed in public. So this needs to be fixed. If i have to find a useable filefile location anyway the whole function does not make any sense, then i can directly use mmap. The purpose is to have a unique name (and in 2009 it is an URI not a file path). Thats how serious non kiddy operating systems are doing like Linux/Solaris/MacOSX-Darwin/HP-UX. And i guess also the accounting functions are wrong then. shm_open does not open a file so the (internal used) file should not add to the file space quota but to the memory allocation quota. -- Best regards, Lothar Scholz mailto:scholz@scriptolutions.com From owner-freebsd-arch@FreeBSD.ORG Mon May 11 10:16:37 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34BBB106564A for ; Mon, 11 May 2009 10:16:37 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id E8C7D8FC15 for ; Mon, 11 May 2009 10:16:35 +0000 (UTC) (envelope-from des@des.no) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id ED5CC6D41B; Mon, 11 May 2009 11:59:10 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id C90AF844B1; Mon, 11 May 2009 11:59:10 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Lothar Scholz References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> <7710650619.20090510075706@scriptolutions.com> <18950.63671.323324.756287@hergotha.csail.mit.edu> <1393224851.20090511112537@scriptolutions.com> Date: Mon, 11 May 2009 11:59:10 +0200 In-Reply-To: <1393224851.20090511112537@scriptolutions.com> (Lothar Scholz's message of "Mon, 11 May 2009 11:25:37 +0200") Message-ID: <86hbzsot8x.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.92 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org, Garrett Wollman Subject: Re: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 10:16:37 -0000 Lothar Scholz writes: > Some idiots started to think about this as a file path. But it isn't > and it shouldn't. Thats what this spec is saying in the typical commitee > polite form when some members made a mistake but are to important to > be blamed in public. You are wrong. The standard says what it says specifically because it makes it possible to implement semaphores and shared memory in terms of file operations. I've been there and done that. > So this needs to be fixed. You've already been told that it *has* been fixed (or rather changed, since it was not broken to begin with) in head. > If i have to find a useable filefile location anyway the whole function d= oes > not make any sense, then i can directly use mmap. The purpose is to > have a unique name (and in 2009 it is an URI not a file path). Thats > how serious non kiddy operating systems are doing like > Linux/Solaris/MacOSX-Darwin/HP-UX. Insulting the developers will get you nowhere. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon May 11 11:06:50 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD1901065672 for ; Mon, 11 May 2009 11:06:50 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7FCFD8FC17 for ; Mon, 11 May 2009 11:06:50 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n4BB6oQf085875 for ; Mon, 11 May 2009 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n4BB6nEv085871 for freebsd-arch@FreeBSD.org; Mon, 11 May 2009 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 11 May 2009 11:06:49 GMT Message-Id: <200905111106.n4BB6nEv085871@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 11:06:50 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Mon May 11 11:10:46 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 160191065875 for ; Mon, 11 May 2009 11:10:46 +0000 (UTC) (envelope-from kmsujit@gmail.com) Received: from ti-out-0910.google.com (ti-out-0910.google.com [209.85.142.189]) by mx1.freebsd.org (Postfix) with ESMTP id 041378FC28 for ; Mon, 11 May 2009 11:10:29 +0000 (UTC) (envelope-from kmsujit@gmail.com) Received: by ti-out-0910.google.com with SMTP id u3so270131tia.3 for ; Mon, 11 May 2009 04:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Ih/D0hBntj13muUSZWgkr4vw7Hxe6N5smuTB+l1/AWM=; b=sOEpT5aYErLWCnFnbKRhtahOcBtY/q1veEKeAjx8aNnkXzs07uy0Mik4PPBTW7Z084 fV7IzyY5gghazp1lywbT532WO+7ge7D63f65tWnUUyqlGwC5OhAQ8Ld89lItLZC6qUku SiUOLlsAp4jJxnrFkHdIR3tHDf2ScAyAGRBnw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=cKvw+kjsF8rRdUxOwGv8Sk+62MzvCk6iD/y1OYDHGIUuQE0BIhRk0+GD6z+OIA7yC3 4hF6n3WWWdHl0tNc8hJIpgK3qpSLljgnG3P6uqrokTmeBOeiBD3+CUDxw2G307zBXnJf THWach54gbgQWdXGJnrgTSFaz0mW9ABhCcUIg= MIME-Version: 1.0 Received: by 10.110.3.15 with SMTP id 15mr462502tic.27.1242040228188; Mon, 11 May 2009 04:10:28 -0700 (PDT) In-Reply-To: <1393224851.20090511112537@scriptolutions.com> References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> <7710650619.20090510075706@scriptolutions.com> <18950.63671.323324.756287@hergotha.csail.mit.edu> <1393224851.20090511112537@scriptolutions.com> Date: Mon, 11 May 2009 16:40:27 +0530 Message-ID: <74fe56020905110410y430bf76yacf5c5a308a99865@mail.gmail.com> From: Sujit K M To: Lothar Scholz Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Garrett Wollman Subject: Re: Re[2]: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 11:11:00 -0000 Any sort of frustration here? > Some idiots started to think about this as a file path. But it isn't > and it shouldn't. Thats what this spec is saying in the typical commitee > polite form when some members made a mistake but are to important to > be blamed in public. > What ever the Idiots are saying is correct. Read up some decent Unix manual. > So this needs to be fixed. What needs to be fixed? Could you be more specific? > If i have to find a useable filefile location anyway the whole function does > not make any sense, then i can directly use mmap. The purpose is to > have a unique name (and in 2009 it is an URI not a file path). Thats > how serious non kiddy operating systems are doing like > Linux/Solaris/MacOSX-Darwin/HP-UX. Try using these giant, great, long lasting things. > > And i guess also the accounting functions are wrong then. shm_open > does not open a file so the (internal used) file should not add to the > file space quota but to the memory allocation quota. I think you need a check up. You seem to be contradicting what ever you said before. From owner-freebsd-arch@FreeBSD.ORG Mon May 11 12:16:01 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EA36106566B for ; Mon, 11 May 2009 12:16:01 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id D4FD48FC1D for ; Mon, 11 May 2009 12:16:00 +0000 (UTC) (envelope-from des@des.no) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 030496D41B; Mon, 11 May 2009 14:15:59 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id 6BAE0844B9; Mon, 11 May 2009 14:15:59 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Nikos Ntarmos References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> <20090510194133.GG20749@ace.cs.uoi.gr> Date: Mon, 11 May 2009 14:15:59 +0200 In-Reply-To: <20090510194133.GG20749@ace.cs.uoi.gr> (Nikos Ntarmos's message of "Sun, 10 May 2009 22:41:33 +0300") Message-ID: <86prefomww.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.92 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org Subject: Re: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 12:16:01 -0000 Nikos Ntarmos writes: > FWIW the test code in the original email still fails even if an absolute > path is used as a sem name, ie: > sem_t *s =3D sem_open("/path/to/foobar", O_CREAT | O_EXCL, S_IWUSR, 0); > with /path/to/foobar pointing to a user writable directory, segfaults > with "invalid system call". As previously mentioned, 'kldload sem'. To forestall any further gripes about the POSIX IPC system calls not being compiled in by default: they are very rarely used, because the SysV IPC API is almost universally available and is generally considered superior to the POSIX API, which it predates by more than ten years. The SysV IPC system calls are in GENERIC, and are used by e.g. Sendmail, X.org and PostgreSQL. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon May 11 15:30:05 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B7471065672 for ; Mon, 11 May 2009 15:30:05 +0000 (UTC) (envelope-from scholz@scriptolutions.com) Received: from dd17920.kasserver.com (dd17920.kasserver.com [85.13.138.236]) by mx1.freebsd.org (Postfix) with ESMTP id 5B40C8FC08 for ; Mon, 11 May 2009 15:30:05 +0000 (UTC) (envelope-from scholz@scriptolutions.com) Received: from X64SSD (unknown [222.123.64.134]) by dd17920.kasserver.com (Postfix) with ESMTP id 3487B181B1137; Mon, 11 May 2009 17:30:01 +0200 (CEST) Date: Mon, 11 May 2009 17:26:05 +0200 From: Lothar Scholz X-Mailer: The Bat! (v3.62.08) Professional Organization: Scriptolutions X-Priority: 3 (Normal) Message-ID: <981850520.20090511172605@scriptolutions.com> To: Sujit K M In-Reply-To: <74fe56020905110410y430bf76yacf5c5a308a99865@mail.gmail.com> References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> <7710650619.20090510075706@scriptolutions.com> <18950.63671.323324.756287@hergotha.csail.mit.edu> <1393224851.20090511112537@scriptolutions.com> <74fe56020905110410y430bf76yacf5c5a308a99865@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Garrett Wollman Subject: Re[4]: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Lothar Scholz List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 15:30:05 -0000 Hello Sujit, Monday, May 11, 2009, 1:10:27 PM, you wrote: >> SKM> What ever the Idiots are saying is correct. Read up some decent Unix manual. I read and even better i ported and finally yes i thought about the function, why it is there and why it is better then System V IPC. SKM> What needs to be fixed? Could you be more specific? That the name argument is just that "a name" (in its own name space) not a path. -- Best regards, Lothar Scholz mailto:scholz@scriptolutions.com From owner-freebsd-arch@FreeBSD.ORG Mon May 11 16:29:36 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5C30106566B for ; Mon, 11 May 2009 16:29:36 +0000 (UTC) (envelope-from zachary.loafman@isilon.com) Received: from seaxch10.isilon.com (seaxch10.isilon.com [74.85.160.26]) by mx1.freebsd.org (Postfix) with ESMTP id 84EC08FC0C for ; Mon, 11 May 2009 16:29:36 +0000 (UTC) (envelope-from zachary.loafman@isilon.com) Received: from famine.isilon.com ([10.54.190.95]) by seaxch10.isilon.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 11 May 2009 09:29:36 -0700 Received: from zloafman by famine.isilon.com with local (Exim 4.69) (envelope-from ) id 1M3YNc-0007Hn-Re for arch@freebsd.org; Mon, 11 May 2009 09:29:28 -0700 Date: Mon, 11 May 2009 09:29:28 -0700 From: Zachary Loafman To: arch@freebsd.org Message-ID: <20090511162928.GD17203@isilon.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="rmUrFcWP4LYae1gV" Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) X-OriginalArrivalTime: 11 May 2009 16:29:36.0306 (UTC) FILETIME=[AC640920:01C9D255] Cc: Subject: FAIL: kernel fault injection X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 16:29:36 -0000 --rmUrFcWP4LYae1gV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Arch - I'd like to contribute the kernel fault injection system that Isilon uses. Before contributing it, I'd like to get approval for the APIs involved. Testing errors is hard. Let's say you have: int foo(void) { [...] error = bar(); if (error) { /* do stuff */ } } .. but some_func() can't reliably be made to fail. How do you test it? We added error injection macros that look like this: int foo(void) { [...] error = bar(); KFAIL_POINT_CODE(FP_KERN, bar_fails_foo, error = RETURN_VALUE); if (error) { /* do stuff */ } } The KFAIL_POINT_CODE macro adds a sysctl MIB that allows you to inject errors into the above code. For example: # sysctl fail_point.kern.bar_fails_foo=".1%return(5)" This says, ".1% of the time, evaluate the fail point code with 5 as the RETURN_VALUE". If this were a standard errno, you could read the above setting as "1/1000th of the time, pretend bar() returned EIO". We also have a few wrappers around KFAIL_POINT_CODE that essentially wrap common uses. For example, the above use can be shorthanded to: KFAIL_POINT_ERROR(FP_KERN, bar_fails_foo, error) Currently, the sysctl parser accepts the following variants: return(x) - triggers the code with RETURN_VALUE set to x sleep(t) - sleep t milliseconds, panic/break - panic or break into the debugger print - print that the fail point was hit In addition to the commands, we have a syntax to express the when to evaluate those commands: p% - evaluate command p% of the time (example above) 5* - evaluate command 5 times, then disable the expression And you can compound with expr1->expr2, so, e.g.: 5%return(5)->1%return(22): 5% of the time, return 5, 1% of the remaining time, return 22 5*return(0)->10*return(5)->1%return(19) return 0 for 5 times, then 5 for 10 times, and after those, return 19 1% of the time. 1%5*return(22): 1/100th of the time, return 22, but only do it 5 times total. I've also attached an ascii rendering of a (rough draft) man page that goes into more detail. Comments? ...Zach --rmUrFcWP4LYae1gV Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="fail.man.txt" FAIL(9) FreeBSD Kernel Developer's Manual FAIL(9) NAME KFAIL_POINT_CODE, KFAIL_POINT_RETURN, KFAIL_POINT_RETURN_VOID, KFAIL_POINT_ERROR, KFAIL_POINT_GOTO, fail_point, FP_KERN -- fail points SYNOPSIS #include KFAIL_POINT_CODE(parent, name, code); KFAIL_POINT_RETURN(parent, name); KFAIL_POINT_RETURN_VOID(parent, name); KFAIL_POINT_ERROR(parent, name, error_var); KFAIL_POINT_GOTO(parent, name, error_var, label); DESCRIPTION Fail points are used to add code points where errors may be injected in a user controlled fashion. Fail points provide a convenient wrapper around user provided error injection code, providing a sysctl(9) MIB, and a parser for that MIB that describes how the error injection code should fire. The base fail point macro is KFAIL_POINT_CODE() where parent is a sysctl tree (frequently FP_KERN for kernel fail points, but various subsystems may wish to provide their own fail point trees), and name is the name of the MIB in that tree, and code is the error injection code. The code argument does not require braces, but it is considered good style to use braces for any multi-line code arguments. Inside the code argument, the evaluation of RETURN_VALUE is derived from the return() value set in the sysctl MIB. See SYSCTL SETTINGS below. The remaining KFAIL_POINT_*() macros are wrappers around common error injection paths: KFAIL_POINT_RETURN(parent, name) is the equivalent of KFAIL_POINT_CODE(..., return RETURN_VALUE) KFAIL_POINT_RETURN_VOID(parent, name) is the equivalent of KFAIL_POINT_CODE(..., return) KFAIL_POINT_ERROR(parent, name, error_var) is the equivalent of KFAIL_POINT_CODE(..., error_var = RETURN_VALUE) KFAIL_POINT_GOTO(parent, name, error_var, label) is the equivalent of KFAIL_POINT_CODE(..., { error_var = RETURN_VALUE; goto label;}) SYSCTL VARIABLES The KFAIL_POINT_*() macros add sysctl MIBs where specified. Many base kernel MIBs can be found in the fail_point.kern tree (referenced in code by FP_KERN ). The sysctl setting recognizes the following grammar: :: ( "->" )* :: ( ( "%") | ( "*" ) )* [ "(" ")" ] :: [ "." ] | "." :: "off" | "return" | "sleep" | "panic" | "break" | "print" The argument specifies which action to take: off Take no action (does not trigger fail point code) return Trigger fail point code with specified argument panic Panic break Break into the debugger. print Print that the fail point executed The % and * modifiers prior to control when is executed. The % form (e.g. "1.2%") can be used to specify a probability that will execute. The * form (e.g. "5*") can be used to specify the number of times should be executed before this is disabled. Only the last probability and the last count are used if multiple are specified, i.e. "1.2%2%" is the same as "2%". When both a probability and a count are specified, the probability is evalu- ated before the count, i.e. "2%5*" means "2% of the time, but only exe- cute it 5 times total". The operator -> can be used to express cascading terms. If you specify ->, it means that if doesn't 'execute', is evaluated. For the purpose of this operator, the return() and print() operators are the only types that cascade. A return() term only cascades if the code executes, and a print() term only cascades when passed a non- zero argument. EXAMPLES sysctl fail_point.kern.foobar="2.1%return(5)" 21/1000ths of the time, execute code with RETURN_VALUE set to 5. sysctl fail_point.kern.foobar="2%return(5)->5%return(22)" 2/100th of the time, execute code with RETURN_VALUE set to 5. If that doesn't happen, 5% of the time execute code with RETURN_VALUE set to 22. sysctl fail_point.kern.foobar="5*return(5)->0.1%return(22)" For 5 times, return 5. After that, 1/1000ths of the time, return 22. sysctl fail_point.kern.foobar="0.1%5*return(5)" Return 5 for 1 in 1000 executions, but only execute 5 times total. CAVEATS It's easy to shoot yourself in the foot by setting fail points too aggressively or setting too many in combination. For example, forcing malloc() to fail consistently is potentially harmful to uptime. The sleep() sysctl setting may not be appropriate in all situations. Cur- rently, fail_point_eval() does not verify whether the context is appro- priate for calling msleep(). FreeBSD 8.0 May 10, 2009 FreeBSD 8.0 --rmUrFcWP4LYae1gV-- From owner-freebsd-arch@FreeBSD.ORG Mon May 11 16:35:44 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 634B61065670 for ; Mon, 11 May 2009 16:35:44 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (hergotha.csail.mit.edu [66.92.79.170]) by mx1.freebsd.org (Postfix) with ESMTP id 0F3788FC1C for ; Mon, 11 May 2009 16:35:43 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.2/8.14.2) with ESMTP id n4BGZeIZ066201; Mon, 11 May 2009 12:35:40 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.2/8.13.8/Submit) id n4BGZerd066198; Mon, 11 May 2009 12:35:40 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18952.21468.748665.878710@hergotha.csail.mit.edu> Date: Mon, 11 May 2009 12:35:40 -0400 From: Garrett Wollman To: Lothar Scholz In-Reply-To: <1393224851.20090511112537@scriptolutions.com> References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> <7710650619.20090510075706@scriptolutions.com> <18950.63671.323324.756287@hergotha.csail.mit.edu> <1393224851.20090511112537@scriptolutions.com> X-Mailer: VM 7.17 under 21.4 (patch 21) "Educational Television" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (hergotha.csail.mit.edu [127.0.0.1]); Mon, 11 May 2009 12:35:41 -0400 (EDT) X-Spam-Status: No, score=-1.4 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on hergotha.csail.mit.edu Cc: arch@freebsd.org Subject: Re: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 16:35:46 -0000 < said: > Some idiots started to think about this as a file path. But it isn't > and it shouldn't. Actually, it really should be. Ask a security person or a virtualization person to explain why an unnecessary multiplicity of namespaces is a bad idea. > If i have to find a useable filefile location anyway the whole > function does not make any sense, then i can directly use mmap. But of course you won't get the same behavior, because open()/mmap() guarantees that the backing store will get updated. That's why there's a separate interface. -GAWollman From owner-freebsd-arch@FreeBSD.ORG Mon May 11 16:49:35 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 082221065677 for ; Mon, 11 May 2009 16:49:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id CD5FC8FC18 for ; Mon, 11 May 2009 16:49:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 6BB3046B2E; Mon, 11 May 2009 12:49:34 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 163938A025; Mon, 11 May 2009 12:49:33 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org, Lothar Scholz Date: Mon, 11 May 2009 08:40:32 -0400 User-Agent: KMail/1.9.7 References: <976698487.20090509182307@scriptolutions.com> <123863765.20090509203610@scriptolutions.com> In-Reply-To: <123863765.20090509203610@scriptolutions.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200905110840.32980.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 11 May 2009 12:49:33 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=BAYES_00, DATE_IN_PAST_03_06, RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: pluknet Subject: Re: Are named posix semaphores not implemented? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 16:49:35 -0000 On Saturday 09 May 2009 2:36:10 pm Lothar Scholz wrote: > Hello pluknet, > > > p> First, you should have sem(4) capacity enabled in kernel > p> (via kldload or statically built). It seems you haven't. > > Yes - but it was a total fresh PC-BSD build. > > I must say that i really don't like this start off with > anything disabled. They will never get a good desktop os > if the user can't run simple programs without the need to > learn about kernel modules. > > Well this is the wrong list to discuss this anyway. They are disabled by default because they used to be badly broken. They have been overhauled in 7.2 and perhaps should be enabled by default now. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon May 11 16:49:36 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C7AB1065674 for ; Mon, 11 May 2009 16:49:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D10DC8FC13 for ; Mon, 11 May 2009 16:49:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 75C5546B38; Mon, 11 May 2009 12:49:35 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 60F358A026; Mon, 11 May 2009 12:49:34 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org, Lothar Scholz Date: Mon, 11 May 2009 08:43:22 -0400 User-Agent: KMail/1.9.7 References: <18950.63671.323324.756287@hergotha.csail.mit.edu> <1393224851.20090511112537@scriptolutions.com> In-Reply-To: <1393224851.20090511112537@scriptolutions.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200905110843.22543.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 11 May 2009 12:49:34 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=BAYES_00, DATE_IN_PAST_03_06, RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Garrett Wollman Subject: Re: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 16:49:36 -0000 On Monday 11 May 2009 5:25:37 am Lothar Scholz wrote: > Some idiots started to think about this as a file path. But it isn't > and it shouldn't. Thats what this spec is saying in the typical commitee > polite form when some members made a mistake but are to important to > be blamed in public. Hmm, why don't you head down to your local bookstore and buy a copy of "Solaris Internals" and then come back and explain to us all how the developers of Solaris are idiots. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue May 12 03:50:26 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52185106566B for ; Tue, 12 May 2009 03:50:26 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.239]) by mx1.freebsd.org (Postfix) with ESMTP id 3211E8FC1B for ; Tue, 12 May 2009 03:50:26 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by rv-out-0506.google.com with SMTP id l9so2107055rvb.3 for ; Mon, 11 May 2009 20:50:25 -0700 (PDT) Received: by 10.141.50.11 with SMTP id c11mr3376260rvk.139.1242098945761; Mon, 11 May 2009 20:29:05 -0700 (PDT) Received: from ?10.0.1.198? (udp016664uds.hawaiiantel.net [72.235.41.117]) by mx.google.com with ESMTPS id b39sm117304rvf.3.2009.05.11.20.29.03 (version=SSLv3 cipher=RC4-MD5); Mon, 11 May 2009 20:29:04 -0700 (PDT) Date: Mon, 11 May 2009 17:32:17 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: arch@freebsd.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Cc: Subject: lockless file descriptor lookup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 03:50:26 -0000 http://people.freebsd.org/~jeff/locklessfd.diff This patch implements a lockless lookup path for file descriptors. The meat of the algorithm is in fget_unlocked(). This returns a referenced file descriptor, unlike fget_locked(). In the common case this reduces the number of atomics required for fget() while allowing for lookups to proceed concurrently with modifications to the table and preventing preemption from causing context switches. Using the libMicro 4.0 benchmarking suite with a thread count of 16 on an 8core box yields improvements by as much as 428% in descriptor heavy tests. There were no performance regressions with this benchmark. The code works by allowing lookup threads to follow two previously unsafe pointers. First, the file descriptor table itself is never freed on expansion until the process exits. That ensures that no pagefaults or random memory access can occur if expansion happens after the table pointer is fetched. Given that the vast majority of processes never expand their descriptor table, it is not any significant memory overhead to save them. I shamelessly stole this idea from NetBSD. The struct files themselves are marked as UMA_ZONE_NOFREE and never reclaimed. This allows us to safely attempt to reference count them without any locks. To prevent fdrop() races fget_unlocked() uses a cmpset loop to ensure that it never raises the reference count above zero. In this way it can never reference a free'd or recently allocated file. Once the file descriptor is resolved, we verify the path via the descriptor table once more to ensure that it has not changed. At this point, we have a valid reference or we drop an invalid reference and retry. This gives us the overhead of only one atomic instruction for common case file access. In the worst case there can be some spinning in the loop in fget_unlocked(), but some thread always makes forward progress for each iteration of the loop. I'm going to see if the usual suspects will stress test this but I'd like to see it in 8.0. This is your chance to make any counter arguments. I'd also appreciate it if someone could look at my volatile cast and make sure I'm actually forcing the compiler to refresh the fd_ofiles array here: + if (fp == ((struct file *volatile*)fdp->fd_ofiles)[fd]) Thanks, Jeff From owner-freebsd-arch@FreeBSD.ORG Tue May 12 09:49:24 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FBF3106564A for ; Tue, 12 May 2009 09:49:22 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail16.syd.optusnet.com.au (mail16.syd.optusnet.com.au [211.29.132.197]) by mx1.freebsd.org (Postfix) with ESMTP id ADDFD8FC12 for ; Tue, 12 May 2009 09:49:21 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c122-106-216-167.belrs3.nsw.optusnet.com.au [122.106.216.167]) by mail16.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n4C9nHdr007747 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 12 May 2009 19:49:19 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id n4C9nHn8041899; Tue, 12 May 2009 19:49:17 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id n4C9nGJv041898; Tue, 12 May 2009 19:49:16 +1000 (EST) (envelope-from peter) Date: Tue, 12 May 2009 19:49:16 +1000 From: peterjeremy@optushome.com.au To: Lothar Scholz Message-ID: <20090512094916.GA41857@server.vk2pj.dyndns.org> References: <976698487.20090509182307@scriptolutions.com> <123863765.20090509203610@scriptolutions.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XsQoSWH+UP9D9v3l" Content-Disposition: inline In-Reply-To: <123863765.20090509203610@scriptolutions.com> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.19 (2009-01-05) Cc: freebsd-arch@freebsd.org Subject: Re: Re[2]: Are named posix semaphores not implemented? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 09:49:24 -0000 --XsQoSWH+UP9D9v3l Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-May-09 20:36:10 +0200, Lothar Scholz wr= ote: >I must say that i really don't like this start off with >anything disabled. OTOH, people run FreeBSD because they don't want a default configuration that is bloated by lots of "features" that they will never use and (in some cases) reduce system performance. > They will never get a good desktop os >if the user can't run simple programs without the need to >learn about kernel modules. This depends on your definition of "simple". I've been running FreeBSD for over 10 years and I don't think I've ever needed sem(4). >Well this is the wrong list to discuss this anyway. It might be relevant if you want to propose the default inclusion of sem(4) into GENERIC (though I'm not sure if PC-BSD is a FreeBSD GENERIC kernel or one that has been adapted for PC-BSD). --=20 Peter Jeremy --XsQoSWH+UP9D9v3l Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAkoJRhwACgkQ/opHv/APuIe9FQCfUV29OTMaro3N1oIMneU66LR2 SVwAnRaUynvm1C0RPj4uIvglDAugoXCh =SyLO -----END PGP SIGNATURE----- --XsQoSWH+UP9D9v3l-- From owner-freebsd-arch@FreeBSD.ORG Tue May 12 10:55:40 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3386106564A for ; Tue, 12 May 2009 10:55:40 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 7FCA68FC1D for ; Tue, 12 May 2009 10:55:40 +0000 (UTC) (envelope-from des@des.no) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 6DC586D41B; Tue, 12 May 2009 12:55:39 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id 33A5B844C0; Tue, 12 May 2009 12:55:39 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Lothar Scholz References: <200905100500.n4A50GOa050728@hergotha.csail.mit.edu> <7710650619.20090510075706@scriptolutions.com> <18950.63671.323324.756287@hergotha.csail.mit.edu> <1393224851.20090511112537@scriptolutions.com> <74fe56020905110410y430bf76yacf5c5a308a99865@mail.gmail.com> <981850520.20090511172605@scriptolutions.com> Date: Tue, 12 May 2009 12:55:39 +0200 In-Reply-To: <981850520.20090511172605@scriptolutions.com> (Lothar Scholz's message of "Mon, 11 May 2009 17:26:05 +0200") Message-ID: <86fxfa615g.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.92 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org, Sujit K M , Garrett Wollman Subject: Re: Posix shared memory problem X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 10:55:41 -0000 Lothar Scholz writes: > Sujit K M writes: > > What needs to be fixed? Could you be more specific? > That the name argument is just that "a name" (in its own name space) > not a path. Allow me to quote from the SUSv3 again: DESCRIPTION The shm_open() function shall establish a connection between a shared memory object and a file descriptor. [...] The name argument points to a string naming a shared memory object. It is unspecified whether the name appears in the file system and is visible to other functions that take pathnames as arguments. The name argument conforms to the construction rules for a pathname. [...] RATIONALE [...] Note that such shared memory objects can actually be implemented as mapped files. In both cases, the size can be set after the open using ftruncate(). The shm_open() function itself does not create a shared object of a specified size because this would duplicate an extant function that set the size of an object referenced by a file descriptor. On implementations where memory objects are implemented using the existing file system, the shm_open() function may be implemented using a macro that invokes open(), and the shm_unlink() function may be implemented using a macro that invokes unlink(). [...] DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Tue May 12 11:02:50 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9F9D1065696 for ; Tue, 12 May 2009 11:02:50 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 7E1928FC0A for ; Tue, 12 May 2009 11:02:50 +0000 (UTC) (envelope-from des@des.no) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id B87166D449; Tue, 12 May 2009 13:02:49 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id 9D87D844BD; Tue, 12 May 2009 13:02:49 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Jeff Roberson References: Date: Tue, 12 May 2009 13:02:49 +0200 In-Reply-To: (Jeff Roberson's message of "Mon, 11 May 2009 17:32:17 -1000 (HST)") Message-ID: <86bppy60ti.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.92 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org Subject: Re: lockless file descriptor lookup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 11:02:51 -0000 Jeff Roberson writes: > I'd also appreciate it if someone could look at my volatile cast and > make sure I'm actually forcing the compiler to refresh the fd_ofiles > array here: > > + if (fp =3D=3D ((struct file *volatile*)fdp->fd_ofiles)[fd]) The problem is that since it is not declared as volatile, some other piece of code may have modified it but not yet flushed it to RAM. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Tue May 12 15:48:13 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2016C106566B for ; Tue, 12 May 2009 15:48:13 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id EBB0C8FC13 for ; Tue, 12 May 2009 15:48:12 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 9FD5B46B03 for ; Tue, 12 May 2009 11:48:12 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 83B4E8A028 for ; Tue, 12 May 2009 11:48:11 -0400 (EDT) From: John Baldwin To: arch@FreeBSD.org Date: Tue, 12 May 2009 10:20:18 -0400 User-Agent: KMail/1.9.7 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200905121020.18497.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 12 May 2009 11:48:11 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Subject: Remove d_thread_t for 8.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 15:48:13 -0000 In the same vein as purging BURN_BRIDGES stuff, is there any objection to removing d_thread_t from 8.0? It is intended as a compat shim to reduce diffs with 4.x. However, at this point drivers are not actively being merged back to 4.x, so I think it is no longer necessary. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue May 12 16:12:34 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D960F106567C for ; Tue, 12 May 2009 16:12:34 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from proxy.meer.net (proxy.meer.net [64.13.141.13]) by mx1.freebsd.org (Postfix) with ESMTP id B3D848FC0C for ; Tue, 12 May 2009 16:12:34 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [64.13.141.3]) by proxy.meer.net (8.14.3/8.14.3) with ESMTP id n4CFjVcr012625; Tue, 12 May 2009 08:45:31 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id n4CFj06m037115; Tue, 12 May 2009 08:45:00 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from gnnmac.hudson-trading.com (209.249.190.8.available.above.net [209.249.190.8] (may be forged)) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.3) with ESMTP id n4CFixLM064874 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 12 May 2009 08:45:00 -0700 (PDT) (envelope-from gnn@neville-neil.com) Message-Id: <62B236A8-E303-4200-A8E4-CFF6C022875D@neville-neil.com> From: George Neville-Neil To: Zachary Loafman In-Reply-To: <20090511162928.GD17203@isilon.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 12 May 2009 11:44:58 -0400 References: <20090511162928.GD17203@isilon.com> X-Mailer: Apple Mail (2.930.3) X-Spam-Score: undef - spam scanning disabled X-CanIt-Geo: ip=64.13.141.3; country=US; region=CA; city=Mountain View; latitude=37.3974; longitude=-122.0732; metrocode=807; areacode=650; http://maps.google.com/maps?q=37.3974,-122.0732&z=6 X-CanItPRO-Stream: default X-Canit-Stats-ID: Bayes signature not available X-Scanned-By: CanIt (www . roaringpenguin . com) on 64.13.141.13 Cc: arch@freebsd.org Subject: Re: FAIL: kernel fault injection X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 16:12:35 -0000 On May 11, 2009, at 12:29 , Zachary Loafman wrote: > Arch - > > I'd like to contribute the kernel fault injection system that Isilon > uses. Before contributing it, I'd like to get approval for the APIs > involved. > > Testing errors is hard. Let's say you have: > > int foo(void) { > [...] > error = bar(); > if (error) { > /* do stuff */ > } > } > > .. but some_func() can't reliably be made to fail. How do you test it? > We added error injection macros that look like this: > > int foo(void) { > [...] > error = bar(); > KFAIL_POINT_CODE(FP_KERN, bar_fails_foo, error = RETURN_VALUE); > if (error) { > /* do stuff */ > } > } > > The KFAIL_POINT_CODE macro adds a sysctl MIB that allows > you to inject errors into the above code. For example: > > # sysctl fail_point.kern.bar_fails_foo=".1%return(5)" > > This says, ".1% of the time, evaluate the fail point code with 5 as > the RETURN_VALUE". If this were a standard errno, you could read the > above setting as "1/1000th of the time, pretend bar() returned EIO". > > We also have a few wrappers around KFAIL_POINT_CODE that essentially > wrap common uses. For example, the above use can be shorthanded to: > KFAIL_POINT_ERROR(FP_KERN, bar_fails_foo, error) > > Currently, the sysctl parser accepts the following variants: > return(x) - triggers the code with RETURN_VALUE set to x > sleep(t) - sleep t milliseconds, > panic/break - panic or break into the debugger > print - print that the fail point was hit > > In addition to the commands, we have a syntax to express the > when to evaluate those commands: > p% - evaluate command p% of the time (example above) > 5* - evaluate command 5 times, then disable the expression > > And you can compound with expr1->expr2, so, e.g.: > 5%return(5)->1%return(22): > 5% of the time, return 5, 1% of the remaining time, return 22 > 5*return(0)->10*return(5)->1%return(19) > return 0 for 5 times, then 5 for 10 times, and after those, > return 19 1% of the time. > 1%5*return(22): > 1/100th of the time, return 22, but only do it 5 times total. > > I've also attached an ascii rendering of a (rough draft) man page that > goes into more detail. > > Comments? > Hi Zach, I've taken a brief look at the email and the man page you have sent. I don't see any glaring problems that would prevent us from using this code. Hopefully others will also see its usefulness. Any idea how soon you'd like to commit this? It would be great to get it in before the 8.0 branch so that the APIs are available throughout the duration of that branch, and then moving forwards. Best, George From owner-freebsd-arch@FreeBSD.ORG Tue May 12 16:19:48 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97B67106567C for ; Tue, 12 May 2009 16:19:48 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.155]) by mx1.freebsd.org (Postfix) with ESMTP id 179DF8FC15 for ; Tue, 12 May 2009 16:19:47 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: by fg-out-1718.google.com with SMTP id e12so764293fga.12 for ; Tue, 12 May 2009 09:19:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-pgp-agent:x-mailer; bh=heupCCtl1vL/rOs5uAxKGbpAXX8S8dOJe8yTmU03XQw=; b=r1T/pW9Gp2aXNbfxa5ua4PO5Vemzx9aBHpzflPbgcrE/kOeYSMs+y9wSeLBCgm5k9V 8bk339L0XHH6xThY3Pxh9DMKJxss/ZS4OJ/NHHMxrzWko3UCdZPD/+dXFGqYvlHHpu3L 7aVmRwvubmD7QznJIkTL4xwoWBOnaxe8JOdTg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-pgp-agent:x-mailer; b=ufJdiupbw0PeAoENbss/E62/49aVNprxxOm4nb1Zvr9BdBqFLnZ3z22TB8bG6qkzjX PLHNqei3XP27hBhZt7vW3JwZWYSFHspvUx7vVftYipImQGT1kBIZ4yJqrgA/P0tt+OAP cixD3xAwqkVVvF9UKm5OpYT8xPn9whdNr52xg= Received: by 10.86.4.7 with SMTP id 7mr7921054fgd.46.1242143316753; Tue, 12 May 2009 08:48:36 -0700 (PDT) Received: from epsilon.lan (bl9-154-171.dsl.telepac.pt [85.242.154.171]) by mx.google.com with ESMTPS id e11sm1238922fga.16.2009.05.12.08.48.35 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 12 May 2009 08:48:36 -0700 (PDT) Sender: Rui Paulo Message-Id: From: Rui Paulo To: Zachary Loafman In-Reply-To: <20090511162928.GD17203@isilon.com> Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-1--307476871" Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 12 May 2009 15:56:51 +0100 References: <20090511162928.GD17203@isilon.com> X-Pgp-Agent: GPGMail 1.2.0 (v56) X-Mailer: Apple Mail (2.930.3) Cc: arch@freebsd.org Subject: Re: FAIL: kernel fault injection X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 16:19:49 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --Apple-Mail-1--307476871 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit On 11 May 2009, at 17:29, Zachary Loafman wrote: > Arch - > > I'd like to contribute the kernel fault injection system that Isilon > uses. Before contributing it, I'd like to get approval for the APIs > involved. > > Testing errors is hard. Let's say you have: > > int foo(void) { > [...] > error = bar(); > if (error) { > /* do stuff */ > } > } > > .. but some_func() can't reliably be made to fail. How do you test it? > We added error injection macros that look like this: > > int foo(void) { > [...] > error = bar(); > KFAIL_POINT_CODE(FP_KERN, bar_fails_foo, error = RETURN_VALUE); > if (error) { > /* do stuff */ > } > } > > The KFAIL_POINT_CODE macro adds a sysctl MIB that allows > you to inject errors into the above code. For example: > > # sysctl fail_point.kern.bar_fails_foo=".1%return(5)" > > This says, ".1% of the time, evaluate the fail point code with 5 as > the RETURN_VALUE". If this were a standard errno, you could read the > above setting as "1/1000th of the time, pretend bar() returned EIO". > > We also have a few wrappers around KFAIL_POINT_CODE that essentially > wrap common uses. For example, the above use can be shorthanded to: > KFAIL_POINT_ERROR(FP_KERN, bar_fails_foo, error) > > Currently, the sysctl parser accepts the following variants: > return(x) - triggers the code with RETURN_VALUE set to x > sleep(t) - sleep t milliseconds, > panic/break - panic or break into the debugger > print - print that the fail point was hit > > In addition to the commands, we have a syntax to express the > when to evaluate those commands: > p% - evaluate command p% of the time (example above) > 5* - evaluate command 5 times, then disable the expression > > And you can compound with expr1->expr2, so, e.g.: > 5%return(5)->1%return(22): > 5% of the time, return 5, 1% of the remaining time, return 22 > 5*return(0)->10*return(5)->1%return(19) > return 0 for 5 times, then 5 for 10 times, and after those, > return 19 1% of the time. > 1%5*return(22): > 1/100th of the time, return 22, but only do it 5 times total. > > I've also attached an ascii rendering of a (rough draft) man page that > goes into more detail. > > Comments? This is great and I would like to see this go in. I just have to minor modifications (possible bikeshed, but whatever): * What about kern.fail_point instead of fail_point.kern ? This framework seems to be only for kernel. * On the man page, you don't explain the 'sleep' type. Is that on purpose? About the CAVEAT section on the man page (second paragraph), do you have any ideas to evaluate if msleep is being called on a correct context? Thanks. -- Rui Paulo --Apple-Mail-1--307476871 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iEYEARECAAYFAkoJjjMACgkQfD8M/ASTygKtDQCgxt4+1uCRY3M7RBL4OqU8JskO 6TQAoKuaC6EEb/YAO19qd6YWlAmUMLnP =H7fV -----END PGP SIGNATURE----- --Apple-Mail-1--307476871-- From owner-freebsd-arch@FreeBSD.ORG Tue May 12 16:26:39 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 83C77106566C for ; Tue, 12 May 2009 16:26:39 +0000 (UTC) (envelope-from zachary.loafman@isilon.com) Received: from seaxch10.isilon.com (seaxch10.isilon.com [74.85.160.26]) by mx1.freebsd.org (Postfix) with ESMTP id 65BEF8FC14 for ; Tue, 12 May 2009 16:26:39 +0000 (UTC) (envelope-from zachary.loafman@isilon.com) Received: from famine.isilon.com ([10.54.190.95]) by seaxch10.isilon.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 12 May 2009 09:26:39 -0700 Received: from zloafman by famine.isilon.com with local (Exim 4.69) (envelope-from ) id 1M3uoJ-0003r6-69; Tue, 12 May 2009 09:26:31 -0700 Date: Tue, 12 May 2009 09:26:31 -0700 From: Zachary Loafman To: Rui Paulo Message-ID: <20090512162630.GC7250@isilon.com> References: <20090511162928.GD17203@isilon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.18 (2008-05-17) X-OriginalArrivalTime: 12 May 2009 16:26:39.0458 (UTC) FILETIME=[6D64EC20:01C9D31E] Cc: arch@freebsd.org Subject: Re: FAIL: kernel fault injection X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 16:26:39 -0000 On Tue, May 12, 2009 at 03:56:51PM +0100, Rui Paulo wrote: > On 11 May 2009, at 17:29, Zachary Loafman wrote: > This is great and I would like to see this go in. I just have to minor > modifications (possible bikeshed, but whatever): > * What about kern.fail_point instead of fail_point.kern ? This framework > seems to be only for kernel. It's only for the kernel, but loadable modules can use it as well. It's kind of a question of whether you want the available fail points in one tree, but away from their respective namespace, or whether you want them in the tree associated with their namespace. Thinking about it more, kern.fail_point might make more sense - I like seeing testing related things grouped with the thing its testing. > * On the man page, you don't explain the 'sleep' type. Is that on > purpose? Oversight, I'll include it in the final. > About the CAVEAT section on the man page (second paragraph), do you have > any ideas to evaluate if msleep is being called on a correct context? I haven't actually thought about it too long. If we factored out the "for (lle = td->td_sleeplocks ..." section from witness_warn, we could potentially make the sleepable lock state. But witness isn't always enabled. Nor are sleepable locks the only real issue with msleep()ing. If you build the fail point manually, you can add fail points that invoke a defined sleep function on timeout() (instead of just calling msleep). You can build fail points manually using a set of lower level fail_point_* routines, one of which is fail_point_set_sleep_fn. I haven't had a chance to write up the man page for those yet. Thanks for the comments! ...Zach From owner-freebsd-arch@FreeBSD.ORG Tue May 12 16:38:40 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26E0B106567E for ; Tue, 12 May 2009 16:38:40 +0000 (UTC) (envelope-from zachary.loafman@isilon.com) Received: from seaxch10.isilon.com (seaxch10.isilon.com [74.85.160.26]) by mx1.freebsd.org (Postfix) with ESMTP id F408B8FC0C for ; Tue, 12 May 2009 16:38:39 +0000 (UTC) (envelope-from zachary.loafman@isilon.com) Received: from famine.isilon.com ([10.54.190.95]) by seaxch10.isilon.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 12 May 2009 09:35:15 -0700 Received: from zloafman by famine.isilon.com with local (Exim 4.69) (envelope-from ) id 1M3uwc-0005LV-Pi; Tue, 12 May 2009 09:35:06 -0700 Date: Tue, 12 May 2009 09:35:06 -0700 From: Zachary Loafman To: George Neville-Neil Message-ID: <20090512163506.GE7250@isilon.com> References: <20090511162928.GD17203@isilon.com> <62B236A8-E303-4200-A8E4-CFF6C022875D@neville-neil.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <62B236A8-E303-4200-A8E4-CFF6C022875D@neville-neil.com> User-Agent: Mutt/1.5.18 (2008-05-17) X-OriginalArrivalTime: 12 May 2009 16:35:15.0099 (UTC) FILETIME=[A0BD7EB0:01C9D31F] Cc: arch@freebsd.org Subject: Re: FAIL: kernel fault injection X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 16:38:40 -0000 On Tue, May 12, 2009 at 11:44:58AM -0400, George Neville-Neil wrote: > Any idea how soon you'd like to commit this? It would be great to > get it in before the 8.0 branch so that the APIs are available > throughout the duration of that branch, and then moving forwards. I'm pretty much ready to commit once we have API consensus, barring code review issues cropping up. Getting it in before the slush doesn't seem like a real issue. I think it's even MFCable to 7.x, given that it's a completely new set of APIs. It would certainly help any 8.x->7.x merges that included new fail points, though it's not like it's hard to delete them on merge. Just sad, especially if you also write unit tests that use them. ...Zach From owner-freebsd-arch@FreeBSD.ORG Tue May 12 16:59:51 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69DD8106566B for ; Tue, 12 May 2009 16:59:51 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from palm.hoeg.nl (mx0.hoeg.nl [IPv6:2001:7b8:613:100::211]) by mx1.freebsd.org (Postfix) with ESMTP id 099978FC0C for ; Tue, 12 May 2009 16:59:50 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: by palm.hoeg.nl (Postfix, from userid 1000) id ED4521D04A; Tue, 12 May 2009 18:59:49 +0200 (CEST) Date: Tue, 12 May 2009 18:59:49 +0200 From: Ed Schouten To: Jeff Roberson Message-ID: <20090512165949.GF58540@hoeg.nl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GCRJOSQGwEYxR+j1" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.19 (2009-01-05) Cc: arch@freebsd.org Subject: Re: lockless file descriptor lookup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 16:59:51 -0000 --GCRJOSQGwEYxR+j1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello Jeff, * Jeff Roberson wrote: > Once the file descriptor is resolved, we verify the path via the =20 > descriptor table once more to ensure that it has not changed. At this = =20 > point, we have a valid reference or we drop an invalid reference and =20 > retry. It's nice to see someone stepped up to implement this. Just out of curiosity, have you done any benchmarks to see how many percent of the time a thread needs more than one attempt to obtain a valid reference on a common workload? Maybe it would be nice for diagnostic purposes to add two sysctls to obtain the amount of successful and unsuccessful attempts. --=20 Ed Schouten WWW: http://80386.nl/ --GCRJOSQGwEYxR+j1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iEYEARECAAYFAkoJqwUACgkQ52SDGA2eCwWFFgCfX+ZV7n/CbAGIx9OhQSQNqE73 KnYAniCy0fesWrS7ByA/efWM/gKrF3nT =WnZE -----END PGP SIGNATURE----- --GCRJOSQGwEYxR+j1-- From owner-freebsd-arch@FreeBSD.ORG Tue May 12 17:10:17 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA385106566B for ; Tue, 12 May 2009 17:10:17 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: from mail-ew0-f159.google.com (mail-ew0-f159.google.com [209.85.219.159]) by mx1.freebsd.org (Postfix) with ESMTP id 4FFD08FC15 for ; Tue, 12 May 2009 17:10:16 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: by ewy3 with SMTP id 3so135750ewy.43 for ; Tue, 12 May 2009 10:10:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-pgp-agent:x-mailer; bh=Z3zBPP9bQxZnn6HEUNMgCOyUClDF90wvzdtBMNU/vPY=; b=LgRmEaYqcFaKPYkUCUThUFvDmr7NnExwhkDyJUqnRWJ2lffbKFR82+PAPO+Ix2HlGQ eJjmlUL6Em/d2esE1jmnk1bHZV15lrO6wgo1gapN7ARWObhFyCmBwkZX1GAa/lZ+KKDK NgJN4+isQo7MgK6Doi6sVC80INcTAmfrF74tw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-pgp-agent:x-mailer; b=mRNWyAiTlh1cASmSTUlbwC0QI4E5OkqN6/0c4ehEON4jhP7GfSeMq3z1Lt1tOvB71g 5uKuz3DiRqoKA1ABfJh/sflKuxWOSLQMZmH5GxXGRDthcj1Wd3AHdOQbKDTLFyqynDyG vsvWR4r1KkpE7s7PZbXXKirZ7d40gAdXWNfkg= Received: by 10.216.46.83 with SMTP id q61mr4071212web.71.1242148215442; Tue, 12 May 2009 10:10:15 -0700 (PDT) Received: from epsilon.lan (bl6-154-7.dsl.telepac.pt [82.155.154.7]) by mx.google.com with ESMTPS id 7sm580490eyb.15.2009.05.12.10.10.14 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 12 May 2009 10:10:14 -0700 (PDT) Message-Id: From: Rui Paulo To: Zachary Loafman In-Reply-To: <20090512163506.GE7250@isilon.com> Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-1--299478854" Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 12 May 2009 18:10:09 +0100 References: <20090511162928.GD17203@isilon.com> <62B236A8-E303-4200-A8E4-CFF6C022875D@neville-neil.com> <20090512163506.GE7250@isilon.com> X-Pgp-Agent: GPGMail 1.2.0 (v56) X-Mailer: Apple Mail (2.930.3) Cc: arch@freebsd.org, George Neville-Neil Subject: Re: FAIL: kernel fault injection X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 17:10:18 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --Apple-Mail-1--299478854 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit On 12 May 2009, at 17:35, Zachary Loafman wrote: > On Tue, May 12, 2009 at 11:44:58AM -0400, George Neville-Neil wrote: > >> Any idea how soon you'd like to commit this? It would be great to >> get it in before the 8.0 branch so that the APIs are available >> throughout the duration of that branch, and then moving forwards. > > I'm pretty much ready to commit once we have API consensus, barring > code > review issues cropping up. Getting it in before the slush doesn't seem > like a real issue. > > I think it's even MFCable to 7.x, given that it's a completely new set > of APIs. It would certainly help any 8.x->7.x merges that included new > fail points, though it's not like it's hard to delete them on merge. > Just sad, especially if you also write unit tests that use them. Ok, so please send the patch for review whenever you're ready. Regards, -- Rui Paulo --Apple-Mail-1--299478854 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iEYEARECAAYFAkoJrXEACgkQfD8M/ASTygKcUQCgiKpfIos5ps6rdCo6XggmSOuA 74AAoKeVb7yoqU2UCgcpzmxyJ6ZDmSWx =8upC -----END PGP SIGNATURE----- --Apple-Mail-1--299478854-- From owner-freebsd-arch@FreeBSD.ORG Tue May 12 20:01:25 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36AF01065691 for ; Tue, 12 May 2009 20:01:25 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail34.syd.optusnet.com.au (mail34.syd.optusnet.com.au [211.29.133.218]) by mx1.freebsd.org (Postfix) with ESMTP id B952C8FC21 for ; Tue, 12 May 2009 20:01:24 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c122-106-216-167.belrs3.nsw.optusnet.com.au [122.106.216.167]) by mail34.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n4CK1K66025601 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 13 May 2009 06:01:22 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id n4CK1Joh071888; Wed, 13 May 2009 06:01:19 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id n4CK1IGO071887; Wed, 13 May 2009 06:01:18 +1000 (EST) (envelope-from peter) Date: Wed, 13 May 2009 06:01:18 +1000 From: Peter Jeremy To: Andriy Gapon Message-ID: <20090512200118.GC99304@server.vk2pj.dyndns.org> References: <4A01B9A3.2030806@icyb.net.ua> <20090507080048.GA64648@server.vk2pj.dyndns.org> <4A0295E0.4020609@icyb.net.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="adJ1OR3c6QgCpb/j" Content-Disposition: inline In-Reply-To: <4A0295E0.4020609@icyb.net.ua> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.19 (2009-01-05) Cc: freebsd-arch@freebsd.org Subject: Re: shutdown_nice during boot X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 20:01:25 -0000 --adJ1OR3c6QgCpb/j Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-May-07 11:03:44 +0300, Andriy Gapon wrote: >on 07/05/2009 11:00 peterjeremy@optushome.com.au said the following: >> On 2009-May-06 19:24:03 +0300, Andriy Gapon wrote: >>> It's possible to re-enable SIGINT right after init is forked, but >>> this way it will be delivered to init before it installs signal >>> handlers and thus init would simply terminate resulting in "Going >>> nowhere without my init!" panic. >>=20 >> The best option would seem to be for init(8) to call sigprocmask(2) >> immediately it starts up and block all signals. > >But a signal still can be delivered after init is exec-ed and before >sigprocmask(2) is called or not? True - there is still a window there where signal dispositiona are inappropriate. Thinking about it some more, maybe the solution is to change the test in shutdown_nice() - rather than testing for the existence of initproc, it could test a sysctl variable that init sets once it has its signal handlers in place. There's already a kern.shutdown node so maybe "kern.shutdown.via_init". (This adds the option for other subsystems to clear it if desired - maybe as part of a watchdog function). Maybe shutdown_nice should also initiate a 30-60 second timer that invokes boot() whether or not init responds. --=20 Peter Jeremy --adJ1OR3c6QgCpb/j Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAkoJ1Y4ACgkQ/opHv/APuIfpewCdH3zlfO7bSTzs2YidNvNiTgkl LjcAn0H20zDPn66+QMAgMDJqXnVLXdc+ =u/pD -----END PGP SIGNATURE----- --adJ1OR3c6QgCpb/j-- From owner-freebsd-arch@FreeBSD.ORG Wed May 13 00:25:55 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8F6A106568B for ; Wed, 13 May 2009 00:25:55 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.233]) by mx1.freebsd.org (Postfix) with ESMTP id A6D5F8FC1B for ; Wed, 13 May 2009 00:25:55 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by rv-out-0506.google.com with SMTP id k40so245621rvb.43 for ; Tue, 12 May 2009 17:25:55 -0700 (PDT) Received: by 10.141.52.6 with SMTP id e6mr107223rvk.133.1242174355215; Tue, 12 May 2009 17:25:55 -0700 (PDT) Received: from ?10.0.1.198? (udp016664uds.hawaiiantel.net [72.235.41.117]) by mx.google.com with ESMTPS id g14sm1073984rvb.22.2009.05.12.17.25.53 (version=SSLv3 cipher=RC4-MD5); Tue, 12 May 2009 17:25:54 -0700 (PDT) Date: Tue, 12 May 2009 14:29:11 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: =?ISO-8859-15?Q?Dag-Erling_Sm=F8rgrav?= In-Reply-To: <86bppy60ti.fsf@ds4.des.no> Message-ID: References: <86bppy60ti.fsf@ds4.des.no> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org Subject: Re: lockless file descriptor lookup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 00:25:56 -0000 On Tue, 12 May 2009, Dag-Erling Sm?rgrav wrote: > Jeff Roberson writes: >> I'd also appreciate it if someone could look at my volatile cast and >> make sure I'm actually forcing the compiler to refresh the fd_ofiles >> array here: >> >> + if (fp == ((struct file *volatile*)fdp->fd_ofiles)[fd]) > > The problem is that since it is not declared as volatile, some other > piece of code may have modified it but not yet flushed it to RAM. That is an acceptable race due to other guarantees. If it hasn't been committed to memory yet, the old table still contains valid data. I only need to be certain that the compiler doesn't cache the original ofiles value. It can't anyway because atomics use inline assembly on all platforms but I'd like it to be explicit anyway. Thanks, Jeff > > DES > -- > Dag-Erling Sm?rgrav - des@des.no > From owner-freebsd-arch@FreeBSD.ORG Wed May 13 00:29:08 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7EFA0106564A for ; Wed, 13 May 2009 00:29:08 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.232]) by mx1.freebsd.org (Postfix) with ESMTP id 5DC0B8FC0C for ; Wed, 13 May 2009 00:29:08 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by rv-out-0506.google.com with SMTP id k40so246522rvb.43 for ; Tue, 12 May 2009 17:29:08 -0700 (PDT) Received: by 10.141.106.12 with SMTP id i12mr108956rvm.59.1242174548008; Tue, 12 May 2009 17:29:08 -0700 (PDT) Received: from ?10.0.1.198? (udp016664uds.hawaiiantel.net [72.235.41.117]) by mx.google.com with ESMTPS id l31sm948927rvb.29.2009.05.12.17.29.05 (version=SSLv3 cipher=RC4-MD5); Tue, 12 May 2009 17:29:06 -0700 (PDT) Date: Tue, 12 May 2009 14:32:24 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Ed Schouten In-Reply-To: <20090512165949.GF58540@hoeg.nl> Message-ID: References: <20090512165949.GF58540@hoeg.nl> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org Subject: Re: lockless file descriptor lookup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 00:29:08 -0000 On Tue, 12 May 2009, Ed Schouten wrote: > Hello Jeff, > > * Jeff Roberson wrote: >> Once the file descriptor is resolved, we verify the path via the >> descriptor table once more to ensure that it has not changed. At this >> point, we have a valid reference or we drop an invalid reference and >> retry. > > It's nice to see someone stepped up to implement this. Just out of > curiosity, have you done any benchmarks to see how many percent of the > time a thread needs more than one attempt to obtain a valid reference on > a common workload? > > Maybe it would be nice for diagnostic purposes to add two sysctls to > obtain the amount of successful and unsuccessful attempts. Hi Ed, I have had trouble triggering it at all in testing. I'd prefer not to commit the counters because they would re-introduce a global point of cache contention unless we made them per-cpu. This effectively implements ll/sc semantics on all architectures via cmpset. I suspect the overhead is minimal even in degenerate cases. Thanks, Jeff > > -- > Ed Schouten > WWW: http://80386.nl/ > From owner-freebsd-arch@FreeBSD.ORG Wed May 13 12:42:24 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6F3C106564A for ; Wed, 13 May 2009 12:42:24 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 506FE8FC13 for ; Wed, 13 May 2009 12:42:24 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1M4Dmu-0006Hi-Lx for freebsd-arch@freebsd.org; Wed, 13 May 2009 12:42:20 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 13 May 2009 12:42:20 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 13 May 2009 12:42:20 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Wed, 13 May 2009 14:42:01 +0200 Lines: 44 Message-ID: References: <976698487.20090509182307@scriptolutions.com> <123863765.20090509203610@scriptolutions.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF3F4D2AF80888078ECBDF59A" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.21 (X11/20090409) In-Reply-To: <123863765.20090509203610@scriptolutions.com> X-Enigmail-Version: 0.95.7 Sender: news Subject: Re: Are named posix semaphores not implemented? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 12:42:25 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF3F4D2AF80888078ECBDF59A Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Lothar Scholz wrote: > Hello pluknet, >=20 >=20 > p> First, you should have sem(4) capacity enabled in kernel > p> (via kldload or statically built). It seems you haven't. >=20 > Yes - but it was a total fresh PC-BSD build. For future troubleshooting, you should get used to reading the man pages - sem(4) describes how to enable it. > I must say that i really don't like this start off with > anything disabled. They will never get a good desktop os > if the user can't run simple programs without the need to > learn about kernel modules. They are disabled by default because they don't get much use - apparently it's not a very popular API. If you make a good case for its inclusion (reference large or popular projects that use it), there is probably no reason not to include it by default. The sem.ko loadable module is something like 20 kB in size. --------------enigF3F4D2AF80888078ECBDF59A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoKwCAACgkQldnAQVacBcgDAgCeOC8y+f6OT4LAYKp0UH9SZS/o 75AAoPjwbDX2ZbaoHMiuX4q+QTd9cRWl =Kziz -----END PGP SIGNATURE----- --------------enigF3F4D2AF80888078ECBDF59A-- From owner-freebsd-arch@FreeBSD.ORG Wed May 13 14:27:28 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC3831065675 for ; Wed, 13 May 2009 14:27:28 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id BF7CE8FC1C for ; Wed, 13 May 2009 14:27:28 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 7526C46B52; Wed, 13 May 2009 10:27:28 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 358738A025; Wed, 13 May 2009 10:27:27 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Wed, 13 May 2009 09:35:42 -0400 User-Agent: KMail/1.9.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200905130935.42795.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 13 May 2009 10:27:27 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Subject: Re: lockless file descriptor lookup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 14:27:29 -0000 On Monday 11 May 2009 11:32:17 pm Jeff Roberson wrote: > http://people.freebsd.org/~jeff/locklessfd.diff > > This patch implements a lockless lookup path for file descriptors. The > meat of the algorithm is in fget_unlocked(). This returns a referenced > file descriptor, unlike fget_locked(). In the common case this reduces > the number of atomics required for fget() while allowing for lookups to > proceed concurrently with modifications to the table and preventing > preemption from causing context switches. Looks good. My only comment would be to not remove the 'hold' comment completely from _fget(), but instead say that it always returns a refcount that must be dropped. Basically: * The file's refcount will be bumped on return. It should be dropped * with fdrop(). or something like that in place of the old paragraph about the 'hold' parameter. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu May 14 01:58:00 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BC54106564A for ; Thu, 14 May 2009 01:58:00 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-pz0-f105.google.com (mail-pz0-f105.google.com [209.85.222.105]) by mx1.freebsd.org (Postfix) with ESMTP id DEC578FC22 for ; Thu, 14 May 2009 01:57:59 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by pzk3 with SMTP id 3so456569pzk.3 for ; Wed, 13 May 2009 18:57:59 -0700 (PDT) Received: by 10.114.25.19 with SMTP id 19mr1554400way.89.1242264999287; Wed, 13 May 2009 18:36:39 -0700 (PDT) Received: from ?10.0.1.198? (udp016664uds.hawaiiantel.net [72.235.41.117]) by mx.google.com with ESMTPS id n20sm1521871pof.27.2009.05.13.18.36.37 (version=SSLv3 cipher=RC4-MD5); Wed, 13 May 2009 18:36:38 -0700 (PDT) Date: Wed, 13 May 2009 15:40:01 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: John Baldwin In-Reply-To: <200905130935.42795.jhb@freebsd.org> Message-ID: References: <200905130935.42795.jhb@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: Re: lockless file descriptor lookup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 01:58:00 -0000 On Wed, 13 May 2009, John Baldwin wrote: > On Monday 11 May 2009 11:32:17 pm Jeff Roberson wrote: >> http://people.freebsd.org/~jeff/locklessfd.diff >> >> This patch implements a lockless lookup path for file descriptors. The >> meat of the algorithm is in fget_unlocked(). This returns a referenced >> file descriptor, unlike fget_locked(). In the common case this reduces >> the number of atomics required for fget() while allowing for lookups to >> proceed concurrently with modifications to the table and preventing >> preemption from causing context switches. > > Looks good. My only comment would be to not remove the 'hold' comment > completely from _fget(), but instead say that it always returns a refcount > that must be dropped. Basically: > > * The file's refcount will be bumped on return. It should be dropped > * with fdrop(). > > or something like that in place of the old paragraph about the 'hold' > parameter. Yeah, I think I'll add a comment in the header too. pho has tested it with some targeted tests and stress2 for a day or two. I'm going to commit this evening so it gets as much exposure as possible before release. Thanks, Jeff > > -- > John Baldwin > From owner-freebsd-arch@FreeBSD.ORG Thu May 14 04:47:51 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8D3E106566C for ; Thu, 14 May 2009 04:47:51 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail11.syd.optusnet.com.au (mail11.syd.optusnet.com.au [211.29.132.192]) by mx1.freebsd.org (Postfix) with ESMTP id 861D68FC0A for ; Thu, 14 May 2009 04:47:50 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c122-107-117-19.carlnfd1.nsw.optusnet.com.au [122.107.117.19]) by mail11.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n4E4lgXH016169 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 14 May 2009 14:47:44 +1000 Date: Thu, 14 May 2009 14:47:41 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Jeff Roberson In-Reply-To: Message-ID: <20090514131613.T1224@besplex.bde.org> References: <86bppy60ti.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: =?ISO-8859-15?Q?Dag-Erling_Sm=F8rgrav?= , arch@freebsd.org Subject: Re: lockless file descriptor lookup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 04:47:52 -0000 On Tue, 12 May 2009, Jeff Roberson wrote: > On Tue, 12 May 2009, Dag-Erling Sm?rgrav wrote: > >> Jeff Roberson writes: >>> I'd also appreciate it if someone could look at my volatile cast and >>> make sure I'm actually forcing the compiler to refresh the fd_ofiles >>> array here: >>> >>> + if (fp == ((struct file *volatile*)fdp->fd_ofiles)[fd]) This has 2 style bugs (missing space after first '*' and missing space before second '*'. It isn't clear whether you want to refresh the fd_ofiles pointer to the (first element of) the array, or the fd'th element. It is clear that you don't want to refresh the whole array. The above refreshes the fd'th element. Strangely, in my tests gcc refreshes the fd'th element even without the cast. E.g., test(fdp->fd_ofiles[fd], fdp->fd_ofiles[fd]); results in 1 memory access for each of the [fd]'s. >> The problem is that since it is not declared as volatile, some other >> piece of code may have modified it but not yet flushed it to RAM. > > That is an acceptable race due to other guarantees. If it hasn't been > committed to memory yet, the old table still contains valid data. I only > need to be certain that the compiler doesn't cache the original ofiles value. > It can't anyway because atomics use inline assembly on all platforms but I'd > like it to be explicit anyway. It shouldn't matter that atomics use inline asm. Non-broken inline asm declares all its inputs and outputs, so compilers can see what it changes just as easily as for C code (and more easily than for non- inline asm or C). Anyway, you probably need atomics that have suitable memory barriers. Memory barriers must affect the compiler and make it perform refreshes for them to work, so you shouldn't need any volatile casts. E.g., all atomic store operations (including cmpset) have release semantics even if they aren't spelled with "_rel" or implemented using inline asm. On amd64 and i386, they happen to be implemented using inline asm with "memory" clobbers. The "memory" clobbers force refreshes of all non-local variables. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu May 14 16:42:48 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58EE6106564A; Thu, 14 May 2009 16:42:48 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (mail.cs.rice.edu [128.42.1.31]) by mx1.freebsd.org (Postfix) with ESMTP id 3121E8FC0A; Thu, 14 May 2009 16:42:47 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (localhost.localdomain [127.0.0.1]) by mail.cs.rice.edu (Postfix) with ESMTP id 34B5A2C2C9B; Thu, 14 May 2009 11:42:47 -0500 (CDT) X-Virus-Scanned: by amavis-2.4.0 at mail.cs.rice.edu Received: from mail.cs.rice.edu ([127.0.0.1]) by mail.cs.rice.edu (mail.cs.rice.edu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 6iHK+oOTSeu3; Thu, 14 May 2009 11:42:39 -0500 (CDT) Received: from adsl-216-63-78-18.dsl.hstntx.swbell.net (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.cs.rice.edu (Postfix) with ESMTP id 678162C2C60; Thu, 14 May 2009 11:42:39 -0500 (CDT) Message-ID: <4A0C49FF.1070707@cs.rice.edu> Date: Thu, 14 May 2009 11:42:39 -0500 From: Alan Cox User-Agent: Thunderbird 2.0.0.21 (X11/20090404) MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <86tz3o4lb9.fsf@ds4.des.no> <86prec4kwj.fsf@ds4.des.no> In-Reply-To: <86prec4kwj.fsf@ds4.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: alc@freebsd.org, arch@freebsd.org Subject: Re: PTE modified bit emulation trap X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 16:42:48 -0000 Dag-Erling Smørgrav wrote: > [from -alpha, -hackers] > > Dag-Erling Smørgrav writes: > >> Coverity complains about the lack of error checking in the following >> code in sys/kern/kern_sysctl.c, around line 1390: >> >> /* >> * Touch all the wired pages to avoid PTE modified >> * bit emulation traps on Alpha while holding locks >> * in the sysctl handler. >> */ >> for (i = (wiredlen + PAGE_SIZE - 1) / PAGE_SIZE, >> cp = req->oldptr; i > 0; i--, cp += PAGE_SIZE) { >> copyin(cp, &dummy, 1); >> copyout(&dummy, cp, 1); >> } >> >> Since Alpha is dead, can we remove this, or is it still needed for other >> platforms? >> > > kmacy suggested you might be the right person to ask... the conclusion > so far is that it *might* be necessary on sparc64 and / or mips. > I think that this code may no longer be needed, but I want to double-check. I faced a related problem implementing superpages support, so I introduced an additional "access type" parameter to pmap_enter(). This parameter was specifically intended to allow a pmap_enter() implementation to preset the PTE's modified bit. I think that the simulated page fault that occurs on vslock()-style wiring passes "write access" to pmap_enter(). If so, then it's just a matter of tweaking the MIPS or any other pmap_enter() to actually do something with the "access type" parameter. Currently, only the architectures that implement the pmap-level support for superpages, i.e., amd64 and i386, do anything with this parameter. Alan From owner-freebsd-arch@FreeBSD.ORG Fri May 15 13:11:06 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11D5D1065679; Fri, 15 May 2009 13:11:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D6ECF8FC08; Fri, 15 May 2009 13:11:05 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 89F4146C16; Fri, 15 May 2009 09:11:05 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 3803C8A026; Fri, 15 May 2009 09:11:04 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Fri, 15 May 2009 08:04:36 -0400 User-Agent: KMail/1.9.7 References: <86tz3o4lb9.fsf@ds4.des.no> <86prec4kwj.fsf@ds4.des.no> <4A0C49FF.1070707@cs.rice.edu> In-Reply-To: <4A0C49FF.1070707@cs.rice.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200905150804.36977.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 15 May 2009 09:11:04 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: alc@freebsd.org, Dag-Erling =?utf-8?q?Sm=C3=B8rgrav?= , Alan Cox , arch@freebsd.org Subject: Re: PTE modified bit emulation trap X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 13:11:06 -0000 On Thursday 14 May 2009 12:42:39 pm Alan Cox wrote: > Dag-Erling Sm=C3=B8rgrav wrote: > > [from -alpha, -hackers] > > > > Dag-Erling Sm=C3=B8rgrav writes: > > =20 > >> Coverity complains about the lack of error checking in the following > >> code in sys/kern/kern_sysctl.c, around line 1390: > >> > >> /* > >> * Touch all the wired pages to avoid PTE modified > >> * bit emulation traps on Alpha while holding locks > >> * in the sysctl handler. > >> */ > >> for (i =3D (wiredlen + PAGE_SIZE - 1) / PAGE_SIZE, > >> cp =3D req->oldptr; i > 0; i--, cp +=3D PAGE_SIZE) { > >> copyin(cp, &dummy, 1); > >> copyout(&dummy, cp, 1); > >> } > >> > >> Since Alpha is dead, can we remove this, or is it still needed for oth= er > >> platforms? > >> =20 > > > > kmacy suggested you might be the right person to ask... the conclusion > > so far is that it *might* be necessary on sparc64 and / or mips. > > =20 >=20 > I think that this code may no longer be needed, but I want to=20 > double-check. I faced a related problem implementing superpages=20 > support, so I introduced an additional "access type" parameter to=20 > pmap_enter(). This parameter was specifically intended to allow a=20 > pmap_enter() implementation to preset the PTE's modified bit. I think=20 > that the simulated page fault that occurs on vslock()-style wiring=20 > passes "write access" to pmap_enter(). If so, then it's just a matter=20 > of tweaking the MIPS or any other pmap_enter() to actually do something=20 > with the "access type" parameter. Currently, only the architectures=20 > that implement the pmap-level support for superpages, i.e., amd64 and=20 > i386, do anything with this parameter. Then it sounds like the code should definitely be removed and that if any=20 problems do crop up, they can be fixed in pmap_enter() instead. =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri May 15 13:11:06 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11D5D1065679; Fri, 15 May 2009 13:11:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D6ECF8FC08; Fri, 15 May 2009 13:11:05 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 89F4146C16; Fri, 15 May 2009 09:11:05 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 3803C8A026; Fri, 15 May 2009 09:11:04 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Fri, 15 May 2009 08:04:36 -0400 User-Agent: KMail/1.9.7 References: <86tz3o4lb9.fsf@ds4.des.no> <86prec4kwj.fsf@ds4.des.no> <4A0C49FF.1070707@cs.rice.edu> In-Reply-To: <4A0C49FF.1070707@cs.rice.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200905150804.36977.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 15 May 2009 09:11:04 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: alc@freebsd.org, Dag-Erling =?utf-8?q?Sm=C3=B8rgrav?= , Alan Cox , arch@freebsd.org Subject: Re: PTE modified bit emulation trap X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 13:11:06 -0000 On Thursday 14 May 2009 12:42:39 pm Alan Cox wrote: > Dag-Erling Sm=C3=B8rgrav wrote: > > [from -alpha, -hackers] > > > > Dag-Erling Sm=C3=B8rgrav writes: > > =20 > >> Coverity complains about the lack of error checking in the following > >> code in sys/kern/kern_sysctl.c, around line 1390: > >> > >> /* > >> * Touch all the wired pages to avoid PTE modified > >> * bit emulation traps on Alpha while holding locks > >> * in the sysctl handler. > >> */ > >> for (i =3D (wiredlen + PAGE_SIZE - 1) / PAGE_SIZE, > >> cp =3D req->oldptr; i > 0; i--, cp +=3D PAGE_SIZE) { > >> copyin(cp, &dummy, 1); > >> copyout(&dummy, cp, 1); > >> } > >> > >> Since Alpha is dead, can we remove this, or is it still needed for oth= er > >> platforms? > >> =20 > > > > kmacy suggested you might be the right person to ask... the conclusion > > so far is that it *might* be necessary on sparc64 and / or mips. > > =20 >=20 > I think that this code may no longer be needed, but I want to=20 > double-check. I faced a related problem implementing superpages=20 > support, so I introduced an additional "access type" parameter to=20 > pmap_enter(). This parameter was specifically intended to allow a=20 > pmap_enter() implementation to preset the PTE's modified bit. I think=20 > that the simulated page fault that occurs on vslock()-style wiring=20 > passes "write access" to pmap_enter(). If so, then it's just a matter=20 > of tweaking the MIPS or any other pmap_enter() to actually do something=20 > with the "access type" parameter. Currently, only the architectures=20 > that implement the pmap-level support for superpages, i.e., amd64 and=20 > i386, do anything with this parameter. Then it sounds like the code should definitely be removed and that if any=20 problems do crop up, they can be fixed in pmap_enter() instead. =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri May 15 16:24:16 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2DFFB106566C; Fri, 15 May 2009 16:24:16 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (mail.cs.rice.edu [128.42.1.31]) by mx1.freebsd.org (Postfix) with ESMTP id E4A4B8FC17; Fri, 15 May 2009 16:24:15 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (localhost.localdomain [127.0.0.1]) by mail.cs.rice.edu (Postfix) with ESMTP id 641D42C2B0E; Fri, 15 May 2009 11:24:15 -0500 (CDT) X-Virus-Scanned: by amavis-2.4.0 at mail.cs.rice.edu Received: from mail.cs.rice.edu ([127.0.0.1]) by mail.cs.rice.edu (mail.cs.rice.edu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Z3VFQTY9kGoN; Fri, 15 May 2009 11:24:07 -0500 (CDT) Received: from adsl-216-63-78-18.dsl.hstntx.swbell.net (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.cs.rice.edu (Postfix) with ESMTP id 8474C2C2AD0; Fri, 15 May 2009 11:24:07 -0500 (CDT) Message-ID: <4A0D9727.6030703@cs.rice.edu> Date: Fri, 15 May 2009 11:24:07 -0500 From: Alan Cox User-Agent: Thunderbird 2.0.0.21 (X11/20090404) MIME-Version: 1.0 To: John Baldwin , =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <86tz3o4lb9.fsf@ds4.des.no> <86prec4kwj.fsf@ds4.des.no> <4A0C49FF.1070707@cs.rice.edu> <200905150804.36977.jhb@freebsd.org> In-Reply-To: <200905150804.36977.jhb@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: alc@freebsd.org, arch@freebsd.org, freebsd-arch@freebsd.org Subject: Re: PTE modified bit emulation trap X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 16:24:16 -0000 John Baldwin wrote: > On Thursday 14 May 2009 12:42:39 pm Alan Cox wrote: > >> Dag-Erling Smørgrav wrote: >> >>> [from -alpha, -hackers] >>> >>> Dag-Erling Smørgrav writes: >>> >>> >>>> Coverity complains about the lack of error checking in the following >>>> code in sys/kern/kern_sysctl.c, around line 1390: >>>> >>>> /* >>>> * Touch all the wired pages to avoid PTE modified >>>> * bit emulation traps on Alpha while holding locks >>>> * in the sysctl handler. >>>> */ >>>> for (i = (wiredlen + PAGE_SIZE - 1) / PAGE_SIZE, >>>> cp = req->oldptr; i > 0; i--, cp += PAGE_SIZE) { >>>> copyin(cp, &dummy, 1); >>>> copyout(&dummy, cp, 1); >>>> } >>>> >>>> Since Alpha is dead, can we remove this, or is it still needed for other >>>> platforms? >>>> >>>> >>> kmacy suggested you might be the right person to ask... the conclusion >>> so far is that it *might* be necessary on sparc64 and / or mips. >>> >>> >> I think that this code may no longer be needed, but I want to >> double-check. I faced a related problem implementing superpages >> support, so I introduced an additional "access type" parameter to >> pmap_enter(). This parameter was specifically intended to allow a >> pmap_enter() implementation to preset the PTE's modified bit. I think >> that the simulated page fault that occurs on vslock()-style wiring >> passes "write access" to pmap_enter(). If so, then it's just a matter >> of tweaking the MIPS or any other pmap_enter() to actually do something >> with the "access type" parameter. Currently, only the architectures >> that implement the pmap-level support for superpages, i.e., amd64 and >> i386, do anything with this parameter. >> > > Then it sounds like the code should definitely be removed and that if any > problems do crop up, they can be fixed in pmap_enter() instead. > > I've had a chance to verify what I said above, so you can remove the code. I don't think that sparc64 will require any changes, but MIPS needs a two-line change to pmap_enter(). I'll see that the change gets made. Alan From owner-freebsd-arch@FreeBSD.ORG Fri May 15 16:54:33 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 005A91065672 for ; Fri, 15 May 2009 16:54:32 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (mail.cs.rice.edu [128.42.1.31]) by mx1.freebsd.org (Postfix) with ESMTP id C019E8FC15 for ; Fri, 15 May 2009 16:54:32 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (localhost.localdomain [127.0.0.1]) by mail.cs.rice.edu (Postfix) with ESMTP id 641D42C2B0E; Fri, 15 May 2009 11:24:15 -0500 (CDT) X-Virus-Scanned: by amavis-2.4.0 at mail.cs.rice.edu Received: from mail.cs.rice.edu ([127.0.0.1]) by mail.cs.rice.edu (mail.cs.rice.edu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Z3VFQTY9kGoN; Fri, 15 May 2009 11:24:07 -0500 (CDT) Received: from adsl-216-63-78-18.dsl.hstntx.swbell.net (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.cs.rice.edu (Postfix) with ESMTP id 8474C2C2AD0; Fri, 15 May 2009 11:24:07 -0500 (CDT) Message-ID: <4A0D9727.6030703@cs.rice.edu> Date: Fri, 15 May 2009 11:24:07 -0500 From: Alan Cox User-Agent: Thunderbird 2.0.0.21 (X11/20090404) MIME-Version: 1.0 To: John Baldwin , =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <86tz3o4lb9.fsf@ds4.des.no> <86prec4kwj.fsf@ds4.des.no> <4A0C49FF.1070707@cs.rice.edu> <200905150804.36977.jhb@freebsd.org> In-Reply-To: <200905150804.36977.jhb@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: alc@freebsd.org, arch@freebsd.org, freebsd-arch@freebsd.org Subject: Re: PTE modified bit emulation trap X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 16:54:33 -0000 John Baldwin wrote: > On Thursday 14 May 2009 12:42:39 pm Alan Cox wrote: > >> Dag-Erling Smørgrav wrote: >> >>> [from -alpha, -hackers] >>> >>> Dag-Erling Smørgrav writes: >>> >>> >>>> Coverity complains about the lack of error checking in the following >>>> code in sys/kern/kern_sysctl.c, around line 1390: >>>> >>>> /* >>>> * Touch all the wired pages to avoid PTE modified >>>> * bit emulation traps on Alpha while holding locks >>>> * in the sysctl handler. >>>> */ >>>> for (i = (wiredlen + PAGE_SIZE - 1) / PAGE_SIZE, >>>> cp = req->oldptr; i > 0; i--, cp += PAGE_SIZE) { >>>> copyin(cp, &dummy, 1); >>>> copyout(&dummy, cp, 1); >>>> } >>>> >>>> Since Alpha is dead, can we remove this, or is it still needed for other >>>> platforms? >>>> >>>> >>> kmacy suggested you might be the right person to ask... the conclusion >>> so far is that it *might* be necessary on sparc64 and / or mips. >>> >>> >> I think that this code may no longer be needed, but I want to >> double-check. I faced a related problem implementing superpages >> support, so I introduced an additional "access type" parameter to >> pmap_enter(). This parameter was specifically intended to allow a >> pmap_enter() implementation to preset the PTE's modified bit. I think >> that the simulated page fault that occurs on vslock()-style wiring >> passes "write access" to pmap_enter(). If so, then it's just a matter >> of tweaking the MIPS or any other pmap_enter() to actually do something >> with the "access type" parameter. Currently, only the architectures >> that implement the pmap-level support for superpages, i.e., amd64 and >> i386, do anything with this parameter. >> > > Then it sounds like the code should definitely be removed and that if any > problems do crop up, they can be fixed in pmap_enter() instead. > > I've had a chance to verify what I said above, so you can remove the code. I don't think that sparc64 will require any changes, but MIPS needs a two-line change to pmap_enter(). I'll see that the change gets made. Alan From owner-freebsd-arch@FreeBSD.ORG Sat May 16 18:26:20 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49F1B106564A for ; Sat, 16 May 2009 18:26:20 +0000 (UTC) (envelope-from do-1976-7037537-904882-12--freebsd-arch.freebsd.org@rt.emm08.net) Received: from iraklia201.emm02.net (iraklia201.emm02.net [80.118.49.201]) by mx1.freebsd.org (Postfix) with ESMTP id B08198FC1F for ; Sat, 16 May 2009 18:26:19 +0000 (UTC) (envelope-from do-1976-7037537-904882-12--freebsd-arch.freebsd.org@rt.emm08.net) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=dk; d=f-j-b.fr; h=Message-Id:Reply-To:Mime-Version:Content-Type:Content-Transfer-Encoding:List-Unsubscribe:From:To:Date:Subject; i=ludivine@f-j-b.fr; bh=g8Y2CvIKIPt2DCgNDKScoVgCkgc=; b=Lmvwcx3PJdHKJFYCbcSsqWInrmMeMX11QVJbnac/ijYE/mia5P1U8WcdjDCPTGBAqeL5iKauxRUY F+nD6ZF/IGBQ8JmbhB0hh82CXV7bHKgF/zAk0MhzIaLxD2ZmXgfMK7XVWJOm9roV5jFJxY4KMYuK j8AYdrFSkrxD3doqjCg= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=dk; d=f-j-b.fr; b=T2MFCnUxj6CujyM/MvLxc+m3rkomvbbta1PD4Zskyo1r2scPDS2aChr7vGaxWLucmbKxkrc2Fg/5 9rAw9rShGeGnbjlCCVWBg5Hv6UnR9WX/uC/Mq7ZE1mXfd8P7aVNqbbyJtxYNmujkQ1LyN1QuduXG kMMkvOCdUKxKGuyiBaM=; Message-Id: Content-Transfer-Encoding: 8bit X-CAMPAIGN-ID: 1976-7037537-904882 X-Mailer: DNET 70375379048821976 From: =?ISO-8859-1?Q?f-j-b?= To: Date: Sat, 16 May 2009 20:15:00 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Envois de S M S en nombre X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: =?ISO-8859-1?Q?f-j-b?= List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2009 18:26:20 -0000 Bonjour, si vous n'arrivez pas à lire ce message, visualisez la version en ligne Merci. Vous recevez ce courriel sur l'adresse freebsd-arch@freebsd.org emailing optin emailing-cible campagnes-emailing cordialement [1]jacqueline@force-marketing.fr [2]http://www.force-marketing.fr This e-mail and any attached documents may contain confidential or proprietary information. If you are not the intended recipient, please advise the sender immediately and delete this e-mail and all attached documents from your computer system. Any unauthorised disclosure, distribution or copying hereof is prohibited." " Ce courriel et les documents qui y sont attaches peuvent contenir des informations confidentielles. Si vous n'etes pas le destinataire escompte, merci d'en informer l'expediteur immediatement et de detruire ce courriel ainsi que tous les documents attaches de votre systeme informatique. Toute divulgation, distribution ou copie du present courriel et des documents attaches sans autorisation prealable de son emetteur est interdite. Pour ne plus recevoir de courriels de notre part, il vous suffit de vous rendre sur [3]cette page. References 1. mailto:jacqueline@force-marketing.fr 2. http://url.f-j-b.fr/id.asp?l=51196-7037537-904882-1976-0 3. http://url.f-j-b.fr/id.asp?l=51197-7037537-904882-1976-0&id=904882-1976-7037537-18b8b3b3&res=fr