From owner-freebsd-arch@FreeBSD.ORG Sun Jan 6 15:23:15 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D22BB22F; Sun, 6 Jan 2013 15:23:15 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 5BF7B400; Sun, 6 Jan 2013 15:23:15 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.5/8.14.5/ALCHEMY.FRANKEN.DE) with ESMTP id r06FNDr7049581; Sun, 6 Jan 2013 16:23:13 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.5/8.14.5/Submit) id r06FND5j049580; Sun, 6 Jan 2013 16:23:13 +0100 (CET) (envelope-from marius) Date: Sun, 6 Jan 2013 16:23:13 +0100 From: Marius Strobl To: Alexander Motin Subject: Re: [RFC/RFT] calloutng Message-ID: <20130106152313.GD26039@alchemy.franken.de> References: <50CCAB99.4040308@FreeBSD.org> <50CE5B54.3050905@FreeBSD.org> <50D03173.9080904@FreeBSD.org> <20121225232126.GA47692@alchemy.franken.de> <50DB4EFE.2020600@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50DB4EFE.2020600@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Davide Italiano , FreeBSD Current , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2013 15:23:15 -0000 On Wed, Dec 26, 2012 at 09:24:46PM +0200, Alexander Motin wrote: > On 26.12.2012 01:21, Marius Strobl wrote: > > On Tue, Dec 18, 2012 at 11:03:47AM +0200, Alexander Motin wrote: > >> Experiments with dummynet shown ineffective support for very short > >> tick-based callouts. New version fixes that, allowing to get as many > >> tick-based callout events as hz value permits, while still be able to > >> aggregate events and generating minimum of interrupts. > >> > >> Also this version modifies system load average calculation to fix some > >> cases existing in HEAD and 9 branches, that could be fixed with new > >> direct callout functionality. > >> > >> http://people.freebsd.org/~mav/calloutng_12_17.patch > >> > >> With several important changes made last time I am going to delay commit > >> to HEAD for another week to do more testing. Comments and new test cases > >> are welcome. Thanks for staying tuned and commenting. > > > > FYI, I gave both calloutng_12_15_1.patch and calloutng_12_17.patch a > > try on sparc64 and it at least survives a buildworld there. However, > > with the patched kernels, buildworld times seem to increase slightly but > > reproducible by 1-2% (I only did four runs but typically buildworld > > times are rather stable and don't vary more than a minute for the > > same kernel and source here). Is this an expected trade-off (system > > time as such doesn't seem to increase)? > > I don't think build process uses significant number of callouts to > affect results directly. I think this additional time could be result of > the deeper next event look up, done by the new code, that is practically > useless for sparc64, which effectively has no cpu_idle() routine. It > wouldn't affect system time and wouldn't show up in any statistics > (except PMC or something alike) because it is executed inside timer > hardware interrupt handler. If my guess is right, that is a part that > probably still could be optimized. I'll look on it. Thanks. > > > Is there anything specific to test? > > Since the most of code is MI, for sparc64 I would mostly look on related > MD parts (eventtimers and timecounters) to make sure they are working > reliably in more stressful conditions. I still have some worries about > possible deadlock on hardware where IPIs are used to fetch present time > from other CPU. Well, I've just learnt two things the hard way: a) We really need the mutex in that path. b) Assuming that the initial synchronization of the counters is good enough and they won't drift considerably accross the CPUs so we can always use the local one makes things go south pretty soon after boot. At least with your calloutng_12_26.patch applied. I'm not really sure what to do about that. Earlier you already said that sched_bind(9) also isn't an option in case if td_critnest > 1. To be honest, I don't really unerstand why using a spin lock in the timecounter path makes sparc64 the only problematic architecture for your changes. The x86 i8254_get_timecount() also uses a spin lock so it should be in the same boat. The affected machines are equipped with a x86-style south bridge which exposes a powermanagment unit (intended to be used as a SMBus bridge only in these machines) on the PCI bus. Actually, this device also includes an ACPI power management timer. However, I've just spent a day trying to get that one working without success - it just doesn't increment. Probably its clock input isn't connected as it's not intended to be used in these machines. That south bridge also includes 8254 compatible timers on the ISA/ LPC side, but are hidden from the OFW device tree. I can hack these devices into existence and give it a try, but even if that works this likely would use the same code as the x86 i8254_get_timecount() so I don't see what would be gained with that. The last thing in order to avoid using the tick counter as timecounter in the MP case I can think of is that the Broadcom MACs in the affected machines also provide a counter driven by a 1 MHz clock. If that's good enough for a timecounter I can hook these up (in case these work ...) and hack bge(4) to not detach in that case (given that we can't detach timecounters ...). > > Here is small tool we are using for test correctness and performance of > different user-level APIs: http://people.freebsd.org/~mav/testsleep.c > I've run Ian's set of tests on a v215 with and without your calloutng_12_26.patch and on a v210 (these uses the IPI approach) with the latter also applied. I'm not really sure what to make out of the numbers. v215 w/o v215 w/ v210 w/ ---------- ---------------- ---------------- ---------------- select 1 1999.61 1 23.87 1 29.97 poll 1 1999.70 1 1069.61 1 1075.24 usleep 1 1999.86 1 23.43 1 28.99 nanosleep 1 999.92 1 23.28 1 28.66 kqueue 1 1000.12 1 1071.13 1 1076.35 kqueueto 1 999.56 1 26.33 1 31.34 syscall 1 1.89 1 1.92 1 2.88 select 300 1999.72 300 326.08 300 332.24 poll 300 1999.12 300 1069.78 300 1075.82 usleep 300 1999.91 300 325.63 300 330.94 nanosleep 300 999.82 300 23.25 300 26.76 kqueue 300 1000.14 300 1071.06 300 1075.96 kqueueto 300 999.53 300 26.32 300 31.42 syscall 300 1.90 300 1.93 300 2.89 select 3000 3998.18 3000 3176.51 3000 3193.86 poll 3000 3999.29 3000 3182.21 3000 3193.12 usleep 3000 3998.46 3000 3191.60 3000 3192.50 nanosleep 3000 1999.79 3000 23.21 3000 27.02 kqueue 3000 3000.12 3000 3189.13 3000 3191.96 kqueueto 3000 1999.99 3000 26.28 3000 31.91 syscall 3000 1.91 3000 1.91 3000 2.90 select 30000 30990.85 30000 31489.18 30000 31548.77 poll 30000 30995.25 30000 31518.80 30000 31487.92 usleep 30000 30992.00 30000 31510.42 30000 31475.50 nanosleep 30000 1999.46 30000 38.67 30000 41.95 kqueue 30000 30006.49 30000 30991.86 30000 30996.77 kqueueto 30000 1999.09 30000 41.67 30000 46.36 syscall 30000 1.91 30000 1.91 30000 2.88 select 300000 300990.65 300000 301864.98 300000 301787.01 poll 300000 300998.09 300000 301831.36 300000 301741.62 usleep 300000 300990.80 300000 301824.67 300000 301770.10 nanosleep 300000 1999.15 300000 325.74 300000 331.01 kqueue 300000 300000.87 300000 301031.11 300000 300992.28 kqueueto 300000 1999.39 300000 328.77 300000 333.45 syscall 300000 1.91 300000 1.91 300000 2.88 Marius From owner-freebsd-arch@FreeBSD.ORG Sun Jan 6 16:29:44 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 43CCD598; Sun, 6 Jan 2013 16:29:44 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 92CCA821; Sun, 6 Jan 2013 16:29:42 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 80B4273027; Sun, 6 Jan 2013 17:20:49 +0100 (CET) Date: Sun, 6 Jan 2013 17:20:49 +0100 From: Luigi Rizzo To: Marius Strobl Subject: Re: [RFC/RFT] calloutng Message-ID: <20130106162049.GA3640@onelab2.iet.unipi.it> References: <50CCAB99.4040308@FreeBSD.org> <50CE5B54.3050905@FreeBSD.org> <50D03173.9080904@FreeBSD.org> <20121225232126.GA47692@alchemy.franken.de> <50DB4EFE.2020600@FreeBSD.org> <20130106152313.GD26039@alchemy.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130106152313.GD26039@alchemy.franken.de> User-Agent: Mutt/1.4.2.3i Cc: Davide Italiano , Alexander Motin , FreeBSD Current , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2013 16:29:44 -0000 On Sun, Jan 06, 2013 at 04:23:13PM +0100, Marius Strobl wrote: > On Wed, Dec 26, 2012 at 09:24:46PM +0200, Alexander Motin wrote: > > On 26.12.2012 01:21, Marius Strobl wrote: > > > On Tue, Dec 18, 2012 at 11:03:47AM +0200, Alexander Motin wrote: > > >> Experiments with dummynet shown ineffective support for very short > > >> tick-based callouts. New version fixes that, allowing to get as many > > >> tick-based callout events as hz value permits, while still be able to > > >> aggregate events and generating minimum of interrupts. > > >> > > >> Also this version modifies system load average calculation to fix some > > >> cases existing in HEAD and 9 branches, that could be fixed with new > > >> direct callout functionality. > > >> > > >> http://people.freebsd.org/~mav/calloutng_12_17.patch > > >> > > >> With several important changes made last time I am going to delay commit > > >> to HEAD for another week to do more testing. Comments and new test cases > > >> are welcome. Thanks for staying tuned and commenting. > > > > > > FYI, I gave both calloutng_12_15_1.patch and calloutng_12_17.patch a > > > try on sparc64 and it at least survives a buildworld there. However, > > > with the patched kernels, buildworld times seem to increase slightly but > > > reproducible by 1-2% (I only did four runs but typically buildworld > > > times are rather stable and don't vary more than a minute for the > > > same kernel and source here). Is this an expected trade-off (system > > > time as such doesn't seem to increase)? > > > > I don't think build process uses significant number of callouts to > > affect results directly. I think this additional time could be result of > > the deeper next event look up, done by the new code, that is practically > > useless for sparc64, which effectively has no cpu_idle() routine. It > > wouldn't affect system time and wouldn't show up in any statistics > > (except PMC or something alike) because it is executed inside timer > > hardware interrupt handler. If my guess is right, that is a part that > > probably still could be optimized. I'll look on it. Thanks. > > > > > Is there anything specific to test? > > > > Since the most of code is MI, for sparc64 I would mostly look on related > > MD parts (eventtimers and timecounters) to make sure they are working > > reliably in more stressful conditions. I still have some worries about > > possible deadlock on hardware where IPIs are used to fetch present time > > from other CPU. > > Well, I've just learnt two things the hard way: > a) We really need the mutex in that path. > b) Assuming that the initial synchronization of the counters is good > enough and they won't drift considerably accross the CPUs so we can > always use the local one makes things go south pretty soon after > boot. At least with your calloutng_12_26.patch applied. > > I'm not really sure what to do about that. Earlier you already said > that sched_bind(9) also isn't an option in case if td_critnest > 1. > To be honest, I don't really unerstand why using a spin lock in the > timecounter path makes sparc64 the only problematic architecture > for your changes. The x86 i8254_get_timecount() also uses a spin lock > so it should be in the same boat. > > The affected machines are equipped with a x86-style south bridge > which exposes a powermanagment unit (intended to be used as a SMBus > bridge only in these machines) on the PCI bus. Actually, this device > also includes an ACPI power management timer. However, I've just > spent a day trying to get that one working without success - it > just doesn't increment. Probably its clock input isn't connected as > it's not intended to be used in these machines. > That south bridge also includes 8254 compatible timers on the ISA/ > LPC side, but are hidden from the OFW device tree. I can hack these > devices into existence and give it a try, but even if that works this > likely would use the same code as the x86 i8254_get_timecount() so I > don't see what would be gained with that. > > The last thing in order to avoid using the tick counter as timecounter > in the MP case I can think of is that the Broadcom MACs in the affected > machines also provide a counter driven by a 1 MHz clock. If that's good > enough for a timecounter I can hook these up (in case these work ...) > and hack bge(4) to not detach in that case (given that we can't detach > timecounters ...). > > > > > Here is small tool we are using for test correctness and performance of > > different user-level APIs: http://people.freebsd.org/~mav/testsleep.c > > > > I've run Ian's set of tests on a v215 with and without your > calloutng_12_26.patch and on a v210 (these uses the IPI approach) > with the latter also applied. > I'm not really sure what to make out of the numbers. > > v215 w/o v215 w/ v210 w/ > ---------- ---------------- ---------------- ---------------- > select 1 1999.61 1 23.87 1 29.97 > poll 1 1999.70 1 1069.61 1 1075.24 > usleep 1 1999.86 1 23.43 1 28.99 > nanosleep 1 999.92 1 23.28 1 28.66 > kqueue 1 1000.12 1 1071.13 1 1076.35 > kqueueto 1 999.56 1 26.33 1 31.34 > syscall 1 1.89 1 1.92 1 2.88 > select 300 1999.72 300 326.08 300 332.24 > poll 300 1999.12 300 1069.78 300 1075.82 > usleep 300 1999.91 300 325.63 300 330.94 > nanosleep 300 999.82 300 23.25 300 26.76 > kqueue 300 1000.14 300 1071.06 300 1075.96 > kqueueto 300 999.53 300 26.32 300 31.42 > syscall 300 1.90 300 1.93 300 2.89 > select 3000 3998.18 3000 3176.51 3000 3193.86 > poll 3000 3999.29 3000 3182.21 3000 3193.12 > usleep 3000 3998.46 3000 3191.60 3000 3192.50 > nanosleep 3000 1999.79 3000 23.21 3000 27.02 > kqueue 3000 3000.12 3000 3189.13 3000 3191.96 > kqueueto 3000 1999.99 3000 26.28 3000 31.91 > syscall 3000 1.91 3000 1.91 3000 2.90 > select 30000 30990.85 30000 31489.18 30000 31548.77 > poll 30000 30995.25 30000 31518.80 30000 31487.92 > usleep 30000 30992.00 30000 31510.42 30000 31475.50 > nanosleep 30000 1999.46 30000 38.67 30000 41.95 > kqueue 30000 30006.49 30000 30991.86 30000 30996.77 > kqueueto 30000 1999.09 30000 41.67 30000 46.36 > syscall 30000 1.91 30000 1.91 30000 2.88 > select 300000 300990.65 300000 301864.98 300000 301787.01 > poll 300000 300998.09 300000 301831.36 300000 301741.62 > usleep 300000 300990.80 300000 301824.67 300000 301770.10 > nanosleep 300000 1999.15 300000 325.74 300000 331.01 > kqueue 300000 300000.87 300000 301031.11 300000 300992.28 > kqueueto 300000 1999.39 300000 328.77 300000 333.45 > syscall 300000 1.91 300000 1.91 300000 2.88 the nanosleep and kqueueto tests are probably passing the wrong argument to the syscall (meant to be microseconds, but nanosleep takes nanosecond so it should probably be multiplied by 1000). I think that for the time being it would be useful to run at least one set of tests with kern.timecounter.alloweddeviation=0 so we can tell how close we get to the required timeouts cheers luigi > Marius > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sun Jan 6 18:02:30 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 733A4638; Sun, 6 Jan 2013 18:02:30 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from adsum.doit.wisc.edu (adsum.doit.wisc.edu [144.92.197.210]) by mx1.freebsd.org (Postfix) with ESMTP id A08AA10F7; Sun, 6 Jan 2013 18:02:30 +0000 (UTC) MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_hv2Jca5yydhH8mRnv/48vw)" Received: from avs-daemon.smtpauth1.wiscmail.wisc.edu by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0MG700100U3ZMM00@smtpauth1.wiscmail.wisc.edu>; Sun, 06 Jan 2013 12:02:23 -0600 (CST) Received: from wanderer.tachypleus.net (dhcp107-17-54-205.hil-sfofhhh.sfo.wayport.net [107.17.54.205]) by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0MG700M2DU3XBW10@smtpauth1.wiscmail.wisc.edu>; Sun, 06 Jan 2013 12:02:22 -0600 (CST) Date: Sun, 06 Jan 2013 10:02:21 -0800 From: Nathan Whitehorn Subject: LLVM Image Activator Sender: whitehorn@wisc.edu To: freebsd-arch@freebsd.org, freebsd-toolchain@freebsd.org Message-id: <50E9BC2D.7000302@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=107.17.54.205 X-Spam-PmxInfo: Server=avs-14, Version=5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2013.1.6.175417, SenderIP=107.17.54.205 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2013 18:02:30 -0000 This is a multi-part message in MIME format. --Boundary_(ID_hv2Jca5yydhH8mRnv/48vw) Content-type: text/plain; CHARSET=US-ASCII Content-transfer-encoding: 7BIT Having LLVM/clang in the base system lets us do some interesting things that we couldn't do with GCC. One is that LLVM ships with a JIT for LLVM IR as well as components of a toolchain for it (this is what Google's pNACL uses) and that you can end up producing binary files that are in IR instead of native code. The IR isn't really cross-platform, but does let you do CPU-specific optimizations when executed by the JIT, etc. The attached patch causes the LLVM JIT (lli) to be built by default (adding ~20 seconds to buildworld on my five-year-old laptop) and adds a kernel image activator that invokes it when passed LLVM bitcode files. It's not completely finished (see the XXX comment in the middle), but it does work, as follows: $ clang -emit-llvm -c -o hw.ll hw.c $ file hw.ll hw.ll: LLVM bitcode $ lli hw.ll Hello world! $ chmod a+x hw.ll $ ./hw.ll Hello world! $ Is there any interest in having features like this? It seems like this could provides some interesting possibilities for us and nice integration from having imported clang into base. -Nathan --Boundary_(ID_hv2Jca5yydhH8mRnv/48vw) Content-type: text/plain; CHARSET=US-ASCII; name=imgact_llvm.diff Content-transfer-encoding: 7BIT Content-disposition: attachment; filename=imgact_llvm.diff Index: sys/kern/imgact_llvm.c =================================================================== --- sys/kern/imgact_llvm.c (revision 0) +++ sys/kern/imgact_llvm.c (working copy) @@ -0,0 +1,78 @@ +/*- + * Copyright (c) 2012 Nathan Whitehorn + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static const char llvm_magic[5] = {'B', 'C', 0xc0, 0xde, 0}; + +static int +exec_llvm_imgact(struct image_params *imgp) +{ + const char *interp = "/usr/bin/lli"; + + /* check for LLVM bitcode magic */ + if (memcmp(imgp->image_header, llvm_magic, 4) != 0) + return (-1); + + if (imgp->args->stringspace < strlen(interp) + 1) + return (ENOMEM); + + imgp->interpreted = 1; + + /* Move over arguments and set argv[0] to the interpreter */ + memmove(imgp->args->begin_argv + strlen(interp) + 1, + imgp->args->begin_argv, imgp->args->endp - imgp->args->begin_argv); + strcpy(imgp->args->begin_argv, interp); + + /* XXX: set argv[1] to fname instead of old argv[0], call -fake-argv0 with old argv[0] */ + + imgp->args->begin_envv += strlen(interp) + 1; + imgp->args->endp += strlen(interp) + 1; + imgp->args->stringspace -= strlen(interp) + 1; + imgp->args->argc++; + + imgp->interpreter_name = imgp->args->begin_argv; + + return (0); +} + +/* + * Tell kern_execve.c about it, with a little help from the linker. + */ +static struct execsw llvm_execsw = { exec_llvm_imgact, llvm_magic }; +EXEC_SET(llvm, llvm_execsw); + Property changes on: sys/kern/imgact_llvm.c ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Index: sys/conf/files =================================================================== --- sys/conf/files (revision 245062) +++ sys/conf/files (working copy) @@ -2586,6 +2586,7 @@ kern/device_if.m standard kern/imgact_elf.c standard kern/imgact_elf32.c optional compat_freebsd32 +kern/imgact_llvm.c standard kern/imgact_shell.c standard kern/inflate.c optional gzip kern/init_main.c standard Index: usr.bin/clang/Makefile =================================================================== --- usr.bin/clang/Makefile (revision 245062) +++ usr.bin/clang/Makefile (working copy) @@ -2,12 +2,11 @@ .include -SUBDIR= clang clang-tblgen tblgen +SUBDIR= clang clang-tblgen tblgen lli .if ${MK_CLANG_EXTRAS} != "no" && !defined(TOOLS_PREFIX) SUBDIR+=bugpoint \ llc \ - lli \ llvm-ar \ llvm-as \ llvm-bcanalyzer \ Index: lib/clang/Makefile =================================================================== --- lib/clang/Makefile (revision 245062) +++ lib/clang/Makefile (working copy) @@ -30,16 +30,23 @@ libllvmbitwriter \ libllvmcodegen \ libllvmcore \ + libllvmdebuginfo \ + libllvmexecutionengine \ libllvminstcombine \ libllvminstrumentation \ + libllvminterpreter \ libllvmipa \ libllvmipo \ + libllvmjit \ libllvmlinker \ libllvmmc \ + libllvmmcdisassembler \ + libllvmmcjit \ libllvmmcparser \ libllvmobject \ libllvmscalaropts \ libllvmselectiondag \ + libllvmruntimedyld \ libllvmsupport \ libllvmtablegen \ libllvmtarget \ @@ -68,18 +75,8 @@ libllvmx86disassembler \ libllvmx86info \ libllvmx86instprinter \ - libllvmx86utils - -.if ${MK_CLANG_EXTRAS} != "no" -SUBDIR+=libllvmdebuginfo \ - libllvmexecutionengine \ - libllvminterpreter \ - libllvmjit \ - libllvmmcdisassembler \ - libllvmmcjit \ - libllvmruntimedyld + libllvmx86utils .endif -.endif SUBDIR+= include Index: lib/clang/libllvmsupport/Makefile =================================================================== --- lib/clang/libllvmsupport/Makefile (revision 245062) +++ lib/clang/libllvmsupport/Makefile (working copy) @@ -16,12 +16,16 @@ ConstantRange.cpp \ CrashRecoveryContext.cpp \ DAGDeltaAlgorithm.cpp \ + DataExtractor.cpp \ + DataStream.cpp \ Debug.cpp \ DeltaAlgorithm.cpp \ + Disassembler.cpp \ Dwarf.cpp \ DynamicLibrary.cpp \ Errno.cpp \ ErrorHandling.cpp \ + FileUtilities.cpp \ FoldingSet.cpp \ FormattedStream.cpp \ GraphWriter.cpp \ @@ -56,6 +60,7 @@ StringMap.cpp \ StringPool.cpp \ StringRef.cpp \ + SystemUtils.cpp \ TargetRegistry.cpp \ ThreadLocal.cpp \ Threading.cpp \ @@ -76,15 +81,4 @@ system_error.cpp LLVM_REQUIRES_RTTI= -.if ${MK_CLANG_EXTRAS} != "no" -SRCS+= BlockFrequency.cpp \ - BranchProbability.cpp \ - DataExtractor.cpp \ - DataStream.cpp \ - Disassembler.cpp \ - FileUtilities.cpp \ - MemoryObject.cpp \ - SystemUtils.cpp -.endif - .include "../clang.lib.mk" --Boundary_(ID_hv2Jca5yydhH8mRnv/48vw)-- From owner-freebsd-arch@FreeBSD.ORG Mon Jan 7 09:18:03 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A8042AD8; Mon, 7 Jan 2013 09:18:03 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 6BFA8396; Mon, 7 Jan 2013 09:18:03 +0000 (UTC) Received: from JRE-MBP-2.local (c-50-143-148-105.hsd1.ca.comcast.net [50.143.148.105]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id r079I1BG066933 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 7 Jan 2013 01:18:02 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <50EA92C4.2080808@freebsd.org> Date: Mon, 07 Jan 2013 01:17:56 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Nathan Whitehorn Subject: Re: LLVM Image Activator References: <50E9BC2D.7000302@freebsd.org> In-Reply-To: <50E9BC2D.7000302@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-toolchain@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 09:18:03 -0000 On 1/6/13 10:02 AM, Nathan Whitehorn wrote: > Having LLVM/clang in the base system lets us do some interesting things > that we couldn't do with GCC. One is that LLVM ships with a JIT for LLVM > IR as well as components of a toolchain for it (this is what Google's > pNACL uses) and that you can end up producing binary files that are in > IR instead of native code. The IR isn't really cross-platform, but does > let you do CPU-specific optimizations when executed by the JIT, etc. > > The attached patch causes the LLVM JIT (lli) to be built by default > (adding ~20 seconds to buildworld on my five-year-old laptop) and adds a > kernel image activator that invokes it when passed LLVM bitcode files. > It's not completely finished (see the XXX comment in the middle), but it > does work, as follows: > > $ clang -emit-llvm -c -o hw.ll hw.c > $ file hw.ll > hw.ll: LLVM bitcode > $ lli hw.ll > Hello world! > $ chmod a+x hw.ll > $ ./hw.ll > Hello world! > $ > > Is there any interest in having features like this? It seems like this > could provides some interesting possibilities for us and nice > integration from having imported clang into base. > -Nathan sounds like a nice idea. I'd put it in if you can find a few other supporters.. (especially ones who can help with maintenance. > > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Jan 7 15:14:58 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 22614505; Mon, 7 Jan 2013 15:14:58 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 01D4BFB5; Mon, 7 Jan 2013 15:14:58 +0000 (UTC) Received: from ralph.baldwin.cx (c-68-39-198-164.hsd1.de.comcast.net [68.39.198.164]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 54263B984; Mon, 7 Jan 2013 10:14:57 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: LLVM Image Activator Date: Mon, 7 Jan 2013 09:36:38 -0500 User-Agent: KMail/1.13.7 (FreeBSD/9.1-PRERELEASE; KDE/4.8.4; amd64; ; ) References: <50E9BC2D.7000302@freebsd.org> In-Reply-To: <50E9BC2D.7000302@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201301070936.39052.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 07 Jan 2013 10:14:57 -0500 (EST) Cc: freebsd-toolchain@freebsd.org, Nathan Whitehorn X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 15:14:58 -0000 On Sunday, January 06, 2013 01:02:21 PM Nathan Whitehorn wrote: > Having LLVM/clang in the base system lets us do some interesting things > that we couldn't do with GCC. One is that LLVM ships with a JIT for LLVM > IR as well as components of a toolchain for it (this is what Google's > pNACL uses) and that you can end up producing binary files that are in > IR instead of native code. The IR isn't really cross-platform, but does > let you do CPU-specific optimizations when executed by the JIT, etc. > > The attached patch causes the LLVM JIT (lli) to be built by default > (adding ~20 seconds to buildworld on my five-year-old laptop) and adds a > kernel image activator that invokes it when passed LLVM bitcode files. > It's not completely finished (see the XXX comment in the middle), but it > does work, as follows: > > $ clang -emit-llvm -c -o hw.ll hw.c > $ file hw.ll > hw.ll: LLVM bitcode > $ lli hw.ll > Hello world! > $ chmod a+x hw.ll > $ ./hw.ll > Hello world! > $ > > Is there any interest in having features like this? It seems like this > could provides some interesting possibilities for us and nice > integration from having imported clang into base. > -Nathan This sounds neat indeed. Does the IR format provide any sort of notation for encoding the path to the interpreter (similar to ELF)? If not, you might want to at least make the path to 'lli' be configurable via a tunable and/or sysctl (e.g. if using a newer version of clang in /usr/local). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Jan 7 16:31:52 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B34FB809; Mon, 7 Jan 2013 16:31:52 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: from mail-wg0-f45.google.com (mail-wg0-f45.google.com [74.125.82.45]) by mx1.freebsd.org (Postfix) with ESMTP id F0308633; Mon, 7 Jan 2013 16:31:51 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id dq12so10159175wgb.24 for ; Mon, 07 Jan 2013 08:31:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id; bh=kUMribF3yu7gycPE1tU3/fD0BBWKU6s4mjyhSWIzMrU=; b=ez2P/4Hw8zHwTfArti+Q2QgWVrL6c0JajtEm9qKmiatKFyk9/XOyJf35hWRgv63H9v lXC1dKyGG7jh0h0myLoCEgEagJDfZNHXSSdkTcY6RufqdFhoi2MGhqe8dhcyZzglP5v3 khakCD170AcBrTVUAnQ7u4i15hfntSlKMwe4mgyKFZc0c2Z6U7spW2h1LR8jBOOFqwb3 EO3D7rjb3YsGwTuLuldc4owkOgGnh0x2lWDlBQ2l6Fiz+iDZ01IZ7KVK+zKc6SoUFst8 JujnICAlD4pYhVzfFi0d85gRAAnqLT22z9AVez9cWoBzFDy3mYXKx3D4tzANB5tEUob6 YWtA== X-Received: by 10.180.78.137 with SMTP id b9mr10112881wix.30.1357575912696; Mon, 07 Jan 2013 08:25:12 -0800 (PST) Received: from dragon.dg (41-135-148-131.dsl.mweb.co.za. [41.135.148.131]) by mx.google.com with ESMTPS id h19sm12811757wiv.7.2013.01.07.08.25.09 (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 07 Jan 2013 08:25:11 -0800 (PST) From: David Naylor To: freebsd-toolchain@freebsd.org Subject: Re: LLVM Image Activator Date: Mon, 7 Jan 2013 18:25:03 +0200 User-Agent: KMail/1.13.7 (FreeBSD/9.1-PRERELEASE; KDE/4.9.1; amd64; ; ) References: <50E9BC2D.7000302@freebsd.org> In-Reply-To: <50E9BC2D.7000302@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1362420.XBIvH0nEZn"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201301071825.06439.naylor.b.david@gmail.com> Cc: Nathan Whitehorn , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 16:31:52 -0000 --nextPart1362420.XBIvH0nEZn Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, Just my 2c On Sunday, 6 January 2013 20:02:21 Nathan Whitehorn wrote: > Having LLVM/clang in the base system lets us do some interesting things > that we couldn't do with GCC. One is that LLVM ships with a JIT for LLVM > IR as well as components of a toolchain for it (this is what Google's > pNACL uses) and that you can end up producing binary files that are in > IR instead of native code. The IR isn't really cross-platform, but does > let you do CPU-specific optimizations when executed by the JIT, etc. >=20 > The attached patch causes the LLVM JIT (lli) to be built by default > (adding ~20 seconds to buildworld on my five-year-old laptop) and adds a > kernel image activator that invokes it when passed LLVM bitcode files. > It's not completely finished (see the XXX comment in the middle), but it > does work, as follows: >=20 > $ clang -emit-llvm -c -o hw.ll hw.c > $ file hw.ll > hw.ll: LLVM bitcode > $ lli hw.ll > Hello world! > $ chmod a+x hw.ll > $ ./hw.ll > Hello world! > $ >=20 > Is there any interest in having features like this? It seems like this > could provides some interesting possibilities for us and nice > integration from having imported clang into base. Would it be possible to have this as a module (and thus in ports)? Or,=20 perhaps, change the sources such these things could be loaded as a module... Also, with modification to LLVM, wouldn't the existing '#!' mechanism work= =20 (aka '#!/usr/bin/lli') at the beginning of the file? =20 On an aside. Could you imagine shipping a 'x86' llvm-byte code and get the= =20 jit (with caching), thus your computer will get full support (speed) from t= he=20 binaries and the binaries will work on all 'x86' related architectures. (M= y=20 thinking is for things like i386 vs pentium4). =20 Regards --nextPart1362420.XBIvH0nEZn Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEABECAAYFAlDq9uIACgkQUaaFgP9pFrKo3ACeNX5ZhxuslKEem1bHkRpR8YQd HDoAn1Ukz4GTpxieoB7u1C7UFCSJsV90 =r9wK -----END PGP SIGNATURE----- --nextPart1362420.XBIvH0nEZn-- From owner-freebsd-arch@FreeBSD.ORG Mon Jan 7 16:46:00 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0045BC34; Mon, 7 Jan 2013 16:45:59 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from agogare.doit.wisc.edu (agogare.doit.wisc.edu [144.92.197.211]) by mx1.freebsd.org (Postfix) with ESMTP id BFC7C6EA; Mon, 7 Jan 2013 16:45:59 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from avs-daemon.smtpauth2.wiscmail.wisc.edu by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0MG900G02L7Z7X00@smtpauth2.wiscmail.wisc.edu>; Mon, 07 Jan 2013 10:45:35 -0600 (CST) Received: from wanderer.tachypleus.net (216-75-226-134.static.wiline.com [216.75.226.134]) by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0MG900BB1L7WDD30@smtpauth2.wiscmail.wisc.edu>; Mon, 07 Jan 2013 10:45:33 -0600 (CST) Date: Mon, 07 Jan 2013 08:45:32 -0800 From: Nathan Whitehorn Subject: Re: LLVM Image Activator In-reply-to: <201301071825.06439.naylor.b.david@gmail.com> Sender: whitehorn@wisc.edu To: David Naylor Message-id: <50EAFBAC.2020808@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=216.75.226.134 X-Spam-PmxInfo: Server=avs-14, Version=5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2013.1.7.163317, SenderIP=216.75.226.134 References: <50E9BC2D.7000302@freebsd.org> <201301071825.06439.naylor.b.david@gmail.com> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 Cc: freebsd-toolchain@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 16:46:00 -0000 On 01/07/13 08:25, David Naylor wrote: > Hi, > > Just my 2c > > On Sunday, 6 January 2013 20:02:21 Nathan Whitehorn wrote: >> Having LLVM/clang in the base system lets us do some interesting things >> that we couldn't do with GCC. One is that LLVM ships with a JIT for LLVM >> IR as well as components of a toolchain for it (this is what Google's >> pNACL uses) and that you can end up producing binary files that are in >> IR instead of native code. The IR isn't really cross-platform, but does >> let you do CPU-specific optimizations when executed by the JIT, etc. >> >> The attached patch causes the LLVM JIT (lli) to be built by default >> (adding ~20 seconds to buildworld on my five-year-old laptop) and adds a >> kernel image activator that invokes it when passed LLVM bitcode files. >> It's not completely finished (see the XXX comment in the middle), but it >> does work, as follows: >> >> $ clang -emit-llvm -c -o hw.ll hw.c >> $ file hw.ll >> hw.ll: LLVM bitcode >> $ lli hw.ll >> Hello world! >> $ chmod a+x hw.ll >> $ ./hw.ll >> Hello world! >> $ >> >> Is there any interest in having features like this? It seems like this >> could provides some interesting possibilities for us and nice >> integration from having imported clang into base. > > Would it be possible to have this as a module (and thus in ports)? Or, > perhaps, change the sources such these things could be loaded as a module... We do support that, but I'm not really sure what would be gained. We have LLVM in base; it seems a shame not to use it. > Also, with modification to LLVM, wouldn't the existing '#!' mechanism work > (aka '#!/usr/bin/lli') at the beginning of the file? In principle, yes, but we'd have to coordinate that with a lot of players since a number of tools operate on LLVM bitcode. The other obstacle is that LLVM bitcode is a giant binary blob, not text, so the standard #! seems a little out of place. > On an aside. Could you imagine shipping a 'x86' llvm-byte code and get the > jit (with caching), thus your computer will get full support (speed) from the > binaries and the binaries will work on all 'x86' related architectures. (My > thinking is for things like i386 vs pentium4). > Yes, this is exactly the kind of thing I had in mind. -Nathan From owner-freebsd-arch@FreeBSD.ORG Mon Jan 7 17:24:40 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5D1A18C4; Mon, 7 Jan 2013 17:24:40 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id AE1228E2; Mon, 7 Jan 2013 17:24:39 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id r07HOXL4060397; Mon, 7 Jan 2013 19:24:33 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.3 kib.kiev.ua r07HOXL4060397 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id r07HOX9J060396; Mon, 7 Jan 2013 19:24:33 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 7 Jan 2013 19:24:33 +0200 From: Konstantin Belousov To: John Baldwin Subject: Re: LLVM Image Activator Message-ID: <20130107172433.GX82219@kib.kiev.ua> References: <50E9BC2D.7000302@freebsd.org> <201301070936.39052.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oDFa8iZvINJ+yMtC" Content-Disposition: inline In-Reply-To: <201301070936.39052.jhb@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-toolchain@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 17:24:40 -0000 --oDFa8iZvINJ+yMtC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 07, 2013 at 09:36:38AM -0500, John Baldwin wrote: > On Sunday, January 06, 2013 01:02:21 PM Nathan Whitehorn wrote: > > Having LLVM/clang in the base system lets us do some interesting things > > that we couldn't do with GCC. One is that LLVM ships with a JIT for LLVM > > IR as well as components of a toolchain for it (this is what Google's > > pNACL uses) and that you can end up producing binary files that are in > > IR instead of native code. The IR isn't really cross-platform, but does > > let you do CPU-specific optimizations when executed by the JIT, etc. > >=20 > > The attached patch causes the LLVM JIT (lli) to be built by default > > (adding ~20 seconds to buildworld on my five-year-old laptop) and adds a > > kernel image activator that invokes it when passed LLVM bitcode files. > > It's not completely finished (see the XXX comment in the middle), but it > > does work, as follows: > >=20 > > $ clang -emit-llvm -c -o hw.ll hw.c > > $ file hw.ll > > hw.ll: LLVM bitcode > > $ lli hw.ll > > Hello world! > > $ chmod a+x hw.ll > > $ ./hw.ll > > Hello world! > > $ > >=20 > > Is there any interest in having features like this? It seems like this > > could provides some interesting possibilities for us and nice > > integration from having imported clang into base. > > -Nathan >=20 > This sounds neat indeed. Does the IR format provide any sort of notation= for=20 > encoding the path to the interpreter (similar to ELF)? If not, you might= want=20 > to at least make the path to 'lli' be configurable via a tunable and/or s= ysctl=20 > (e.g. if using a newer version of clang in /usr/local). I do not like one-purpose write-once never-used image activators. I still do remember the buzz about the binary format 0xCAFEBABE, which AFAIR gained image activator support on several OSes, to be garbage collected. For one thing, such hacks are very system-specific and thus are never used, causing quick bit rot. Second, more important, is that parsing of yet another format opens an attack vector on kernel, and due to the nature of the activators, on the user space. Look at the PE image activator history, including at least one SA, to see what I mean. And PE is real image format instead of hack. Even for ELF, there were recent panic-level bugs. Note that several language runtimes are happy to use the shebang for bytecode. For examples, look at the ocaml and lisps. If adding despite these arguments, please do not make the thing 'standard'. Provide the option to compile it in, and may be make it a module. --oDFa8iZvINJ+yMtC Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ6wTQAAoJEJDCuSvBvK1BM4sP/1zs6BH/0cXbfHukn+M0TOLz S5+Dn+FTd9K4FmXS2CQpQdcZDCE1uBTQ8j6pIyUCO4rOSaGmdYTCOij2TRnEs+Rf v1wdr49/H7iO1uDAG2l32WPSmvd2K6tFilDRiLvSsBaH63uI2+xGhq0jPfSXPbv+ jnsBDOGSoex2+rIaDvoV9TZ/2ShWEDp2SBnPPBYTa515JmO0ypw2FBZd5+9wbW7v oaLNhQQz7eB1nrlxCs2MlQB4PR94DH9xAmkdRp+Q/AJ58icOiZOBiEH2L9ESYF+n UFP32ESKJrABTvzoS+nusRtmQwWZrymFUpb1AxqXVbi8fm2/9dOf1aLqbH9dO2vT IiuzC/C14yvp01DSs66ryD7ODogEaTEIMbBx9CHOkmcka4vxxPhqhX3is7xMnnVs ipcb0DeKiv7FIYwinTkJzrXhS4gi5N6nR+1RqEsq2LZW6BywERhVR3S4wNgWJWJH VisAiQUrUlwxpEwm6OJaA+PQ1OKlx80ZRYVJwC9Wr6gml9TRqjlOvO7VLXx4+K1v 3T2u8ha0qPEApKvIM5s2Ws2Npj4qkAbjHl8hxfTOYe++HQgjbNYVnJnAZ+7nm0CN u9YsM61C+u9W5X3ROmXiKcqsZf2VSekJhQNXXebgqs5BB+rG+tiIWijuzst9IF5S 3gpSF0hDaZf3X4suEe6w =i6JE -----END PGP SIGNATURE----- --oDFa8iZvINJ+yMtC-- From owner-freebsd-arch@FreeBSD.ORG Mon Jan 7 18:22:48 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 83877917; Mon, 7 Jan 2013 18:22:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id C02D3B3D; Mon, 7 Jan 2013 18:22:47 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id r07IMaZe066239; Mon, 7 Jan 2013 20:22:36 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.3 kib.kiev.ua r07IMaZe066239 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id r07IMZkb066238; Mon, 7 Jan 2013 20:22:35 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 7 Jan 2013 20:22:35 +0200 From: Konstantin Belousov To: arch@freebsd.org Subject: Fast sigblock (AKA rtld speedup) Message-ID: <20130107182235.GA65279@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sdtB3X0nJg68CQEu" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 18:22:48 -0000 --sdtB3X0nJg68CQEu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Below is the forward of the patch for which I failed to obtain a private review. Might be, the list generates more responses. Our rtld has a performance bootleneck, typically exposed by the images with the lot of the run-time relocation processing, and by the C++ exception handling. We block the signals delivery during the rtld performing the relocations, as well as for the dl_iterate_phdr(3) (the later is used for finding the dwarf unwinding tables). The signal blocking is needed to allow the rtld to resolve the symbols for the signal handlers in the safe way, but also causes 2 syscalls overhead per each rtld entry. The proposed approach allows to shave off those two syscalls, doubling the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. Date: Mon, 13 Aug 2012 15:26:00 +0300 =46rom: Konstantin Belousov =2E.. The basic idea is to implement sigprocmask() as single write into usermode address. If kernel needs to calculate the signal mask for current thread, it takes into the consideration non-zero value of the word at the agreed address. Also, usermode is informed about signals which were put on hold due to fast sigblock active. As I said, on my measurements in microbenchmark that did throw/catch in a loop, I see equal user and system time spent for unpatched system, and same user time with zero system time on patched system. Patch can be improved further, e.g. it would be nice to allow rtld to fall back to sigprocmask(2) if kernel does not support fast sigblock, to prevent flag day. Also, the mask enforced by fast sigblock can be made configurable. Note that libthr already blocks signals by catching them, and not using rtld service in the first line handler. I tried to make the change in the spirit of libthr interceptors, but handoff to libthr appears too complicated to work. In fact, libthr can be changed to start using fast sigblock instead of wrapping sigaction, but this is out of scope of the proposal right now. Please comment. diff --git a/lib/libc/sys/Symbol.map b/lib/libc/sys/Symbol.map index 6888ea0..3b75539 100644 --- a/lib/libc/sys/Symbol.map +++ b/lib/libc/sys/Symbol.map @@ -546,6 +546,7 @@ FBSDprivate_1.0 { __sys_extattr_set_link; _extattrctl; __sys_extattrctl; + __sys_fast_sigblock; _fchdir; __sys_fchdir; _fchflags; diff --git a/libexec/rtld-elf/rtld_lock.c b/libexec/rtld-elf/rtld_lock.c index d1563e5..50c52c6 100644 --- a/libexec/rtld-elf/rtld_lock.c +++ b/libexec/rtld-elf/rtld_lock.c @@ -43,6 +43,7 @@ */ =20 #include +#include #include #include #include @@ -59,8 +60,8 @@ typedef struct Struct_Lock { void *base; } Lock; =20 -static sigset_t fullsigmask, oldsigmask; static int thread_flag; +static uint32_t fsigblock; =20 static void * def_lock_create() @@ -111,18 +112,26 @@ def_rlock_acquire(void *lock) } =20 static void +sig_fastunblock(void) +{ + uint32_t oldval; + + oldval =3D atomic_fetchadd_32(&fsigblock, -FAST_SIGBLOCK_INC); + if (oldval =3D=3D (FAST_SIGBLOCK_PEND | FAST_SIGBLOCK_INC)) + __sys_fast_sigblock(FAST_SIGBLOCK_UNBLOCK, NULL); +} + +static void def_wlock_acquire(void *lock) { Lock *l =3D (Lock *)lock; - sigset_t tmp_oldsigmask; =20 for ( ; ; ) { - sigprocmask(SIG_BLOCK, &fullsigmask, &tmp_oldsigmask); + atomic_add_rel_32(&fsigblock, FAST_SIGBLOCK_INC); if (atomic_cmpset_acq_int(&l->lock, 0, WAFLAG)) break; - sigprocmask(SIG_SETMASK, &tmp_oldsigmask, NULL); + sig_fastunblock(); } - oldsigmask =3D tmp_oldsigmask; } =20 static void @@ -134,7 +143,7 @@ def_lock_release(void *lock) atomic_add_rel_int(&l->lock, -RC_INCR); else { atomic_add_rel_int(&l->lock, -WAFLAG); - sigprocmask(SIG_SETMASK, &oldsigmask, NULL); + sig_fastunblock(); } } =20 @@ -286,19 +295,7 @@ lockdflt_init() =20 memcpy(&lockinfo, &deflockinfo, sizeof(lockinfo)); _rtld_thread_init(NULL); - /* - * Construct a mask to block all signals except traps which might - * conceivably be generated within the dynamic linker itself. - */ - sigfillset(&fullsigmask); - sigdelset(&fullsigmask, SIGILL); - sigdelset(&fullsigmask, SIGTRAP); - sigdelset(&fullsigmask, SIGABRT); - sigdelset(&fullsigmask, SIGEMT); - sigdelset(&fullsigmask, SIGFPE); - sigdelset(&fullsigmask, SIGBUS); - sigdelset(&fullsigmask, SIGSEGV); - sigdelset(&fullsigmask, SIGSYS); + __sys_fast_sigblock(FAST_SIGBLOCK_SETPTR, &fsigblock); } =20 /* @@ -319,7 +316,10 @@ _rtld_thread_init(struct RtldLockInfo *pli) =20 if (pli =3D=3D NULL) pli =3D &deflockinfo; - + else { + fsigblock =3D 0; + __sys_fast_sigblock(FAST_SIGBLOCK_UNSETPTR, NULL); + } =20 for (i =3D 0; i < RTLD_LOCK_CNT; i++) if ((locks[i] =3D pli->lock_create()) =3D=3D NULL) diff --git a/sys/compat/freebsd32/syscalls.master b/sys/compat/freebsd32/sy= scalls.master index 0891e41..f9e8b9e 100644 --- a/sys/compat/freebsd32/syscalls.master +++ b/sys/compat/freebsd32/syscalls.master @@ -997,3 +997,4 @@ uint32_t offset1, uint32_t offset2,\ uint32_t len1, uint32_t len2, \ int advice); } +532 AUE_NULL NOPROTO { int fast_sigblock(int cmd, uint32_t *ptr); } diff --git a/sys/kern/kern_exec.c b/sys/kern/kern_exec.c index 90f7311..8a3cd15 100644 --- a/sys/kern/kern_exec.c +++ b/sys/kern/kern_exec.c @@ -1031,6 +1031,7 @@ exec_new_vmspace(imgp, sv) int error; struct proc *p =3D imgp->proc; struct vmspace *vmspace =3D p->p_vmspace; + struct thread *td =3D curthread; vm_object_t obj; vm_offset_t sv_minuser, stack_addr; vm_map_t map; @@ -1039,6 +1040,10 @@ exec_new_vmspace(imgp, sv) imgp->vmspace_destroyed =3D 1; imgp->sysent =3D sv; =20 + td->td_pflags &=3D ~TDP_FAST_SIGBLOCK; + td->td_sigblock_ptr =3D NULL; + td->td_sigblock_val =3D 0; + /* May be called with Giant held */ EVENTHANDLER_INVOKE(process_exec, p, imgp); =20 diff --git a/sys/kern/kern_sig.c b/sys/kern/kern_sig.c index 2685a8b..3c8a2f7 100644 --- a/sys/kern/kern_sig.c +++ b/sys/kern/kern_sig.c @@ -230,6 +230,7 @@ static int sigproptbl[NSIG] =3D { }; =20 static void reschedule_signals(struct proc *p, sigset_t block, int flags); +static sigset_t fastblock_mask; =20 static void sigqueue_start(void) @@ -240,6 +241,16 @@ sigqueue_start(void) p31b_setcfg(CTL_P1003_1B_REALTIME_SIGNALS, _POSIX_REALTIME_SIGNALS); p31b_setcfg(CTL_P1003_1B_RTSIG_MAX, SIGRTMAX - SIGRTMIN + 1); p31b_setcfg(CTL_P1003_1B_SIGQUEUE_MAX, max_pending_per_proc); + SIGFILLSET(fastblock_mask); + SIG_CANTMASK(fastblock_mask); + SIGDELSET(fastblock_mask, SIGILL); + SIGDELSET(fastblock_mask, SIGTRAP); + SIGDELSET(fastblock_mask, SIGABRT); + SIGDELSET(fastblock_mask, SIGEMT); + SIGDELSET(fastblock_mask, SIGFPE); + SIGDELSET(fastblock_mask, SIGBUS); + SIGDELSET(fastblock_mask, SIGSEGV); + SIGDELSET(fastblock_mask, SIGSYS); } =20 ksiginfo_t * @@ -2525,6 +2536,7 @@ issignal(struct thread *td, int stop_allowed) struct sigqueue *queue; sigset_t sigpending; int sig, prop, newsig; + uint32_t oldval; =20 p =3D td->td_proc; ps =3D p->p_sigacts; @@ -2541,6 +2553,32 @@ issignal(struct thread *td, int stop_allowed) SIG_STOPSIGMASK(sigpending); if (SIGISEMPTY(sigpending)) /* no signal to send */ return (0); + + /* + * Do fast sigblock if requested by usermode. Since + * we do know that there was a signal pending at this + * point, set the FAST_SIGBLOCK_PEND as indicator for + * usermode to perform a dummy call to + * FAST_SIGBLOCK_UNBLOCK, which causes immediate + * delivery of postponed pending signal. + */ + if ((td->td_pflags & TDP_FAST_SIGBLOCK) !=3D 0) { + if (td->td_sigblock_val !=3D 0) + SIGSETNAND(sigpending, fastblock_mask); + if (SIGISEMPTY(sigpending)) { + oldval =3D fuword32(td->td_sigblock_ptr); + if (oldval =3D=3D -1) { + fetch_fast_sigblock_failed(td, 0); + return (0); + } + oldval |=3D FAST_SIGBLOCK_PEND; + if (suword32(td->td_sigblock_ptr, oldval) =3D=3D -1) + fetch_fast_sigblock_failed(td, 1); + td->td_sigblock_val =3D oldval; + return (0); + } + } + sig =3D sig_ffs(&sigpending); =20 if (p->p_stops & S_SIG) { @@ -3456,3 +3494,92 @@ sigacts_shared(struct sigacts *ps) mtx_unlock(&ps->ps_mtx); return (shared); } + +int +sys_fast_sigblock(struct thread *td, struct fast_sigblock_args *uap) +{ + struct proc *p; + int error; + uint32_t oldval; + + error =3D 0; + switch (uap->cmd) { + case FAST_SIGBLOCK_SETPTR: + if ((td->td_pflags & TDP_FAST_SIGBLOCK) !=3D 0) + error =3D EBUSY; + else if (((uintptr_t)(uap->ptr) & (sizeof(uint32_t) - 1)) !=3D 0) + error =3D EINVAL; + else { + td->td_pflags |=3D TDP_FAST_SIGBLOCK; + td->td_sigblock_ptr =3D uap->ptr; + } + break; + + case FAST_SIGBLOCK_UNBLOCK: + if ((td->td_pflags & TDP_FAST_SIGBLOCK) !=3D 0) { + oldval =3D casuword32(td->td_sigblock_ptr, + FAST_SIGBLOCK_PEND, 0); + if (oldval =3D=3D (uint32_t)-1) + error =3D EFAULT; + else if (oldval !=3D FAST_SIGBLOCK_PEND) + error =3D EBUSY; + else + td->td_sigblock_val =3D 0; + } else + error =3D EDOOFUS; + + /* + * Rely on normal ast mechanism to deliver pending + * signals to current thread. But notify others about + * fake unblock. + */ + p =3D td->td_proc; + if (error =3D=3D 0 && p->p_numthreads !=3D 1) { + PROC_LOCK(p); + reschedule_signals(p, td->td_sigmask, 0); + PROC_UNLOCK(p); + } + break; + + case FAST_SIGBLOCK_UNSETPTR: + if ((td->td_pflags & TDP_FAST_SIGBLOCK) !=3D 0) { + error =3D copyin(td->td_sigblock_ptr, &oldval, + sizeof(uint32_t)); + if (error !=3D 0) + break; + if (oldval !=3D 0 && oldval !=3D FAST_SIGBLOCK_PEND) + error =3D EBUSY; + else { + td->td_pflags &=3D ~TDP_FAST_SIGBLOCK; + td->td_sigblock_val =3D 0; + } + } else + error =3D EDOOFUS; + break; + } + return (error); +} + +void +fetch_fast_sigblock(struct thread *td) +{ + + if ((td->td_pflags & TDP_FAST_SIGBLOCK) =3D=3D 0) + return; + td->td_sigblock_val =3D fuword32(td->td_sigblock_ptr); + if (td->td_sigblock_val =3D=3D -1) + fetch_fast_sigblock_failed(td, 0); +} + +void +fetch_fast_sigblock_failed(struct thread *td, int write) +{ + ksiginfo_t ksi; + + td->td_sigblock_val =3D 0; + ksiginfo_init_trap(&ksi); + ksi.ksi_signo =3D SIGSEGV; + ksi.ksi_code =3D write ? SEGV_ACCERR : SEGV_MAPERR; + ksi.ksi_addr =3D td->td_sigblock_ptr; + trapsignal(td, &ksi); +} diff --git a/sys/kern/subr_syscall.c b/sys/kern/subr_syscall.c index 5aee684..77b250d 100644 --- a/sys/kern/subr_syscall.c +++ b/sys/kern/subr_syscall.c @@ -131,6 +131,12 @@ syscallenter(struct thread *td, struct syscall_args *s= a) sa->callp, sa->args, 0); #endif =20 + /* + * Fetch fast sigblock value at the time of syscall + * entry because sleepqueue primitives might call + * cursig(). + */ + fetch_fast_sigblock(td); AUDIT_SYSCALL_ENTER(sa->code, td); error =3D (sa->callp->sy_call)(td, sa->args); AUDIT_SYSCALL_EXIT(error, td); diff --git a/sys/kern/subr_trap.c b/sys/kern/subr_trap.c index 095bbdf..66e485b 100644 --- a/sys/kern/subr_trap.c +++ b/sys/kern/subr_trap.c @@ -237,6 +237,7 @@ ast(struct trapframe *framep) */ if (flags & TDF_NEEDSIGCHK || p->p_pendingcnt > 0 || !SIGISEMPTY(p->p_siglist)) { + fetch_fast_sigblock(td); PROC_LOCK(p); mtx_lock(&p->p_sigacts->ps_mtx); while ((sig =3D cursig(td, SIG_STOP_ALLOWED)) !=3D 0) diff --git a/sys/kern/syscalls.master b/sys/kern/syscalls.master index f62dad7..28a9393 100644 --- a/sys/kern/syscalls.master +++ b/sys/kern/syscalls.master @@ -951,5 +951,6 @@ off_t offset, off_t len); } 531 AUE_NULL STD { int posix_fadvise(int fd, off_t offset, \ off_t len, int advice); } +532 AUE_NULL STD { int fast_sigblock(int cmd, uint32_t *ptr); } ; Please copy any additions and changes to the following compatability tab= les: ; sys/compat/freebsd32/syscalls.master diff --git a/sys/sys/proc.h b/sys/sys/proc.h index 06df632..4899ca2 100644 --- a/sys/sys/proc.h +++ b/sys/sys/proc.h @@ -272,6 +272,9 @@ struct thread { struct osd td_osd; /* (k) Object specific data. */ struct vm_map_entry *td_map_def_user; /* (k) Deferred entries. */ pid_t td_dbg_forked; /* (c) Child pid for debugger. */ + void *td_sigblock_ptr; /* (k) uptr for fast sigblock. */ + uint32_t td_sigblock_val; /* (k) fast sigblock value at + kernel entry. */ #define td_endzero td_sigmask =20 /* Copied during fork1() or create_thread(). */ @@ -424,6 +427,7 @@ do { \ #define TDP_RESETSPUR 0x04000000 /* Reset spurious page fault history. */ #define TDP_NERRNO 0x08000000 /* Last errno is already in td_errno */ #define TDP_UIOHELD 0x10000000 /* Current uio has pages held in td_ma */ +#define TDP_FAST_SIGBLOCK 0x20000000 /* Fast sigblock active */ =20 /* * Reasons that the current thread can not be run yet. diff --git a/sys/sys/signalvar.h b/sys/sys/signalvar.h index 71685e7..68cca58 100644 --- a/sys/sys/signalvar.h +++ b/sys/sys/signalvar.h @@ -250,6 +250,20 @@ typedef struct sigqueue { /* Flags for ksi_flags */ #define SQ_INIT 0x01 =20 +/* + * Fast_sigblock + */ +#define FAST_SIGBLOCK_SETPTR 1 +#define FAST_SIGBLOCK_UNBLOCK 2 +#define FAST_SIGBLOCK_UNSETPTR 3 + +#define FAST_SIGBLOCK_PEND 0x1 +#define FAST_SIGBLOCK_INC 0x10 + +#ifndef _KERNEL +int __sys_fast_sigblock(int cmd, void *ptr); +#endif + #ifdef _KERNEL =20 /* Return nonzero if process p has an unmasked pending signal. */ @@ -329,6 +343,8 @@ extern struct mtx sigio_lock; =20 int cursig(struct thread *td, int stop_allowed); void execsigs(struct proc *p); +void fetch_fast_sigblock(struct thread *td); +void fetch_fast_sigblock_failed(struct thread *td, int write); void gsignal(int pgid, int sig, ksiginfo_t *ksi); void killproc(struct proc *p, char *why); ksiginfo_t * ksiginfo_alloc(int wait); ----- End forwarded message ----- --sdtB3X0nJg68CQEu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ6xJqAAoJEJDCuSvBvK1BiSMP/RrPt9uEu5jsbL+GbZ4EPALD veW06QvySP6TQ65jBFHoimZGXn3uSdduXEPBTqXsd5ECgbuptcy+luWORkc3b8Yz NuoTczRd6WJCC0J9WsxC3FgBVynq1iBsARfe4sg+SvpmlwnIkKON22E3FSy6k42z eYN/IDtYrC0nRzDSbBtkuIJ7QSXs2btY6vJHQhBMjhM/O7UtoZMF8tx/2T0khbvR Dt/HAM6HJBUYDOuYMvUJCx7NgogXcbHYDHD4FHpGriBmFNXxtAy/RkP0Xuv+9MO9 gQ5S22Uj/hkxwiz/nHMm+QNRNtqokywATLPm7LIwykMvPiwc0AwhXkKXFu87myjI Lk7aKFk/MuXERwZ6X8FU9ieodO4eJECgXF3Xsmkw3QxMk1KyhrslXbmyl7wLnuel dFHlOMAMjKx7Xka7sm3hZNIpztDJKS6cdys+VYYxHj2j0T45hEOO4Qk643dFb2sa a+DQdRi1p/dBY4fu1nmdRgGtbzl3u7ClqyEf9j+lWrLawgQ4QkWtUNlBBX8hohLz FuGNzMDRQs7UupUVHJfaHpoBKzRg1hhKczGux1pdd4wLTZBtvb47At4tVGquGyMT sDrN8vK/jPofhqTPg2IkRI1LnRQ6BZ5anwRiDmewO7QTyBkngO7Cdd58hirwQC2o 0PU6sx0kaeTntQH7hMVL =G0Up -----END PGP SIGNATURE----- --sdtB3X0nJg68CQEu-- From owner-freebsd-arch@FreeBSD.ORG Mon Jan 7 20:18:51 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 12DCA58C; Mon, 7 Jan 2013 20:18:51 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id AB44733C; Mon, 7 Jan 2013 20:18:50 +0000 (UTC) Received: from JRE-MBP-2.local (c-50-143-148-105.hsd1.ca.comcast.net [50.143.148.105]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id r07KImEt070010 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 7 Jan 2013 12:18:49 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <50EB2DA1.5090306@freebsd.org> Date: Mon, 07 Jan 2013 12:18:41 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> In-Reply-To: <20130107182235.GA65279@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 20:18:51 -0000 On 1/7/13 10:22 AM, Konstantin Belousov wrote: > Below is the forward of the patch for which I failed to obtain a private > review. Might be, the list generates more responses. > > Our rtld has a performance bootleneck, typically exposed by the images > with the lot of the run-time relocation processing, and by the C++ > exception handling. We block the signals delivery during the rtld > performing the relocations, as well as for the dl_iterate_phdr(3) (the > later is used for finding the dwarf unwinding tables). > > The signal blocking is needed to allow the rtld to resolve the symbols > for the signal handlers in the safe way, but also causes 2 syscalls > overhead per each rtld entry. > > The proposed approach allows to shave off those two syscalls, doubling > the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. > > Date: Mon, 13 Aug 2012 15:26:00 +0300 > From: Konstantin Belousov > > ... > > The basic idea is to implement sigprocmask() as single write into usermode > address. If kernel needs to calculate the signal mask for current thread, > it takes into the consideration non-zero value of the word at the agreed > address. Also, usermode is informed about signals which were put on hold > due to fast sigblock active. > > As I said, on my measurements in microbenchmark that did throw/catch in > a loop, I see equal user and system time spent for unpatched system, and > same user time with zero system time on patched system. > > Patch can be improved further, e.g. it would be nice to allow rtld to fall > back to sigprocmask(2) if kernel does not support fast sigblock, to prevent > flag day. Also, the mask enforced by fast sigblock can be made configurable. > > Note that libthr already blocks signals by catching them, and not using rtld > service in the first line handler. I tried to make the change in the spirit > of libthr interceptors, but handoff to libthr appears too complicated to > work. In fact, libthr can be changed to start using fast sigblock instead > of wrapping sigaction, but this is out of scope of the proposal right now. Is there any danger that an upriveliged user program can trick the kernel into doing anything it shouldn't by manipulating either the agreed upon address or the contents of the mask at the address? (even reading something by seeing what sigs get masked) This was an issue with the M:N threading package and resulted in extra syscalls to get around the problem. (I forget the details). From owner-freebsd-arch@FreeBSD.ORG Mon Jan 7 20:27:19 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DFBDE88D; Mon, 7 Jan 2013 20:27:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 3AA493DB; Mon, 7 Jan 2013 20:27:19 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id r07KREqv078357; Mon, 7 Jan 2013 22:27:14 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.3 kib.kiev.ua r07KREqv078357 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id r07KREiw078356; Mon, 7 Jan 2013 22:27:14 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 7 Jan 2013 22:27:14 +0200 From: Konstantin Belousov To: Julian Elischer Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130107202714.GY82219@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <50EB2DA1.5090306@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6BJIGhnDLgQ8Wrmm" Content-Disposition: inline In-Reply-To: <50EB2DA1.5090306@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 20:27:19 -0000 --6BJIGhnDLgQ8Wrmm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 07, 2013 at 12:18:41PM -0800, Julian Elischer wrote: > On 1/7/13 10:22 AM, Konstantin Belousov wrote: > > Below is the forward of the patch for which I failed to obtain a private > > review. Might be, the list generates more responses. > > > > Our rtld has a performance bootleneck, typically exposed by the images > > with the lot of the run-time relocation processing, and by the C++ > > exception handling. We block the signals delivery during the rtld > > performing the relocations, as well as for the dl_iterate_phdr(3) (the > > later is used for finding the dwarf unwinding tables). > > > > The signal blocking is needed to allow the rtld to resolve the symbols > > for the signal handlers in the safe way, but also causes 2 syscalls > > overhead per each rtld entry. > > > > The proposed approach allows to shave off those two syscalls, doubling > > the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. > > > > Date: Mon, 13 Aug 2012 15:26:00 +0300 > > From: Konstantin Belousov > > > > ... > > > > The basic idea is to implement sigprocmask() as single write into userm= ode > > address. If kernel needs to calculate the signal mask for current threa= d, > > it takes into the consideration non-zero value of the word at the agreed > > address. Also, usermode is informed about signals which were put on hold > > due to fast sigblock active. > > > > As I said, on my measurements in microbenchmark that did throw/catch in > > a loop, I see equal user and system time spent for unpatched system, and > > same user time with zero system time on patched system. > > > > Patch can be improved further, e.g. it would be nice to allow rtld to f= all > > back to sigprocmask(2) if kernel does not support fast sigblock, to pre= vent > > flag day. Also, the mask enforced by fast sigblock can be made configur= able. > > > > Note that libthr already blocks signals by catching them, and not using= rtld > > service in the first line handler. I tried to make the change in the sp= irit > > of libthr interceptors, but handoff to libthr appears too complicated to > > work. In fact, libthr can be changed to start using fast sigblock inste= ad > > of wrapping sigaction, but this is out of scope of the proposal right n= ow. > Is there any danger that an upriveliged user program can trick the kernel > into doing anything it shouldn't by manipulating either the agreed upon > address or the contents of the mask at the address? (even reading > something by seeing what sigs get masked) I do not see how would anything like this possible. >=20 > This was an issue with the M:N threading package and resulted in extra=20 > syscalls > to get around the problem. (I forget the details). Would be great to dig up the details indeed. I cannot comment otherwise. --6BJIGhnDLgQ8Wrmm Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ6y+iAAoJEJDCuSvBvK1BaL4P/1vDmzlkJ6vxLZOwmTGbnoRe kaidamizUqMmlB2Fp4ruR2+6QeLn7oY2J0wSi7vtH/s+tynuQVSgfUgfxrqo68Lw 0cI4IWu/8TekYFLKlk3rDvpeHz8u3A3gzQOPXXwkutv07ZswJFN9moevMeAp+FA6 Kjh8VGzMaJACXnKk3qeSZEKWuyGYkcNA0xuY0TxtL4z6G6R856u2owmM1KA4iyC5 L9ZdaH2ZPCM8FWxJOBYbM7sIT4M99X9WB86RKGTBM7BgZdXG2yBU8f9KCTz4wkpP E9k985BJAcHLV8sf/1NQKG6ozMGRvCe1WAL2FoQ/7Cadr55ZbT8aPgh4xKcfKmWG v7FX3xhMZ11pufznNaJ05VChK/dnOM7EhXqIaV7nA99NTt5PebGbNVqCu16RxaUo hWuBDwN//JUmKGQz4Gzu/eVJLsOlKFGUX5AooskaG3+Dl6OL9sPcG9EfhAxLQRm6 VFOJ1jrfb+Ab3PEu44CYugl7OHIu3+rcXGNPI+mTLp3DmNEYes7u234fmO6aAqBl 8KWKf23WTdJjYRTrUSP6R/EKnfOi8yNVdCKXERn5ZPutYIQpTF+LuQRwyE4ct7do UMYTjXsVvu1DdY9LpMDp4fnXd4yKZaKXDs6pxkhvNriqvZvXB4Nmf/JCpxZEY1h5 JXDUqsOBRBj4FBKNV9A5 =JIfl -----END PGP SIGNATURE----- --6BJIGhnDLgQ8Wrmm-- From owner-freebsd-arch@FreeBSD.ORG Mon Jan 7 20:42:04 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2EE15D7F; Mon, 7 Jan 2013 20:42:04 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 1AA496A1; Mon, 7 Jan 2013 20:42:03 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (unknown [207.238.187.242]) by elvis.mu.org (Postfix) with ESMTPSA id 4209E1A3EA7; Mon, 7 Jan 2013 12:42:02 -0800 (PST) Message-ID: <50EB3319.6060303@mu.org> Date: Mon, 07 Jan 2013 15:42:01 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> In-Reply-To: <20130107182235.GA65279@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 20:42:04 -0000 On 1/7/13 1:22 PM, Konstantin Belousov wrote: > Below is the forward of the patch for which I failed to obtain a private > review. Might be, the list generates more responses. Very cool. Sorry for being rusty here, but is it safe to call fuword in the middle of issignal()? The reason I ask is because it looks like proc lock is required for this block, and I forget it fuword(9) can fault. If fuword(9) can fault then I don't think you can do this. You'll need to use something like fuswintr(9) or wire the page down using vslock(9) or something. I am probably missing something? -Alfred From owner-freebsd-arch@FreeBSD.ORG Tue Jan 8 00:08:47 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 12DA7870; Tue, 8 Jan 2013 00:08:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 80799EAC; Tue, 8 Jan 2013 00:08:46 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id r0808fMG010778; Tue, 8 Jan 2013 02:08:41 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r0808fMG010778 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id r0808e4J010777; Tue, 8 Jan 2013 02:08:40 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Jan 2013 02:08:40 +0200 From: Konstantin Belousov To: Alfred Perlstein Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130108000840.GA82219@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <50EB3319.6060303@mu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/SXoEQDAiXmUSAYo" Content-Disposition: inline In-Reply-To: <50EB3319.6060303@mu.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 00:08:47 -0000 --/SXoEQDAiXmUSAYo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 07, 2013 at 03:42:01PM -0500, Alfred Perlstein wrote: > On 1/7/13 1:22 PM, Konstantin Belousov wrote: > > Below is the forward of the patch for which I failed to obtain a private > > review. Might be, the list generates more responses. > Very cool. >=20 > Sorry for being rusty here, but is it safe to call fuword in the middle= =20 > of issignal()? >=20 > The reason I ask is because it looks like proc lock is required for this= =20 > block, and I forget it fuword(9) can fault. >=20 > If fuword(9) can fault then I don't think you can do this. >=20 > You'll need to use something like fuswintr(9) or wire the page down=20 > using vslock(9) or something. >=20 > I am probably missing something? No, you are completely right. This is a serious bug. Thank you. I was careful to code the td_sigblock_val prefetch etc, but failed to do the update properly. There is no reason to execute the update in the issignal() at all, the setting of the pending bit can be postponed to the moment after the postsig() loop was executed. I uploaded the updated but not yet tested patch at http://people.freebsd.org/~kib/misc/rtld-sigblock.1.patch --/SXoEQDAiXmUSAYo Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ62OIAAoJEJDCuSvBvK1B3UwP/AtqY913PEsaOOuAuMCBfgo2 yYoBB/d4xaDzMNglQ6TxTNiqy4tN6Wd3vki38VrW4Kc4QbAmL4Yi+w08D8LkXqQ0 OuwP3PuHz6s9eoSOY+rIkHVhxG624SxnzCiKrSS6X8wT1Mky/dvYH/YsfgkXUTsH gbVJdP7P8jIkst9Rajepq6n9jjHqaWSJZ7JIMrQenVRy/x3PteidWGhL+UjIPBrw pOAXuwsvmr+NSqNhsiPCD7Q7yDOKBz3SALLjI/llc4cr1lC5j9C65bAa7gJKTHWZ Gj0jDz2+GI7CyM18TrUqMNspnh4WqXCSlAzhfsSG5LmS9Zl2FlI0T2UvOxLm9kl4 S5w4YofRVRUIjLyaXtebTGY29HE2mDripJkE6dYnU8sH0t7vShoLSfEvrHalBPNy px/Qtq8pRZWzPrnq/maFXCgqBwSfGEmuRhLX6e8EFHw35rnaN0f4p+cbqArJLpqB GIXksbN19Pre8LQfK4iNJxNqhUd0OniRqMP1qbqBU2/IrDTEmdhnT411fr1Wtvtz N1bLmEgou0Pgm5JC3h+FxbpnevvHaXgJGS01rt5L4GC/yAHmO60qf0KqbyRveMqu rxVaO7g1W+VeRpIxW9MUsk1dtyJosu/+higMc2iI063+YeZcvjRimLFOByHW7FXH BMfeFRMFZV7/ocl1IRdV =7eJr -----END PGP SIGNATURE----- --/SXoEQDAiXmUSAYo-- From owner-freebsd-arch@FreeBSD.ORG Tue Jan 8 05:09:24 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 99CBD333; Tue, 8 Jan 2013 05:09:24 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 746C8A7C; Tue, 8 Jan 2013 05:09:24 +0000 (UTC) Received: from xyf.my.dom (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r0859MdR082717; Tue, 8 Jan 2013 05:09:23 GMT (envelope-from davidxu@freebsd.org) Message-ID: <50EBAA1F.9070303@freebsd.org> Date: Tue, 08 Jan 2013 13:09:51 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:14.0) Gecko/20120822 Thunderbird/14.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> In-Reply-To: <20130107182235.GA65279@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 05:09:24 -0000 On 2013/01/08 02:22, Konstantin Belousov wrote: > Below is the forward of the patch for which I failed to obtain a private > review. Might be, the list generates more responses. > > Our rtld has a performance bootleneck, typically exposed by the images > with the lot of the run-time relocation processing, and by the C++ > exception handling. We block the signals delivery during the rtld > performing the relocations, as well as for the dl_iterate_phdr(3) (the > later is used for finding the dwarf unwinding tables). > > The signal blocking is needed to allow the rtld to resolve the symbols > for the signal handlers in the safe way, but also causes 2 syscalls > overhead per each rtld entry. > > The proposed approach allows to shave off those two syscalls, doubling > the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. > > Date: Mon, 13 Aug 2012 15:26:00 +0300 > From: Konstantin Belousov > > ... > > The basic idea is to implement sigprocmask() as single write into usermode > address. If kernel needs to calculate the signal mask for current thread, > it takes into the consideration non-zero value of the word at the agreed > address. Also, usermode is informed about signals which were put on hold > due to fast sigblock active. > > As I said, on my measurements in microbenchmark that did throw/catch in > a loop, I see equal user and system time spent for unpatched system, and > same user time with zero system time on patched system. > > Patch can be improved further, e.g. it would be nice to allow rtld to fall > back to sigprocmask(2) if kernel does not support fast sigblock, to prevent > flag day. Also, the mask enforced by fast sigblock can be made configurable. > > Note that libthr already blocks signals by catching them, and not using rtld > service in the first line handler. I tried to make the change in the spirit > of libthr interceptors, but handoff to libthr appears too complicated to > work. In fact, libthr can be changed to start using fast sigblock instead > of wrapping sigaction, but this is out of scope of the proposal right now. > > Please comment. > > diff --git a/lib/libc/sys/Symbol.map b/lib/libc/sys/Symbol.map > index 6888ea0..3b75539 100644 > --- a/lib/libc/sys/Symbol.map > +++ b/lib/libc/sys/Symbol.map > @@ -546,6 +546,7 @@ FBSDprivate_1.0 { > __sys_extattr_set_link; > _extattrctl; > __sys_extattrctl; > + __sys_fast_sigblock; > _fchdir; > __sys_fchdir; > _fchflags; > diff --git a/libexec/rtld-elf/rtld_lock.c b/libexec/rtld-elf/rtld_lock.c > index d1563e5..50c52c6 100644 > --- a/libexec/rtld-elf/rtld_lock.c > +++ b/libexec/rtld-elf/rtld_lock.c > @@ -43,6 +43,7 @@ > */ > > #include > +#include > #include > #include > #include > @@ -59,8 +60,8 @@ typedef struct Struct_Lock { > void *base; > } Lock; > > -static sigset_t fullsigmask, oldsigmask; > static int thread_flag; > +static uint32_t fsigblock; > > static void * > def_lock_create() > @@ -111,18 +112,26 @@ def_rlock_acquire(void *lock) > } > > static void > +sig_fastunblock(void) > +{ > + uint32_t oldval; > + > + oldval = atomic_fetchadd_32(&fsigblock, -FAST_SIGBLOCK_INC); > + if (oldval == (FAST_SIGBLOCK_PEND | FAST_SIGBLOCK_INC)) > + __sys_fast_sigblock(FAST_SIGBLOCK_UNBLOCK, NULL); > +} > + > +static void > def_wlock_acquire(void *lock) > { > Lock *l = (Lock *)lock; > - sigset_t tmp_oldsigmask; > > for ( ; ; ) { > - sigprocmask(SIG_BLOCK, &fullsigmask, &tmp_oldsigmask); > + atomic_add_rel_32(&fsigblock, FAST_SIGBLOCK_INC); > if (atomic_cmpset_acq_int(&l->lock, 0, WAFLAG)) > break; > - sigprocmask(SIG_SETMASK, &tmp_oldsigmask, NULL); > + sig_fastunblock(); > } > - oldsigmask = tmp_oldsigmask; > } > > static void > @@ -134,7 +143,7 @@ def_lock_release(void *lock) > atomic_add_rel_int(&l->lock, -RC_INCR); > else { > atomic_add_rel_int(&l->lock, -WAFLAG); > - sigprocmask(SIG_SETMASK, &oldsigmask, NULL); > + sig_fastunblock(); > } > } > > @@ -286,19 +295,7 @@ lockdflt_init() > > memcpy(&lockinfo, &deflockinfo, sizeof(lockinfo)); > _rtld_thread_init(NULL); > - /* > - * Construct a mask to block all signals except traps which might > - * conceivably be generated within the dynamic linker itself. > - */ > - sigfillset(&fullsigmask); > - sigdelset(&fullsigmask, SIGILL); > - sigdelset(&fullsigmask, SIGTRAP); > - sigdelset(&fullsigmask, SIGABRT); > - sigdelset(&fullsigmask, SIGEMT); > - sigdelset(&fullsigmask, SIGFPE); > - sigdelset(&fullsigmask, SIGBUS); > - sigdelset(&fullsigmask, SIGSEGV); > - sigdelset(&fullsigmask, SIGSYS); > + __sys_fast_sigblock(FAST_SIGBLOCK_SETPTR, &fsigblock); > } > > /* > @@ -319,7 +316,10 @@ _rtld_thread_init(struct RtldLockInfo *pli) > > if (pli == NULL) > pli = &deflockinfo; > - > + else { > + fsigblock = 0; > + __sys_fast_sigblock(FAST_SIGBLOCK_UNSETPTR, NULL); > + } > > for (i = 0; i < RTLD_LOCK_CNT; i++) > if ((locks[i] = pli->lock_create()) == NULL) > diff --git a/sys/compat/freebsd32/syscalls.master b/sys/compat/freebsd32/syscalls.master > index 0891e41..f9e8b9e 100644 > --- a/sys/compat/freebsd32/syscalls.master > +++ b/sys/compat/freebsd32/syscalls.master > @@ -997,3 +997,4 @@ > uint32_t offset1, uint32_t offset2,\ > uint32_t len1, uint32_t len2, \ > int advice); } > +532 AUE_NULL NOPROTO { int fast_sigblock(int cmd, uint32_t *ptr); } > diff --git a/sys/kern/kern_exec.c b/sys/kern/kern_exec.c > index 90f7311..8a3cd15 100644 > --- a/sys/kern/kern_exec.c > +++ b/sys/kern/kern_exec.c > @@ -1031,6 +1031,7 @@ exec_new_vmspace(imgp, sv) > int error; > struct proc *p = imgp->proc; > struct vmspace *vmspace = p->p_vmspace; > + struct thread *td = curthread; > vm_object_t obj; > vm_offset_t sv_minuser, stack_addr; > vm_map_t map; > @@ -1039,6 +1040,10 @@ exec_new_vmspace(imgp, sv) > imgp->vmspace_destroyed = 1; > imgp->sysent = sv; > > + td->td_pflags &= ~TDP_FAST_SIGBLOCK; > + td->td_sigblock_ptr = NULL; > + td->td_sigblock_val = 0; > + > /* May be called with Giant held */ > EVENTHANDLER_INVOKE(process_exec, p, imgp); > > diff --git a/sys/kern/kern_sig.c b/sys/kern/kern_sig.c > index 2685a8b..3c8a2f7 100644 > --- a/sys/kern/kern_sig.c > +++ b/sys/kern/kern_sig.c > @@ -230,6 +230,7 @@ static int sigproptbl[NSIG] = { > }; > > static void reschedule_signals(struct proc *p, sigset_t block, int flags); > +static sigset_t fastblock_mask; > > static void > sigqueue_start(void) > @@ -240,6 +241,16 @@ sigqueue_start(void) > p31b_setcfg(CTL_P1003_1B_REALTIME_SIGNALS, _POSIX_REALTIME_SIGNALS); > p31b_setcfg(CTL_P1003_1B_RTSIG_MAX, SIGRTMAX - SIGRTMIN + 1); > p31b_setcfg(CTL_P1003_1B_SIGQUEUE_MAX, max_pending_per_proc); > + SIGFILLSET(fastblock_mask); > + SIG_CANTMASK(fastblock_mask); > + SIGDELSET(fastblock_mask, SIGILL); > + SIGDELSET(fastblock_mask, SIGTRAP); > + SIGDELSET(fastblock_mask, SIGABRT); > + SIGDELSET(fastblock_mask, SIGEMT); > + SIGDELSET(fastblock_mask, SIGFPE); > + SIGDELSET(fastblock_mask, SIGBUS); > + SIGDELSET(fastblock_mask, SIGSEGV); > + SIGDELSET(fastblock_mask, SIGSYS); > } > > ksiginfo_t * > @@ -2525,6 +2536,7 @@ issignal(struct thread *td, int stop_allowed) > struct sigqueue *queue; > sigset_t sigpending; > int sig, prop, newsig; > + uint32_t oldval; > > p = td->td_proc; > ps = p->p_sigacts; > @@ -2541,6 +2553,32 @@ issignal(struct thread *td, int stop_allowed) > SIG_STOPSIGMASK(sigpending); > if (SIGISEMPTY(sigpending)) /* no signal to send */ > return (0); > + > + /* > + * Do fast sigblock if requested by usermode. Since > + * we do know that there was a signal pending at this > + * point, set the FAST_SIGBLOCK_PEND as indicator for > + * usermode to perform a dummy call to > + * FAST_SIGBLOCK_UNBLOCK, which causes immediate > + * delivery of postponed pending signal. > + */ > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) != 0) { > + if (td->td_sigblock_val != 0) > + SIGSETNAND(sigpending, fastblock_mask); > + if (SIGISEMPTY(sigpending)) { > + oldval = fuword32(td->td_sigblock_ptr); > + if (oldval == -1) { > + fetch_fast_sigblock_failed(td, 0); > + return (0); > + } > + oldval |= FAST_SIGBLOCK_PEND; > + if (suword32(td->td_sigblock_ptr, oldval) == -1) > + fetch_fast_sigblock_failed(td, 1); > + td->td_sigblock_val = oldval; > + return (0); > + } > + } > + > sig = sig_ffs(&sigpending); > > if (p->p_stops & S_SIG) { > @@ -3456,3 +3494,92 @@ sigacts_shared(struct sigacts *ps) > mtx_unlock(&ps->ps_mtx); > return (shared); > } > + > +int > +sys_fast_sigblock(struct thread *td, struct fast_sigblock_args *uap) > +{ > + struct proc *p; > + int error; > + uint32_t oldval; > + > + error = 0; > + switch (uap->cmd) { > + case FAST_SIGBLOCK_SETPTR: > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) != 0) > + error = EBUSY; > + else if (((uintptr_t)(uap->ptr) & (sizeof(uint32_t) - 1)) != 0) > + error = EINVAL; > + else { > + td->td_pflags |= TDP_FAST_SIGBLOCK; > + td->td_sigblock_ptr = uap->ptr; > + } > + break; > + > + case FAST_SIGBLOCK_UNBLOCK: > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) != 0) { > + oldval = casuword32(td->td_sigblock_ptr, > + FAST_SIGBLOCK_PEND, 0); > + if (oldval == (uint32_t)-1) > + error = EFAULT; > + else if (oldval != FAST_SIGBLOCK_PEND) > + error = EBUSY; > + else > + td->td_sigblock_val = 0; > + } else > + error = EDOOFUS; > + > + /* > + * Rely on normal ast mechanism to deliver pending > + * signals to current thread. But notify others about > + * fake unblock. > + */ > + p = td->td_proc; > + if (error == 0 && p->p_numthreads != 1) { > + PROC_LOCK(p); > + reschedule_signals(p, td->td_sigmask, 0); > + PROC_UNLOCK(p); > + } > + break; > + > + case FAST_SIGBLOCK_UNSETPTR: > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) != 0) { > + error = copyin(td->td_sigblock_ptr, &oldval, > + sizeof(uint32_t)); > + if (error != 0) > + break; > + if (oldval != 0 && oldval != FAST_SIGBLOCK_PEND) > + error = EBUSY; > + else { > + td->td_pflags &= ~TDP_FAST_SIGBLOCK; > + td->td_sigblock_val = 0; > + } > + } else > + error = EDOOFUS; > + break; > + } > + return (error); > +} > + > +void > +fetch_fast_sigblock(struct thread *td) > +{ > + > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) == 0) > + return; > + td->td_sigblock_val = fuword32(td->td_sigblock_ptr); > + if (td->td_sigblock_val == -1) > + fetch_fast_sigblock_failed(td, 0); > +} > + > +void > +fetch_fast_sigblock_failed(struct thread *td, int write) > +{ > + ksiginfo_t ksi; > + > + td->td_sigblock_val = 0; > + ksiginfo_init_trap(&ksi); > + ksi.ksi_signo = SIGSEGV; > + ksi.ksi_code = write ? SEGV_ACCERR : SEGV_MAPERR; > + ksi.ksi_addr = td->td_sigblock_ptr; > + trapsignal(td, &ksi); > +} > diff --git a/sys/kern/subr_syscall.c b/sys/kern/subr_syscall.c > index 5aee684..77b250d 100644 > --- a/sys/kern/subr_syscall.c > +++ b/sys/kern/subr_syscall.c > @@ -131,6 +131,12 @@ syscallenter(struct thread *td, struct syscall_args *sa) > sa->callp, sa->args, 0); > #endif > > + /* > + * Fetch fast sigblock value at the time of syscall > + * entry because sleepqueue primitives might call > + * cursig(). > + */ > + fetch_fast_sigblock(td); > AUDIT_SYSCALL_ENTER(sa->code, td); > error = (sa->callp->sy_call)(td, sa->args); > AUDIT_SYSCALL_EXIT(error, td); > diff --git a/sys/kern/subr_trap.c b/sys/kern/subr_trap.c > index 095bbdf..66e485b 100644 > --- a/sys/kern/subr_trap.c > +++ b/sys/kern/subr_trap.c > @@ -237,6 +237,7 @@ ast(struct trapframe *framep) > */ > if (flags & TDF_NEEDSIGCHK || p->p_pendingcnt > 0 || > !SIGISEMPTY(p->p_siglist)) { > + fetch_fast_sigblock(td); > PROC_LOCK(p); > mtx_lock(&p->p_sigacts->ps_mtx); > while ((sig = cursig(td, SIG_STOP_ALLOWED)) != 0) > diff --git a/sys/kern/syscalls.master b/sys/kern/syscalls.master > index f62dad7..28a9393 100644 > --- a/sys/kern/syscalls.master > +++ b/sys/kern/syscalls.master > @@ -951,5 +951,6 @@ > off_t offset, off_t len); } > 531 AUE_NULL STD { int posix_fadvise(int fd, off_t offset, \ > off_t len, int advice); } > +532 AUE_NULL STD { int fast_sigblock(int cmd, uint32_t *ptr); } > ; Please copy any additions and changes to the following compatability tables: > ; sys/compat/freebsd32/syscalls.master > diff --git a/sys/sys/proc.h b/sys/sys/proc.h > index 06df632..4899ca2 100644 > --- a/sys/sys/proc.h > +++ b/sys/sys/proc.h > @@ -272,6 +272,9 @@ struct thread { > struct osd td_osd; /* (k) Object specific data. */ > struct vm_map_entry *td_map_def_user; /* (k) Deferred entries. */ > pid_t td_dbg_forked; /* (c) Child pid for debugger. */ > + void *td_sigblock_ptr; /* (k) uptr for fast sigblock. */ > + uint32_t td_sigblock_val; /* (k) fast sigblock value at > + kernel entry. */ > #define td_endzero td_sigmask > > /* Copied during fork1() or create_thread(). */ > @@ -424,6 +427,7 @@ do { \ > #define TDP_RESETSPUR 0x04000000 /* Reset spurious page fault history. */ > #define TDP_NERRNO 0x08000000 /* Last errno is already in td_errno */ > #define TDP_UIOHELD 0x10000000 /* Current uio has pages held in td_ma */ > +#define TDP_FAST_SIGBLOCK 0x20000000 /* Fast sigblock active */ > > /* > * Reasons that the current thread can not be run yet. > diff --git a/sys/sys/signalvar.h b/sys/sys/signalvar.h > index 71685e7..68cca58 100644 > --- a/sys/sys/signalvar.h > +++ b/sys/sys/signalvar.h > @@ -250,6 +250,20 @@ typedef struct sigqueue { > /* Flags for ksi_flags */ > #define SQ_INIT 0x01 > > +/* > + * Fast_sigblock > + */ > +#define FAST_SIGBLOCK_SETPTR 1 > +#define FAST_SIGBLOCK_UNBLOCK 2 > +#define FAST_SIGBLOCK_UNSETPTR 3 > + > +#define FAST_SIGBLOCK_PEND 0x1 > +#define FAST_SIGBLOCK_INC 0x10 > + > +#ifndef _KERNEL > +int __sys_fast_sigblock(int cmd, void *ptr); > +#endif > + > #ifdef _KERNEL > > /* Return nonzero if process p has an unmasked pending signal. */ > @@ -329,6 +343,8 @@ extern struct mtx sigio_lock; > > int cursig(struct thread *td, int stop_allowed); > void execsigs(struct proc *p); > +void fetch_fast_sigblock(struct thread *td); > +void fetch_fast_sigblock_failed(struct thread *td, int write); > void gsignal(int pgid, int sig, ksiginfo_t *ksi); > void killproc(struct proc *p, char *why); > ksiginfo_t * ksiginfo_alloc(int wait); > > > > ----- End forwarded message ----- > So you want to make kernel share data with userland. The only thing I don't like is it adds overhead to syscall, rumor said that our syscall is slow than others, this might not be a problem ? Long time ago, I proposed a schedctl() syscall to make kernel share data with userland, some old patches are still there: http://people.freebsd.org/~davidxu/schedctl/ Mine does not have such an overhead, but it has another one: memory page is allocated in kernel which can not be swapped out and can not be freed until process is exited, the page is doubly mapped into in kernel and userland, accessing the shared data in kernel has zero overhead though. From owner-freebsd-arch@FreeBSD.ORG Tue Jan 8 06:21:54 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DB7C570D; Tue, 8 Jan 2013 06:21:54 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id CAE67D30; Tue, 8 Jan 2013 06:21:54 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (unknown [207.238.187.242]) by elvis.mu.org (Postfix) with ESMTPSA id 501121A3CC4; Mon, 7 Jan 2013 22:21:51 -0800 (PST) Message-ID: <50EBBAFC.9070803@mu.org> Date: Tue, 08 Jan 2013 01:21:48 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> <50EB3319.6060303@mu.org> <20130108000840.GA82219@kib.kiev.ua> In-Reply-To: <20130108000840.GA82219@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 06:21:54 -0000 On 1/7/13 7:08 PM, Konstantin Belousov wrote: > On Mon, Jan 07, 2013 at 03:42:01PM -0500, Alfred Perlstein wrote: >> On 1/7/13 1:22 PM, Konstantin Belousov wrote: >>> Below is the forward of the patch for which I failed to obtain a private >>> review. Might be, the list generates more responses. >> Very cool. >> >> Sorry for being rusty here, but is it safe to call fuword in the middle >> of issignal()? >> >> The reason I ask is because it looks like proc lock is required for this >> block, and I forget it fuword(9) can fault. >> >> If fuword(9) can fault then I don't think you can do this. >> >> You'll need to use something like fuswintr(9) or wire the page down >> using vslock(9) or something. >> >> I am probably missing something? > No, you are completely right. This is a serious bug. Thank you. > > I was careful to code the td_sigblock_val prefetch etc, but failed > to do the update properly. There is no reason to execute the update > in the issignal() at all, the setting of the pending bit can be postponed > to the moment after the postsig() loop was executed. > > I uploaded the updated but not yet tested patch at > http://people.freebsd.org/~kib/misc/rtld-sigblock.1.patch This new patch looks like it may have issues with PSTOP. there is a lot of code in issignal() that is missed when this code is in place, I have not audited the effect of this, are you sure this is what will work for the majority of cases? Honestly, if I were coding it for correctness I would vslock the pages (or otherwise wire them in) and let issignal() behave normally and not early exit from it. That said I may be missing something. -Alfred From owner-freebsd-arch@FreeBSD.ORG Tue Jan 8 10:47:04 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 72ACA63A; Tue, 8 Jan 2013 10:47:04 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-f45.google.com (mail-bk0-f45.google.com [209.85.214.45]) by mx1.freebsd.org (Postfix) with ESMTP id 7A15890E; Tue, 8 Jan 2013 10:47:03 +0000 (UTC) Received: by mail-bk0-f45.google.com with SMTP id jk13so141215bkc.4 for ; Tue, 08 Jan 2013 02:47:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=w3tRLkMQks8wQZDzW9pxUxxx+ggw2dsVaHaLhGz/AgM=; b=WuY+fskh+9PPS2YT5CM6vT2574rf8OK6QBOi39+CGTnhHpMnB7iDJoSTnBp9dw10c3 4bpoTu6LXeKVkdnrOektPG/iAWHzanlmNpA8J5CW6iJC8iMAGV94sYGXKDlThAZoIUuf m/kfx1ueByP7mCY7NkIl3hzCS+inwNQNlxpiNeTkiNNMoxBIo1YN+1KebqnB6eBKzoIh bfC/LC+qRQkIynJ+G7rMEhC6jgCV/6Yq3Ym7y3EndIlsFVNobr7ycgvSHXyTD/fOlrjj tfVaWlEU2gzjIuwTv4BAqTKo2OL179T6lmc8jZaA0cJzxnGHxh2NDMdd1h5lW6lDt7vO Jtnw== X-Received: by 10.204.154.202 with SMTP id p10mr31709514bkw.29.1357642022301; Tue, 08 Jan 2013 02:47:02 -0800 (PST) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPS id z5sm44885552bkv.11.2013.01.08.02.46.59 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 08 Jan 2013 02:47:01 -0800 (PST) Sender: Alexander Motin Message-ID: <50EBF921.2000304@FreeBSD.org> Date: Tue, 08 Jan 2013 12:46:57 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Marius Strobl Subject: Re: [RFC/RFT] calloutng References: <50CCAB99.4040308@FreeBSD.org> <50CE5B54.3050905@FreeBSD.org> <50D03173.9080904@FreeBSD.org> <20121225232126.GA47692@alchemy.franken.de> <50DB4EFE.2020600@FreeBSD.org> <20130106152313.GD26039@alchemy.franken.de> In-Reply-To: <20130106152313.GD26039@alchemy.franken.de> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Davide Italiano , FreeBSD Current , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 10:47:04 -0000 On 06.01.2013 17:23, Marius Strobl wrote: > On Wed, Dec 26, 2012 at 09:24:46PM +0200, Alexander Motin wrote: >> On 26.12.2012 01:21, Marius Strobl wrote: >>> On Tue, Dec 18, 2012 at 11:03:47AM +0200, Alexander Motin wrote: >>>> Experiments with dummynet shown ineffective support for very short >>>> tick-based callouts. New version fixes that, allowing to get as many >>>> tick-based callout events as hz value permits, while still be able to >>>> aggregate events and generating minimum of interrupts. >>>> >>>> Also this version modifies system load average calculation to fix some >>>> cases existing in HEAD and 9 branches, that could be fixed with new >>>> direct callout functionality. >>>> >>>> http://people.freebsd.org/~mav/calloutng_12_17.patch >>>> >>>> With several important changes made last time I am going to delay commit >>>> to HEAD for another week to do more testing. Comments and new test cases >>>> are welcome. Thanks for staying tuned and commenting. >>> >>> FYI, I gave both calloutng_12_15_1.patch and calloutng_12_17.patch a >>> try on sparc64 and it at least survives a buildworld there. However, >>> with the patched kernels, buildworld times seem to increase slightly but >>> reproducible by 1-2% (I only did four runs but typically buildworld >>> times are rather stable and don't vary more than a minute for the >>> same kernel and source here). Is this an expected trade-off (system >>> time as such doesn't seem to increase)? >> >> I don't think build process uses significant number of callouts to >> affect results directly. I think this additional time could be result of >> the deeper next event look up, done by the new code, that is practically >> useless for sparc64, which effectively has no cpu_idle() routine. It >> wouldn't affect system time and wouldn't show up in any statistics >> (except PMC or something alike) because it is executed inside timer >> hardware interrupt handler. If my guess is right, that is a part that >> probably still could be optimized. I'll look on it. Thanks. >> >>> Is there anything specific to test? >> >> Since the most of code is MI, for sparc64 I would mostly look on related >> MD parts (eventtimers and timecounters) to make sure they are working >> reliably in more stressful conditions. I still have some worries about >> possible deadlock on hardware where IPIs are used to fetch present time >> from other CPU. > > Well, I've just learnt two things the hard way: > a) We really need the mutex in that path. > b) Assuming that the initial synchronization of the counters is good > enough and they won't drift considerably accross the CPUs so we can > always use the local one makes things go south pretty soon after > boot. At least with your calloutng_12_26.patch applied. Do you think it means they are not really synchronized for some reason? > I'm not really sure what to do about that. Earlier you already said > that sched_bind(9) also isn't an option in case if td_critnest > 1. > To be honest, I don't really unerstand why using a spin lock in the > timecounter path makes sparc64 the only problematic architecture > for your changes. The x86 i8254_get_timecount() also uses a spin lock > so it should be in the same boat. The problem is not in using spinlock, but in waiting for other CPU while spinlock is held. Other CPU may also hold spinlock and wait for something, causing deadlock. i8254 code uses spinlock just to atomically access hardware registers, so it causes no problems. > The affected machines are equipped with a x86-style south bridge > which exposes a powermanagment unit (intended to be used as a SMBus > bridge only in these machines) on the PCI bus. Actually, this device > also includes an ACPI power management timer. However, I've just > spent a day trying to get that one working without success - it > just doesn't increment. Probably its clock input isn't connected as > it's not intended to be used in these machines. > That south bridge also includes 8254 compatible timers on the ISA/ > LPC side, but are hidden from the OFW device tree. I can hack these > devices into existence and give it a try, but even if that works this > likely would use the same code as the x86 i8254_get_timecount() so I > don't see what would be gained with that. > > The last thing in order to avoid using the tick counter as timecounter > in the MP case I can think of is that the Broadcom MACs in the affected > machines also provide a counter driven by a 1 MHz clock. If that's good > enough for a timecounter I can hook these up (in case these work ...) > and hack bge(4) to not detach in that case (given that we can't detach > timecounters ...). i8254 on x86 is also just a bit above 1MHz. >> Here is small tool we are using for test correctness and performance of >> different user-level APIs: http://people.freebsd.org/~mav/testsleep.c >> > > I've run Ian's set of tests on a v215 with and without your > calloutng_12_26.patch and on a v210 (these uses the IPI approach) > with the latter also applied. > I'm not really sure what to make out of the numbers. > > v215 w/o v215 w/ v210 w/ > ---------- ---------------- ---------------- ---------------- > select 1 1999.61 1 23.87 1 29.97 > poll 1 1999.70 1 1069.61 1 1075.24 > usleep 1 1999.86 1 23.43 1 28.99 > nanosleep 1 999.92 1 23.28 1 28.66 > kqueue 1 1000.12 1 1071.13 1 1076.35 > kqueueto 1 999.56 1 26.33 1 31.34 > syscall 1 1.89 1 1.92 1 2.88 Numbers are not bad, respecting the fact that to protect from lost interrupts eventtimer code on sparc64 now sets minimal programming interval to 15us. It was made to reduce race window between the timer read-modify-write and some long NMIs. May be with rereading counter after programming comparator (same as done for HPET, reading which is probably much more expensive) this value could be reduced. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Tue Jan 8 10:55:38 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C6DE595C; Tue, 8 Jan 2013 10:55:38 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-wg0-f53.google.com (mail-wg0-f53.google.com [74.125.82.53]) by mx1.freebsd.org (Postfix) with ESMTP id ED5BF96E; Tue, 8 Jan 2013 10:55:37 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id fn15so200606wgb.8 for ; Tue, 08 Jan 2013 02:55:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=M3EwzbqQ3rMCyMxQ2hP3jyAajEx/QVYxap8URdgdNiI=; b=FNjSSGycSAwaLILf2sONvCXHDsl8cgTXyRHhFRfKkPYctSkJAWklUhdXu73F6xm27b 3LsQttQDfXjMqYQsKxZEl8UL/CPRELw6L7AVeSh5xK20AR7deguov+BOpM2bxCLfNd9B 8xgwfMMUjWd30GU8NHcK6yiG04baAa8S9/PXwOHl0OqMAK3k5tCOO2u0a3nn8+k8JCT+ AoBbsDAjY3WRZM2HukNs0af/jxvRdXOJfcY5q0reUi1ZsRF2Rb4mI3qKWFr8fFDx1pQv vDq6UWziR8VcHfs58F2oKHXtKxZH/n9f4RCFAIkl8lhFJ0fXqOMeK4CcTZ34Ere/KJ9q McGA== X-Received: by 10.180.33.44 with SMTP id o12mr14204490wii.28.1357642530948; Tue, 08 Jan 2013 02:55:30 -0800 (PST) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPS id p3sm16024526wic.8.2013.01.08.02.55.28 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jan 2013 02:55:29 -0800 (PST) Sender: Alexander Motin Message-ID: <50EBFB1F.2080708@FreeBSD.org> Date: Tue, 08 Jan 2013 12:55:27 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: [RFC/RFT] calloutng References: <50CCAB99.4040308@FreeBSD.org> <50CE5B54.3050905@FreeBSD.org> <50D03173.9080904@FreeBSD.org> <20121225232126.GA47692@alchemy.franken.de> <50DB4EFE.2020600@FreeBSD.org> <20130106152313.GD26039@alchemy.franken.de> <20130106162049.GA3640@onelab2.iet.unipi.it> In-Reply-To: <20130106162049.GA3640@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Davide Italiano , freebsd-arch@FreeBSD.org, FreeBSD Current , Marius Strobl X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 10:55:38 -0000 On 06.01.2013 18:20, Luigi Rizzo wrote: > On Sun, Jan 06, 2013 at 04:23:13PM +0100, Marius Strobl wrote: >> On Wed, Dec 26, 2012 at 09:24:46PM +0200, Alexander Motin wrote: >>> Here is small tool we are using for test correctness and performance of >>> different user-level APIs: http://people.freebsd.org/~mav/testsleep.c >>> >> >> I've run Ian's set of tests on a v215 with and without your >> calloutng_12_26.patch and on a v210 (these uses the IPI approach) >> with the latter also applied. >> I'm not really sure what to make out of the numbers. >> >> v215 w/o v215 w/ v210 w/ >> ---------- ---------------- ---------------- ---------------- >> select 1 1999.61 1 23.87 1 29.97 >> poll 1 1999.70 1 1069.61 1 1075.24 >> usleep 1 1999.86 1 23.43 1 28.99 >> nanosleep 1 999.92 1 23.28 1 28.66 >> kqueue 1 1000.12 1 1071.13 1 1076.35 >> kqueueto 1 999.56 1 26.33 1 31.34 >> syscall 1 1.89 1 1.92 1 2.88 >> select 300 1999.72 300 326.08 300 332.24 >> poll 300 1999.12 300 1069.78 300 1075.82 >> usleep 300 1999.91 300 325.63 300 330.94 >> nanosleep 300 999.82 300 23.25 300 26.76 >> kqueue 300 1000.14 300 1071.06 300 1075.96 >> kqueueto 300 999.53 300 26.32 300 31.42 >> syscall 300 1.90 300 1.93 300 2.89 >> select 3000 3998.18 3000 3176.51 3000 3193.86 >> poll 3000 3999.29 3000 3182.21 3000 3193.12 >> usleep 3000 3998.46 3000 3191.60 3000 3192.50 >> nanosleep 3000 1999.79 3000 23.21 3000 27.02 >> kqueue 3000 3000.12 3000 3189.13 3000 3191.96 >> kqueueto 3000 1999.99 3000 26.28 3000 31.91 >> syscall 3000 1.91 3000 1.91 3000 2.90 >> select 30000 30990.85 30000 31489.18 30000 31548.77 >> poll 30000 30995.25 30000 31518.80 30000 31487.92 >> usleep 30000 30992.00 30000 31510.42 30000 31475.50 >> nanosleep 30000 1999.46 30000 38.67 30000 41.95 >> kqueue 30000 30006.49 30000 30991.86 30000 30996.77 >> kqueueto 30000 1999.09 30000 41.67 30000 46.36 >> syscall 30000 1.91 30000 1.91 30000 2.88 >> select 300000 300990.65 300000 301864.98 300000 301787.01 >> poll 300000 300998.09 300000 301831.36 300000 301741.62 >> usleep 300000 300990.80 300000 301824.67 300000 301770.10 >> nanosleep 300000 1999.15 300000 325.74 300000 331.01 >> kqueue 300000 300000.87 300000 301031.11 300000 300992.28 >> kqueueto 300000 1999.39 300000 328.77 300000 333.45 >> syscall 300000 1.91 300000 1.91 300000 2.88 > > the nanosleep and kqueueto tests are probably passing the wrong > argument to the syscall (meant to be microseconds, but nanosleep > takes nanosecond so it should probably be multiplied by 1000). Yes, you are right. I've used it in such way to see what will happen if I request sub-microsecond interval. In this test these rows are definitely incorrect. > I think that for the time being it would be useful to run at least > one set of tests with kern.timecounter.alloweddeviation=0 so we can > tell how close we get to the required timeouts May be just to be sure, because it should not significantly affect results of the 1us tests, as 5% of 1us is much less then numbers we see there. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Tue Jan 8 11:16:57 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 04BDAE2F; Tue, 8 Jan 2013 11:16:57 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id AE5FBA34; Tue, 8 Jan 2013 11:16:56 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 23E747300A; Tue, 8 Jan 2013 12:16:03 +0100 (CET) Date: Tue, 8 Jan 2013 12:16:03 +0100 From: Luigi Rizzo To: Alexander Motin Subject: Re: [RFC/RFT] calloutng Message-ID: <20130108111603.GA30469@onelab2.iet.unipi.it> References: <50CCAB99.4040308@FreeBSD.org> <50CE5B54.3050905@FreeBSD.org> <50D03173.9080904@FreeBSD.org> <20121225232126.GA47692@alchemy.franken.de> <50DB4EFE.2020600@FreeBSD.org> <20130106152313.GD26039@alchemy.franken.de> <20130106162049.GA3640@onelab2.iet.unipi.it> <50EBFB1F.2080708@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50EBFB1F.2080708@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: Davide Italiano , freebsd-arch@FreeBSD.org, FreeBSD Current , Marius Strobl X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 11:16:57 -0000 On Tue, Jan 08, 2013 at 12:55:27PM +0200, Alexander Motin wrote: > On 06.01.2013 18:20, Luigi Rizzo wrote: ... > > I think that for the time being it would be useful to run at least > > one set of tests with kern.timecounter.alloweddeviation=0 so we can > > tell how close we get to the required timeouts > > May be just to be sure, because it should not significantly affect > results of the 1us tests, as 5% of 1us is much less then numbers we see > there. to clarify - i don't mind if we are 50-100us (absolute error) off the requested timeout for short intervals, but i want to be sure that this error can be achieved also for large requests. cheers luigi From owner-freebsd-arch@FreeBSD.ORG Tue Jan 8 14:56:39 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id CC1A078; Tue, 8 Jan 2013 14:56:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id EE3CB729; Tue, 8 Jan 2013 14:56:38 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id r08EuVV9096773; Tue, 8 Jan 2013 16:56:31 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r08EuVV9096773 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id r08EuVef096772; Tue, 8 Jan 2013 16:56:31 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Jan 2013 16:56:31 +0200 From: Konstantin Belousov To: David Xu Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130108145631.GD82219@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <50EBAA1F.9070303@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="em00FKA6+IJh3lGW" Content-Disposition: inline In-Reply-To: <50EBAA1F.9070303@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 14:56:39 -0000 --em00FKA6+IJh3lGW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 08, 2013 at 01:09:51PM +0800, David Xu wrote: > On 2013/01/08 02:22, Konstantin Belousov wrote: > > Below is the forward of the patch for which I failed to obtain a private > > review. Might be, the list generates more responses. > > > > Our rtld has a performance bootleneck, typically exposed by the images > > with the lot of the run-time relocation processing, and by the C++ > > exception handling. We block the signals delivery during the rtld > > performing the relocations, as well as for the dl_iterate_phdr(3) (the > > later is used for finding the dwarf unwinding tables). > > > > The signal blocking is needed to allow the rtld to resolve the symbols > > for the signal handlers in the safe way, but also causes 2 syscalls > > overhead per each rtld entry. > > > > The proposed approach allows to shave off those two syscalls, doubling > > the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. > > > > Date: Mon, 13 Aug 2012 15:26:00 +0300 > > From: Konstantin Belousov > > > > ... > > > > The basic idea is to implement sigprocmask() as single write into userm= ode > > address. If kernel needs to calculate the signal mask for current threa= d, > > it takes into the consideration non-zero value of the word at the agreed > > address. Also, usermode is informed about signals which were put on hold > > due to fast sigblock active. > > > > As I said, on my measurements in microbenchmark that did throw/catch in > > a loop, I see equal user and system time spent for unpatched system, and > > same user time with zero system time on patched system. > > > > Patch can be improved further, e.g. it would be nice to allow rtld to f= all > > back to sigprocmask(2) if kernel does not support fast sigblock, to pre= vent > > flag day. Also, the mask enforced by fast sigblock can be made configur= able. > > > > Note that libthr already blocks signals by catching them, and not using= rtld > > service in the first line handler. I tried to make the change in the sp= irit > > of libthr interceptors, but handoff to libthr appears too complicated to > > work. In fact, libthr can be changed to start using fast sigblock inste= ad > > of wrapping sigaction, but this is out of scope of the proposal right n= ow. > > > > Please comment. > > > > diff --git a/lib/libc/sys/Symbol.map b/lib/libc/sys/Symbol.map > > index 6888ea0..3b75539 100644 > > --- a/lib/libc/sys/Symbol.map > > +++ b/lib/libc/sys/Symbol.map > > @@ -546,6 +546,7 @@ FBSDprivate_1.0 { > > __sys_extattr_set_link; > > _extattrctl; > > __sys_extattrctl; > > + __sys_fast_sigblock; > > _fchdir; > > __sys_fchdir; > > _fchflags; > > diff --git a/libexec/rtld-elf/rtld_lock.c b/libexec/rtld-elf/rtld_lock.c > > index d1563e5..50c52c6 100644 > > --- a/libexec/rtld-elf/rtld_lock.c > > +++ b/libexec/rtld-elf/rtld_lock.c > > @@ -43,6 +43,7 @@ > > */ > > > > #include > > +#include > > #include > > #include > > #include > > @@ -59,8 +60,8 @@ typedef struct Struct_Lock { > > void *base; > > } Lock; > > > > -static sigset_t fullsigmask, oldsigmask; > > static int thread_flag; > > +static uint32_t fsigblock; > > > > static void * > > def_lock_create() > > @@ -111,18 +112,26 @@ def_rlock_acquire(void *lock) > > } > > > > static void > > +sig_fastunblock(void) > > +{ > > + uint32_t oldval; > > + > > + oldval =3D atomic_fetchadd_32(&fsigblock, -FAST_SIGBLOCK_INC); > > + if (oldval =3D=3D (FAST_SIGBLOCK_PEND | FAST_SIGBLOCK_INC)) > > + __sys_fast_sigblock(FAST_SIGBLOCK_UNBLOCK, NULL); > > +} > > + > > +static void > > def_wlock_acquire(void *lock) > > { > > Lock *l =3D (Lock *)lock; > > - sigset_t tmp_oldsigmask; > > > > for ( ; ; ) { > > - sigprocmask(SIG_BLOCK, &fullsigmask, &tmp_oldsigmask); > > + atomic_add_rel_32(&fsigblock, FAST_SIGBLOCK_INC); > > if (atomic_cmpset_acq_int(&l->lock, 0, WAFLAG)) > > break; > > - sigprocmask(SIG_SETMASK, &tmp_oldsigmask, NULL); > > + sig_fastunblock(); > > } > > - oldsigmask =3D tmp_oldsigmask; > > } > > > > static void > > @@ -134,7 +143,7 @@ def_lock_release(void *lock) > > atomic_add_rel_int(&l->lock, -RC_INCR); > > else { > > atomic_add_rel_int(&l->lock, -WAFLAG); > > - sigprocmask(SIG_SETMASK, &oldsigmask, NULL); > > + sig_fastunblock(); > > } > > } > > > > @@ -286,19 +295,7 @@ lockdflt_init() > > > > memcpy(&lockinfo, &deflockinfo, sizeof(lockinfo)); > > _rtld_thread_init(NULL); > > - /* > > - * Construct a mask to block all signals except traps which might > > - * conceivably be generated within the dynamic linker itself. > > - */ > > - sigfillset(&fullsigmask); > > - sigdelset(&fullsigmask, SIGILL); > > - sigdelset(&fullsigmask, SIGTRAP); > > - sigdelset(&fullsigmask, SIGABRT); > > - sigdelset(&fullsigmask, SIGEMT); > > - sigdelset(&fullsigmask, SIGFPE); > > - sigdelset(&fullsigmask, SIGBUS); > > - sigdelset(&fullsigmask, SIGSEGV); > > - sigdelset(&fullsigmask, SIGSYS); > > + __sys_fast_sigblock(FAST_SIGBLOCK_SETPTR, &fsigblock); > > } > > > > /* > > @@ -319,7 +316,10 @@ _rtld_thread_init(struct RtldLockInfo *pli) > > > > if (pli =3D=3D NULL) > > pli =3D &deflockinfo; > > - > > + else { > > + fsigblock =3D 0; > > + __sys_fast_sigblock(FAST_SIGBLOCK_UNSETPTR, NULL); > > + } > > > > for (i =3D 0; i < RTLD_LOCK_CNT; i++) > > if ((locks[i] =3D pli->lock_create()) =3D=3D NULL) > > diff --git a/sys/compat/freebsd32/syscalls.master b/sys/compat/freebsd3= 2/syscalls.master > > index 0891e41..f9e8b9e 100644 > > --- a/sys/compat/freebsd32/syscalls.master > > +++ b/sys/compat/freebsd32/syscalls.master > > @@ -997,3 +997,4 @@ > > uint32_t offset1, uint32_t offset2,\ > > uint32_t len1, uint32_t len2, \ > > int advice); } > > +532 AUE_NULL NOPROTO { int fast_sigblock(int cmd, uint32_t *ptr); } > > diff --git a/sys/kern/kern_exec.c b/sys/kern/kern_exec.c > > index 90f7311..8a3cd15 100644 > > --- a/sys/kern/kern_exec.c > > +++ b/sys/kern/kern_exec.c > > @@ -1031,6 +1031,7 @@ exec_new_vmspace(imgp, sv) > > int error; > > struct proc *p =3D imgp->proc; > > struct vmspace *vmspace =3D p->p_vmspace; > > + struct thread *td =3D curthread; > > vm_object_t obj; > > vm_offset_t sv_minuser, stack_addr; > > vm_map_t map; > > @@ -1039,6 +1040,10 @@ exec_new_vmspace(imgp, sv) > > imgp->vmspace_destroyed =3D 1; > > imgp->sysent =3D sv; > > > > + td->td_pflags &=3D ~TDP_FAST_SIGBLOCK; > > + td->td_sigblock_ptr =3D NULL; > > + td->td_sigblock_val =3D 0; > > + > > /* May be called with Giant held */ > > EVENTHANDLER_INVOKE(process_exec, p, imgp); > > > > diff --git a/sys/kern/kern_sig.c b/sys/kern/kern_sig.c > > index 2685a8b..3c8a2f7 100644 > > --- a/sys/kern/kern_sig.c > > +++ b/sys/kern/kern_sig.c > > @@ -230,6 +230,7 @@ static int sigproptbl[NSIG] =3D { > > }; > > > > static void reschedule_signals(struct proc *p, sigset_t block, int fl= ags); > > +static sigset_t fastblock_mask; > > > > static void > > sigqueue_start(void) > > @@ -240,6 +241,16 @@ sigqueue_start(void) > > p31b_setcfg(CTL_P1003_1B_REALTIME_SIGNALS, _POSIX_REALTIME_SIGNALS); > > p31b_setcfg(CTL_P1003_1B_RTSIG_MAX, SIGRTMAX - SIGRTMIN + 1); > > p31b_setcfg(CTL_P1003_1B_SIGQUEUE_MAX, max_pending_per_proc); > > + SIGFILLSET(fastblock_mask); > > + SIG_CANTMASK(fastblock_mask); > > + SIGDELSET(fastblock_mask, SIGILL); > > + SIGDELSET(fastblock_mask, SIGTRAP); > > + SIGDELSET(fastblock_mask, SIGABRT); > > + SIGDELSET(fastblock_mask, SIGEMT); > > + SIGDELSET(fastblock_mask, SIGFPE); > > + SIGDELSET(fastblock_mask, SIGBUS); > > + SIGDELSET(fastblock_mask, SIGSEGV); > > + SIGDELSET(fastblock_mask, SIGSYS); > > } > > > > ksiginfo_t * > > @@ -2525,6 +2536,7 @@ issignal(struct thread *td, int stop_allowed) > > struct sigqueue *queue; > > sigset_t sigpending; > > int sig, prop, newsig; > > + uint32_t oldval; > > > > p =3D td->td_proc; > > ps =3D p->p_sigacts; > > @@ -2541,6 +2553,32 @@ issignal(struct thread *td, int stop_allowed) > > SIG_STOPSIGMASK(sigpending); > > if (SIGISEMPTY(sigpending)) /* no signal to send */ > > return (0); > > + > > + /* > > + * Do fast sigblock if requested by usermode. Since > > + * we do know that there was a signal pending at this > > + * point, set the FAST_SIGBLOCK_PEND as indicator for > > + * usermode to perform a dummy call to > > + * FAST_SIGBLOCK_UNBLOCK, which causes immediate > > + * delivery of postponed pending signal. > > + */ > > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) !=3D 0) { > > + if (td->td_sigblock_val !=3D 0) > > + SIGSETNAND(sigpending, fastblock_mask); > > + if (SIGISEMPTY(sigpending)) { > > + oldval =3D fuword32(td->td_sigblock_ptr); > > + if (oldval =3D=3D -1) { > > + fetch_fast_sigblock_failed(td, 0); > > + return (0); > > + } > > + oldval |=3D FAST_SIGBLOCK_PEND; > > + if (suword32(td->td_sigblock_ptr, oldval) =3D=3D -1) > > + fetch_fast_sigblock_failed(td, 1); > > + td->td_sigblock_val =3D oldval; > > + return (0); > > + } > > + } > > + > > sig =3D sig_ffs(&sigpending); > > > > if (p->p_stops & S_SIG) { > > @@ -3456,3 +3494,92 @@ sigacts_shared(struct sigacts *ps) > > mtx_unlock(&ps->ps_mtx); > > return (shared); > > } > > + > > +int > > +sys_fast_sigblock(struct thread *td, struct fast_sigblock_args *uap) > > +{ > > + struct proc *p; > > + int error; > > + uint32_t oldval; > > + > > + error =3D 0; > > + switch (uap->cmd) { > > + case FAST_SIGBLOCK_SETPTR: > > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) !=3D 0) > > + error =3D EBUSY; > > + else if (((uintptr_t)(uap->ptr) & (sizeof(uint32_t) - 1)) !=3D 0) > > + error =3D EINVAL; > > + else { > > + td->td_pflags |=3D TDP_FAST_SIGBLOCK; > > + td->td_sigblock_ptr =3D uap->ptr; > > + } > > + break; > > + > > + case FAST_SIGBLOCK_UNBLOCK: > > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) !=3D 0) { > > + oldval =3D casuword32(td->td_sigblock_ptr, > > + FAST_SIGBLOCK_PEND, 0); > > + if (oldval =3D=3D (uint32_t)-1) > > + error =3D EFAULT; > > + else if (oldval !=3D FAST_SIGBLOCK_PEND) > > + error =3D EBUSY; > > + else > > + td->td_sigblock_val =3D 0; > > + } else > > + error =3D EDOOFUS; > > + > > + /* > > + * Rely on normal ast mechanism to deliver pending > > + * signals to current thread. But notify others about > > + * fake unblock. > > + */ > > + p =3D td->td_proc; > > + if (error =3D=3D 0 && p->p_numthreads !=3D 1) { > > + PROC_LOCK(p); > > + reschedule_signals(p, td->td_sigmask, 0); > > + PROC_UNLOCK(p); > > + } > > + break; > > + > > + case FAST_SIGBLOCK_UNSETPTR: > > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) !=3D 0) { > > + error =3D copyin(td->td_sigblock_ptr, &oldval, > > + sizeof(uint32_t)); > > + if (error !=3D 0) > > + break; > > + if (oldval !=3D 0 && oldval !=3D FAST_SIGBLOCK_PEND) > > + error =3D EBUSY; > > + else { > > + td->td_pflags &=3D ~TDP_FAST_SIGBLOCK; > > + td->td_sigblock_val =3D 0; > > + } > > + } else > > + error =3D EDOOFUS; > > + break; > > + } > > + return (error); > > +} > > + > > +void > > +fetch_fast_sigblock(struct thread *td) > > +{ > > + > > + if ((td->td_pflags & TDP_FAST_SIGBLOCK) =3D=3D 0) > > + return; > > + td->td_sigblock_val =3D fuword32(td->td_sigblock_ptr); > > + if (td->td_sigblock_val =3D=3D -1) > > + fetch_fast_sigblock_failed(td, 0); > > +} > > + > > +void > > +fetch_fast_sigblock_failed(struct thread *td, int write) > > +{ > > + ksiginfo_t ksi; > > + > > + td->td_sigblock_val =3D 0; > > + ksiginfo_init_trap(&ksi); > > + ksi.ksi_signo =3D SIGSEGV; > > + ksi.ksi_code =3D write ? SEGV_ACCERR : SEGV_MAPERR; > > + ksi.ksi_addr =3D td->td_sigblock_ptr; > > + trapsignal(td, &ksi); > > +} > > diff --git a/sys/kern/subr_syscall.c b/sys/kern/subr_syscall.c > > index 5aee684..77b250d 100644 > > --- a/sys/kern/subr_syscall.c > > +++ b/sys/kern/subr_syscall.c > > @@ -131,6 +131,12 @@ syscallenter(struct thread *td, struct syscall_arg= s *sa) > > sa->callp, sa->args, 0); > > #endif > > > > + /* > > + * Fetch fast sigblock value at the time of syscall > > + * entry because sleepqueue primitives might call > > + * cursig(). > > + */ > > + fetch_fast_sigblock(td); > > AUDIT_SYSCALL_ENTER(sa->code, td); > > error =3D (sa->callp->sy_call)(td, sa->args); > > AUDIT_SYSCALL_EXIT(error, td); > > diff --git a/sys/kern/subr_trap.c b/sys/kern/subr_trap.c > > index 095bbdf..66e485b 100644 > > --- a/sys/kern/subr_trap.c > > +++ b/sys/kern/subr_trap.c > > @@ -237,6 +237,7 @@ ast(struct trapframe *framep) > > */ > > if (flags & TDF_NEEDSIGCHK || p->p_pendingcnt > 0 || > > !SIGISEMPTY(p->p_siglist)) { > > + fetch_fast_sigblock(td); > > PROC_LOCK(p); > > mtx_lock(&p->p_sigacts->ps_mtx); > > while ((sig =3D cursig(td, SIG_STOP_ALLOWED)) !=3D 0) > > diff --git a/sys/kern/syscalls.master b/sys/kern/syscalls.master > > index f62dad7..28a9393 100644 > > --- a/sys/kern/syscalls.master > > +++ b/sys/kern/syscalls.master > > @@ -951,5 +951,6 @@ > > off_t offset, off_t len); } > > 531 AUE_NULL STD { int posix_fadvise(int fd, off_t offset, \ > > off_t len, int advice); } > > +532 AUE_NULL STD { int fast_sigblock(int cmd, uint32_t *ptr); } > > ; Please copy any additions and changes to the following compatabilit= y tables: > > ; sys/compat/freebsd32/syscalls.master > > diff --git a/sys/sys/proc.h b/sys/sys/proc.h > > index 06df632..4899ca2 100644 > > --- a/sys/sys/proc.h > > +++ b/sys/sys/proc.h > > @@ -272,6 +272,9 @@ struct thread { > > struct osd td_osd; /* (k) Object specific data. */ > > struct vm_map_entry *td_map_def_user; /* (k) Deferred entries. */ > > pid_t td_dbg_forked; /* (c) Child pid for debugger. */ > > + void *td_sigblock_ptr; /* (k) uptr for fast sigblock. */ > > + uint32_t td_sigblock_val; /* (k) fast sigblock value at > > + kernel entry. */ > > #define td_endzero td_sigmask > > > > /* Copied during fork1() or create_thread(). */ > > @@ -424,6 +427,7 @@ do { \ > > #define TDP_RESETSPUR 0x04000000 /* Reset spurious page fault history= =2E */ > > #define TDP_NERRNO 0x08000000 /* Last errno is already in td_errno */ > > #define TDP_UIOHELD 0x10000000 /* Current uio has pages held in td_ma= */ > > +#define TDP_FAST_SIGBLOCK 0x20000000 /* Fast sigblock active */ > > > > /* > > * Reasons that the current thread can not be run yet. > > diff --git a/sys/sys/signalvar.h b/sys/sys/signalvar.h > > index 71685e7..68cca58 100644 > > --- a/sys/sys/signalvar.h > > +++ b/sys/sys/signalvar.h > > @@ -250,6 +250,20 @@ typedef struct sigqueue { > > /* Flags for ksi_flags */ > > #define SQ_INIT 0x01 > > > > +/* > > + * Fast_sigblock > > + */ > > +#define FAST_SIGBLOCK_SETPTR 1 > > +#define FAST_SIGBLOCK_UNBLOCK 2 > > +#define FAST_SIGBLOCK_UNSETPTR 3 > > + > > +#define FAST_SIGBLOCK_PEND 0x1 > > +#define FAST_SIGBLOCK_INC 0x10 > > + > > +#ifndef _KERNEL > > +int __sys_fast_sigblock(int cmd, void *ptr); > > +#endif > > + > > #ifdef _KERNEL > > > > /* Return nonzero if process p has an unmasked pending signal. */ > > @@ -329,6 +343,8 @@ extern struct mtx sigio_lock; > > > > int cursig(struct thread *td, int stop_allowed); > > void execsigs(struct proc *p); > > +void fetch_fast_sigblock(struct thread *td); > > +void fetch_fast_sigblock_failed(struct thread *td, int write); > > void gsignal(int pgid, int sig, ksiginfo_t *ksi); > > void killproc(struct proc *p, char *why); > > ksiginfo_t * ksiginfo_alloc(int wait); > > > > > > > > ----- End forwarded message ----- > > > So you want to make kernel share data with userland. The only thing I > don't like is it adds overhead to syscall, rumor said that our syscall > is slow than others, this might not be a problem ? >=20 > Long time ago, I proposed a schedctl() syscall to make kernel share > data with userland, some old patches are still there: > http://people.freebsd.org/~davidxu/schedctl/ >=20 > Mine does not have such an overhead, but it has another one: > memory page is allocated in kernel which can not be swapped out > and can not be freed until process is exited, the page is doubly > mapped into in kernel and userland, accessing the shared data > in kernel has zero overhead though. Initial version of the changes indeed used the remap of the user page into the KVA. Besides the issues you described regarding the page being wired for the whole life of the thread, as well as the one page frame in the KVA, there were other non-trivial problems. It was mostly related to the fork(2) copying the address spaces, but not limited too. The use of the direct user address access appeared to be much less intrusive. I completely agree with your note about the additional memory access in the syscall entry path. I do believe that it does not incur a overhead in the real-world loads, but might make some skews in the microbenchmarks. I did not noted any in the testing I did. Anyway, I am still not sure is it worth optimizing for the throw/catch microbenchmark at ll. If general public consequence is yes, the patch needs some work before being committable anyway, mostly to allow patched rtld to work on the pre-patch kernels. Thank you for the reply. --em00FKA6+IJh3lGW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ7DOeAAoJEJDCuSvBvK1B2gAP/0J5YxA+X//ck3myMOOFSGWE Tai1FZe99/kWOiq5Hz6ytPmY1N2enNIePAwboPEPXcqRRu15xG5Pa4nmJqv68a98 Ue3cd6Jf2M/wARRwjWBUjbd/unQWFzQqnXRxVRZJ0F7wFwFS+49hpxLx90UHK3eM g0u4/WCKhf2Czg4rzLSo4L+IhIJ5y+Pw3tkyEfKFe3phJaFdZEhkhZ36WpDqwC6P IQI3kYbKdKIs74zOVpyFn4LBvCk67lGAVF/UF2XVsDreiGzE4w6TfUShmcXAlc+5 nU7la9yvVItmwu7QoONVdi5ccRmO/yZVClGGxPCI/yH/JCdbzNnnqHXlUjRJnI2K U7uUEniyb8NblmK7j1OhouoyjukcspVL/oFhsh+1SAaMVSfY1IXNVvGkN8OqG5ID fgVMtwesYQM+ajGK7CMubA+1e7R2RYbfMrCZ2RuCSDEUqKc2zBH2blr3LfN0Hfnd REtIqtQ2GgA0TdZghT1f+MyNrKpYSyAAxQc/C4FbLdtUzPTotAMDOOE+HIYDxGs9 CnCPmJi7CuDLCjjOGFmQmt3iXMZ3iB96W1mt71931n+V3x6kuY56sU3mYQ0rWIJn jFFkmJyFnfaSmTgAcPCjNuQB5fL4HmgIwSnkTNofYnMvrHPVZJKrF3wAc74lNlG9 YSmACN9ERrLG1Blenu0D =gbJr -----END PGP SIGNATURE----- --em00FKA6+IJh3lGW-- From owner-freebsd-arch@FreeBSD.ORG Tue Jan 8 15:00:35 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 112AC1CA; Tue, 8 Jan 2013 15:00:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 7ED3D74F; Tue, 8 Jan 2013 15:00:34 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id r08F0PSP097265; Tue, 8 Jan 2013 17:00:25 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r08F0PSP097265 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id r08F0OwQ097264; Tue, 8 Jan 2013 17:00:24 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Jan 2013 17:00:24 +0200 From: Konstantin Belousov To: Alfred Perlstein Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130108150024.GE82219@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <50EB3319.6060303@mu.org> <20130108000840.GA82219@kib.kiev.ua> <50EBBAFC.9070803@mu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="prKJgtdAmn5IQEeG" Content-Disposition: inline In-Reply-To: <50EBBAFC.9070803@mu.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 15:00:35 -0000 --prKJgtdAmn5IQEeG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 08, 2013 at 01:21:48AM -0500, Alfred Perlstein wrote: > On 1/7/13 7:08 PM, Konstantin Belousov wrote: > > On Mon, Jan 07, 2013 at 03:42:01PM -0500, Alfred Perlstein wrote: > >> On 1/7/13 1:22 PM, Konstantin Belousov wrote: > >>> Below is the forward of the patch for which I failed to obtain a priv= ate > >>> review. Might be, the list generates more responses. > >> Very cool. > >> > >> Sorry for being rusty here, but is it safe to call fuword in the middle > >> of issignal()? > >> > >> The reason I ask is because it looks like proc lock is required for th= is > >> block, and I forget it fuword(9) can fault. > >> > >> If fuword(9) can fault then I don't think you can do this. > >> > >> You'll need to use something like fuswintr(9) or wire the page down > >> using vslock(9) or something. > >> > >> I am probably missing something? > > No, you are completely right. This is a serious bug. Thank you. > > > > I was careful to code the td_sigblock_val prefetch etc, but failed > > to do the update properly. There is no reason to execute the update > > in the issignal() at all, the setting of the pending bit can be postpon= ed > > to the moment after the postsig() loop was executed. > > > > I uploaded the updated but not yet tested patch at > > http://people.freebsd.org/~kib/misc/rtld-sigblock.1.patch >=20 > This new patch looks like it may have issues with PSTOP. >=20 > there is a lot of code in issignal() that is missed when this code is in= =20 > place, I have not audited the effect of this, are you sure this is what= =20 > will work for the majority of cases? What exactly you are concerned with ? Note that fastblock_mask manipulations right before setting of TDP_FAST_SIGPENDING is to disallow the postponing of the delivery of (potentially) synchronous signals caused by traps, as well as SIGKILL/SIGSTOP. >=20 > Honestly, if I were coding it for correctness I would vslock the pages=20 > (or otherwise wire them in) and let issignal() behave normally and not=20 > early exit from it. vslock() is racy, causes user address space fragmentation and was simply impossibly to use correctly until very recent. --prKJgtdAmn5IQEeG Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ7DSHAAoJEJDCuSvBvK1BRvwP/01DeP1jOWHj4dsGCavZPXKh D3yI6BYAaPxiQihncwRQg8FO/eSoXlAx16wRbrZz5aj7FUq6taOEJ5mcf6jkp1mr IM47VZVQnYWunDP7rrLP4EsurIPfn5OVyazx/hXQO4lD+6JFpWczGP5FQUjJDWK4 2TdMvYa/Q3Wfj73xzVuvtKncrwEx3klT0LOq+fPvUBP/I2AWHI/Wh2+yZVcHD3PL 9mihrB5JuK+h6chbKYmNGrNSoAZQqNBXDHWoGtSfIlPr1jfzzl+Pdq9Z0cOJWYyY qZOs5GksPHbwNWQc2qJrKgLA/ckZaZF8CLAlxWpspizsBoqztQGE9lVPQGP9rVOz J2PbUzPZsfUNzjHvsT1mIoicC/rl1Q98IgKc521npXgCKza9nzQXOxi4rCNZ4Uoo wueGh/y+OP75cTSn88/QZS46pM3gXi+G2Udx46hRJO0rrCYKocGlEG2nHRtULE8S 45VwUciiswuVVwWoKw9w93MCIzCtvX0Hv0GaRoJrz8lNJ9Dzo/uqGrxXkUdv8BbT Z2rFwRoeddXi8j8cmKmAg/bSfooKJAocndPSleaakmSG54oijAxSt3KIqf4QJrw4 ZuTmY240QUZaBbQkkqUltlEL06dF4G+vZ1pvKcZ5KCtr4T9ICp5VCVacQMcsYQ9Q 7/t9PLEpX6cmyozwydb9 =5W7k -----END PGP SIGNATURE----- --prKJgtdAmn5IQEeG-- From owner-freebsd-arch@FreeBSD.ORG Tue Jan 8 15:02:30 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5170239B; Tue, 8 Jan 2013 15:02:30 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 2C12677D; Tue, 8 Jan 2013 15:02:29 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (unknown [207.238.187.242]) by elvis.mu.org (Postfix) with ESMTPSA id F06371A3D38; Tue, 8 Jan 2013 07:02:25 -0800 (PST) Message-ID: <50EC34FE.4070804@mu.org> Date: Tue, 08 Jan 2013 10:02:22 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> <50EBAA1F.9070303@freebsd.org> <20130108145631.GD82219@kib.kiev.ua> In-Reply-To: <20130108145631.GD82219@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, David Xu , toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 15:02:30 -0000 On 1/8/13 9:56 AM, Konstantin Belousov wrote: > On Tue, Jan 08, 2013 at 01:09:51PM +0800, David Xu wrote: >> On 2013/01/08 02:22, Konstantin Belousov wrote: >>> Below is the forward of the patch for which I failed to obtain a private >>> review. Might be, the list generates more responses. >>> >>> Our rtld has a performance bootleneck, typically exposed by the images >>> with the lot of the run-time relocation processing, and by the C++ >>> exception handling. We block the signals delivery during the rtld >>> performing the relocations, as well as for the dl_iterate_phdr(3) (the >>> later is used for finding the dwarf unwinding tables). >>> >>> The signal blocking is needed to allow the rtld to resolve the symbols >>> for the signal handlers in the safe way, but also causes 2 syscalls >>> overhead per each rtld entry. >>> >>> The proposed approach allows to shave off those two syscalls, doubling >>> the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. >>> >>> Date: Mon, 13 Aug 2012 15:26:00 +0300 >>> From: Konstantin Belousov >>> >>> ... >>> >>> The basic idea is to implement sigprocmask() as single write into usermode >>> address. If kernel needs to calculate the signal mask for current thread, >>> it takes into the consideration non-zero value of the word at the agreed >>> address. Also, usermode is informed about signals which were put on hold >>> due to fast sigblock active. >>> >>> As I said, on my measurements in microbenchmark that did throw/catch in >>> a loop, I see equal user and system time spent for unpatched system, and >>> same user time with zero system time on patched system. >>> >>> Patch can be improved further, e.g. it would be nice to allow rtld to fall >>> back to sigprocmask(2) if kernel does not support fast sigblock, to prevent >>> flag day. Also, the mask enforced by fast sigblock can be made configurable. >>> >>> Note that libthr already blocks signals by catching them, and not using rtld >>> service in the first line handler. I tried to make the change in the spirit >>> of libthr interceptors, but handoff to libthr appears too complicated to >>> work. In fact, libthr can be changed to start using fast sigblock instead >>> of wrapping sigaction, but this is out of scope of the proposal right now. >>> >>> Please comment. >>> >>> diff --git a/lib/libc/sys/Symbol.map b/lib/libc/sys/Symbol.map >>> index 6888ea0..3b75539 100644 >>> --- a/lib/libc/sys/Symbol.map >>> +++ b/lib/libc/sys/Symbol.map >>> @@ -546,6 +546,7 @@ FBSDprivate_1.0 { >>> __sys_extattr_set_link; >>> _extattrctl; >>> __sys_extattrctl; >>> + __sys_fast_sigblock; >>> _fchdir; >>> __sys_fchdir; >>> _fchflags; >>> diff --git a/libexec/rtld-elf/rtld_lock.c b/libexec/rtld-elf/rtld_lock.c >>> index d1563e5..50c52c6 100644 >>> --- a/libexec/rtld-elf/rtld_lock.c >>> +++ b/libexec/rtld-elf/rtld_lock.c >>> @@ -43,6 +43,7 @@ >>> */ >>> >>> #include >>> +#include >>> #include >>> #include >>> #include >>> @@ -59,8 +60,8 @@ typedef struct Struct_Lock { >>> void *base; >>> } Lock; >>> >>> -static sigset_t fullsigmask, oldsigmask; >>> static int thread_flag; >>> +static uint32_t fsigblock; >>> >>> static void * >>> def_lock_create() >>> @@ -111,18 +112,26 @@ def_rlock_acquire(void *lock) >>> } >>> >>> static void >>> +sig_fastunblock(void) >>> +{ >>> + uint32_t oldval; >>> + >>> + oldval = atomic_fetchadd_32(&fsigblock, -FAST_SIGBLOCK_INC); >>> + if (oldval == (FAST_SIGBLOCK_PEND | FAST_SIGBLOCK_INC)) >>> + __sys_fast_sigblock(FAST_SIGBLOCK_UNBLOCK, NULL); >>> +} >>> + >>> +static void >>> def_wlock_acquire(void *lock) >>> { >>> Lock *l = (Lock *)lock; >>> - sigset_t tmp_oldsigmask; >>> >>> for ( ; ; ) { >>> - sigprocmask(SIG_BLOCK, &fullsigmask, &tmp_oldsigmask); >>> + atomic_add_rel_32(&fsigblock, FAST_SIGBLOCK_INC); >>> if (atomic_cmpset_acq_int(&l->lock, 0, WAFLAG)) >>> break; >>> - sigprocmask(SIG_SETMASK, &tmp_oldsigmask, NULL); >>> + sig_fastunblock(); >>> } >>> - oldsigmask = tmp_oldsigmask; >>> } >>> >>> static void >>> @@ -134,7 +143,7 @@ def_lock_release(void *lock) >>> atomic_add_rel_int(&l->lock, -RC_INCR); >>> else { >>> atomic_add_rel_int(&l->lock, -WAFLAG); >>> - sigprocmask(SIG_SETMASK, &oldsigmask, NULL); >>> + sig_fastunblock(); >>> } >>> } >>> >>> @@ -286,19 +295,7 @@ lockdflt_init() >>> >>> memcpy(&lockinfo, &deflockinfo, sizeof(lockinfo)); >>> _rtld_thread_init(NULL); >>> - /* >>> - * Construct a mask to block all signals except traps which might >>> - * conceivably be generated within the dynamic linker itself. >>> - */ >>> - sigfillset(&fullsigmask); >>> - sigdelset(&fullsigmask, SIGILL); >>> - sigdelset(&fullsigmask, SIGTRAP); >>> - sigdelset(&fullsigmask, SIGABRT); >>> - sigdelset(&fullsigmask, SIGEMT); >>> - sigdelset(&fullsigmask, SIGFPE); >>> - sigdelset(&fullsigmask, SIGBUS); >>> - sigdelset(&fullsigmask, SIGSEGV); >>> - sigdelset(&fullsigmask, SIGSYS); >>> + __sys_fast_sigblock(FAST_SIGBLOCK_SETPTR, &fsigblock); >>> } >>> >>> /* >>> @@ -319,7 +316,10 @@ _rtld_thread_init(struct RtldLockInfo *pli) >>> >>> if (pli == NULL) >>> pli = &deflockinfo; >>> - >>> + else { >>> + fsigblock = 0; >>> + __sys_fast_sigblock(FAST_SIGBLOCK_UNSETPTR, NULL); >>> + } >>> >>> for (i = 0; i < RTLD_LOCK_CNT; i++) >>> if ((locks[i] = pli->lock_create()) == NULL) >>> diff --git a/sys/compat/freebsd32/syscalls.master b/sys/compat/freebsd32/syscalls.master >>> index 0891e41..f9e8b9e 100644 >>> --- a/sys/compat/freebsd32/syscalls.master >>> +++ b/sys/compat/freebsd32/syscalls.master >>> @@ -997,3 +997,4 @@ >>> uint32_t offset1, uint32_t offset2,\ >>> uint32_t len1, uint32_t len2, \ >>> int advice); } >>> +532 AUE_NULL NOPROTO { int fast_sigblock(int cmd, uint32_t *ptr); } >>> diff --git a/sys/kern/kern_exec.c b/sys/kern/kern_exec.c >>> index 90f7311..8a3cd15 100644 >>> --- a/sys/kern/kern_exec.c >>> +++ b/sys/kern/kern_exec.c >>> @@ -1031,6 +1031,7 @@ exec_new_vmspace(imgp, sv) >>> int error; >>> struct proc *p = imgp->proc; >>> struct vmspace *vmspace = p->p_vmspace; >>> + struct thread *td = curthread; >>> vm_object_t obj; >>> vm_offset_t sv_minuser, stack_addr; >>> vm_map_t map; >>> @@ -1039,6 +1040,10 @@ exec_new_vmspace(imgp, sv) >>> imgp->vmspace_destroyed = 1; >>> imgp->sysent = sv; >>> >>> + td->td_pflags &= ~TDP_FAST_SIGBLOCK; >>> + td->td_sigblock_ptr = NULL; >>> + td->td_sigblock_val = 0; >>> + >>> /* May be called with Giant held */ >>> EVENTHANDLER_INVOKE(process_exec, p, imgp); >>> >>> diff --git a/sys/kern/kern_sig.c b/sys/kern/kern_sig.c >>> index 2685a8b..3c8a2f7 100644 >>> --- a/sys/kern/kern_sig.c >>> +++ b/sys/kern/kern_sig.c >>> @@ -230,6 +230,7 @@ static int sigproptbl[NSIG] = { >>> }; >>> >>> static void reschedule_signals(struct proc *p, sigset_t block, int flags); >>> +static sigset_t fastblock_mask; >>> >>> static void >>> sigqueue_start(void) >>> @@ -240,6 +241,16 @@ sigqueue_start(void) >>> p31b_setcfg(CTL_P1003_1B_REALTIME_SIGNALS, _POSIX_REALTIME_SIGNALS); >>> p31b_setcfg(CTL_P1003_1B_RTSIG_MAX, SIGRTMAX - SIGRTMIN + 1); >>> p31b_setcfg(CTL_P1003_1B_SIGQUEUE_MAX, max_pending_per_proc); >>> + SIGFILLSET(fastblock_mask); >>> + SIG_CANTMASK(fastblock_mask); >>> + SIGDELSET(fastblock_mask, SIGILL); >>> + SIGDELSET(fastblock_mask, SIGTRAP); >>> + SIGDELSET(fastblock_mask, SIGABRT); >>> + SIGDELSET(fastblock_mask, SIGEMT); >>> + SIGDELSET(fastblock_mask, SIGFPE); >>> + SIGDELSET(fastblock_mask, SIGBUS); >>> + SIGDELSET(fastblock_mask, SIGSEGV); >>> + SIGDELSET(fastblock_mask, SIGSYS); >>> } >>> >>> ksiginfo_t * >>> @@ -2525,6 +2536,7 @@ issignal(struct thread *td, int stop_allowed) >>> struct sigqueue *queue; >>> sigset_t sigpending; >>> int sig, prop, newsig; >>> + uint32_t oldval; >>> >>> p = td->td_proc; >>> ps = p->p_sigacts; >>> @@ -2541,6 +2553,32 @@ issignal(struct thread *td, int stop_allowed) >>> SIG_STOPSIGMASK(sigpending); >>> if (SIGISEMPTY(sigpending)) /* no signal to send */ >>> return (0); >>> + >>> + /* >>> + * Do fast sigblock if requested by usermode. Since >>> + * we do know that there was a signal pending at this >>> + * point, set the FAST_SIGBLOCK_PEND as indicator for >>> + * usermode to perform a dummy call to >>> + * FAST_SIGBLOCK_UNBLOCK, which causes immediate >>> + * delivery of postponed pending signal. >>> + */ >>> + if ((td->td_pflags & TDP_FAST_SIGBLOCK) != 0) { >>> + if (td->td_sigblock_val != 0) >>> + SIGSETNAND(sigpending, fastblock_mask); >>> + if (SIGISEMPTY(sigpending)) { >>> + oldval = fuword32(td->td_sigblock_ptr); >>> + if (oldval == -1) { >>> + fetch_fast_sigblock_failed(td, 0); >>> + return (0); >>> + } >>> + oldval |= FAST_SIGBLOCK_PEND; >>> + if (suword32(td->td_sigblock_ptr, oldval) == -1) >>> + fetch_fast_sigblock_failed(td, 1); >>> + td->td_sigblock_val = oldval; >>> + return (0); >>> + } >>> + } >>> + >>> sig = sig_ffs(&sigpending); >>> >>> if (p->p_stops & S_SIG) { >>> @@ -3456,3 +3494,92 @@ sigacts_shared(struct sigacts *ps) >>> mtx_unlock(&ps->ps_mtx); >>> return (shared); >>> } >>> + >>> +int >>> +sys_fast_sigblock(struct thread *td, struct fast_sigblock_args *uap) >>> +{ >>> + struct proc *p; >>> + int error; >>> + uint32_t oldval; >>> + >>> + error = 0; >>> + switch (uap->cmd) { >>> + case FAST_SIGBLOCK_SETPTR: >>> + if ((td->td_pflags & TDP_FAST_SIGBLOCK) != 0) >>> + error = EBUSY; >>> + else if (((uintptr_t)(uap->ptr) & (sizeof(uint32_t) - 1)) != 0) >>> + error = EINVAL; >>> + else { >>> + td->td_pflags |= TDP_FAST_SIGBLOCK; >>> + td->td_sigblock_ptr = uap->ptr; >>> + } >>> + break; >>> + >>> + case FAST_SIGBLOCK_UNBLOCK: >>> + if ((td->td_pflags & TDP_FAST_SIGBLOCK) != 0) { >>> + oldval = casuword32(td->td_sigblock_ptr, >>> + FAST_SIGBLOCK_PEND, 0); >>> + if (oldval == (uint32_t)-1) >>> + error = EFAULT; >>> + else if (oldval != FAST_SIGBLOCK_PEND) >>> + error = EBUSY; >>> + else >>> + td->td_sigblock_val = 0; >>> + } else >>> + error = EDOOFUS; >>> + >>> + /* >>> + * Rely on normal ast mechanism to deliver pending >>> + * signals to current thread. But notify others about >>> + * fake unblock. >>> + */ >>> + p = td->td_proc; >>> + if (error == 0 && p->p_numthreads != 1) { >>> + PROC_LOCK(p); >>> + reschedule_signals(p, td->td_sigmask, 0); >>> + PROC_UNLOCK(p); >>> + } >>> + break; >>> + >>> + case FAST_SIGBLOCK_UNSETPTR: >>> + if ((td->td_pflags & TDP_FAST_SIGBLOCK) != 0) { >>> + error = copyin(td->td_sigblock_ptr, &oldval, >>> + sizeof(uint32_t)); >>> + if (error != 0) >>> + break; >>> + if (oldval != 0 && oldval != FAST_SIGBLOCK_PEND) >>> + error = EBUSY; >>> + else { >>> + td->td_pflags &= ~TDP_FAST_SIGBLOCK; >>> + td->td_sigblock_val = 0; >>> + } >>> + } else >>> + error = EDOOFUS; >>> + break; >>> + } >>> + return (error); >>> +} >>> + >>> +void >>> +fetch_fast_sigblock(struct thread *td) >>> +{ >>> + >>> + if ((td->td_pflags & TDP_FAST_SIGBLOCK) == 0) >>> + return; >>> + td->td_sigblock_val = fuword32(td->td_sigblock_ptr); >>> + if (td->td_sigblock_val == -1) >>> + fetch_fast_sigblock_failed(td, 0); >>> +} >>> + >>> +void >>> +fetch_fast_sigblock_failed(struct thread *td, int write) >>> +{ >>> + ksiginfo_t ksi; >>> + >>> + td->td_sigblock_val = 0; >>> + ksiginfo_init_trap(&ksi); >>> + ksi.ksi_signo = SIGSEGV; >>> + ksi.ksi_code = write ? SEGV_ACCERR : SEGV_MAPERR; >>> + ksi.ksi_addr = td->td_sigblock_ptr; >>> + trapsignal(td, &ksi); >>> +} >>> diff --git a/sys/kern/subr_syscall.c b/sys/kern/subr_syscall.c >>> index 5aee684..77b250d 100644 >>> --- a/sys/kern/subr_syscall.c >>> +++ b/sys/kern/subr_syscall.c >>> @@ -131,6 +131,12 @@ syscallenter(struct thread *td, struct syscall_args *sa) >>> sa->callp, sa->args, 0); >>> #endif >>> >>> + /* >>> + * Fetch fast sigblock value at the time of syscall >>> + * entry because sleepqueue primitives might call >>> + * cursig(). >>> + */ >>> + fetch_fast_sigblock(td); >>> AUDIT_SYSCALL_ENTER(sa->code, td); >>> error = (sa->callp->sy_call)(td, sa->args); >>> AUDIT_SYSCALL_EXIT(error, td); >>> diff --git a/sys/kern/subr_trap.c b/sys/kern/subr_trap.c >>> index 095bbdf..66e485b 100644 >>> --- a/sys/kern/subr_trap.c >>> +++ b/sys/kern/subr_trap.c >>> @@ -237,6 +237,7 @@ ast(struct trapframe *framep) >>> */ >>> if (flags & TDF_NEEDSIGCHK || p->p_pendingcnt > 0 || >>> !SIGISEMPTY(p->p_siglist)) { >>> + fetch_fast_sigblock(td); >>> PROC_LOCK(p); >>> mtx_lock(&p->p_sigacts->ps_mtx); >>> while ((sig = cursig(td, SIG_STOP_ALLOWED)) != 0) >>> diff --git a/sys/kern/syscalls.master b/sys/kern/syscalls.master >>> index f62dad7..28a9393 100644 >>> --- a/sys/kern/syscalls.master >>> +++ b/sys/kern/syscalls.master >>> @@ -951,5 +951,6 @@ >>> off_t offset, off_t len); } >>> 531 AUE_NULL STD { int posix_fadvise(int fd, off_t offset, \ >>> off_t len, int advice); } >>> +532 AUE_NULL STD { int fast_sigblock(int cmd, uint32_t *ptr); } >>> ; Please copy any additions and changes to the following compatability tables: >>> ; sys/compat/freebsd32/syscalls.master >>> diff --git a/sys/sys/proc.h b/sys/sys/proc.h >>> index 06df632..4899ca2 100644 >>> --- a/sys/sys/proc.h >>> +++ b/sys/sys/proc.h >>> @@ -272,6 +272,9 @@ struct thread { >>> struct osd td_osd; /* (k) Object specific data. */ >>> struct vm_map_entry *td_map_def_user; /* (k) Deferred entries. */ >>> pid_t td_dbg_forked; /* (c) Child pid for debugger. */ >>> + void *td_sigblock_ptr; /* (k) uptr for fast sigblock. */ >>> + uint32_t td_sigblock_val; /* (k) fast sigblock value at >>> + kernel entry. */ >>> #define td_endzero td_sigmask >>> >>> /* Copied during fork1() or create_thread(). */ >>> @@ -424,6 +427,7 @@ do { \ >>> #define TDP_RESETSPUR 0x04000000 /* Reset spurious page fault history. */ >>> #define TDP_NERRNO 0x08000000 /* Last errno is already in td_errno */ >>> #define TDP_UIOHELD 0x10000000 /* Current uio has pages held in td_ma */ >>> +#define TDP_FAST_SIGBLOCK 0x20000000 /* Fast sigblock active */ >>> >>> /* >>> * Reasons that the current thread can not be run yet. >>> diff --git a/sys/sys/signalvar.h b/sys/sys/signalvar.h >>> index 71685e7..68cca58 100644 >>> --- a/sys/sys/signalvar.h >>> +++ b/sys/sys/signalvar.h >>> @@ -250,6 +250,20 @@ typedef struct sigqueue { >>> /* Flags for ksi_flags */ >>> #define SQ_INIT 0x01 >>> >>> +/* >>> + * Fast_sigblock >>> + */ >>> +#define FAST_SIGBLOCK_SETPTR 1 >>> +#define FAST_SIGBLOCK_UNBLOCK 2 >>> +#define FAST_SIGBLOCK_UNSETPTR 3 >>> + >>> +#define FAST_SIGBLOCK_PEND 0x1 >>> +#define FAST_SIGBLOCK_INC 0x10 >>> + >>> +#ifndef _KERNEL >>> +int __sys_fast_sigblock(int cmd, void *ptr); >>> +#endif >>> + >>> #ifdef _KERNEL >>> >>> /* Return nonzero if process p has an unmasked pending signal. */ >>> @@ -329,6 +343,8 @@ extern struct mtx sigio_lock; >>> >>> int cursig(struct thread *td, int stop_allowed); >>> void execsigs(struct proc *p); >>> +void fetch_fast_sigblock(struct thread *td); >>> +void fetch_fast_sigblock_failed(struct thread *td, int write); >>> void gsignal(int pgid, int sig, ksiginfo_t *ksi); >>> void killproc(struct proc *p, char *why); >>> ksiginfo_t * ksiginfo_alloc(int wait); >>> >>> >>> >>> ----- End forwarded message ----- >>> >> So you want to make kernel share data with userland. The only thing I >> don't like is it adds overhead to syscall, rumor said that our syscall >> is slow than others, this might not be a problem ? >> >> Long time ago, I proposed a schedctl() syscall to make kernel share >> data with userland, some old patches are still there: >> http://people.freebsd.org/~davidxu/schedctl/ >> >> Mine does not have such an overhead, but it has another one: >> memory page is allocated in kernel which can not be swapped out >> and can not be freed until process is exited, the page is doubly >> mapped into in kernel and userland, accessing the shared data >> in kernel has zero overhead though. > Initial version of the changes indeed used the remap of the user page into > the KVA. Besides the issues you described regarding the page being wired > for the whole life of the thread, as well as the one page frame in the KVA, > there were other non-trivial problems. It was mostly related to the fork(2) > copying the address spaces, but not limited too. The use of the direct > user address access appeared to be much less intrusive. > > I completely agree with your note about the additional memory access in > the syscall entry path. I do believe that it does not incur a overhead > in the real-world loads, but might make some skews in the microbenchmarks. > I did not noted any in the testing I did. > > Anyway, I am still not sure is it worth optimizing for the throw/catch > microbenchmark at ll. If general public consequence is yes, the patch > needs some work before being committable anyway, mostly to allow patched > rtld to work on the pre-patch kernels. > > Thank you for the reply. I think it would be great if it works, I just haven't had time to fully understand your more recent patch. If you feel it's safe and maybe do a few test programs to see what happens, that would be best. -Alfred From owner-freebsd-arch@FreeBSD.ORG Wed Jan 9 05:17:38 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 265281F8 for ; Wed, 9 Jan 2013 05:17:38 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id D6B61187 for ; Wed, 9 Jan 2013 05:17:37 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id C948311BE4 for ; Wed, 9 Jan 2013 15:17:35 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro.local (c-75-71-5-126.hsd1.co.comcast.net [75.71.5.126]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BJE32338 (AUTH peterg@ptree32.com.au); Wed, 9 Jan 2013 15:17:34 +1000 Message-ID: <50ECFD6C.4000408@freebsd.org> Date: Tue, 08 Jan 2013 22:17:32 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.28) Gecko/20120306 Thunderbird/3.1.20 MIME-Version: 1.0 To: "freebsd-arch@freebsd.org" Subject: [RFC] Moving bhyve to head Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2013 05:17:38 -0000 Neel and I would like to move bhyve development from the projects/bhyve branch into CURRENT. This will allow the code to reach a wider audience before 10, and provide us with better feedback on what features should be prioritized. The intent of bhyve is to provide a small, extendible codebase that allows FreeBSD users to easily run virtual machines. Currently, bhyve supports running FreeBSD/amd64 guests on FreeBSD/amd64 hosts with Intel VT-x and EPT CPU support. Additional guest operating systems should be available in the near future, as will AMD-SVM CPU support. bhyve is implemented as a kernel module and user-level utilities. Note that it has zero impact on the system until the module is loaded. The raw diff against CURRENT can be viewed at http://people.freebsd.org/~neel/bhyve/diff.txt (A sanitized diff, without the svn mergeinfo, is at: http://people.freebsd.org/~neel/bhyve/diff_without_mergeinfo.txt A listing of modified and added files with annotations is at: http://people.freebsd.org/~neel/bhyve/diff_filenames_only.txt) Info on bhyve and installation instructions can be found at http://wiki.freebsd.org/BHyVe http://bhyve.org Comments and review requested :) later, Peter & Neel. From owner-freebsd-arch@FreeBSD.ORG Wed Jan 9 05:45:16 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B8E127AF; Wed, 9 Jan 2013 05:45:16 +0000 (UTC) (envelope-from hufeng1987@gmail.com) Received: from mail-wg0-f53.google.com (mail-wg0-f53.google.com [74.125.82.53]) by mx1.freebsd.org (Postfix) with ESMTP id 3188824F; Wed, 9 Jan 2013 05:45:15 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id fn15so844913wgb.20 for ; Tue, 08 Jan 2013 21:45:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=J34xuXpdk5Tc7kv5IS5jXqIWDGQiWTEcjG5TjOoonFI=; b=wnrlL9PznjfnZ+qmGzDTdUQKBXuypJ+KMvhyFW5m+LkfBNl8VOCC/4chsiT25JJoE7 zkbuRHdsANrDS7jcuBWvncqJScNGSDKq7abSDq5Xx6lr7Ufsxfb/aruEZr6+ac5O3afW bTIWa698EXZTJzlsWGR6lxpspuTdD+BRb+I5BNuy4VO12Z7oTDSyUTt61dIwFVxH2thK /bRUcZFuVwW7DxYWo7t2Mr3ADjskv0tAYwcJJTcI1fz/7TBpvJpeQkhLGcExlfUMP0Ds G8TYGYkEdQxzyMyC6WUFtIvRDERU8S/tWMn8NjF6M/rPgAGaPpr6vI/Ff2+qb7obkUQu WTYQ== X-Received: by 10.180.14.10 with SMTP id l10mr1055165wic.7.1357710315069; Tue, 08 Jan 2013 21:45:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.125.98 with HTTP; Tue, 8 Jan 2013 21:44:55 -0800 (PST) In-Reply-To: <50ECFD6C.4000408@freebsd.org> References: <50ECFD6C.4000408@freebsd.org> From: Netroby Date: Wed, 9 Jan 2013 13:44:55 +0800 Message-ID: Subject: Re: [RFC] Moving bhyve to head To: Peter Grehan Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2013 05:45:16 -0000 Great news , happy to see these changes. Appreciate your time. ---------------------------- Netroby 2013/1/9 Peter Grehan > > Neel and I would like to move bhyve development from the > projects/bhyve branch into CURRENT. This will allow the code > to reach a wider audience before 10, and provide us with better > feedback on what features should be prioritized. > > The intent of bhyve is to provide a small, extendible codebase > that allows FreeBSD users to easily run virtual machines. Currently, > bhyve supports running FreeBSD/amd64 guests on FreeBSD/amd64 > hosts with Intel VT-x and EPT CPU support. Additional guest operating > systems should be available in the near future, as will AMD-SVM CPU support. > > bhyve is implemented as a kernel module and user-level utilities. Note > that it has zero impact on the system until the module is loaded. > > The raw diff against CURRENT can be viewed at > http://people.freebsd.org/~neel/bhyve/diff.txt > > (A sanitized diff, without the svn mergeinfo, is at: > http://people.freebsd.org/~neel/bhyve/diff_without_mergeinfo.txt > > A listing of modified and added files with annotations is at: > http://people.freebsd.org/~neel/bhyve/diff_filenames_only.txt) > > Info on bhyve and installation instructions can be found at > http://wiki.freebsd.org/BHyVe > http://bhyve.org > > Comments and review requested :) > > later, > > Peter & Neel. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Jan 9 07:38:50 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 979B49BF; Wed, 9 Jan 2013 07:38:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 0EF967AF; Wed, 9 Jan 2013 07:38:49 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r097chMb015543; Wed, 9 Jan 2013 09:38:43 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r097chMb015543 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r097cgIJ015542; Wed, 9 Jan 2013 09:38:42 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 9 Jan 2013 09:38:42 +0200 From: Konstantin Belousov To: Alfred Perlstein Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130109073841.GH2561@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <50EBAA1F.9070303@freebsd.org> <20130108145631.GD82219@kib.kiev.ua> <50EC34FE.4070804@mu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="PNpeiK4tTqhYOExY" Content-Disposition: inline In-Reply-To: <50EC34FE.4070804@mu.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, David Xu , toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2013 07:38:50 -0000 --PNpeiK4tTqhYOExY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 08, 2013 at 10:02:22AM -0500, Alfred Perlstein wrote: > I think it would be great if it works, I just haven't had time to fully= =20 > understand your more recent patch. If you feel it's safe and maybe do a= =20 > few test programs to see what happens, that would be best. I revived the test programs I used during the development of the patch, see http://people.freebsd.org/~kib/misc/fast_sigblock.c http://people.freebsd.org/~kib/misc/fast_sigblock2.c Also, I adapted the code to the specific case you asked about, in http://people.freebsd.org/~kib/misc/fast_sigblock3.c When run from the shell, it reports 'Killed'. --PNpeiK4tTqhYOExY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ7R6BAAoJEJDCuSvBvK1BzNcP/Aw2npROk0aBG7KzwWyLEWaB 2kI1ABq4nLTuPjhXlBCYOp+NZTJbPMYJuC6p7D+rmWm80sUN4nbaFsNPzGrbglHZ qGwavL0YDvJsCJ3JRfwe2SNYRP4Y1Hfb3D27TQLnYQOw3j2gETqaZTLGs4NJ6X8D R7BIho6Tkx/W7JVgK7CkYUSYPe7YXAu9HfInIN9X9B8WT4NkcgWUKTZTNkF/veBE WA32QPtrcOYR3r1Pf3OOnejRAipl45XV+OaglemrIcTwMYAGszgMdpmTAVEhaxuj 6xSit+8K0aFaZi98vrbMygeZo7Ghpkt257Q8Q2Pn8R0SpKcb1FiNQ2uRcgKARYx1 NPytWMjMdNYeR+ddYdAKm5Q8ibN0EFTmHxJFt7YngmEQ9nbmfcJRArY4iIYyGAeR ECAJxl5xe9zq8a9hY8CYVQt4pmf05NvdR4yEcwZk6p1sNEdA1wTr4/7lVKQgNJEg 64ot+aLHD9+GnpTU3ITZVPOk19hLGCG2buBF/1XG/hex4MELFiam6z+JwKN3ZD7m 5Em/A7OdBwrfUNu/hhZZziryIFPWoN7prOFJIDlMHlP3hPalm5JjBUJEr2Mo6lk6 04KzeH26/7qBmpyQqyT3RUGg4tQ3dvQTRefqjJfi4ESNJM5A+lY9wwc2uYPu3gkF pUVK4yxRIuiz0QFHi5GB =hGET -----END PGP SIGNATURE----- --PNpeiK4tTqhYOExY-- From owner-freebsd-arch@FreeBSD.ORG Wed Jan 9 15:52:22 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 70EB36EA; Wed, 9 Jan 2013 15:52:22 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5F28F9CF; Wed, 9 Jan 2013 15:52:22 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (unknown [207.238.187.242]) by elvis.mu.org (Postfix) with ESMTPSA id C78F01A3DBF; Wed, 9 Jan 2013 07:52:20 -0800 (PST) Message-ID: <50ED9232.8020606@mu.org> Date: Wed, 09 Jan 2013 10:52:18 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> <50EBAA1F.9070303@freebsd.org> <20130108145631.GD82219@kib.kiev.ua> <50EC34FE.4070804@mu.org> <20130109073841.GH2561@kib.kiev.ua> In-Reply-To: <20130109073841.GH2561@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, David Xu , toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2013 15:52:22 -0000 On 1/9/13 2:38 AM, Konstantin Belousov wrote: > On Tue, Jan 08, 2013 at 10:02:22AM -0500, Alfred Perlstein wrote: >> I think it would be great if it works, I just haven't had time to fully >> understand your more recent patch. If you feel it's safe and maybe do a >> few test programs to see what happens, that would be best. > I revived the test programs I used during the development of the patch, see > http://people.freebsd.org/~kib/misc/fast_sigblock.c > http://people.freebsd.org/~kib/misc/fast_sigblock2.c > > Also, I adapted the code to the specific case you asked about, in > http://people.freebsd.org/~kib/misc/fast_sigblock3.c > When run from the shell, it reports 'Killed'. This looks very good, thank you for addressing my concerns. I think you should go forward with it then. -Alfred From owner-freebsd-arch@FreeBSD.ORG Wed Jan 9 17:44:25 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3C831A73; Wed, 9 Jan 2013 17:44:25 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 156DE18A; Wed, 9 Jan 2013 17:44:24 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (unknown [207.238.187.242]) by elvis.mu.org (Postfix) with ESMTPSA id 609EE1A3D01; Wed, 9 Jan 2013 09:44:24 -0800 (PST) Message-ID: <50EDAC76.2090802@mu.org> Date: Wed, 09 Jan 2013 12:44:22 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: [RFC] Moving bhyve to head References: <50ECFD6C.4000408@freebsd.org> In-Reply-To: <50ECFD6C.4000408@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2013 17:44:25 -0000 This is great news. Thanks very much to Peter, Neel and Netapp. I've been using bhyve for a few weeks now and it has made booting kernels and testing various things much easier. I hope eventually to have some time to use bhyve to create a test harness for booting FreeBSD with low memory and various tuning parameters as well as hooking it to tinderbox for testing "the daily build". -Alfred On 1/9/13 12:17 AM, Peter Grehan wrote: > Neel and I would like to move bhyve development from the > projects/bhyve branch into CURRENT. This will allow the code > to reach a wider audience before 10, and provide us with better > feedback on what features should be prioritized. > > The intent of bhyve is to provide a small, extendible codebase > that allows FreeBSD users to easily run virtual machines. Currently, > bhyve supports running FreeBSD/amd64 guests on FreeBSD/amd64 > hosts with Intel VT-x and EPT CPU support. Additional guest operating > systems should be available in the near future, as will AMD-SVM CPU > support. > > bhyve is implemented as a kernel module and user-level utilities. Note > that it has zero impact on the system until the module is loaded. > > The raw diff against CURRENT can be viewed at > http://people.freebsd.org/~neel/bhyve/diff.txt > > (A sanitized diff, without the svn mergeinfo, is at: > http://people.freebsd.org/~neel/bhyve/diff_without_mergeinfo.txt > > A listing of modified and added files with annotations is at: > http://people.freebsd.org/~neel/bhyve/diff_filenames_only.txt) > > Info on bhyve and installation instructions can be found at > http://wiki.freebsd.org/BHyVe > http://bhyve.org > > Comments and review requested :) > > later, > > Peter & Neel. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Thu Jan 10 16:26:28 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id F1263282 for ; Thu, 10 Jan 2013 16:26:28 +0000 (UTC) (envelope-from lidl@pix.net) Received: from Mail.Fairview-Park.Com (Mail.Fairview-Park.Com [98.141.206.6]) by mx1.freebsd.org (Postfix) with ESMTP id 9AC2E7FB for ; Thu, 10 Jan 2013 16:26:27 +0000 (UTC) Received: from [192.168.8.101] (Kurt.Fairview-Park.Com [192.168.8.101]) (authenticated bits=0) by Mail.Fairview-Park.Com (8.14.5/8.14.5) with ESMTP id r0AGOuLh023426 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 10 Jan 2013 11:24:56 -0500 (EST) (envelope-from lidl@pix.net) X-FVP-rcvd: Kurt.Fairview-Park.Com [192.168.8.101] Thu, 10 Jan 2013 11:24:56 -0500 (EST) Message-ID: <50EEEB70.509@pix.net> Date: Thu, 10 Jan 2013 11:25:20 -0500 From: Kurt Lidl User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: [RFC] Moving bhyve to head References: <50ECFD6C.4000408@freebsd.org> In-Reply-To: <50ECFD6C.4000408@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jan 2013 16:26:29 -0000 > Neel and I would like to move bhyve development from the > projects/bhyve branch into CURRENT. This will allow the code > to reach a wider audience before 10, and provide us with better > feedback on what features should be prioritized. > > The intent of bhyve is to provide a small, extendible codebase > that allows FreeBSD users to easily run virtual machines. Currently, > bhyve supports running FreeBSD/amd64 guests on FreeBSD/amd64 > hosts with Intel VT-x and EPT CPU support. Additional guest operating > systems should be available in the near future, as will AMD-SVM CPU support. > > bhyve is implemented as a kernel module and user-level utilities. Note > that it has zero impact on the system until the module is loaded. This is exciting news. I would be very happy to see this in CURRENT. -Kurt From owner-freebsd-arch@FreeBSD.ORG Fri Jan 11 09:55:11 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id F3650AED; Fri, 11 Jan 2013 09:55:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 515A6FFF; Fri, 11 Jan 2013 09:55:10 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r0B9sxiX045840; Fri, 11 Jan 2013 11:54:59 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r0B9sxiX045840 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r0B9sxTZ045839; Fri, 11 Jan 2013 11:54:59 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 11 Jan 2013 11:54:59 +0200 From: Konstantin Belousov To: arch@freebsd.org, toolchain@freebsd.org Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130111095459.GZ2561@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NSDslKtQVY7LeFYu" Content-Disposition: inline In-Reply-To: <20130107182235.GA65279@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2013 09:55:11 -0000 --NSDslKtQVY7LeFYu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 07, 2013 at 08:22:35PM +0200, Konstantin Belousov wrote: > Below is the forward of the patch for which I failed to obtain a private > review. Might be, the list generates more responses. >=20 > Our rtld has a performance bootleneck, typically exposed by the images > with the lot of the run-time relocation processing, and by the C++ > exception handling. We block the signals delivery during the rtld > performing the relocations, as well as for the dl_iterate_phdr(3) (the > later is used for finding the dwarf unwinding tables). >=20 > The signal blocking is needed to allow the rtld to resolve the symbols > for the signal handlers in the safe way, but also causes 2 syscalls > overhead per each rtld entry. >=20 > The proposed approach allows to shave off those two syscalls, doubling > the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. >=20 > Date: Mon, 13 Aug 2012 15:26:00 +0300 > From: Konstantin Belousov >=20 > ... >=20 > The basic idea is to implement sigprocmask() as single write into usermode > address. If kernel needs to calculate the signal mask for current thread, > it takes into the consideration non-zero value of the word at the agreed > address. Also, usermode is informed about signals which were put on hold > due to fast sigblock active. >=20 > As I said, on my measurements in microbenchmark that did throw/catch in > a loop, I see equal user and system time spent for unpatched system, and > same user time with zero system time on patched system. >=20 > Patch can be improved further, e.g. it would be nice to allow rtld to fall > back to sigprocmask(2) if kernel does not support fast sigblock, to preve= nt > flag day. Also, the mask enforced by fast sigblock can be made configurab= le. >=20 > Note that libthr already blocks signals by catching them, and not using r= tld > service in the first line handler. I tried to make the change in the spir= it > of libthr interceptors, but handoff to libthr appears too complicated to > work. In fact, libthr can be changed to start using fast sigblock instead > of wrapping sigaction, but this is out of scope of the proposal right now. >=20 > Please comment. So there were no overly negative comments, and thanks to Alfred and David for useful notes. The patch at http://people.freebsd.org/~kib/misc/rtld-sigblock.2.patch is the commit candidate. Now kernel informs rtld about supprt for fast_sigblock with new auxv flag. Rtld can operate on old (and possibly future) kernels without fast_sigblock, rtld checks the auxv for presence of the ELF_BSDF_FASTSIGBLK flag before use. --NSDslKtQVY7LeFYu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ7+FyAAoJEJDCuSvBvK1BsJYP/iWcrdR8vkZY7F0vNFXpBxjR ut2gpD7pkI4biwsZECz8HzyR6nOceyxENv7ufyVu9M2bkIvwb7mIIfgnjAkBzvik yRh/LOl7eY0GBJpYtykF3m9dp3s+hPvF+SBg1NjQD5x8snC8uIWlToNRxms24P9i 0sUn4KosmntfedrVlLgf2cuXR3QJzOs7h9A1O/RBjelcMVMB1E1SmatrrCqAVWhz EwwKBdOhtyOWtKXEY3tkcsjsjI5nlsUnBOs17mwO1vqID956xZmCQGXM1/sM8yWH 2TAzKa89l06E9ql11lM+z6gzQ6oJb587S8VRPXgwaZU4pIKbgq3jbz2uFcsQKUiz IONGZyZBhO2VrvJWccE/5uASnc24yMck4eHztkibm6SqyrG0qnUZLxNmL2bUQXiN yg+U/DRxXjBBCFWiDnD6U3x+JlVbv91vIsOtrOGbmsIpXeZJ8ggybDD1ejR7n8fN 7V9UXneHrqM2/j7xFwZElqL7DfALVYv5pSqe/n0kf0/RkOZk3GMo3qZLf48im1A+ 8+2Iwez9MZDeWlIAJJPyrIPitZppjLCOb2NJZKFznc/tTQVg8Ug3GM8EnlzvRbw/ XltAhJilvGXGrZrrqxLekFeV9UUc0rQldZ/Wel9xFmrS5DrYNd94PME/lbRdzrEY ilQiq6EeRxZ893rISj3Q =66cr -----END PGP SIGNATURE----- --NSDslKtQVY7LeFYu-- From owner-freebsd-arch@FreeBSD.ORG Fri Jan 11 10:23:17 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id ABBC4973; Fri, 11 Jan 2013 10:23:17 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 833211C7; Fri, 11 Jan 2013 10:23:17 +0000 (UTC) Received: from xyf.my.dom (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r0BANGKh059347; Fri, 11 Jan 2013 10:23:16 GMT (envelope-from davidxu@freebsd.org) Message-ID: <50EFE830.3030500@freebsd.org> Date: Fri, 11 Jan 2013 18:23:44 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:14.0) Gecko/20120822 Thunderbird/14.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> In-Reply-To: <20130111095459.GZ2561@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2013 10:23:17 -0000 On 2013/01/11 17:54, Konstantin Belousov wrote: > On Mon, Jan 07, 2013 at 08:22:35PM +0200, Konstantin Belousov wrote: >> Below is the forward of the patch for which I failed to obtain a private >> review. Might be, the list generates more responses. >> >> Our rtld has a performance bootleneck, typically exposed by the images >> with the lot of the run-time relocation processing, and by the C++ >> exception handling. We block the signals delivery during the rtld >> performing the relocations, as well as for the dl_iterate_phdr(3) (the >> later is used for finding the dwarf unwinding tables). >> >> The signal blocking is needed to allow the rtld to resolve the symbols >> for the signal handlers in the safe way, but also causes 2 syscalls >> overhead per each rtld entry. >> >> The proposed approach allows to shave off those two syscalls, doubling >> the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. >> >> Date: Mon, 13 Aug 2012 15:26:00 +0300 >> From: Konstantin Belousov >> >> ... >> >> The basic idea is to implement sigprocmask() as single write into usermode >> address. If kernel needs to calculate the signal mask for current thread, >> it takes into the consideration non-zero value of the word at the agreed >> address. Also, usermode is informed about signals which were put on hold >> due to fast sigblock active. >> >> As I said, on my measurements in microbenchmark that did throw/catch in >> a loop, I see equal user and system time spent for unpatched system, and >> same user time with zero system time on patched system. >> >> Patch can be improved further, e.g. it would be nice to allow rtld to fall >> back to sigprocmask(2) if kernel does not support fast sigblock, to prevent >> flag day. Also, the mask enforced by fast sigblock can be made configurable. >> >> Note that libthr already blocks signals by catching them, and not using rtld >> service in the first line handler. I tried to make the change in the spirit >> of libthr interceptors, but handoff to libthr appears too complicated to >> work. In fact, libthr can be changed to start using fast sigblock instead >> of wrapping sigaction, but this is out of scope of the proposal right now. >> >> Please comment. > > So there were no overly negative comments, and thanks to Alfred and David > for useful notes. > > The patch at > http://people.freebsd.org/~kib/misc/rtld-sigblock.2.patch > is the commit candidate. Now kernel informs rtld about supprt for > fast_sigblock with new auxv flag. Rtld can operate on old (and possibly > future) kernels without fast_sigblock, rtld checks the auxv for > presence of the ELF_BSDF_FASTSIGBLK flag before use. > The patch looks fine to me. Sorry, I still have a question: if I found a process does not responsing to a signal, now should we check both its signal mask and fast block value ? but is there such a tool available ? for example, in gdb. From owner-freebsd-arch@FreeBSD.ORG Fri Jan 11 20:49:53 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 872CF55A; Fri, 11 Jan 2013 20:49:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id D7A23CD; Fri, 11 Jan 2013 20:49:52 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r0BKndIO014116; Fri, 11 Jan 2013 22:49:39 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r0BKndIO014116 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r0BKnc8c014115; Fri, 11 Jan 2013 22:49:38 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 11 Jan 2013 22:49:38 +0200 From: Konstantin Belousov To: David Xu Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130111204938.GD2561@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jYUWvSWTDpDT74zJ" Content-Disposition: inline In-Reply-To: <50EFE830.3030500@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2013 20:49:53 -0000 --jYUWvSWTDpDT74zJ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jan 11, 2013 at 06:23:44PM +0800, David Xu wrote: > On 2013/01/11 17:54, Konstantin Belousov wrote: > > On Mon, Jan 07, 2013 at 08:22:35PM +0200, Konstantin Belousov wrote: > >> Below is the forward of the patch for which I failed to obtain a priva= te > >> review. Might be, the list generates more responses. > >> > >> Our rtld has a performance bootleneck, typically exposed by the images > >> with the lot of the run-time relocation processing, and by the C++ > >> exception handling. We block the signals delivery during the rtld > >> performing the relocations, as well as for the dl_iterate_phdr(3) (the > >> later is used for finding the dwarf unwinding tables). > >> > >> The signal blocking is needed to allow the rtld to resolve the symbols > >> for the signal handlers in the safe way, but also causes 2 syscalls > >> overhead per each rtld entry. > >> > >> The proposed approach allows to shave off those two syscalls, doubling > >> the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. > >> > >> Date: Mon, 13 Aug 2012 15:26:00 +0300 > >> From: Konstantin Belousov > >> > >> ... > >> > >> The basic idea is to implement sigprocmask() as single write into user= mode > >> address. If kernel needs to calculate the signal mask for current thre= ad, > >> it takes into the consideration non-zero value of the word at the agre= ed > >> address. Also, usermode is informed about signals which were put on ho= ld > >> due to fast sigblock active. > >> > >> As I said, on my measurements in microbenchmark that did throw/catch in > >> a loop, I see equal user and system time spent for unpatched system, a= nd > >> same user time with zero system time on patched system. > >> > >> Patch can be improved further, e.g. it would be nice to allow rtld to = fall > >> back to sigprocmask(2) if kernel does not support fast sigblock, to pr= event > >> flag day. Also, the mask enforced by fast sigblock can be made configu= rable. > >> > >> Note that libthr already blocks signals by catching them, and not usin= g rtld > >> service in the first line handler. I tried to make the change in the s= pirit > >> of libthr interceptors, but handoff to libthr appears too complicated = to > >> work. In fact, libthr can be changed to start using fast sigblock inst= ead > >> of wrapping sigaction, but this is out of scope of the proposal right = now. > >> > >> Please comment. > > > > So there were no overly negative comments, and thanks to Alfred and Dav= id > > for useful notes. > > > > The patch at > > http://people.freebsd.org/~kib/misc/rtld-sigblock.2.patch > > is the commit candidate. Now kernel informs rtld about supprt for > > fast_sigblock with new auxv flag. Rtld can operate on old (and possibly > > future) kernels without fast_sigblock, rtld checks the auxv for > > presence of the ELF_BSDF_FASTSIGBLK flag before use. > > >=20 > The patch looks fine to me. Sorry, I still have a question: > if I found a process does not responsing to a signal, now should we > check both its signal mask and fast block value ? but is there such > a tool available ? for example, in gdb. Fair enough. I added the facility to export the address of the fast sigblock word to the tools, and implemented the procstat extension to print the address with -j. You can indeed use gdb to look up the content of the word when debugging, after getting it address from procstat. http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch --jYUWvSWTDpDT74zJ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ8HriAAoJEJDCuSvBvK1BI/wP/i56RdjaKBiS48mkS24o2YHr yQ+NCpZE6Wj1PI37MnL/gPOPZyqoGEdsqdhDRfG0+dhEf2X1NCnOhWWHIaq4V2BJ CDhvpqLLU/Fd3N9N/JmaZvrHpZF7Bsvs36PLCAD7EChoX4KTnC9lEUqsfa9YUNuR 9eT7DmfSqGAcfUTlIs2EbudDI/EWkhXdx6PVAKGd6lBW74UQH6yeLrFJoB5ly6b9 dVEbufpcqy1OxbniM0BX9UgZUmbb3d3bE/tgrS/wrxo1gh4iMu5a2Ll/8zS6+PZm kcJ6pvp+4rTdmUxrgfmSlwgN+NIQpXGFOWtTR4zgyjOgK2uUkbUd9qujBHwmLoFS fJ7UIQCCyra3+GFPoEuRaHdrCMvEoR10J6zEHWN6TceIfei7FY0Qb2fbfRtW1If5 4p1tWoL0D+3lmO8aitdWQNIGEir4qXHQMFYW6zIBJC2qV8eMvxLW8uyCnq809vd7 5dmmwkybRR8n5vJQ2dXZwiWPF/K6o0WF+wI8bGMTAa4TO/TdKsy6AW1AtHOv33nO 8NChYxPM3s3YmjjKwp0qfrhHw7zacrooBPR3sHGpoDEMHSHsQESk5yOy0o63vSqG jJWAus44Lij0aQOCy4KJmkE7K41g0bUuUumSW4+D0+noDegykndojE5ulvj7RmPN z2GrZFFWTZd3O9NIbo7m =1Olc -----END PGP SIGNATURE----- --jYUWvSWTDpDT74zJ-- From owner-freebsd-arch@FreeBSD.ORG Fri Jan 11 20:52:10 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id AADB47DA; Fri, 11 Jan 2013 20:52:10 +0000 (UTC) (envelope-from lev@serebryakov.spb.ru) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id 59DBBFE; Fri, 11 Jan 2013 20:52:10 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:a15f:ca89:8194:fe2e]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 5D8A64AC2D; Sat, 12 Jan 2013 00:52:02 +0400 (MSK) Date: Sat, 12 Jan 2013 00:51:56 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1533659601.20130112005156@serebryakov.spb.ru> To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) In-Reply-To: <20130111095459.GZ2561@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2013 20:52:10 -0000 Hello, Konstantin. You wrote 11 =D1=8F=D0=BD=D0=B2=D0=B0=D1=80=D1=8F 2013 =D0=B3., 13:54:59: KB> http://people.freebsd.org/~kib/misc/rtld-sigblock.2.patch KB> is the commit candidate. Now kernel informs rtld about supprt for Is it first patch in this state, which implements idea of shared memory between kernel and userland to spped up some syscalls? There were discussions about this approach for speedup of getpid()/getppid()/gettimeofday(), but without code ready for commit, am I right? --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Fri Jan 11 21:15:29 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2C60FE16; Fri, 11 Jan 2013 21:15:29 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 990AF1D8; Fri, 11 Jan 2013 21:15:28 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r0BLFMos017352; Fri, 11 Jan 2013 23:15:22 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r0BLFMos017352 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r0BLFMbi017351; Fri, 11 Jan 2013 23:15:22 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 11 Jan 2013 23:15:22 +0200 From: Konstantin Belousov To: Lev Serebryakov Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130111211522.GG2561@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <1533659601.20130112005156@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OWym5pP/pDvoNOpg" Content-Disposition: inline In-Reply-To: <1533659601.20130112005156@serebryakov.spb.ru> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2013 21:15:29 -0000 --OWym5pP/pDvoNOpg Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 12, 2013 at 12:51:56AM +0400, Lev Serebryakov wrote: > Hello, Konstantin. > You wrote 11 =D1=CE=D7=C1=D2=D1 2013 =C7., 13:54:59: >=20 >=20 > KB> http://people.freebsd.org/~kib/misc/rtld-sigblock.2.patch > KB> is the commit candidate. Now kernel informs rtld about supprt for > Is it first patch in this state, which implements idea of shared > memory between kernel and userland to spped up some syscalls? There > were discussions about this approach for speedup of > getpid()/getppid()/gettimeofday(), but without code ready for commit, > am I right? No, you are wrong. FreeBSD have shared page for very long time, and it is used for syscall-less gettimeofday for quite some time. Non-shared page which could carry per-thread or per-process information, neccessary for looking good in the trivial benchmarks, like getpid(), is different case. The patch does not implement the idea of speeding up some syscalls at all. It provides the special purpose one-off implementation of the alternate mechanism, for the internal consumption of rtld. --OWym5pP/pDvoNOpg Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ8IDoAAoJEJDCuSvBvK1BZm8P/jfGlOBCq+iVPm29a0Uz4vW/ ssRW030DgmxZJkTALoH2Cq1HiU5jc2DmSqQL80UufQ97vKO3zWcIpU/iP3LbKuSF C+g+YOMPwbGFwAq2zWuSbQ8l8WNgxNivbWdiqR8UJguiZRJNm3ea88RXoRVGzOqV p1TwM4K/pEuyGqNGvUorZwAhVAI2d5YCL+UapGKFUX2hipg3G/F4Ruddnjn7Iwxi st8Bn+BnuIO/baHzIfleSC0xOaShnVOTv6bRTIxl8iKi7bzdZQ/0NEZSzl+v/yly mH18PxZB9s3Q/dAJ/HBHV0qwP4d59HvICWT078Pa/ePTqVqHjktOvQv7aG40WcH1 cWoZcKkjn0kzUk3iv9bu0G4p7an3AYhocwzBfWgyRKQoK8upzQBy2AFcA3jhEzLl qsjUzyBO2tMi4VxViySJr9rf8ilSDQDAG3GZtc0cG9XdxN93Z1et0v74J6FQZqt6 m2aSbkUtks4VqiPrC6Owf3DSdlWHKvIh36FnxnNWn2Vz2DvvEUm+sFCE5I3HwCU2 EutouzVdn+o167trHFHgASUBMdELShGNLh6ThN+fqVd9jn5b6CiBT18iMYf5JPU4 m88t1Gm/jN1RqwNnGV9TvlJXeefvUxWqZHEQb4phlILAae6z2k02x2wzB6rH2KKH HyQozZBs9LriINdOXz1A =sm/a -----END PGP SIGNATURE----- --OWym5pP/pDvoNOpg-- From owner-freebsd-arch@FreeBSD.ORG Fri Jan 11 23:29:21 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id F2CF016D; Fri, 11 Jan 2013 23:29:20 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (unknown [IPv6:2001:610:1108:5012::107]) by mx1.freebsd.org (Postfix) with ESMTP id B370093F; Fri, 11 Jan 2013 23:29:20 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id C402F1203DE; Sat, 12 Jan 2013 00:29:06 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id A4F582848C; Sat, 12 Jan 2013 00:29:06 +0100 (CET) Date: Sat, 12 Jan 2013 00:29:06 +0100 From: Jilles Tjoelker To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130111232906.GA29017@stack.nl> References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> <20130111204938.GD2561@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130111204938.GD2561@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2013 23:29:21 -0000 On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: > http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch The new fields td_sigblock_ptr and td_sigblock_val are in the part that is zeroed for new threads, while the code in rtld appears to expect them to be copied (on fork, vfork and pthread_create). The fields are correctly zeroed on exec. Sharing the magic variable between threads means that one thread holding an rtld lock will block signals for all other threads as well. This is different from how the normal signal mask works but I don't know whether it may break things. However, the patch depends on it in some way because sigtd() does not know about fast sigblock and will therefore happily select a thread that is fast-sigblocking to handle a signal directed at the process. Although libthr's postpone approach is somewhat ugly, it does not depend on any non-standard kernel features and does not delay the default action. Would it be possible to move that code to libc to make things easier for rtld? It looks like this requires teaching libc about various threading concepts, though. Something feels ugly about not allowing applications to use this, although I don't know how it would work. On the other hand, the strange signal state created by this and implicit assumptions that the duration will be short may be a reason to restrict its use to a relatively small piece of code. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 00:27:12 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 886372C5; Sat, 12 Jan 2013 00:27:12 +0000 (UTC) (envelope-from listlog2011@gmail.com) Received: from mail-gg0-f173.google.com (mail-gg0-f173.google.com [209.85.161.173]) by mx1.freebsd.org (Postfix) with ESMTP id 27BBEB43; Sat, 12 Jan 2013 00:27:11 +0000 (UTC) Received: by mail-gg0-f173.google.com with SMTP id f2so437687ggn.18 for ; Fri, 11 Jan 2013 16:27:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:reply-to:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=rPSsYlKDdjJKAN6OxfLjsSv8ldhkvqciiCO6YXHO3jk=; b=kijs77MHQYJHyA6ISRj0q32xVUtucxtq8AfSGbr4UektWEKwIwh/80m0iKi9OXdODH CT6rp922iHv1T5KO2uArsB2IeZUWdPAACGE7DZBOvkqpdNruw2m23XVZJ/hFXVFLWGAK aOdW58wwOq+8kQT/6qKMdrWEV/r9/n75seqVyAGA5fqXs5m4qGzWkdq0FwOHO4evlHh+ W4bFCJhDqh/m41qmIHhgu1G3baLkYXAk5iERCXK2RiLYfxXKRz72czKNAB1LDHP14G7D VnZKNzIDQ8JaRI9H6OjNlAmbh4PAHXru7hctA8SILcmbMpWWzCDPQ5USR2ZoCZCC3qtf rwUQ== X-Received: by 10.236.93.69 with SMTP id k45mr11007395yhf.28.1357950431277; Fri, 11 Jan 2013 16:27:11 -0800 (PST) Received: from xp5k.my.domain ([115.192.135.231]) by mx.google.com with ESMTPS id r9sm5910613yhm.14.2013.01.11.16.27.08 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Jan 2013 16:27:10 -0800 (PST) Message-ID: <50F0ADDA.4000801@gmail.com> Date: Sat, 12 Jan 2013 08:27:06 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Jilles Tjoelker Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> <20130111204938.GD2561@kib.kiev.ua> <20130111232906.GA29017@stack.nl> In-Reply-To: <20130111232906.GA29017@stack.nl> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: davidxu@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 00:27:12 -0000 On 2013/01/12 07:29, Jilles Tjoelker wrote: > On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: >> http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch > The new fields td_sigblock_ptr and td_sigblock_val are in the part that > is zeroed for new threads, while the code in rtld appears to expect them > to be copied (on fork, vfork and pthread_create). The fields are > correctly zeroed on exec. > > Sharing the magic variable between threads means that one thread holding > an rtld lock will block signals for all other threads as well. This is > different from how the normal signal mask works but I don't know whether > it may break things. However, the patch depends on it in some way > because sigtd() does not know about fast sigblock and will therefore > happily select a thread that is fast-sigblocking to handle a signal > directed at the process. > > Although libthr's postpone approach is somewhat ugly, it does not depend > on any non-standard kernel features and does not delay the default > action. Would it be possible to move that code to libc to make things > easier for rtld? It looks like this requires teaching libc about various > threading concepts, though. Long time ago, if i remembered correctly, kib said that he wanted to merge libthr code into libc, I don't know its state. > Something feels ugly about not allowing applications to use this, > although I don't know how it would work. On the other hand, the strange > signal state created by this and implicit assumptions that the duration > will be short may be a reason to restrict its use to a relatively small > piece of code. > True, it seems it is for short duration. From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 05:32:03 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 89561A79; Sat, 12 Jan 2013 05:32:03 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id DB09A655; Sat, 12 Jan 2013 05:32:02 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r0C5VmH5033003; Sat, 12 Jan 2013 07:31:48 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r0C5VmH5033003 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r0C5VmRC033002; Sat, 12 Jan 2013 07:31:48 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 12 Jan 2013 07:31:48 +0200 From: Konstantin Belousov To: Jilles Tjoelker Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130112053147.GH2561@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> <20130111204938.GD2561@kib.kiev.ua> <20130111232906.GA29017@stack.nl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="1vEdgMWljt97yA53" Content-Disposition: inline In-Reply-To: <20130111232906.GA29017@stack.nl> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 05:32:03 -0000 --1vEdgMWljt97yA53 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 12, 2013 at 12:29:06AM +0100, Jilles Tjoelker wrote: > On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: > > http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch >=20 > The new fields td_sigblock_ptr and td_sigblock_val are in the part that > is zeroed for new threads, while the code in rtld appears to expect them > to be copied (on fork, vfork and pthread_create). The fields are > correctly zeroed on exec. Thank you for noting this. Should be fixed in http://people.freebsd.org/~kib/misc/rtld-sigblock.4.patch >=20 > Sharing the magic variable between threads means that one thread holding > an rtld lock will block signals for all other threads as well. This is > different from how the normal signal mask works but I don't know whether > it may break things. However, the patch depends on it in some way > because sigtd() does not know about fast sigblock and will therefore > happily select a thread that is fast-sigblocking to handle a signal > directed at the process. Hm, I do not quite follow, at least not the first part of the paragraph. The fast sigblock pointer is per-thread, so it is not shared in the kernel. Regardless of the kernel side, rtld is only supposed to use the fast block in the default implementation of the rtld locks, which are overriden by the libthr implementation on libthr initialization. There is also an explicit hand-off from the default locks to the external (libthr), and rtld cares to turn off fast sigblock mechanism when the handoff is performed. The selection of the thread for the delivery of signal in the mt process should not matter then, due to the mechanism not supposed to be used in the mt process. >=20 > Although libthr's postpone approach is somewhat ugly, it does not depend > on any non-standard kernel features and does not delay the default > action. Would it be possible to move that code to libc to make things > easier for rtld? It looks like this requires teaching libc about various > threading concepts, though. The concern there is that rtld would need to have a tight coupling with libc, and possibly with libthr too. >=20 > Something feels ugly about not allowing applications to use this, > although I don't know how it would work. On the other hand, the strange > signal state created by this and implicit assumptions that the duration > will be short may be a reason to restrict its use to a relatively small > piece of code. Application use of the interface is equivalent to the application willingly corrupt its own state. --1vEdgMWljt97yA53 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ8PVDAAoJEJDCuSvBvK1BoRkQAI5otpa95LCidyeDwAW/TN2w R+VMZahZeIGNS8SWABgZnDqBfDHmwnIUDmg4Em/2z/yutPbBr74cl2h7JCCXLwaj FidmiaC18+O8AY68ir9uqt4QGt7VlNbcDutb9KSghdbFVVl0qF18OBNckv5jhltl sQu53uKng5x4DMP3yGxn/7icQVRr1koOwQJku1H7af1c39VIvpjbgLFBucfKlwVX LQkpk5EcjjvjL+SSTVzQJrSzvulj+bE2OXH+24QN+mClY07SjjAgHrASH/SHulAl UkN7H8XqoAEK7YYdz2lDedSOlYmkstykoFuoVbQhTvU+7PN23aEvNkOpi+UUFcZN ZHd45SiQ1/LHNC8W3CUnPzDNEcTMxkdYoNe0qKw43ZkaLgIrFVgM9Je9Ouy+QRCX ij/Y55X4UdqHhBG5dOGg/cvYgjkStAPEVGW+Of0L1YlGxvUGizVU82FTsSszBrH7 Qv1o13mhs1Mas/NXCaP37G4hSJzmYjMBEXcgTYSKb3po07uDErilBx/hkCl+EGBD wU5e25CTQT284fWMB8xxgx7FQ6c0gTLkWeTHf0TBYHEWyt3qT//GrhJoFOm4uCJ8 vgyt+sAbaPLg8rWt26kUZuHnVn3e+n51aPk4hIFBreCAt6mg4HH6OV1yDVPwmwHW Ri1zZ8wEM3SAvomdsBwN =clJL -----END PGP SIGNATURE----- --1vEdgMWljt97yA53-- From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 05:32:58 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DCAC8B37; Sat, 12 Jan 2013 05:32:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 7F03265D; Sat, 12 Jan 2013 05:32:58 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r0C5Wq1Z033020; Sat, 12 Jan 2013 07:32:52 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r0C5Wq1Z033020 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r0C5WqBT033019; Sat, 12 Jan 2013 07:32:52 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 12 Jan 2013 07:32:52 +0200 From: Konstantin Belousov To: davidxu@freebsd.org Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130112053252.GI2561@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> <20130111204938.GD2561@kib.kiev.ua> <20130111232906.GA29017@stack.nl> <50F0ADDA.4000801@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="O2lvLMQR9k8T/RyB" Content-Disposition: inline In-Reply-To: <50F0ADDA.4000801@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, Jilles Tjoelker , toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 05:32:58 -0000 --O2lvLMQR9k8T/RyB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 12, 2013 at 08:27:06AM +0800, David Xu wrote: > On 2013/01/12 07:29, Jilles Tjoelker wrote: > > On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: > >> http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch > > The new fields td_sigblock_ptr and td_sigblock_val are in the part that > > is zeroed for new threads, while the code in rtld appears to expect them > > to be copied (on fork, vfork and pthread_create). The fields are > > correctly zeroed on exec. > > > > Sharing the magic variable between threads means that one thread holding > > an rtld lock will block signals for all other threads as well. This is > > different from how the normal signal mask works but I don't know whether > > it may break things. However, the patch depends on it in some way > > because sigtd() does not know about fast sigblock and will therefore > > happily select a thread that is fast-sigblocking to handle a signal > > directed at the process. > > > > Although libthr's postpone approach is somewhat ugly, it does not depend > > on any non-standard kernel features and does not delay the default > > action. Would it be possible to move that code to libc to make things > > easier for rtld? It looks like this requires teaching libc about various > > threading concepts, though. > Long time ago, if i remembered correctly, kib said that he wanted to > merge libthr code into libc, I don't know its state. Yes, this is correct, and I still want this. >=20 > > Something feels ugly about not allowing applications to use this, > > although I don't know how it would work. On the other hand, the strange > > signal state created by this and implicit assumptions that the duration > > will be short may be a reason to restrict its use to a relatively small > > piece of code. > > > True, it seems it is for short duration. --O2lvLMQR9k8T/RyB Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ8PWDAAoJEJDCuSvBvK1BDgUP/iWJXQukG9XrXeguc+y+bAOf VcWtQfyWdFz0ZsOLdKH6z/gnRF/yiQynvp+na+rtq0rgWDBXIQ7uP0ZKItAevfkW nFM7s12Gv1bQNQq59uFeyshU97ORHCk11ZpjLs84mcKPDWEyAsZQAxRXhz5oTSSL SAkV3ooYeE6eSuFA6ZQgUT3WKDbTwpTTm1mTjGjnPWrRYNrKkrNNZ2P6E7uANawi GWMX0sMbrhtrrmrwUPNdR7ny8jqcaQrCvLxZ+73cVEQBIZIhEuHt9QxqEwZm+OYX v9xV4cSZfCLC2HA/zVxzch5HL+ZExBKDVkSx0K394giPzjlymJePq18VgiBTcozj 6tRPNaWNzUZj9ACwbXsr9sGPuv+/e3VqAR0b5nREOatqy2ha04Q80PZpyIf9Umwx 7D8qsRxmhQJhWfvPevaOeD22OuCy2Wv9qQ76p6OnprsNt8VMStMnBKoxpBL024Na 6VsUaK3QpI3zI1NLdzQyvbfXDd8pQBjYm6NC2Jv3ukX7uc1D3JXJtOESr1TnWyWe QwT8Nf1hLM24mtqJQRipYE4wYSEYXNK3A5v1eXCF3PG/VipQ/I4j+3WPOmDdS6xu E7y5hTNBlQvZuZIFb2FVcuvTuu3AgOB9hdmheZA/xJzsiHUy6dEYWrHST3FUhxHr d7lyOHIZ890gPcsouqy7 =YOKS -----END PGP SIGNATURE----- --O2lvLMQR9k8T/RyB-- From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 09:17:33 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 148A8C34 for ; Sat, 12 Jan 2013 09:17:33 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 03412CD0 for ; Sat, 12 Jan 2013 09:17:32 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (unknown [207.238.187.242]) by elvis.mu.org (Postfix) with ESMTPSA id C5CD51A3C1D for ; Sat, 12 Jan 2013 01:17:29 -0800 (PST) Message-ID: <50F12A27.1000103@mu.org> Date: Sat, 12 Jan 2013 04:17:27 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> <20130111204938.GD2561@kib.kiev.ua> <20130111232906.GA29017@stack.nl> <50F0ADDA.4000801@gmail.com> <20130112053252.GI2561@kib.kiev.ua> In-Reply-To: <20130112053252.GI2561@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 09:17:33 -0000 On 1/12/13 12:32 AM, Konstantin Belousov wrote: > On Sat, Jan 12, 2013 at 08:27:06AM +0800, David Xu wrote: >> On 2013/01/12 07:29, Jilles Tjoelker wrote: >>> On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: >>>> http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch >>> The new fields td_sigblock_ptr and td_sigblock_val are in the part that >>> is zeroed for new threads, while the code in rtld appears to expect them >>> to be copied (on fork, vfork and pthread_create). The fields are >>> correctly zeroed on exec. >>> >>> Sharing the magic variable between threads means that one thread holding >>> an rtld lock will block signals for all other threads as well. This is >>> different from how the normal signal mask works but I don't know whether >>> it may break things. However, the patch depends on it in some way >>> because sigtd() does not know about fast sigblock and will therefore >>> happily select a thread that is fast-sigblocking to handle a signal >>> directed at the process. >>> >>> Although libthr's postpone approach is somewhat ugly, it does not depend >>> on any non-standard kernel features and does not delay the default >>> action. Would it be possible to move that code to libc to make things >>> easier for rtld? It looks like this requires teaching libc about various >>> threading concepts, though. >> Long time ago, if i remembered correctly, kib said that he wanted to >> merge libthr code into libc, I don't know its state. > Yes, this is correct, and I still want this. Can we just do this already? No offense intended if the challenge is technical, I'm assuming it's political? It's only 10 years overdue. -Alfred From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 09:38:46 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BB7C2FE2 for ; Sat, 12 Jan 2013 09:38:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 1AAD9D71 for ; Sat, 12 Jan 2013 09:38:45 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r0C9cVZi058589; Sat, 12 Jan 2013 11:38:31 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r0C9cVZi058589 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r0C9cVrk058588; Sat, 12 Jan 2013 11:38:31 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 12 Jan 2013 11:38:31 +0200 From: Konstantin Belousov To: Alfred Perlstein Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130112093831.GK2561@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> <20130111204938.GD2561@kib.kiev.ua> <20130111232906.GA29017@stack.nl> <50F0ADDA.4000801@gmail.com> <20130112053252.GI2561@kib.kiev.ua> <50F12A27.1000103@mu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="PFhoI1KhOsYPpkJR" Content-Disposition: inline In-Reply-To: <50F12A27.1000103@mu.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 09:38:46 -0000 --PFhoI1KhOsYPpkJR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 12, 2013 at 04:17:27AM -0500, Alfred Perlstein wrote: > On 1/12/13 12:32 AM, Konstantin Belousov wrote: > > On Sat, Jan 12, 2013 at 08:27:06AM +0800, David Xu wrote: > >> On 2013/01/12 07:29, Jilles Tjoelker wrote: > >>> On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: > >>>> http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch > >>> The new fields td_sigblock_ptr and td_sigblock_val are in the part th= at > >>> is zeroed for new threads, while the code in rtld appears to expect t= hem > >>> to be copied (on fork, vfork and pthread_create). The fields are > >>> correctly zeroed on exec. > >>> > >>> Sharing the magic variable between threads means that one thread hold= ing > >>> an rtld lock will block signals for all other threads as well. This is > >>> different from how the normal signal mask works but I don't know whet= her > >>> it may break things. However, the patch depends on it in some way > >>> because sigtd() does not know about fast sigblock and will therefore > >>> happily select a thread that is fast-sigblocking to handle a signal > >>> directed at the process. > >>> > >>> Although libthr's postpone approach is somewhat ugly, it does not dep= end > >>> on any non-standard kernel features and does not delay the default > >>> action. Would it be possible to move that code to libc to make things > >>> easier for rtld? It looks like this requires teaching libc about vari= ous > >>> threading concepts, though. > >> Long time ago, if i remembered correctly, kib said that he wanted to > >> merge libthr code into libc, I don't know its state. > > Yes, this is correct, and I still want this. >=20 > Can we just do this already? I estimate the initial efforts for this as 2-3 weeks of full-time work. >=20 > No offense intended if the challenge is technical, I'm assuming it's=20 > political? >=20 > It's only 10 years overdue. The KSE was removed in 2008, so it could only be done for the 5 years. That said, the hardest issue with the merge is to preserve the ABI, esp. in regard to the libthr and its namespace. For the binaries, the threading symbols still must come from the libthr, while the merge left as an implementation detail. I implemented the required features for rtld, namely ELF filters support, not so long time ago, and there were still bug fixes as recent as of 2012, after libstdc++ was split. Also, we do not want the threading initialized before first additional thread is created, as well as we do not want libc to use real locking primitives before that. The same holds for rtld locks. The main motivation for the merge is the loadable modules which are linked against -lpthread, while the process which loads them is not. Since libthr interposes several libc interfaces, such processes end up with the completely broken linkage. As an example, the errno is still resolved to the single-threaded function, so the checks for the error cause return garbage after libthr load. I tried to cover the issue by marking libthr as not dlopen-able, but the outcry was too loud. --PFhoI1KhOsYPpkJR Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ8S8XAAoJEJDCuSvBvK1BFWAP/jqbY6Cxt3+PxxHReF3wggVf iBbKyRBGcsNuyBe5qPJuZWAQiQuO8aiCr3y5bbDTLampfbcY61NyWNhK4I7DlrXB E+++9AZAi0v9jPyXvblmNahBTxcBWpJuE2UX8OqPSU1kM8gCBCfgPqSUSoycSEWi uedzQ1ozN3jkmgiQCfPoq5FMVKzqMe9s2Co27d2wx8ZRR3oVijaIq2bfyRcu5J7T N1SuvNUZJYBxy6lFq8FA3CNmJLRmDlpfStS+zmROAJE+FDOio2x8C4FKa2U3zBa2 apDQiqOD5j47/OB7/dCmBT6MwORtzohAgOf7WEdGnhhV6MrTSUkExLIiWLNQ63Y8 0GBY4KK3+ewM8RXSSd2dsOeieub8F+nCgwelg2ePN/SicvYtKTO2iprFfYD2nUNt WVjQKliiOESZ8E7HxX3AIseexHYlUptiVTOLLKF80T/lal4cWnkDncbkBCQ3osqs 2XbWaj/qSAncCt82VfQucFksZwLI3i1A7zI5MOS9H1Llb3gZdsIOihuibGbj7O7X fgvRL/9H0+qm3tJZwjolXWdJiSEwkkzZFM96oRle29z8DH411de+tFBkX3fNVIMl paKi8ZeMyEGrVVzFH79UshEyJFpWLBApxqN+E+Wy/UaOoQfiUvzK38DsN5EaUU0/ 9VEZZwz83EKkxmEdAhju =5lYX -----END PGP SIGNATURE----- --PFhoI1KhOsYPpkJR-- From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 11:56:41 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 97BBD8E1; Sat, 12 Jan 2013 11:56:41 +0000 (UTC) (envelope-from theraven@theravensnest.org) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) by mx1.freebsd.org (Postfix) with ESMTP id 4810A2C0; Sat, 12 Jan 2013 11:56:40 +0000 (UTC) Received: from [192.168.0.2] (cpc10-cmbg15-2-0-cust123.5-4.cable.virginmedia.com [86.30.246.124]) (authenticated bits=0) by theravensnest.org (8.14.5/8.14.5) with ESMTP id r0CBuU1c001928 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Sat, 12 Jan 2013 11:56:33 GMT (envelope-from theraven@theravensnest.org) Subject: Re: LLVM Image Activator Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: David Chisnall In-Reply-To: <50E9BC2D.7000302@freebsd.org> Date: Sat, 12 Jan 2013 11:56:24 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <7ADA6363-E714-42E7-82FA-BDE6C64C75A9@theravensnest.org> References: <50E9BC2D.7000302@freebsd.org> To: Nathan Whitehorn X-Mailer: Apple Mail (2.1278) Cc: freebsd-toolchain@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 11:56:41 -0000 Hi Nathan, I'm very interested in making better use of LLVM in our infrastructure. = It would also be nice to be able to integrate this with rtld, so that we = can load shared libraries that contain LLVM IR. Beyond that, there are a few long-term projects that I have in mind: - Run passes that add code for collecting profiling information before = running code invoked in this way. - Recompile on subsequent invocations based on the collected profiling = information. - Specialise installed shared libraries based on the programs that link = to them, so that if foo band bar both link to libbaz, but use it in very = different ways, they get different versions compiled specifically for = them. - Statically compile and cache programs invoked with the JIT, so that = they are very fast on their second run. This would probably want a = periodic job to remove old caches. =20 I have written code in LanguageKit for doing several of these things, = but it would be nicer to have them done at the system level, rather than = at the level of a front end. I'd also like to start using the LLVM JIT for accelerating various other = functions. I have a proof-of-concept JIT for BPF that uses LLVM and I'd = like to extend this to pf/ipf firewall rules. David On 6 Jan 2013, at 18:02, Nathan Whitehorn wrote: > Having LLVM/clang in the base system lets us do some interesting = things > that we couldn't do with GCC. One is that LLVM ships with a JIT for = LLVM > IR as well as components of a toolchain for it (this is what Google's > pNACL uses) and that you can end up producing binary files that are in > IR instead of native code. The IR isn't really cross-platform, but = does > let you do CPU-specific optimizations when executed by the JIT, etc. >=20 > The attached patch causes the LLVM JIT (lli) to be built by default > (adding ~20 seconds to buildworld on my five-year-old laptop) and adds = a > kernel image activator that invokes it when passed LLVM bitcode files. > It's not completely finished (see the XXX comment in the middle), but = it > does work, as follows: >=20 > $ clang -emit-llvm -c -o hw.ll hw.c > $ file hw.ll > hw.ll: LLVM bitcode > $ lli hw.ll > Hello world! > $ chmod a+x hw.ll > $ ./hw.ll > Hello world! > $ >=20 > Is there any interest in having features like this? It seems like this > could provides some interesting possibilities for us and nice > integration from having imported clang into base. > -Nathan > _______________________________________________ > freebsd-toolchain@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain > To unsubscribe, send any mail to = "freebsd-toolchain-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 16:25:51 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5404CA58; Sat, 12 Jan 2013 16:25:51 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id DE92CEFF; Sat, 12 Jan 2013 16:25:50 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id F1529359313; Sat, 12 Jan 2013 17:25:47 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id DA5F62848C; Sat, 12 Jan 2013 17:25:47 +0100 (CET) Date: Sat, 12 Jan 2013 17:25:47 +0100 From: Jilles Tjoelker To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130112162547.GA54954@stack.nl> References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> <20130111204938.GD2561@kib.kiev.ua> <20130111232906.GA29017@stack.nl> <20130112053147.GH2561@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130112053147.GH2561@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 16:25:51 -0000 On Sat, Jan 12, 2013 at 07:31:48AM +0200, Konstantin Belousov wrote: > On Sat, Jan 12, 2013 at 12:29:06AM +0100, Jilles Tjoelker wrote: > > On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: > > > http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch > > The new fields td_sigblock_ptr and td_sigblock_val are in the part that > > is zeroed for new threads, while the code in rtld appears to expect them > > to be copied (on fork, vfork and pthread_create). The fields are > > correctly zeroed on exec. > Thank you for noting this. Should be fixed in > http://people.freebsd.org/~kib/misc/rtld-sigblock.4.patch > > Sharing the magic variable between threads means that one thread holding > > an rtld lock will block signals for all other threads as well. This is > > different from how the normal signal mask works but I don't know whether > > it may break things. However, the patch depends on it in some way > > because sigtd() does not know about fast sigblock and will therefore > > happily select a thread that is fast-sigblocking to handle a signal > > directed at the process. > Hm, I do not quite follow, at least not the first part of the paragraph. > The fast sigblock pointer is per-thread, so it is not shared in the kernel. > Regardless of the kernel side, rtld is only supposed to use the fast > block in the default implementation of the rtld locks, which are overriden > by the libthr implementation on libthr initialization. There is also an > explicit hand-off from the default locks to the external (libthr), and > rtld cares to turn off fast sigblock mechanism when the handoff is > performed. > The selection of the thread for the delivery of signal in the mt process > should not matter then, due to the mechanism not supposed to be used > in the mt process. OK, you are right. If multiple threads exist, the patched code disables fast sigblock and the current unpatched code already postpones signal handlers without sigprocmask() syscalls. This suggests a different rather simpler change. Libthr can always use its rtld lock implementation instead of only when multiple threads exist. This avoids the sigprocmask() syscalls and should not be much slower than the default implementation in rtld because that also contains atomic operations (both the unpatched and the patched version). People that care about performance of exceptions can then link in libthr (in many cases, it is already linked in to allow for (the possibility of) threading). I have tested this and exceptions were indeed more than twice as fast in my test program if I created an extra thread doing nothing than in the fully single-threaded version (with or without libthr). With that, I think fast sigblock is too much code and complication for a niche case. Most of the extra atomics in multi-threaded applications are conditional on __isthreaded (or can be made so); therefore, performance loss from linking in libthr should be negligible in most cases. > > Although libthr's postpone approach is somewhat ugly, it does not depend > > on any non-standard kernel features and does not delay the default > > action. Would it be possible to move that code to libc to make things > > easier for rtld? It looks like this requires teaching libc about various > > threading concepts, though. > The concern there is that rtld would need to have a tight coupling > with libc, and possibly with libthr too. Libc could call _rtld_thread_init() during initialization, somewhat like libthr does now. The problems here are in libc and libthr, not in rtld. Before libc is ready, the current code can be used; if libc is always initialized first, the sigprocmask() calls can be removed because there is no way a signal handler can be installed that early. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 16:56:24 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 760E6373; Sat, 12 Jan 2013 16:56:24 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-x229.google.com (wi-in-x0229.1e100.net [IPv6:2a00:1450:400c:c05::229]) by mx1.freebsd.org (Postfix) with ESMTP id 2DE9BFF3; Sat, 12 Jan 2013 16:56:23 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id hq12so482973wib.0 for ; Sat, 12 Jan 2013 08:56:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=q+q0YjXkji2n4a1qWatdaZ3BA1azA7I4bjuw/XOLA0Q=; b=YN/nf2FrMlS3kfOrCtgYj3RNT2gFqD4lt49zdD9Xv0rzMii+vb447u1UM9UzrzHjSm V5F0fzw1wG+2uQwCsrzeutwENKDnnrj39Pn9F4DlcSK0H4sKs9xS4VE4TSmCUcoitWeZ 7Q+9s/YXWGZjle4WOYrEwOTzUdiwUSogw/2EI29EXSlrUxZ2SO86bOu6CZHvd/YusIG+ KEovTPBeNnaR3tZT1SnoXPDph8PhcKN92IolsTiqGXMcQTiw+X7BX54F89Bsqd65ZPmF aDjRDLBeLU4+5E5rd9Cfz1RJCzmeB0aHP4hVIBOwI5twnYeT7KZyxBjFxB4RMVMPtgrl lIyQ== MIME-Version: 1.0 Received: by 10.180.88.40 with SMTP id bd8mr4537826wib.33.1358009782041; Sat, 12 Jan 2013 08:56:22 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.217.57.9 with HTTP; Sat, 12 Jan 2013 08:56:21 -0800 (PST) In-Reply-To: References: <201211272119.qARLJxXV061083@svn.freebsd.org> <50C1BC90.90106@freebsd.org> <50C25A27.4060007@bluezbox.com> <50C26331.6030504@freebsd.org> <50C26AE9.4020600@bluezbox.com> <50C3A3D3.9000804@freebsd.org> <50C3AF72.4010902@rice.edu> <330405A1-312A-45A5-BB86-4969478D8BBD@bluezbox.com> <50D03E83.8060908@rice.edu> <50DD081E.8000409@bluezbox.com> <50EB1841.5030006@bluezbox.com> <50EB22D2.6090103@rice.edu> <50EB415F.8020405@freebsd.org> <50F04FE5.7010406@rice.edu> Date: Sat, 12 Jan 2013 08:56:21 -0800 X-Google-Sender-Auth: 2CBnz6-loTgqGKTy30-VMnQ25Jo Message-ID: Subject: Re: svn commit: r243631 - in head/sys: kern sys From: Adrian Chadd To: "Jayachandran C." Content-Type: text/plain; charset=ISO-8859-1 Cc: src-committers@freebsd.org, Andre Oppermann , Alan Cox , svn-src-all@freebsd.org, Oleksandr Tymoshenko , freebsd-arch@freebsd.org, svn-src-head@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 16:56:24 -0000 Hi, I think this outlines a larger scale problem here, which is that way, way too many things are relying on maxfiles here and it wasn't properly reviewed or thought out before it made it into the tree. So, can we either: * review _all_ the places maxfiles is being used, and finally fix those; * .. or revert this work until said review and fixup is done? This is the kind of thing that we should just not mess up at this point.. Thanks, Adrian On 12 January 2013 07:46, Jayachandran C. wrote: > On Fri, Jan 11, 2013 at 11:16 PM, Alan Cox wrote: >> On 01/11/2013 05:38, Jayachandran C. wrote: > [...] >>> I see an issue with commit on MIPS XLP platform as well. >>> >>> With 16 GB physical memory, the ncallout is calculated to be 538881 >>> (since it is based on maxfiles - which is now based on the physical >>> memory). Due to this, the callwheel allocation per cpu is 16MB >>> (callwheelsize is 1MB). And on a 32 CPU machine, the total allocation >>> for callouts comes to 32*16MB = 512MB. >>> >>> I have worked around this issue for now by increasing VM_KMEM_SIZE_MAX >>> (which is 200MB now) - but I think a better fix is needed for this. >>> >> >> MIPS should use a definition for VM_KMEM_SIZE_MAX that scales with the >> kernel address space size, like amd64, i386, and sparc64, and not a >> fixed number. I think that the following should work for both 32- and >> 64-bit processors: >> >> Index: mips/include/vmparam.h >> =================================================================== >> --- mips/include/vmparam.h (revision 245229) >> +++ mips/include/vmparam.h (working copy) >> @@ -130,10 +130,11 @@ >> #endif >> >> /* >> - * Ceiling on amount of kmem_map kva space. >> + * Ceiling on the amount of kmem_map KVA space: 40% of the entire KVA >> space. >> */ >> #ifndef VM_KMEM_SIZE_MAX >> -#define VM_KMEM_SIZE_MAX (200 * 1024 * 1024) >> +#define VM_KMEM_SIZE_MAX ((VM_MAX_KERNEL_ADDRESS - \ >> + VM_MIN_KERNEL_ADDRESS + 1) * 2 / 5) >> #endif >> >> /* initial pagein size of beginning of executable file */ > > This fix is needed, can you please check it in? I have tested it for > 32 and 64 bit. > > But the second part of the problem - allocating 512MB out of 16GB at > boot-time for callouts - might need a fix as well. > > Thanks, > JC. From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 17:22:10 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BECBEB18; Sat, 12 Jan 2013 17:22:10 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 8AA40123; Sat, 12 Jan 2013 17:22:10 +0000 (UTC) Received: from JRE-MBP-2.local (c-50-143-148-105.hsd1.ca.comcast.net [50.143.148.105]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id r0CHM1JN020230 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 12 Jan 2013 09:22:01 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <50F19BB4.6080309@freebsd.org> Date: Sat, 12 Jan 2013 09:21:56 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: Fast sigblock (AKA rtld speedup) References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> <20130111204938.GD2561@kib.kiev.ua> <20130111232906.GA29017@stack.nl> <20130112053147.GH2561@kib.kiev.ua> In-Reply-To: <20130112053147.GH2561@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Jilles Tjoelker , toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 17:22:10 -0000 On 1/11/13 9:31 PM, Konstantin Belousov wrote: > On Sat, Jan 12, 2013 at 12:29:06AM +0100, Jilles Tjoelker wrote: >> On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: >>> http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch >> The new fields td_sigblock_ptr and td_sigblock_val are in the part that >> is zeroed for new threads, while the code in rtld appears to expect them >> to be copied (on fork, vfork and pthread_create). The fields are >> correctly zeroed on exec. > Thank you for noting this. Should be fixed in > http://people.freebsd.org/~kib/misc/rtld-sigblock.4.patch > >> Sharing the magic variable between threads means that one thread holding >> an rtld lock will block signals for all other threads as well. This is >> different from how the normal signal mask works but I don't know whether >> it may break things. However, the patch depends on it in some way >> because sigtd() does not know about fast sigblock and will therefore >> happily select a thread that is fast-sigblocking to handle a signal >> directed at the process. > Hm, I do not quite follow, at least not the first part of the paragraph. > > The fast sigblock pointer is per-thread, so it is not shared in the kernel. > Regardless of the kernel side, rtld is only supposed to use the fast > block in the default implementation of the rtld locks, which are overriden > by the libthr implementation on libthr initialization. There is also an > explicit hand-off from the default locks to the external (libthr), and > rtld cares to turn off fast sigblock mechanism when the handoff is > performed. I assume that the thread notifies the kernel of the location.. Might I suggest as a saftybelt that on notification of this address that the kernel requires that a known "magic" value be in this location as proof that all is as expected.? just a thought. > > The selection of the thread for the delivery of signal in the mt process > should not matter then, due to the mechanism not supposed to be used > in the mt process. >> Although libthr's postpone approach is somewhat ugly, it does not depend >> on any non-standard kernel features and does not delay the default >> action. Would it be possible to move that code to libc to make things >> easier for rtld? It looks like this requires teaching libc about various >> threading concepts, though. > The concern there is that rtld would need to have a tight coupling > with libc, and possibly with libthr too. > >> Something feels ugly about not allowing applications to use this, >> although I don't know how it would work. On the other hand, the strange >> signal state created by this and implicit assumptions that the duration >> will be short may be a reason to restrict its use to a relatively small >> piece of code. > Application use of the interface is equivalent to the application > willingly corrupt its own state. From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 19:02:28 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 362D1217; Sat, 12 Jan 2013 19:02:28 +0000 (UTC) (envelope-from alc@rice.edu) Received: from mh10.mail.rice.edu (mh10.mail.rice.edu [128.42.201.30]) by mx1.freebsd.org (Postfix) with ESMTP id E0DC961B; Sat, 12 Jan 2013 19:02:27 +0000 (UTC) Received: from mh10.mail.rice.edu (localhost.localdomain [127.0.0.1]) by mh10.mail.rice.edu (Postfix) with ESMTP id E9123603DF; Sat, 12 Jan 2013 13:02:20 -0600 (CST) Received: from mh10.mail.rice.edu (localhost.localdomain [127.0.0.1]) by mh10.mail.rice.edu (Postfix) with ESMTP id E719B603DB; Sat, 12 Jan 2013 13:02:20 -0600 (CST) X-Virus-Scanned: by amavis-2.7.0 at mh10.mail.rice.edu, auth channel Received: from mh10.mail.rice.edu ([127.0.0.1]) by mh10.mail.rice.edu (mh10.mail.rice.edu [127.0.0.1]) (amavis, port 10026) with ESMTP id J9iE2IBZuJ6e; Sat, 12 Jan 2013 13:02:20 -0600 (CST) Received: from adsl-216-63-78-18.dsl.hstntx.swbell.net (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) (Authenticated sender: alc) by mh10.mail.rice.edu (Postfix) with ESMTPSA id 3AAF8603D8; Sat, 12 Jan 2013 13:02:20 -0600 (CST) Message-ID: <50F1B33B.70401@rice.edu> Date: Sat, 12 Jan 2013 13:02:19 -0600 From: Alan Cox User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: svn commit: r243631 - in head/sys: kern sys References: <201211272119.qARLJxXV061083@svn.freebsd.org> <50C1BC90.90106@freebsd.org> <50C25A27.4060007@bluezbox.com> <50C26331.6030504@freebsd.org> <50C26AE9.4020600@bluezbox.com> <50C3A3D3.9000804@freebsd.org> <50C3AF72.4010902@rice.edu> <330405A1-312A-45A5-BB86-4969478D8BBD@bluezbox.com> <50D03E83.8060908@rice.edu> <50DD081E.8000409@bluezbox.com> <50EB1841.5030006@bluezbox.com> <50EB22D2.6090103@rice.edu> <50EB415F.8020405@freebsd.org> <50F04FE5.7010406@rice.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: src-committers@freebsd.org, Andre Oppermann , "Jayachandran C." , svn-src-all@freebsd.org, Oleksandr Tymoshenko , freebsd-arch@freebsd.org, svn-src-head@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 19:02:28 -0000 On 01/12/2013 10:56, Adrian Chadd wrote: > Hi, > > I think this outlines a larger scale problem here, which is that way, > way too many things are relying on maxfiles here and it wasn't > properly reviewed or thought out before it made it into the tree. > > So, can we either: > > * review _all_ the places maxfiles is being used, and finally fix those; > * .. or revert this work until said review and fixup is done? > > This is the kind of thing that we should just not mess up at this point.. At this point, I think reverting the changes would be an overreaction. Moreover, the change that I just made to vmparam.h on MIPS should arguably have been made when 64-bit support was added to MIPS. For example, if someone had tried running ZFS on a 64-bit MIPS processor just a few weeks or months ago, we would have discovered that the same change was required. In other words, the autoconfiguration changes are bringing to light some problems that have been lurking within the param.h and vmparam.h files on some of the non-x86/sparc64 platforms. Alan P.S. That said, I think we should exercise much caution in MFCing these changes. From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 19:45:52 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4CCAAADB; Sat, 12 Jan 2013 19:45:52 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 1EE6C729; Sat, 12 Jan 2013 19:45:51 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (unknown [207.238.187.242]) by elvis.mu.org (Postfix) with ESMTPSA id 9026A1A3C55; Sat, 12 Jan 2013 11:45:46 -0800 (PST) Message-ID: <50F1BD69.4060104@mu.org> Date: Sat, 12 Jan 2013 14:45:45 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: svn commit: r243631 - in head/sys: kern sys References: <201211272119.qARLJxXV061083@svn.freebsd.org> <50C1BC90.90106@freebsd.org> <50C25A27.4060007@bluezbox.com> <50C26331.6030504@freebsd.org> <50C26AE9.4020600@bluezbox.com> <50C3A3D3.9000804@freebsd.org> <50C3AF72.4010902@rice.edu> <330405A1-312A-45A5-BB86-4969478D8BBD@bluezbox.com> <50D03E83.8060908@rice.edu> <50DD081E.8000409@bluezbox.com> <50EB1841.5030006@bluezbox.com> <50EB22D2.6090103@rice.edu> <50EB415F.8020405@freebsd.org> <50F04FE5.7010406@rice.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: src-committers@freebsd.org, Andre Oppermann , Alan Cox , "Jayachandran C." , svn-src-all@freebsd.org, Oleksandr Tymoshenko , freebsd-arch@freebsd.org, svn-src-head@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 19:45:52 -0000 On 1/12/13 11:56 AM, Adrian Chadd wrote: > Hi, > > I think this outlines a larger scale problem here, which is that way, > way too many things are relying on maxfiles here and it wasn't > properly reviewed or thought out before it made it into the tree. > > So, can we either: > > * review _all_ the places maxfiles is being used, and finally fix those; > * .. or revert this work until said review and fixup is done? > > This is the kind of thing that we should just not mess up at this point.. I'm not sure if regressing to the waterfall method of development is a good idea at this point. I see a light at the end of the tunnel and we to continue to just handle these minor corner cases as we progress. If we move to a model where a minor bug is grounds to completely remove helpful code then nothing will ever get done. -Alfred > Thanks, > > > > Adrian > > On 12 January 2013 07:46, Jayachandran C. wrote: >> On Fri, Jan 11, 2013 at 11:16 PM, Alan Cox wrote: >>> On 01/11/2013 05:38, Jayachandran C. wrote: >> [...] >>>> I see an issue with commit on MIPS XLP platform as well. >>>> >>>> With 16 GB physical memory, the ncallout is calculated to be 538881 >>>> (since it is based on maxfiles - which is now based on the physical >>>> memory). Due to this, the callwheel allocation per cpu is 16MB >>>> (callwheelsize is 1MB). And on a 32 CPU machine, the total allocation >>>> for callouts comes to 32*16MB = 512MB. >>>> >>>> I have worked around this issue for now by increasing VM_KMEM_SIZE_MAX >>>> (which is 200MB now) - but I think a better fix is needed for this. >>>> >>> MIPS should use a definition for VM_KMEM_SIZE_MAX that scales with the >>> kernel address space size, like amd64, i386, and sparc64, and not a >>> fixed number. I think that the following should work for both 32- and >>> 64-bit processors: >>> >>> Index: mips/include/vmparam.h >>> =================================================================== >>> --- mips/include/vmparam.h (revision 245229) >>> +++ mips/include/vmparam.h (working copy) >>> @@ -130,10 +130,11 @@ >>> #endif >>> >>> /* >>> - * Ceiling on amount of kmem_map kva space. >>> + * Ceiling on the amount of kmem_map KVA space: 40% of the entire KVA >>> space. >>> */ >>> #ifndef VM_KMEM_SIZE_MAX >>> -#define VM_KMEM_SIZE_MAX (200 * 1024 * 1024) >>> +#define VM_KMEM_SIZE_MAX ((VM_MAX_KERNEL_ADDRESS - \ >>> + VM_MIN_KERNEL_ADDRESS + 1) * 2 / 5) >>> #endif >>> >>> /* initial pagein size of beginning of executable file */ >> This fix is needed, can you please check it in? I have tested it for >> 32 and 64 bit. >> >> But the second part of the problem - allocating 512MB out of 16GB at >> boot-time for callouts - might need a fix as well. >> >> Thanks, >> JC. From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 21:16:48 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 95663E5B; Sat, 12 Jan 2013 21:16:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 09B7A9C6; Sat, 12 Jan 2013 21:16:47 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r0CLGe0e031517; Sat, 12 Jan 2013 23:16:40 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r0CLGe0e031517 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r0CLGelQ031516; Sat, 12 Jan 2013 23:16:40 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 12 Jan 2013 23:16:40 +0200 From: Konstantin Belousov To: Jilles Tjoelker Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130112211640.GN2561@kib.kiev.ua> References: <20130107182235.GA65279@kib.kiev.ua> <20130111095459.GZ2561@kib.kiev.ua> <50EFE830.3030500@freebsd.org> <20130111204938.GD2561@kib.kiev.ua> <20130111232906.GA29017@stack.nl> <20130112053147.GH2561@kib.kiev.ua> <20130112162547.GA54954@stack.nl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4cokgWgqjr3t8EL1" Content-Disposition: inline In-Reply-To: <20130112162547.GA54954@stack.nl> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 21:16:48 -0000 --4cokgWgqjr3t8EL1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 12, 2013 at 05:25:47PM +0100, Jilles Tjoelker wrote: > On Sat, Jan 12, 2013 at 07:31:48AM +0200, Konstantin Belousov wrote: > > On Sat, Jan 12, 2013 at 12:29:06AM +0100, Jilles Tjoelker wrote: > > > On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: > > > > http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch >=20 > > > The new fields td_sigblock_ptr and td_sigblock_val are in the part th= at > > > is zeroed for new threads, while the code in rtld appears to expect t= hem > > > to be copied (on fork, vfork and pthread_create). The fields are > > > correctly zeroed on exec. > > Thank you for noting this. Should be fixed in > > http://people.freebsd.org/~kib/misc/rtld-sigblock.4.patch >=20 > > > Sharing the magic variable between threads means that one thread hold= ing > > > an rtld lock will block signals for all other threads as well. This is > > > different from how the normal signal mask works but I don't know whet= her > > > it may break things. However, the patch depends on it in some way > > > because sigtd() does not know about fast sigblock and will therefore > > > happily select a thread that is fast-sigblocking to handle a signal > > > directed at the process. >=20 > > Hm, I do not quite follow, at least not the first part of the paragraph. >=20 > > The fast sigblock pointer is per-thread, so it is not shared in the ker= nel. > > Regardless of the kernel side, rtld is only supposed to use the fast > > block in the default implementation of the rtld locks, which are overri= den > > by the libthr implementation on libthr initialization. There is also an > > explicit hand-off from the default locks to the external (libthr), and > > rtld cares to turn off fast sigblock mechanism when the handoff is > > performed. >=20 > > The selection of the thread for the delivery of signal in the mt process > > should not matter then, due to the mechanism not supposed to be used > > in the mt process. >=20 > OK, you are right. If multiple threads exist, the patched code disables > fast sigblock and the current unpatched code already postpones signal > handlers without sigprocmask() syscalls. >=20 > This suggests a different rather simpler change. Libthr can always use > its rtld lock implementation instead of only when multiple threads > exist. This avoids the sigprocmask() syscalls and should not be much > slower than the default implementation in rtld because that also > contains atomic operations (both the unpatched and the patched version). > People that care about performance of exceptions can then link in libthr > (in many cases, it is already linked in to allow for (the possibility > of) threading). >=20 > I have tested this and exceptions were indeed more than twice as fast in > my test program if I created an extra thread doing nothing than in the > fully single-threaded version (with or without libthr). >=20 > With that, I think fast sigblock is too much code and complication for a > niche case. Ok, thank you for the review. I am reverting the branch. I think a solution would need to wait while the work to merge libc and libthr is done. >=20 > Most of the extra atomics in multi-threaded applications are conditional > on __isthreaded (or can be made so); therefore, performance loss from > linking in libthr should be negligible in most cases. >=20 > > > Although libthr's postpone approach is somewhat ugly, it does not dep= end > > > on any non-standard kernel features and does not delay the default > > > action. Would it be possible to move that code to libc to make things > > > easier for rtld? It looks like this requires teaching libc about vari= ous > > > threading concepts, though. >=20 > > The concern there is that rtld would need to have a tight coupling > > with libc, and possibly with libthr too. >=20 > Libc could call _rtld_thread_init() during initialization, somewhat like > libthr does now. The problems here are in libc and libthr, not in rtld. > Before libc is ready, the current code can be used; if libc is always > initialized first, the sigprocmask() calls can be removed because there > is no way a signal handler can be installed that early. >=20 > --=20 > Jilles Tjoelker --4cokgWgqjr3t8EL1 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ8dK2AAoJEJDCuSvBvK1BYkMP/i+lV09PclmW2T/9PjvEm8rZ E4xLnrt4S5tBt1OZLqlSnSRsZ4KUUg0PfOIq7YODHkE6YjaonUb2wVcxa5hPHST3 ea+7waOb0ifKIBgVZ6pkQtHG3IbqRimiJrxVIg7jnz+BSGgA1vpz6shWLT9+aV/A 524lTC6zUiRj9JyAYDlevQ2VTDMAW/BcYS8vI9+9Qmg4nIxsSYTWLcpOlr/iJGCd fxp431WOv+XYlSaknzx+fEBrelXEEEfgY1DbW3BjYH/zqZvceZvQN4fCWmSlJ/uf 5dS86FiykyJb1KqKvQy00esVHIwsMlmhgb4BSzsNuhBPZZTvxlA69FkUONP/s9jb ht8j2FeCaGE3He80IaORWJCAjpPbdeYlBM1Q3D0Rwz3R8zXrjHGbBMhgEKXvEcq3 ZPouvvsuMX6NYlLlCVpgxTOfxmdN4tdCXZolGyb7VAMi9bjCSVrpEJlflxJAv65t JUvJ1+kvcCb7/Bb+oAvC35SpUB7tMX4zIS+0S9XkJCAk/9JF5PgLvKrbN+aRK5Bz bqEIE2gSU4QfVOCigJv3QAt7ZIFpGiRZe5mkcKximPuVaQRBNFKzrt1so2hxfuHE 5Sjsit8zujDlNGejtLM5/Yl+uJYoNick+icMk7/8ubCzQyPyS152ZkfOfax4S8// PI1LW0Kv6qMyukPKjJcA =FUvI -----END PGP SIGNATURE----- --4cokgWgqjr3t8EL1-- From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 23:04:42 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0DA1A579; Sat, 12 Jan 2013 23:04:42 +0000 (UTC) (envelope-from jmg@h2.funkthat.com) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) by mx1.freebsd.org (Postfix) with ESMTP id C9C91CF3; Sat, 12 Jan 2013 23:04:41 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id r0CN4ZVb081699 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 12 Jan 2013 15:04:35 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id r0CN4ZSM081698; Sat, 12 Jan 2013 15:04:35 -0800 (PST) (envelope-from jmg) Date: Sat, 12 Jan 2013 15:04:35 -0800 From: John-Mark Gurney To: David Xu Subject: Re: Fast sigblock (AKA rtld speedup) Message-ID: <20130112230435.GJ1410@funkthat.com> Mail-Followup-To: David Xu , Konstantin Belousov , arch@freebsd.org, toolchain@freebsd.org References: <20130107182235.GA65279@kib.kiev.ua> <50EBAA1F.9070303@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50EBAA1F.9070303@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 12 Jan 2013 15:04:35 -0800 (PST) Cc: Konstantin Belousov , arch@freebsd.org, toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 23:04:42 -0000 David Xu wrote this message on Tue, Jan 08, 2013 at 13:09 +0800: > and can not be freed until process is exited, the page is doubly > mapped into in kernel and userland, accessing the shared data > in kernel has zero overhead though. Don't forget that there are arches out there w/ VIVT caches which will probably eliminate most of the performance benifits if we have the same page mapped writable in two different virtual addresses.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Sat Jan 12 23:31:48 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BD93FAC7 for ; Sat, 12 Jan 2013 23:31:48 +0000 (UTC) (envelope-from jmg@h2.funkthat.com) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) by mx1.freebsd.org (Postfix) with ESMTP id 88277DB4 for ; Sat, 12 Jan 2013 23:31:48 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id r0CNVl5M082083 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sat, 12 Jan 2013 15:31:47 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id r0CNVlsT082082 for freebsd-arch@FreeBSD.org; Sat, 12 Jan 2013 15:31:47 -0800 (PST) (envelope-from jmg) Date: Sat, 12 Jan 2013 15:31:47 -0800 From: John-Mark Gurney To: freebsd-arch@FreeBSD.org Subject: how long to keep support for gcc on x86? Message-ID: <20130112233147.GK1410@funkthat.com> Mail-Followup-To: freebsd-arch@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 12 Jan 2013 15:31:47 -0800 (PST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jan 2013 23:31:48 -0000 So, now that -current x86 is defaulting to clang, how much longer do we need to support gcc on platforms that default to clang? I'm asking because clang support AES-NI, but gcc does not... So, can I commit clang only x86 code to the kernel now? And not provide support for gcc? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."