From owner-freebsd-arch@FreeBSD.ORG Mon Jan 15 14:16:15 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6805616A407 for ; Mon, 15 Jan 2007 14:16:15 +0000 (UTC) (envelope-from dumbbell@freebsd.org) Received: from mail.ilius.net (mail.ilius.net [62.23.30.92]) by mx1.freebsd.org (Postfix) with ESMTP id CED0513C45D for ; Mon, 15 Jan 2007 14:16:14 +0000 (UTC) (envelope-from dumbbell@freebsd.org) Received: from [192.168.192.55] [62.23.30.90] by mail.ilius.net with ESMTP (SMTPD-9.10) id A3AE023C; Mon, 15 Jan 2007 14:37:50 +0100 Message-ID: <45AB83A9.1090107@freebsd.org> Date: Mon, 15 Jan 2007 14:37:45 +0100 From: =?UTF-8?B?SmVhbi1Tw6liYXN0aWVuIFDDqWRyb24=?= User-Agent: Thunderbird 1.5.0.9 (X11/20070108) MIME-Version: 1.0 To: freebsd-arch@freebsd.org X-Enigmail-Version: 0.94.1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Subject: psm(4): Synaptics touchpads initialisation X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2007 14:16:15 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, The Synaptics touchpad on my Asus V6 laptop never worked on FreeBSD when `hw.psm.synaptics_support' was set. The problem is that the touchpad returns to Relative Mode (to act as an ordinary mouse) after initialisation, but the driver expects Absolute Mode packets (to provide advanced features support). In the user point of view, the mouse pointer never move because all packets are discarded. The following patch fixes the problem with my laptop: http://people.freebsd.org/~dumbbell/synaptics/psm-synaptics-abs_mode-c.patch I had a few success reports, but I'm not very happy with it because there's Synaptics-specific code in the middle of a generic function. I'd like to have comments about it before committing anything. Regards, - -- Jean-Sébastien Pédron http://www.dumbbell.fr/ PGP Key: http://www.dumbbell.fr/pgp/pubkey.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFq4Ooa+xGJsFYOlMRAkRiAJ9Ymey7LvuWXCOGh8l05RfCBDg9bACeN+9q 11jZ0LbD/fEOrBFCBfMCySM= =2A1j -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 17:20:01 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0A54916A407 for ; Tue, 16 Jan 2007 17:20:00 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.191]) by mx1.freebsd.org (Postfix) with ESMTP id 7409313C468 for ; Tue, 16 Jan 2007 17:20:00 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so2450959nfc for ; Tue, 16 Jan 2007 09:19:58 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=MJqDbKN+1J/WogPf7CmW4GhVSTNysIIcc1+XzrrhnTaHCYZVp4ULcE3cbHTqJgDNOn+nXmi9dXL4WbTu2rtAW9KA/zYDlZx3wUhjDbiCvlvMFy8NWwfaOppMeoiP99pqKTZishA82AVQ5L+6+Eneaxh0zZ4jvWG193+laGJDuJY= Received: by 10.49.80.12 with SMTP id h12mr3001101nfl.1168966264243; Tue, 16 Jan 2007 08:51:04 -0800 (PST) Received: by 10.48.238.9 with HTTP; Tue, 16 Jan 2007 08:51:03 -0800 (PST) Message-ID: <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> Date: Tue, 16 Jan 2007 17:51:03 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "John Baldwin" In-Reply-To: <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607250814m1a476f09p2d962dedc0c99be1@mail.gmail.com> <200607251232.51230.jhb@freebsd.org> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> X-Google-Sender-Auth: b8864b2be4837ff1 Cc: Kip Macy , freebsd-current@freebsd.org, Suleiman Souhlal , Pawel Jakub Dawidek , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 17:20:01 -0000 2006/7/28, Attilio Rao : > > After some thinking, I think it's better using init/fini methods > (since they hide the sizeof(struct turnstile) with size parameter). > > Feedbacks and comments are welcome: > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff [CC'ed all the interested people] Even if a long time is passed I did some benchmarks based on ebizzy tool. This program claims to reproduce a real httpd server behaviour and is used into the Linux world for benchmarks, AFAIK. I think that results of the comparison on this patch is very interesting, and I think it worths a commit :) I think that results can be even better on a Xeon machine (I had no chance to reproduce this on some of these). (Results taken in consideration have been measured after some starts, in order to minimize caching differences). The patch: http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff The benchmark results: http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.benchmark The kernel options file: http://users.gufi.org/~rookie/works/patches/ts-sq/CURRENT For any information, comment, etc. please feel free to contact me. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 18:39:39 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 288A116A47E for ; Tue, 16 Jan 2007 18:39:39 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.189]) by mx1.freebsd.org (Postfix) with ESMTP id B866A13C4A7 for ; Tue, 16 Jan 2007 18:39:37 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so2474461nfc for ; Tue, 16 Jan 2007 10:39:35 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=UTl/lgoYGJ2QtrSspPNXs2Pu6MP12LIvOFcrKSpTvxH0QVu2e/wNYLuTKdn29pZXOskpwJmLNZt9vS77apI5Tv2dZJsIT+2/1wIrG7AAP7fDeusQk9Tz0EzhpCqTkCctUUWYI4dl31Z57JjH8GH+zQn64pRIB+DdOvAjpDpT9LY= Received: by 10.48.245.17 with SMTP id s17mr6333351nfh.1168972769649; Tue, 16 Jan 2007 10:39:29 -0800 (PST) Received: by 10.48.238.9 with HTTP; Tue, 16 Jan 2007 10:39:29 -0800 (PST) Message-ID: <3bbf2fe10701161039g4b1572a1qf8b97f2389b31a0e@mail.gmail.com> Date: Tue, 16 Jan 2007 19:39:29 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Julian Elischer" In-Reply-To: <45AD19B2.6010806@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607250814m1a476f09p2d962dedc0c99be1@mail.gmail.com> <200607251232.51230.jhb@freebsd.org> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <45AD19B2.6010806@elischer.org> X-Google-Sender-Auth: dcfd126d2e293cf6 Cc: Kip Macy , freebsd-current@freebsd.org, Pawel Jakub Dawidek , Suleiman Souhlal , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 18:39:39 -0000 2007/1/16, Julian Elischer : > Attilio Rao wrote: > > 2006/7/28, Attilio Rao : > >> > >> After some thinking, I think it's better using init/fini methods > >> (since they hide the sizeof(struct turnstile) with size parameter). > >> > >> Feedbacks and comments are welcome: > >> http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > > > [CC'ed all the interested people] > > > > Even if a long time is passed I did some benchmarks based on ebizzy tool. > > This program claims to reproduce a real httpd server behaviour and is > > used into the Linux world for benchmarks, AFAIK. > > I think that results of the comparison on this patch is very > > interesting, and I think it worths a commit :) > > I think that results can be even better on a Xeon machine (I had no > > chance to reproduce this on some of these). > > (Results taken in consideration have been measured after some starts, > > in order to minimize caching differences). > > > > The patch: > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > > > The benchmark results: > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.benchmark > > those are very big differences! > what does the benchmark actually measure? just time ./ebizzy (which create a lot of contention inside the kernel). Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 18:42:04 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F155516A47C for ; Tue, 16 Jan 2007 18:42:04 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outP.internet-mail-service.net (outP.internet-mail-service.net [216.240.47.239]) by mx1.freebsd.org (Postfix) with ESMTP id C003813C4C8 for ; Tue, 16 Jan 2007 18:42:04 +0000 (UTC) (envelope-from julian@elischer.org) Received: from shell.idiom.com (HELO idiom.com) (216.240.47.20) by out.internet-mail-service.net (qpsmtpd/0.32) with ESMTP; Tue, 16 Jan 2007 10:10:43 -0800 Received: from [10.251.23.190] (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id 69794125B2B; Tue, 16 Jan 2007 10:30:11 -0800 (PST) Message-ID: <45AD19B2.6010806@elischer.org> Date: Tue, 16 Jan 2007 10:30:10 -0800 From: Julian Elischer User-Agent: Thunderbird 1.5.0.9 (Macintosh/20061207) MIME-Version: 1.0 To: Attilio Rao References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607250814m1a476f09p2d962dedc0c99be1@mail.gmail.com> <200607251232.51230.jhb@freebsd.org> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> In-Reply-To: <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Pawel Jakub Dawidek , Kip Macy , Suleiman Souhlal , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 18:42:05 -0000 Attilio Rao wrote: > 2006/7/28, Attilio Rao : >> >> After some thinking, I think it's better using init/fini methods >> (since they hide the sizeof(struct turnstile) with size parameter). >> >> Feedbacks and comments are welcome: >> http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > [CC'ed all the interested people] > > Even if a long time is passed I did some benchmarks based on ebizzy tool. > This program claims to reproduce a real httpd server behaviour and is > used into the Linux world for benchmarks, AFAIK. > I think that results of the comparison on this patch is very > interesting, and I think it worths a commit :) > I think that results can be even better on a Xeon machine (I had no > chance to reproduce this on some of these). > (Results taken in consideration have been measured after some starts, > in order to minimize caching differences). > > The patch: > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > The benchmark results: > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.benchmark those are very big differences! what does the benchmark actually measure? > > The kernel options file: > http://users.gufi.org/~rookie/works/patches/ts-sq/CURRENT > > For any information, comment, etc. please feel free to contact me. > > Attilio > > From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 19:02:47 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EF6C616A4B3 for ; Tue, 16 Jan 2007 19:02:46 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.187]) by mx1.freebsd.org (Postfix) with ESMTP id 47ED013C44C for ; Tue, 16 Jan 2007 19:02:46 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so2481075nfc for ; Tue, 16 Jan 2007 11:02:45 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=YZE0tLgPRXi+hfCE3w1kfRdpT8GDWp3BAzGffR6YFMLIM34rfy7wI6OrtglSL0Kj/GiQVHkwoFuumYaaNBZjNeW1r0D9xNsLUCro2NCqmZfNADKCoyZBLqsosSDLwe29d0qhmMJD05HcZ4Ign0pFMmEwBEPRGRlAQKJ/RpiJzTo= Received: by 10.49.80.12 with SMTP id h12mr3173846nfl.1168974160335; Tue, 16 Jan 2007 11:02:40 -0800 (PST) Received: by 10.48.238.9 with HTTP; Tue, 16 Jan 2007 11:02:40 -0800 (PST) Message-ID: <3bbf2fe10701161102w5a094dfge7faf641ab1a0425@mail.gmail.com> Date: Tue, 16 Jan 2007 20:02:40 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Nick Evans" In-Reply-To: <20070116131836.0681a51d@pleiades.nextvenue.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607250814m1a476f09p2d962dedc0c99be1@mail.gmail.com> <200607251232.51230.jhb@freebsd.org> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116122925.5cdb1ded@pleiades.nextvenue.com> <3bbf2fe10701160951x1cb4cd75pd843b453a8f4a96@mail.gmail.com> <20070116131836.0681a51d@pleiades.nextvenue.com> X-Google-Sender-Auth: 2259c4ab2e877946 Cc: Kip Macy , freebsd-current@freebsd.org, Suleiman Souhlal , Pawel Jakub Dawidek , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 19:02:47 -0000 2007/1/16, Nick Evans : > On Tue, 16 Jan 2007 18:51:30 +0100 > "Attilio Rao" wrote: > > > 2007/1/16, Nick Evans : > > > On Tue, 16 Jan 2007 17:51:03 +0100 > > > "Attilio Rao" wrote: > > > > > > > 2006/7/28, Attilio Rao : > > > > > > > > > > After some thinking, I think it's better using init/fini methods > > > > > (since they hide the sizeof(struct turnstile) with size parameter). > > > > > > > > > > Feedbacks and comments are welcome: > > > > > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > > > > > > > [CC'ed all the interested people] > > > > > > > > Even if a long time is passed I did some benchmarks based on ebizzy > > > > tool. This program claims to reproduce a real httpd server behaviour > > > > and is used into the Linux world for benchmarks, AFAIK. > > > > I think that results of the comparison on this patch is very > > > > interesting, and I think it worths a commit :) > > > > I think that results can be even better on a Xeon machine (I had no > > > > chance to reproduce this on some of these). > > > > (Results taken in consideration have been measured after some starts, > > > > in order to minimize caching differences). > > > > > > > > The patch: > > > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > > > > > > > The benchmark results: > > > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.benchmark > > > > > > > > The kernel options file: > > > > http://users.gufi.org/~rookie/works/patches/ts-sq/CURRENT > > > > > > > > For any information, comment, etc. please feel free to contact me. > > > > > > > > Attilio > > > > > > > > > > > > -- > > > > Peace can only be achieved by understanding - A. Einstein > > > > _______________________________________________ > > > > freebsd-current@freebsd.org mailing list > > > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > > > To unsubscribe, send any mail to > > > > "freebsd-current-unsubscribe@freebsd.org" > > > > > > Attilio, > > > > > > What class of Xeon do you need this tested on, P3 or the newer P4+ stuff? > > > I have a few systems here I can test this on including a quad P3-Xeon box > > > that I've been testing Jeff's ULE 2.0 on. Do you have specific extra test > > > points or are the before and after for the (non)preemption cases > > > sufficient? I can also give you access to the systems if that is easier. > > > > Hi Nick, > > thanks a lot for responsivness and help. > > I think that P3 Xeon alredy should show some speedup, in particular I > > would see some tests on the P3-Xeon quad. > > It would be enough reproduce the test I did (using the same options > > file and starting ebizzy for 1-2 times before the results gathering). > > I hope you are suitable for doing this benchmarks alone (FreeBSD + > > university + job don't leave me too much time ATM :(( ). > > > > Thanks a lot for your efforts, > > Attilio > > > > > > -- > > Peace can only be achieved by understanding - A. Einstein > > > Yea, we should be able to handle benchmarking this if you can help me > through the first hurdle. Do you have a version that compiles cleanly on > -CURRENT? The version I dug up off lkml via google complains: > > root@current[13:10]# make > gcc -Wall -lpthread -o ebizzy ebizzy.c > In file included from ebizzy.c:43: > /usr/include/malloc.h:3:2: #error " has been replaced by " > ebizzy.c: In function `read_options': > ebizzy.c:212: warning: implicit declaration of function `mallopt' > ebizzy.c:212: error: `M_MMAP_MAX' undeclared (first use in this function) > ebizzy.c:212: error: (Each undeclared identifier is reported only once > ebizzy.c:212: error: for each function it appears in.) > ebizzy.c: In function `alloc_mem': > ebizzy.c:224: error: `MAP_ANONYMOUS' undeclared (first use in this function) > *** Error code 1 > > Stop in /root/ebizzy. > > > I didn't see an updated version listed in our archives anywhere. Since ebizzy doesn't compile natively on FreeBSD try this patch (it disables the -m option, but it doesn't matter currently): http://users.gufi.org/~rookie/works/patches/ts-sq/ebizzy.diff Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 20:27:20 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A668916A40F; Tue, 16 Jan 2007 20:27:20 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.freebsd.org (Postfix) with ESMTP id 1080613C519; Tue, 16 Jan 2007 20:27:17 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id l0GKR5Hp046593; Tue, 16 Jan 2007 15:27:06 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: "Attilio Rao" Date: Tue, 16 Jan 2007 14:38:51 -0500 User-Agent: KMail/1.9.1 References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> In-Reply-To: <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200701161438.52481.jhb@freebsd.org> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 16 Jan 2007 15:27:06 -0500 (EST) X-Virus-Scanned: ClamAV 0.88.3/2457/Tue Jan 16 06:53:04 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Kip Macy , freebsd-current@freebsd.org, Suleiman Souhlal , Pawel Jakub Dawidek , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 20:27:20 -0000 On Tuesday 16 January 2007 11:51, Attilio Rao wrote: > 2006/7/28, Attilio Rao : > > > > After some thinking, I think it's better using init/fini methods > > (since they hide the sizeof(struct turnstile) with size parameter). > > > > Feedbacks and comments are welcome: > > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > [CC'ed all the interested people] > > Even if a long time is passed I did some benchmarks based on ebizzy tool. > This program claims to reproduce a real httpd server behaviour and is > used into the Linux world for benchmarks, AFAIK. > I think that results of the comparison on this patch is very > interesting, and I think it worths a commit :) > I think that results can be even better on a Xeon machine (I had no > chance to reproduce this on some of these). > (Results taken in consideration have been measured after some starts, > in order to minimize caching differences). > > The patch: > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff Looks good. Some minor nits are that in subr_turnstile.c in the comment I would say "a turnstile is allocated" rather than "a turnstile is got from a specific UMA zone" as it reads a little bit clearer. Also, I would say "Allocate a" rather than "Get a" for the two _alloc() functions. Also, why not just use UMA_ALIGN_CACHE and make UMA_ALIGN_CACHE (128 - 1) on i386 and amd64 rather than adding a new UMA_ALIGN_SYNC? -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 20:34:29 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 92EAC16A47C; Tue, 16 Jan 2007 20:34:29 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from blake.polstra.com (blake.polstra.com [64.81.189.66]) by mx1.freebsd.org (Postfix) with ESMTP id 3240013C428; Tue, 16 Jan 2007 20:34:29 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [64.81.189.67]) by blake.polstra.com (8.13.8/8.13.8) with ESMTP id l0GKOB4r077462; Tue, 16 Jan 2007 12:24:11 -0800 (PST) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.5 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> Date: Tue, 16 Jan 2007 12:24:11 -0800 (PST) From: John Polstra To: Attilio Rao X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (blake.polstra.com [64.81.189.66]); Tue, 16 Jan 2007 12:24:14 -0800 (PST) Cc: Pawel Jakub Dawidek , Kip Macy , Suleiman Souhlal , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 20:34:29 -0000 On 16-Jan-2007 Attilio Rao wrote: > The patch: > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > The benchmark results: > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.benchmark > > The kernel options file: > http://users.gufi.org/~rookie/works/patches/ts-sq/CURRENT This is good stuff! I tried your patch on a performance-critical system that I've been working on. Without going into a lot of detail, it's a bunch of in-kernel code that blasts packets back and forth between pairs of gigabit interfaces. Userland isn't involved at all. Running 4 gigabit ports in this way on a Dell 1950 with 4 CPU cores running at 3.0 GHz, I got about 4% better performance (in terms of packets per second) using your patch. That's a pretty good improvement, considering that the design makes some effort to avoid lock contention. John From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 20:36:21 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4860A16A49E for ; Tue, 16 Jan 2007 20:36:21 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.245]) by mx1.freebsd.org (Postfix) with ESMTP id 0696113C461 for ; Tue, 16 Jan 2007 20:36:20 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by an-out-0708.google.com with SMTP id c24so1124344ana for ; Tue, 16 Jan 2007 12:36:20 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=e4X8wT0Si9S6zAUEmRhUhH4qfZ1Yrrnfh2/pl8CiZEo1EGlRNGR6Ztl2vrfNN9BzjNA4/KP1e17TRckRRZKtyYRbmUOoHGR0UopQ33RGYD5ii/x8vwMob4UAWnrV5uHzPTacdYyE7q6OcHG4D4K1yZx3qQce59Q0Yo3oDA8Llvg= Received: by 10.49.19.5 with SMTP id w5mr6498912nfi.1168979776259; Tue, 16 Jan 2007 12:36:16 -0800 (PST) Received: by 10.48.238.9 with HTTP; Tue, 16 Jan 2007 12:36:16 -0800 (PST) Message-ID: <3bbf2fe10701161236s48e6cc16p99c8c38c1d7becde@mail.gmail.com> Date: Tue, 16 Jan 2007 21:36:16 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "John Baldwin" In-Reply-To: <200701161438.52481.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <200701161438.52481.jhb@freebsd.org> X-Google-Sender-Auth: 1ee207f6857c0685 Cc: Kip Macy , freebsd-current@freebsd.org, Suleiman Souhlal , Pawel Jakub Dawidek , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 20:36:21 -0000 2007/1/16, John Baldwin : > On Tuesday 16 January 2007 11:51, Attilio Rao wrote: > > 2006/7/28, Attilio Rao : > > > > > > After some thinking, I think it's better using init/fini methods > > > (since they hide the sizeof(struct turnstile) with size parameter). > > > > > > Feedbacks and comments are welcome: > > > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > > > [CC'ed all the interested people] > > > > Even if a long time is passed I did some benchmarks based on ebizzy tool. > > This program claims to reproduce a real httpd server behaviour and is > > used into the Linux world for benchmarks, AFAIK. > > I think that results of the comparison on this patch is very > > interesting, and I think it worths a commit :) > > I think that results can be even better on a Xeon machine (I had no > > chance to reproduce this on some of these). > > (Results taken in consideration have been measured after some starts, > > in order to minimize caching differences). > > > > The patch: > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > Looks good. Some minor nits are that in subr_turnstile.c in the comment I > would say "a turnstile is allocated" rather than "a turnstile is got from a > specific UMA zone" as it reads a little bit clearer. Also, I would > say "Allocate a" rather than "Get a" for the two _alloc() functions. Also, > why not just use UMA_ALIGN_CACHE and make UMA_ALIGN_CACHE (128 - 1) on i386 > and amd64 rather than adding a new UMA_ALIGN_SYNC? I was thinking that in this way anyone who wants to replace the syncronizing primitive boundary to an appropriate value can do it. I just used UMA_ALIGN_CACHE as default value beacause I don't know the better boundary (for syncronizing primitives) for other arches. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 20:43:15 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7210816A40F; Tue, 16 Jan 2007 20:43:15 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from blake.polstra.com (blake.polstra.com [64.81.189.66]) by mx1.freebsd.org (Postfix) with ESMTP id 111EA13C4A7; Tue, 16 Jan 2007 20:43:14 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [64.81.189.67]) by blake.polstra.com (8.13.8/8.13.8) with ESMTP id l0GKhEZd077880; Tue, 16 Jan 2007 12:43:14 -0800 (PST) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.5 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200701161438.52481.jhb@freebsd.org> Date: Tue, 16 Jan 2007 12:43:14 -0800 (PST) From: John Polstra To: John Baldwin X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (blake.polstra.com [64.81.189.66]); Tue, 16 Jan 2007 12:43:14 -0800 (PST) Cc: Pawel Jakub Dawidek , Kip Macy , Suleiman Souhlal , Attilio Rao , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 20:43:15 -0000 On 16-Jan-2007 John Baldwin wrote: > On Tuesday 16 January 2007 11:51, Attilio Rao wrote: >> The patch: >> http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > Looks good. Some minor nits are that in subr_turnstile.c in the comment I > would say "a turnstile is allocated" rather than "a turnstile is got from a > specific UMA zone" as it reads a little bit clearer. Also, I would > say "Allocate a" rather than "Get a" for the two _alloc() functions. Also, > why not just use UMA_ALIGN_CACHE and make UMA_ALIGN_CACHE (128 - 1) on i386 > and amd64 rather than adding a new UMA_ALIGN_SYNC? Also, instead of calling bzero in the _init functions, I think you could pass UMA_ZONE_ZINIT to uma_zcreate. John From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 21:00:28 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AA63916A4E5; Tue, 16 Jan 2007 21:00:28 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from relay.talkpoint.com (pobox.talkpoint.com [204.141.15.158]) by mx1.freebsd.org (Postfix) with ESMTP id 79FD313C43E; Tue, 16 Jan 2007 21:00:05 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from ASSP-nospam ([127.0.0.1]) by relay.talkpoint.com with Microsoft SMTPSVC(5.0.2195.6713); Tue, 16 Jan 2007 15:42:58 -0500 Received: from 204.141.15.136 ([204.141.15.136] helo=postal.talkpoint.com) by ASSP-nospam ; 16 Jan 07 20:42:58 -0000 Received: from pleiades.nextvenue.com ([204.141.15.194]) by postal.talkpoint.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id ZN3V7H6Q; Tue, 16 Jan 2007 15:42:58 -0500 Date: Tue, 16 Jan 2007 15:42:58 -0500 From: Nick Evans To: "Attilio Rao" Message-ID: <20070116154258.568e1aaf@pleiades.nextvenue.com> In-Reply-To: <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607250814m1a476f09p2d962dedc0c99be1@mail.gmail.com> <200607251232.51230.jhb@freebsd.org> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> X-Mailer: Sylpheed-Claws 2.4.0 (GTK+ 2.8.20; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 16 Jan 2007 20:42:58.0756 (UTC) FILETIME=[E8495040:01C739AE] Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 21:00:28 -0000 On Tue, 16 Jan 2007 17:51:03 +0100 "Attilio Rao" wrote: > 2006/7/28, Attilio Rao : > > > > After some thinking, I think it's better using init/fini methods > > (since they hide the sizeof(struct turnstile) with size parameter). > > > > Feedbacks and comments are welcome: > > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > [CC'ed all the interested people] > > Even if a long time is passed I did some benchmarks based on ebizzy tool. > This program claims to reproduce a real httpd server behaviour and is > used into the Linux world for benchmarks, AFAIK. > I think that results of the comparison on this patch is very > interesting, and I think it worths a commit :) > I think that results can be even better on a Xeon machine (I had no > chance to reproduce this on some of these). > (Results taken in consideration have been measured after some starts, > in order to minimize caching differences). > > The patch: > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > The benchmark results: > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.benchmark > > The kernel options file: > http://users.gufi.org/~rookie/works/patches/ts-sq/CURRENT > > For any information, comment, etc. please feel free to contact me. > > Attilio > > > -- > Peace can only be achieved by understanding - A. Einstein > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" Some preliminary results: PREEMPTION: 4BSD, Quad P3-Xeon, 2GB ram pre-patch 1. 176.36 real 703.75 user 0.01 sys 2. 176.73 real 704.34 user 0.03 sys 3. 176.49 real 703.72 user 0.04 sys 4. 175.81 real 701.36 user 0.03 sys 5. 176.57 real 700.98 user 0.02 sys post-patch 1. 179.17 real 714.39 user 0.01 sys 2. 178.33 real 711.50 user 0.04 sys 3. 178.32 real 711.04 user 0.03 sys 4. 177.34 real 707.51 user 0.03 sys 5. 178.25 real 710.17 user 0.03 sys From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 21:00:59 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4573D16A66F for ; Tue, 16 Jan 2007 21:00:59 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.189]) by mx1.freebsd.org (Postfix) with ESMTP id B382A13C4BA for ; Tue, 16 Jan 2007 21:00:56 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so2516009nfc for ; Tue, 16 Jan 2007 13:00:55 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=Q12o8UA/Wy2JUq7tjb8PCVX9cKBUmeqIUnn6w8TePJo3cO79yjdceWit2T3/M4yQd40PLgKybfnEpYu0Tebz+NCxmiieBdZZdkDqDqC6L/RafyCkVhJJhFujmHTL9T1L2sFmrPUiw7K3ie/Um0jxgr5dv8SzeUD2OlEKEmlyEGE= Received: by 10.49.27.17 with SMTP id e17mr5583403nfj.1168981254082; Tue, 16 Jan 2007 13:00:54 -0800 (PST) Received: by 10.48.238.9 with HTTP; Tue, 16 Jan 2007 13:00:54 -0800 (PST) Message-ID: <3bbf2fe10701161300jc53b707h408fc0848767511f@mail.gmail.com> Date: Tue, 16 Jan 2007 22:00:54 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "John Polstra" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200701161438.52481.jhb@freebsd.org> X-Google-Sender-Auth: 89b7436090da4470 Cc: Pawel Jakub Dawidek , Kip Macy , Suleiman Souhlal , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 21:00:59 -0000 2007/1/16, John Polstra : > On 16-Jan-2007 John Baldwin wrote: > > On Tuesday 16 January 2007 11:51, Attilio Rao wrote: > >> The patch: > >> http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > > > Looks good. Some minor nits are that in subr_turnstile.c in the comment I > > would say "a turnstile is allocated" rather than "a turnstile is got from a > > specific UMA zone" as it reads a little bit clearer. Also, I would > > say "Allocate a" rather than "Get a" for the two _alloc() functions. Also, > > why not just use UMA_ALIGN_CACHE and make UMA_ALIGN_CACHE (128 - 1) on i386 > > and amd64 rather than adding a new UMA_ALIGN_SYNC? > > Also, instead of calling bzero in the _init functions, I think you > could pass UMA_ZONE_ZINIT to uma_zcreate. Since it doesn't seem to be documented, it automatically zeros all the initializations or just the first one? Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 21:10:23 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EF31716A58C; Tue, 16 Jan 2007 21:10:22 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.freebsd.org (Postfix) with ESMTP id 6E46013C4A6; Tue, 16 Jan 2007 21:10:16 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id l0GLA1MB047029; Tue, 16 Jan 2007 16:10:14 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: "Attilio Rao" Date: Tue, 16 Jan 2007 16:05:21 -0500 User-Agent: KMail/1.9.1 References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <200701161438.52481.jhb@freebsd.org> <3bbf2fe10701161236s48e6cc16p99c8c38c1d7becde@mail.gmail.com> In-Reply-To: <3bbf2fe10701161236s48e6cc16p99c8c38c1d7becde@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200701161605.22394.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 16 Jan 2007 16:10:14 -0500 (EST) X-Virus-Scanned: ClamAV 0.88.3/2457/Tue Jan 16 06:53:04 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Kip Macy , freebsd-current@freebsd.org, Suleiman Souhlal , Pawel Jakub Dawidek , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 21:10:23 -0000 On Tuesday 16 January 2007 15:36, Attilio Rao wrote: > 2007/1/16, John Baldwin : > > On Tuesday 16 January 2007 11:51, Attilio Rao wrote: > > > 2006/7/28, Attilio Rao : > > > > > > > > After some thinking, I think it's better using init/fini methods > > > > (since they hide the sizeof(struct turnstile) with size parameter). > > > > > > > > Feedbacks and comments are welcome: > > > > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > > > > > [CC'ed all the interested people] > > > > > > Even if a long time is passed I did some benchmarks based on ebizzy tool. > > > This program claims to reproduce a real httpd server behaviour and is > > > used into the Linux world for benchmarks, AFAIK. > > > I think that results of the comparison on this patch is very > > > interesting, and I think it worths a commit :) > > > I think that results can be even better on a Xeon machine (I had no > > > chance to reproduce this on some of these). > > > (Results taken in consideration have been measured after some starts, > > > in order to minimize caching differences). > > > > > > The patch: > > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > > > Looks good. Some minor nits are that in subr_turnstile.c in the comment I > > would say "a turnstile is allocated" rather than "a turnstile is got from a > > specific UMA zone" as it reads a little bit clearer. Also, I would > > say "Allocate a" rather than "Get a" for the two _alloc() functions. Also, > > why not just use UMA_ALIGN_CACHE and make UMA_ALIGN_CACHE (128 - 1) on i386 > > and amd64 rather than adding a new UMA_ALIGN_SYNC? > > I was thinking that in this way anyone who wants to replace the > syncronizing primitive boundary to an appropriate value can do it. > I just used UMA_ALIGN_CACHE as default value beacause I don't know the > better boundary (for syncronizing primitives) for other arches. Is there a good reason to not cache-align synch primitives? That is, why would an arch not use cache-align? Also, is there a reason to not update UMA_ALIGN_CACHE on x86? -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 21:10:23 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 144DE16A596; Tue, 16 Jan 2007 21:10:23 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.freebsd.org (Postfix) with ESMTP id 3030813C4BB; Tue, 16 Jan 2007 21:10:18 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id l0GLA1MC047029; Tue, 16 Jan 2007 16:10:16 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 16 Jan 2007 16:07:43 -0500 User-Agent: KMail/1.9.1 References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> In-Reply-To: <20070116154258.568e1aaf@pleiades.nextvenue.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200701161607.43725.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 16 Jan 2007 16:10:16 -0500 (EST) X-Virus-Scanned: ClamAV 0.88.3/2457/Tue Jan 16 06:53:04 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Nick Evans , Attilio Rao Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 21:10:23 -0000 On Tuesday 16 January 2007 15:42, Nick Evans wrote: > On Tue, 16 Jan 2007 17:51:03 +0100 > "Attilio Rao" wrote: > > > 2006/7/28, Attilio Rao : > > > > > > After some thinking, I think it's better using init/fini methods > > > (since they hide the sizeof(struct turnstile) with size parameter). > > > > > > Feedbacks and comments are welcome: > > > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > > > [CC'ed all the interested people] > > > > Even if a long time is passed I did some benchmarks based on ebizzy tool. > > This program claims to reproduce a real httpd server behaviour and is > > used into the Linux world for benchmarks, AFAIK. > > I think that results of the comparison on this patch is very > > interesting, and I think it worths a commit :) > > I think that results can be even better on a Xeon machine (I had no > > chance to reproduce this on some of these). > > (Results taken in consideration have been measured after some starts, > > in order to minimize caching differences). > > > > The patch: > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > > > The benchmark results: > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.benchmark > > > > The kernel options file: > > http://users.gufi.org/~rookie/works/patches/ts-sq/CURRENT > > > > For any information, comment, etc. please feel free to contact me. > > > > Attilio > > > > > > -- > > Peace can only be achieved by understanding - A. Einstein > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > Some preliminary results: > > PREEMPTION: 4BSD, Quad P3-Xeon, 2GB ram > > pre-patch > > 1. 176.36 real 703.75 user 0.01 sys > 2. 176.73 real 704.34 user 0.03 sys > 3. 176.49 real 703.72 user 0.04 sys > 4. 175.81 real 701.36 user 0.03 sys > 5. 176.57 real 700.98 user 0.02 sys > > post-patch > > 1. 179.17 real 714.39 user 0.01 sys > 2. 178.33 real 711.50 user 0.04 sys > 3. 178.32 real 711.04 user 0.03 sys > 4. 177.34 real 707.51 user 0.03 sys > 5. 178.25 real 710.17 user 0.03 sys What did you use to do your benchmark? Also, have you tried adjusting UMA_ALIGN_SYNC (maybe use 64 - 1)? -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 21:24:05 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2481B16A415; Tue, 16 Jan 2007 21:24:05 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from relay.talkpoint.com (relay.talkpoint.com [204.141.15.158]) by mx1.freebsd.org (Postfix) with ESMTP id DB31713C461; Tue, 16 Jan 2007 21:24:04 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from ASSP-nospam ([127.0.0.1]) by relay.talkpoint.com with Microsoft SMTPSVC(5.0.2195.6713); Tue, 16 Jan 2007 16:24:03 -0500 Received: from 204.141.15.136 ([204.141.15.136] helo=postal.talkpoint.com) by ASSP-nospam ; 16 Jan 07 21:24:03 -0000 Received: from pleiades.nextvenue.com ([204.141.15.194]) by postal.talkpoint.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id ZN3V72DX; Tue, 16 Jan 2007 16:24:03 -0500 Date: Tue, 16 Jan 2007 16:24:03 -0500 From: Nick Evans To: John Baldwin Message-ID: <20070116162403.70e2ec8a@pleiades.nextvenue.com> In-Reply-To: <200701161607.43725.jhb@freebsd.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <200701161607.43725.jhb@freebsd.org> X-Mailer: Sylpheed-Claws 2.4.0 (GTK+ 2.8.20; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 16 Jan 2007 21:24:04.0023 (UTC) FILETIME=[A5B33470:01C739B4] Cc: Attilio Rao , McKnight , Justin, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 21:24:05 -0000 On Tue, 16 Jan 2007 16:07:43 -0500 John Baldwin wrote: > On Tuesday 16 January 2007 15:42, Nick Evans wrote: > > On Tue, 16 Jan 2007 17:51:03 +0100 > > "Attilio Rao" wrote: > > > > > 2006/7/28, Attilio Rao : > > > > > > > > After some thinking, I think it's better using init/fini methods > > > > (since they hide the sizeof(struct turnstile) with size parameter). > > > > > > > > Feedbacks and comments are welcome: > > > > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > > > > > [CC'ed all the interested people] > > > > > > Even if a long time is passed I did some benchmarks based on ebizzy > > > tool. This program claims to reproduce a real httpd server behaviour > > > and is used into the Linux world for benchmarks, AFAIK. > > > I think that results of the comparison on this patch is very > > > interesting, and I think it worths a commit :) > > > I think that results can be even better on a Xeon machine (I had no > > > chance to reproduce this on some of these). > > > (Results taken in consideration have been measured after some starts, > > > in order to minimize caching differences). > > > > > > The patch: > > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > > > > > The benchmark results: > > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.benchmark > > > > > > The kernel options file: > > > http://users.gufi.org/~rookie/works/patches/ts-sq/CURRENT > > > > > > For any information, comment, etc. please feel free to contact me. > > > > > > Attilio > > > > > > > > > -- > > > Peace can only be achieved by understanding - A. Einstein > > > _______________________________________________ > > > freebsd-current@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to > > > "freebsd-current-unsubscribe@freebsd.org" > > > > Some preliminary results: > > > > PREEMPTION: 4BSD, Quad P3-Xeon, 2GB ram > > > > pre-patch > > > > 1. 176.36 real 703.75 user 0.01 sys > > 2. 176.73 real 704.34 user 0.03 sys > > 3. 176.49 real 703.72 user 0.04 sys > > 4. 175.81 real 701.36 user 0.03 sys > > 5. 176.57 real 700.98 user 0.02 sys > > > > post-patch > > > > 1. 179.17 real 714.39 user 0.01 sys > > 2. 178.33 real 711.50 user 0.04 sys > > 3. 178.32 real 711.04 user 0.03 sys > > 4. 177.34 real 707.51 user 0.03 sys > > 5. 178.25 real 710.17 user 0.03 sys > > What did you use to do your benchmark? Also, have you tried adjusting > UMA_ALIGN_SYNC (maybe use 64 - 1)? > > -- > John Baldwin Tested with ebizzy, default runtime options, no WITNESS or INVARIANTS. I haven't tried variations on UMA_ALIGN_SYNC, that's next on the list. I also have a single core P4-Xeon and dual core Presler 915 that I'm going to test this with as soon as they're done building world with the latest -CURRENT. When ULE 2.0 shakes out a bit more I plan on testing with that too. I'll post results on my other hardware sometime tonight. Nick From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 21:41:12 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CFECA16A529; Tue, 16 Jan 2007 21:41:12 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id E747713C442; Tue, 16 Jan 2007 21:41:11 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id l0GLJ8j9067886; Tue, 16 Jan 2007 14:19:13 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <45AD4148.90002@samsco.org> Date: Tue, 16 Jan 2007 14:19:04 -0700 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.2pre) Gecko/20070111 SeaMonkey/1.1 MIME-Version: 1.0 To: John Baldwin References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <200701161438.52481.jhb@freebsd.org> <3bbf2fe10701161236s48e6cc16p99c8c38c1d7becde@mail.gmail.com> <200701161605.22394.jhb@freebsd.org> In-Reply-To: <200701161605.22394.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Tue, 16 Jan 2007 14:19:13 -0700 (MST) X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: Pawel Jakub Dawidek , Kip Macy , Suleiman Souhlal , Attilio Rao , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 21:41:12 -0000 John Baldwin wrote: > On Tuesday 16 January 2007 15:36, Attilio Rao wrote: >> 2007/1/16, John Baldwin : >>> On Tuesday 16 January 2007 11:51, Attilio Rao wrote: >>>> 2006/7/28, Attilio Rao : >>>>> After some thinking, I think it's better using init/fini methods >>>>> (since they hide the sizeof(struct turnstile) with size parameter). >>>>> >>>>> Feedbacks and comments are welcome: >>>>> http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff >>>> [CC'ed all the interested people] >>>> >>>> Even if a long time is passed I did some benchmarks based on ebizzy > tool. >>>> This program claims to reproduce a real httpd server behaviour and is >>>> used into the Linux world for benchmarks, AFAIK. >>>> I think that results of the comparison on this patch is very >>>> interesting, and I think it worths a commit :) >>>> I think that results can be even better on a Xeon machine (I had no >>>> chance to reproduce this on some of these). >>>> (Results taken in consideration have been measured after some starts, >>>> in order to minimize caching differences). >>>> >>>> The patch: >>>> http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff >>> Looks good. Some minor nits are that in subr_turnstile.c in the comment I >>> would say "a turnstile is allocated" rather than "a turnstile is got from > a >>> specific UMA zone" as it reads a little bit clearer. Also, I would >>> say "Allocate a" rather than "Get a" for the two _alloc() functions. > Also, >>> why not just use UMA_ALIGN_CACHE and make UMA_ALIGN_CACHE (128 - 1) on > i386 >>> and amd64 rather than adding a new UMA_ALIGN_SYNC? >> I was thinking that in this way anyone who wants to replace the >> syncronizing primitive boundary to an appropriate value can do it. >> I just used UMA_ALIGN_CACHE as default value beacause I don't know the >> better boundary (for syncronizing primitives) for other arches. > > Is there a good reason to not cache-align synch primitives? That is, why > would an arch not use cache-align? Also, is there a reason to not update > UMA_ALIGN_CACHE on x86? > If you always cache-line-align them, that also addresses the Intel recommendation to always keep them from sharing cache lines. Scott From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 21:41:14 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D33F216A47C; Tue, 16 Jan 2007 21:41:14 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from blake.polstra.com (blake.polstra.com [64.81.189.66]) by mx1.freebsd.org (Postfix) with ESMTP id 78EF713C4F9; Tue, 16 Jan 2007 21:41:14 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [64.81.189.67]) by blake.polstra.com (8.13.8/8.13.8) with ESMTP id l0GLfCa6078806; Tue, 16 Jan 2007 13:41:12 -0800 (PST) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.5 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <3bbf2fe10701161300jc53b707h408fc0848767511f@mail.gmail.com> Date: Tue, 16 Jan 2007 13:41:12 -0800 (PST) From: John Polstra To: Attilio Rao X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (blake.polstra.com [64.81.189.66]); Tue, 16 Jan 2007 13:41:12 -0800 (PST) Cc: Kip Macy , freebsd-current@freebsd.org, Suleiman Souhlal , Pawel Jakub Dawidek , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 21:41:15 -0000 On 16-Jan-2007 Attilio Rao wrote: > 2007/1/16, John Polstra : >> Also, instead of calling bzero in the _init functions, I think you >> could pass UMA_ZONE_ZINIT to uma_zcreate. > > Since it doesn't seem to be documented, it automatically zeros all the > initializations or just the first one? It looks like it acts upon whole chunks of memory, just like the user _init function. But after unsuccessfully trying to wind my way through the UMA code for a few minutes, it's not totally clear to me whether or not UMA_ZONE_ZINIT overrides the user _init function entirely. Maybe somebody who has used it will clear up the confusion. John From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 22:21:47 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 479AB16A416 for ; Tue, 16 Jan 2007 22:21:47 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.175]) by mx1.freebsd.org (Postfix) with ESMTP id 22C2813C4C3 for ; Tue, 16 Jan 2007 22:21:43 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by ug-out-1314.google.com with SMTP id o2so1663363uge for ; Tue, 16 Jan 2007 14:21:42 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=MkyNchNm1YRdR3U+I/6oZfrJv9PVpaOmlnklZxupmy4zhDqsAfTjSuVDVaa8UzoP/2IRMGbGtGAd9kiaqnazOXE22qMp5N/vj4CGyX7F+LDOxiRAaD7AcYzZXA8Nwv/EKShW2DCt2PS6uynJEGJ1vPcbCLSZjt85jz22QVpMN38= Received: by 10.82.139.17 with SMTP id m17mr1365704bud.1168984536548; Tue, 16 Jan 2007 13:55:36 -0800 (PST) Received: by 10.82.191.16 with HTTP; Tue, 16 Jan 2007 13:55:36 -0800 (PST) Message-ID: Date: Tue, 16 Jan 2007 13:55:36 -0800 From: "Kip Macy" To: "Nick Evans" In-Reply-To: <20070116154258.568e1aaf@pleiades.nextvenue.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607250814m1a476f09p2d962dedc0c99be1@mail.gmail.com> <200607251232.51230.jhb@freebsd.org> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> Cc: Attilio Rao , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 22:21:47 -0000 x86 pre-P4 had 32-byte cache lines. Thus older processors will not benefit. -Kip On 1/16/07, Nick Evans wrote: > On Tue, 16 Jan 2007 17:51:03 +0100 > "Attilio Rao" wrote: > > > 2006/7/28, Attilio Rao : > > > > > > After some thinking, I think it's better using init/fini methods > > > (since they hide the sizeof(struct turnstile) with size parameter). > > > > > > Feedbacks and comments are welcome: > > > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > > > [CC'ed all the interested people] > > > > Even if a long time is passed I did some benchmarks based on ebizzy tool. > > This program claims to reproduce a real httpd server behaviour and is > > used into the Linux world for benchmarks, AFAIK. > > I think that results of the comparison on this patch is very > > interesting, and I think it worths a commit :) > > I think that results can be even better on a Xeon machine (I had no > > chance to reproduce this on some of these). > > (Results taken in consideration have been measured after some starts, > > in order to minimize caching differences). > > > > The patch: > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > > > The benchmark results: > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.benchmark > > > > The kernel options file: > > http://users.gufi.org/~rookie/works/patches/ts-sq/CURRENT > > > > For any information, comment, etc. please feel free to contact me. > > > > Attilio > > > > > > -- > > Peace can only be achieved by understanding - A. Einstein > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > Some preliminary results: > > PREEMPTION: 4BSD, Quad P3-Xeon, 2GB ram > > pre-patch > > 1. 176.36 real 703.75 user 0.01 sys > 2. 176.73 real 704.34 user 0.03 sys > 3. 176.49 real 703.72 user 0.04 sys > 4. 175.81 real 701.36 user 0.03 sys > 5. 176.57 real 700.98 user 0.02 sys > > post-patch > > 1. 179.17 real 714.39 user 0.01 sys > 2. 178.33 real 711.50 user 0.04 sys > 3. 178.32 real 711.04 user 0.03 sys > 4. 177.34 real 707.51 user 0.03 sys > 5. 178.25 real 710.17 user 0.03 sys > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 22:25:08 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A855516A559 for ; Tue, 16 Jan 2007 22:25:08 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.171]) by mx1.freebsd.org (Postfix) with ESMTP id 2F64613C455 for ; Tue, 16 Jan 2007 22:25:08 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by ug-out-1314.google.com with SMTP id o2so1664035uge for ; Tue, 16 Jan 2007 14:25:07 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=qPQWnL4/4RRCvudoZHOUzW6JnkRNyO7wdLL98jxX+P92M1Ei+7Ys8VjWWoMqk1T+YJannuYUjOajeW7ZWZsKTFkEYmUmLj0Ruf5IkblP3h4fCQ0CeKFBVJfrEZWSczrenbXCIZYsaiqN/+4qxNpf9aBKc7HirnGoVcxO/OFLNP8= Received: by 10.82.107.15 with SMTP id f15mr1381445buc.1168986306133; Tue, 16 Jan 2007 14:25:06 -0800 (PST) Received: by 10.82.191.16 with HTTP; Tue, 16 Jan 2007 14:25:06 -0800 (PST) Message-ID: Date: Tue, 16 Jan 2007 14:25:06 -0800 From: "Kip Macy" To: "Ivan Voras" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607250814m1a476f09p2d962dedc0c99be1@mail.gmail.com> <200607251232.51230.jhb@freebsd.org> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 22:25:08 -0000 On 1/16/07, Ivan Voras wrote: > Kip Macy wrote: > > x86 pre-P4 had 32-byte cache lines. Thus older processors will not benefit. > > But it does seem to hurt the performance a bit - maybe it's time to add > another CPU option like I586_CPU and I686_CPU? Unless there is a compelling reason not to do so, I think that that would be a good idea. -Kip From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 22:52:43 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AB73316A47C for ; Tue, 16 Jan 2007 22:52:43 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.187]) by mx1.freebsd.org (Postfix) with ESMTP id 1AC5813C47E for ; Tue, 16 Jan 2007 22:52:42 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so19363nfc for ; Tue, 16 Jan 2007 14:52:42 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=EK2dpa7MVFcU4pzYzUezhJIupCn4voKCLw2IdiUI+kDO8Y9LweK7wi4j8zkM4fffmf7v7pDXSC1WvLJfVaxCeD2dE67DPAOs3plv+n8Rr4qcclogjQ352X1Uv0WnabIGcC/OgrS9u0c+O7wwkEtxNdpwkzAKS8/RR2JYAsDa4oA= Received: by 10.49.94.18 with SMTP id w18mr9798nfl.1168987956450; Tue, 16 Jan 2007 14:52:36 -0800 (PST) Received: by 10.48.238.9 with HTTP; Tue, 16 Jan 2007 14:52:36 -0800 (PST) Message-ID: <3bbf2fe10701161452v1ca60470q3c26c31dd4bb4785@mail.gmail.com> Date: Tue, 16 Jan 2007 23:52:36 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "John Baldwin" In-Reply-To: <200701161605.22394.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <200701161438.52481.jhb@freebsd.org> <3bbf2fe10701161236s48e6cc16p99c8c38c1d7becde@mail.gmail.com> <200701161605.22394.jhb@freebsd.org> X-Google-Sender-Auth: 23d6ed650ff311a4 Cc: Kip Macy , reebsd-current@freebsd.org, Pawel Jakub Dawidek , Suleiman Souhlal , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 22:52:43 -0000 2007/1/16, John Baldwin : > On Tuesday 16 January 2007 15:36, Attilio Rao wrote: > > 2007/1/16, John Baldwin : > > > On Tuesday 16 January 2007 11:51, Attilio Rao wrote: > > > > 2006/7/28, Attilio Rao : > > > > > > > > > > After some thinking, I think it's better using init/fini methods > > > > > (since they hide the sizeof(struct turnstile) with size parameter). > > > > > > > > > > Feedbacks and comments are welcome: > > > > > http://users.gufi.org/~rookie/works/patches/uma_sync_init.diff > > > > > > > > [CC'ed all the interested people] > > > > > > > > Even if a long time is passed I did some benchmarks based on ebizzy > tool. > > > > This program claims to reproduce a real httpd server behaviour and is > > > > used into the Linux world for benchmarks, AFAIK. > > > > I think that results of the comparison on this patch is very > > > > interesting, and I think it worths a commit :) > > > > I think that results can be even better on a Xeon machine (I had no > > > > chance to reproduce this on some of these). > > > > (Results taken in consideration have been measured after some starts, > > > > in order to minimize caching differences). > > > > > > > > The patch: > > > > http://users.gufi.org/~rookie/works/patches/ts-sq/ts-sq.diff > > > > > > Looks good. Some minor nits are that in subr_turnstile.c in the comment I > > > would say "a turnstile is allocated" rather than "a turnstile is got from > a > > > specific UMA zone" as it reads a little bit clearer. Also, I would > > > say "Allocate a" rather than "Get a" for the two _alloc() functions. > Also, > > > why not just use UMA_ALIGN_CACHE and make UMA_ALIGN_CACHE (128 - 1) on > i386 > > > and amd64 rather than adding a new UMA_ALIGN_SYNC? > > > > I was thinking that in this way anyone who wants to replace the > > syncronizing primitive boundary to an appropriate value can do it. > > I just used UMA_ALIGN_CACHE as default value beacause I don't know the > > better boundary (for syncronizing primitives) for other arches. > > Is there a good reason to not cache-align synch primitives? That is, why > would an arch not use cache-align? Also, is there a reason to not update > UMA_ALIGN_CACHE on x86? Beacause the cache line varies between different CPU models of the same family. For example, L1, L2, L3 cache lines are 64 bytes wide on P4 and Xeon. L1 and L2 caches are 32 bytes wide on the other CPUs (P3, P2, etc.) and in particular they have nothing to do with trace cache line size that takes advanteges from this code. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 22:54:05 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B7CDE16A416 for ; Tue, 16 Jan 2007 22:54:05 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.184]) by mx1.freebsd.org (Postfix) with ESMTP id 4C4AB13C44B for ; Tue, 16 Jan 2007 22:54:05 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so19646nfc for ; Tue, 16 Jan 2007 14:54:04 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=FQXxjNI+hmujRtHN43EE6O3G07cq3vnDWSzdVlOwM/oqSL488XRwzoR9Fgf+P9kBLDsrjjFv0/CCBAR8yWM6thRDvc2zFFHzWSNX7krMvsKyM3VDNIeFXJAiHiWbHnQcNweGCopyEOvSMM5ET1NLP8VBmvjaVjvtlhPgN8mkeUs= Received: by 10.49.12.4 with SMTP id p4mr33046nfi.1168988043791; Tue, 16 Jan 2007 14:54:03 -0800 (PST) Received: by 10.48.238.9 with HTTP; Tue, 16 Jan 2007 14:54:03 -0800 (PST) Message-ID: <3bbf2fe10701161454m74bd9356i3999187515c60596@mail.gmail.com> Date: Tue, 16 Jan 2007 23:54:03 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Ivan Voras" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607250814m1a476f09p2d962dedc0c99be1@mail.gmail.com> <200607251232.51230.jhb@freebsd.org> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> X-Google-Sender-Auth: c9bda9979a1f9f15 Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 22:54:05 -0000 2007/1/16, Ivan Voras : > Kip Macy wrote: > > x86 pre-P4 had 32-byte cache lines. Thus older processors will not benefit. > > But it does seem to hurt the performance a bit - maybe it's time to add > another CPU option like I586_CPU and I686_CPU? Well, it is my feeling that probabilly the align_cache parameter should be a run-time settled parameter (in particular for ia32 CPUs which changed a lot along the years). Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 23:06:25 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6043616A47E; Tue, 16 Jan 2007 23:06:25 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.freebsd.org (Postfix) with ESMTP id 0C0AB13C44B; Tue, 16 Jan 2007 23:06:24 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id l0GN6Ca1047863; Tue, 16 Jan 2007 18:06:12 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 16 Jan 2007 18:06:26 -0500 User-Agent: KMail/1.9.1 References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10701161454m74bd9356i3999187515c60596@mail.gmail.com> In-Reply-To: <3bbf2fe10701161454m74bd9356i3999187515c60596@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200701161806.27245.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 16 Jan 2007 18:06:13 -0500 (EST) X-Virus-Scanned: ClamAV 0.88.3/2457/Tue Jan 16 06:53:04 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Attilio Rao , Ivan Voras Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 23:06:25 -0000 On Tuesday 16 January 2007 17:54, Attilio Rao wrote: > 2007/1/16, Ivan Voras : > > Kip Macy wrote: > > > x86 pre-P4 had 32-byte cache lines. Thus older processors will not benefit. > > > > But it does seem to hurt the performance a bit - maybe it's time to add > > another CPU option like I586_CPU and I686_CPU? > > Well, it is my feeling that probabilly the align_cache parameter > should be a run-time settled parameter (in particular for ia32 CPUs > which changed a lot along the years). Yes. I think UMA_ALIGN_CACHE should be a magic cookie value and that the MD code should provide a cache line size to uma(4) during bootup, and I think you can probably just axe UMA_ALIGN_SYNC and use UMA_ALIGN_CACHE for the turnstile and sleep queue zones. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 23:25:19 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7C13F16A494 for ; Tue, 16 Jan 2007 23:25:19 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.186]) by mx1.freebsd.org (Postfix) with ESMTP id 1085813C457 for ; Tue, 16 Jan 2007 23:25:18 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so25918nfc for ; Tue, 16 Jan 2007 15:25:18 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=unHSmBAgHBl2oawC1ES6U9y1AHkT+ydZlxTw7hRj/fNl9TncmL0vLUd0KMpElLvdSzj4QkZaa6C1cv+HLfJBMIdYXsJmu23u1wwdyTH7P97Sxn/dOtLg80KoKHblMY77yuQlBhsceo/vvFtcJqdzrAd2jzyTKI6rq9aXY/418/4= Received: by 10.49.28.3 with SMTP id f3mr32690nfj.1168989914143; Tue, 16 Jan 2007 15:25:14 -0800 (PST) Received: by 10.48.238.9 with HTTP; Tue, 16 Jan 2007 15:25:14 -0800 (PST) Message-ID: <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> Date: Wed, 17 Jan 2007 00:25:14 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Ivan Voras" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> X-Google-Sender-Auth: 81be34d649c48d7f Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 23:25:19 -0000 2007/1/17, Ivan Voras : > Kip Macy wrote: > > On 1/16/07, Ivan Voras wrote: > >> But it does seem to hurt the performance a bit - maybe it's time to add > >> another CPU option like I586_CPU and I686_CPU? > > > > Unless there is a compelling reason not to do so, I think that that > > would be a good idea. > > Maybe even someone finds a way to get optimized versions of memcpy in > the kernel :) > > I was thinking: AFAIK the only major stopper is context saving of the > various "auxiliary" registers - FPU, MMX, SSE, right? But is it an > all-or-nothing situation? I.e. does it make sense (can it be done?) to > just elect to save the MMX context? (AFAIK they are different registers > than SSE, but overlay FPU registers?) The idea is to save something > smaller than the full set. When I implemented fpu copy into the kernel I had a lot of thinking about this and I think it is possible at least with some restrictions. For example, for an xmm copy you would just save 8 registers content but you have to ensure no pending FPU exceptions will break your kernel and so you should preserve a clean copy of FPU state or, treact the corner cases you can get. For xmm, after some very productive discussions with bde@, we arrived at the conclusion that should be pretty safe to just have an 16 byte aligned buffer for registers saving (in this way you can use 8 movdqa for saving them) but I didn't end to play with it. (My implementation should deal with the problem of pinning the scheduler too, in order to avoid a wrong reading of per-cpu datas). Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 23:34:34 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1D90916A40F for ; Tue, 16 Jan 2007 23:34:34 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.170]) by mx1.freebsd.org (Postfix) with ESMTP id A931C13C457 for ; Tue, 16 Jan 2007 23:34:33 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by ug-out-1314.google.com with SMTP id o2so1676849uge for ; Tue, 16 Jan 2007 15:34:32 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=fjFFFicid4J/XuJ2aXt+yqFDvE31qfqqMiQjz3JEoKG2p7A3UvBVYmXfyGy2YTHefsrOb07BTEamNCZezDDEjZueHSgHk8dPr9rx8wtOywkDrOD/9VlLD0ce45FWSG9FQ+rsrN1eJ0yYfQd1n16NW3okE9XkdibYjJtR+RqMlzs= Received: by 10.82.138.6 with SMTP id l6mr1382937bud.1168990471744; Tue, 16 Jan 2007 15:34:31 -0800 (PST) Received: by 10.82.191.16 with HTTP; Tue, 16 Jan 2007 15:34:31 -0800 (PST) Message-ID: Date: Tue, 16 Jan 2007 15:34:31 -0800 From: "Kip Macy" To: "Ivan Voras" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 23:34:34 -0000 > Maybe even someone finds a way to get optimized versions of memcpy in > the kernel :) > > I was thinking: AFAIK the only major stopper is context saving of the > various "auxiliary" registers - FPU, MMX, SSE, right? But is it an > all-or-nothing situation? I.e. does it make sense (can it be done?) to > just elect to save the MMX context? (AFAIK they are different registers > than SSE, but overlay FPU registers?) The idea is to save something > smaller than the full set. It makes a huge difference in a proprietary file serving appliance that I know of. However, past measurements on FreeBSD have supposedly indicated that it isn't that big win as a result of increased context switch time. -Kip From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 23:39:30 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4C5E516A494 for ; Tue, 16 Jan 2007 23:39:30 +0000 (UTC) (envelope-from mikej@rogers.com) Received: from smtp102.rog.mail.re2.yahoo.com (smtp102.rog.mail.re2.yahoo.com [206.190.36.80]) by mx1.freebsd.org (Postfix) with SMTP id A436813C442 for ; Tue, 16 Jan 2007 23:39:29 +0000 (UTC) (envelope-from mikej@rogers.com) Received: (qmail 35412 invoked from network); 16 Jan 2007 23:12:48 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=Received:X-YMail-OSG:Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=zSFLsghiXjulBTZCcIRVCjplH+OWHij39z72w2IuJoltkw/RlgspnswMeIZGI1q1zJmgIqmHdZ0IkMH6S/KdZj/cGIiXXFhIQKJm02iLZAD7CB+VYZSOt5e0FsWFjzhaOoi5QByXihZNywbteMy3TdEtimClkFFLQsbd9vihnx8= ; Received: from unknown (HELO ?172.16.0.200?) (mikej@rogers.com@74.111.253.239 with plain) by smtp102.rog.mail.re2.yahoo.com with SMTP; 16 Jan 2007 23:12:48 -0000 X-YMail-OSG: ChVx5UEVM1nYP7s3yXEkA45MXBPi3bOsH.GWJSV_5KF1E4SJD21vhF9TfYFtgbuefA-- Message-ID: <45AD5BEF.4070406@rogers.com> Date: Tue, 16 Jan 2007 18:12:47 -0500 From: Mike Jakubik User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: Kip Macy References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607250814m1a476f09p2d962dedc0c99be1@mail.gmail.com> <200607251232.51230.jhb@freebsd.org> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Nick Evans , Attilio Rao , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 23:39:30 -0000 Kip Macy wrote: > x86 pre-P4 had 32-byte cache lines. Thus older processors will not > benefit. What about AMDs line of processors? From owner-freebsd-arch@FreeBSD.ORG Tue Jan 16 23:55:04 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 83AEC16A415 for ; Tue, 16 Jan 2007 23:55:04 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id B4DFB13C465 for ; Tue, 16 Jan 2007 23:55:03 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1H6y8n-0000qI-Gy for freebsd-arch@freebsd.org; Wed, 17 Jan 2007 00:54:57 +0100 Received: from 83-131-108-141.adsl.net.t-com.hr ([83.131.108.141]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Jan 2007 00:54:57 +0100 Received: from ivoras by 83-131-108-141.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Jan 2007 00:54:57 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Wed, 17 Jan 2007 00:58:44 +0100 Lines: 30 Message-ID: References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig2A79E6FD5277C51720F8CAB8" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 83-131-108-141.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: X-Enigmail-Version: 0.94.1.2 Sender: news Cc: freebsd-current@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 23:55:04 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig2A79E6FD5277C51720F8CAB8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Kip Macy wrote: >> Maybe even someone finds a way to get optimized versions of memcpy in >> the kernel :) > It makes a huge difference in a proprietary file serving appliance > that I know of.=20 Beneficial difference? --------------enig2A79E6FD5277C51720F8CAB8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFrWa0ldnAQVacBcgRAmrGAKCfKW9XslZV3yt8JcyfSa+YdOLvBACfbEml 1UDzZAIknm/byuMk9vIrOUE= =cPvE -----END PGP SIGNATURE----- --------------enig2A79E6FD5277C51720F8CAB8-- From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 00:52:05 2007 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D7FB316A407; Wed, 17 Jan 2007 00:52:05 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk.360sip.com [72.236.70.226]) by mx1.freebsd.org (Postfix) with ESMTP id 281E313C457; Wed, 17 Jan 2007 00:52:05 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from [192.168.1.47] ([204.244.149.125]) (authenticated bits=0) by sippysoft.com (8.13.8/8.13.6) with ESMTP id l0H0U0NS095908 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 16 Jan 2007 16:30:04 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <45AD6DFA.6030808@FreeBSD.org> Date: Tue, 16 Jan 2007 16:29:46 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: Attilio Rao References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> In-Reply-To: <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@FreeBSD.org, Ivan Voras , freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 00:52:06 -0000 Attilio Rao wrote: > 2007/1/17, Ivan Voras : >> Kip Macy wrote: >> > On 1/16/07, Ivan Voras wrote: >> >> But it does seem to hurt the performance a bit - maybe it's time to >> add >> >> another CPU option like I586_CPU and I686_CPU? >> > >> > Unless there is a compelling reason not to do so, I think that that >> > would be a good idea. >> >> Maybe even someone finds a way to get optimized versions of memcpy in >> the kernel :) >> >> I was thinking: AFAIK the only major stopper is context saving of the >> various "auxiliary" registers - FPU, MMX, SSE, right? But is it an >> all-or-nothing situation? I.e. does it make sense (can it be done?) to >> just elect to save the MMX context? (AFAIK they are different registers >> than SSE, but overlay FPU registers?) The idea is to save something >> smaller than the full set. > > When I implemented fpu copy into the kernel I had a lot of thinking > about this and I think it is possible at least with some restrictions. > For example, for an xmm copy you would just save 8 registers content > but you have to ensure no pending FPU exceptions will break your > kernel and so you should preserve a clean copy of FPU state or, treact > the corner cases you can get. > For xmm, after some very productive discussions with bde@, we arrived > at the conclusion that should be pretty safe to just have an 16 byte > aligned buffer for registers saving (in this way you can use 8 movdqa > for saving them) but I didn't end to play with it. > (My implementation should deal with the problem of pinning the > scheduler too, in order to avoid a wrong reading of per-cpu datas). I might be wrong, but I think the DragonFly has solved this issue (i.e. optimized memcpy in the kernel) somehow quite some time ago. -Maxim From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 00:55:12 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BD9D716A40F for ; Wed, 17 Jan 2007 00:55:12 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.190]) by mx1.freebsd.org (Postfix) with ESMTP id 3CD4B13C468 for ; Wed, 17 Jan 2007 00:55:12 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so42430nfc for ; Tue, 16 Jan 2007 16:55:11 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=UiWS5XqcE4hMR3V9L1sJu0rEyJ43vIfQY1xCZqsykuUJFCH64ppkLmql5nbeHtAYPLjGJLDhzwuPsaHGcisnLBC1mzda88EWN9xjuonM1rOwyPQYRR4DaIJhxT/40VrHZKzEs8pfFQz6U4OL6byknKGo8GTbCLzuzJA7NOuSVIE= Received: by 10.49.93.4 with SMTP id v4mr109303nfl.1168995309825; Tue, 16 Jan 2007 16:55:09 -0800 (PST) Received: by 10.48.238.9 with HTTP; Tue, 16 Jan 2007 16:55:09 -0800 (PST) Message-ID: <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> Date: Wed, 17 Jan 2007 01:55:09 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Maxim Sobolev" In-Reply-To: <45AD6DFA.6030808@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> X-Google-Sender-Auth: 156349cc4e226549 Cc: freebsd-current@freebsd.org, Ivan Voras , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 00:55:12 -0000 2007/1/17, Maxim Sobolev : > Attilio Rao wrote: > > 2007/1/17, Ivan Voras : > >> Kip Macy wrote: > >> > On 1/16/07, Ivan Voras wrote: > >> >> But it does seem to hurt the performance a bit - maybe it's time to > >> add > >> >> another CPU option like I586_CPU and I686_CPU? > >> > > >> > Unless there is a compelling reason not to do so, I think that that > >> > would be a good idea. > >> > >> Maybe even someone finds a way to get optimized versions of memcpy in > >> the kernel :) > >> > >> I was thinking: AFAIK the only major stopper is context saving of the > >> various "auxiliary" registers - FPU, MMX, SSE, right? But is it an > >> all-or-nothing situation? I.e. does it make sense (can it be done?) to > >> just elect to save the MMX context? (AFAIK they are different registers > >> than SSE, but overlay FPU registers?) The idea is to save something > >> smaller than the full set. > > > > When I implemented fpu copy into the kernel I had a lot of thinking > > about this and I think it is possible at least with some restrictions. > > For example, for an xmm copy you would just save 8 registers content > > but you have to ensure no pending FPU exceptions will break your > > kernel and so you should preserve a clean copy of FPU state or, treact > > the corner cases you can get. > > For xmm, after some very productive discussions with bde@, we arrived > > at the conclusion that should be pretty safe to just have an 16 byte > > aligned buffer for registers saving (in this way you can use 8 movdqa > > for saving them) but I didn't end to play with it. > > (My implementation should deal with the problem of pinning the > > scheduler too, in order to avoid a wrong reading of per-cpu datas). > > I might be wrong, but I think the DragonFly has solved this issue (i.e. > optimized memcpy in the kernel) somehow quite some time ago. Dragonfly saves the whole context (xmm + mmx + fpu state). It is a too heavy mechanism ATM for us (and for them too I suspect). The don't need to deal with pinning too, BTW. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 04:50:45 2007 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9EDD416A40F; Wed, 17 Jan 2007 04:50:45 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2-3.pacific.net.au [61.8.2.226]) by mx1.freebsd.org (Postfix) with ESMTP id 3A90F13C428; Wed, 17 Jan 2007 04:50:45 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout2.pacific.net.au (Postfix) with ESMTP id D19F46E2C7; Wed, 17 Jan 2007 15:50:41 +1100 (EST) Received: from besplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id C6C8A2741B; Wed, 17 Jan 2007 15:50:42 +1100 (EST) Date: Wed, 17 Jan 2007 15:50:41 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Ivan Voras In-Reply-To: Message-ID: <20070117134022.V18339@besplex.bde.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 04:50:45 -0000 On Wed, 17 Jan 2007, Ivan Voras wrote: > Kip Macy wrote: >>> Maybe even someone finds a way to get optimized versions of memcpy in >>> the kernel :) > >> It makes a huge difference in a proprietary file serving appliance >> that I know of. > > Beneficial difference? Heheh. >> However, past measurements on FreeBSD have supposedly >> indicated that it isn't that big win as a result of increased context >> switch time. No, they indicated that the win is not very large (sometimes negative), and is very machine dependent. E.g., it is a small pessimization all 64 bit i386's running 64-bit mode -- that's just all i386's you would want to buy now. On other CPU classes: P2 (my old Celeron): +- epsilon difference P3 (freefall): +- epsilon difference P4 (nosedive's Xeon): movdqa 17% faster than movsl, but all other cached moves slower using MMX or SSE[1-2]; movnt with block prefetch 60% faster than movsl with no prefetch, but < 5% faster with no prefetch for both. AXP: (my 5 year old system with a newer CPU): movq through MMX is 60% faster than movsl for cached moves, but movdqa through XMM is only 4% faster. movnt with block prefetch is 155% faster than movsl with no prefetch, and 73% faster with no prefetch for both. A64 in 32-bit mode: in between P4 and AXP (closer to AXP). movsl doesn't lose by so much, and prefetchnta actually works so block prefetch is not needed and there is a better chance of prefetching helping more than benchmarks. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 12:00:46 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5D73B16A5B3; Wed, 17 Jan 2007 12:00:46 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2-3.pacific.net.au [61.8.2.226]) by mx1.freebsd.org (Postfix) with ESMTP id EDACB13C44B; Wed, 17 Jan 2007 12:00:45 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.2.162]) by mailout2.pacific.net.au (Postfix) with ESMTP id 5DC1D6E17F; Wed, 17 Jan 2007 23:00:42 +1100 (EST) Received: from besplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailproxy1.pacific.net.au (Postfix) with ESMTP id 580638C02; Wed, 17 Jan 2007 23:00:43 +1100 (EST) Date: Wed, 17 Jan 2007 23:00:42 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Ivan Voras In-Reply-To: <20070117134022.V18339@besplex.bde.org> Message-ID: <20070117224812.Q23194@besplex.bde.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <20070117134022.V18339@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 12:00:46 -0000 On Wed, 17 Jan 2007, I wrote: > ... > P4 (nosedive's Xeon): movdqa 17% faster than movsl, but all other cached > moves slower using MMX or SSE[1-2]; movnt with block prefetch 60% faster > than movsl with no prefetch, but < 5% faster with no prefetch for both. > AXP: (my 5 year old system with a newer CPU): movq through MMX is 60% > faster than movsl for cached moves, but movdqa through XMM is only 4% > faster. movnt with block prefetch is 155% faster than movsl with no > prefetch, and 73% faster with no prefetch for both. > A64 in 32-bit mode: in between P4 and AXP (closer to AXP). movsl doesn't > lose by so much, and prefetchnta actually works so block prefetch is > not needed and there is a better chance of prefetching helping more > than benchmarks. And MMX/XMM registers ar not needed to get movnt on machines with SSE2, since movnti is part of SSE2. This reduces the advantages of using MMX/XMM registers on P4's and A64's in 32-bit mode to the non-nt parts of the above (fully cached case), which I think are less important than the nt parts. Another complication with movnt is that its semantics are very machine- dependent. On AXP, movnt to a target that happens to be in the L1 cache goes at L1 cache speed, so it is probably good to use movnt blindly (except movnti doesn't exist so you can't just substitute movl with movnti and must use XMM registers with all their complications), but on P4 and A64, movnt to a cached target goes at main memory speed so you only want to use it intentionally to avoid thrashing the caches. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 15:58:34 2007 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 97AD916A4DF for ; Wed, 17 Jan 2007 15:58:34 +0000 (UTC) (envelope-from rnsanchez@wait4.org) Received: from sumo.dreamhost.com (sumo.dreamhost.com [66.33.216.29]) by mx1.freebsd.org (Postfix) with ESMTP id 7C45713C448 for ; Wed, 17 Jan 2007 15:58:34 +0000 (UTC) (envelope-from rnsanchez@wait4.org) Received: from spunkymail-a19.dreamhost.com (sd-green-bigip-74.dreamhost.com [208.97.132.74]) by sumo.dreamhost.com (Postfix) with ESMTP id A66771873BC for ; Wed, 17 Jan 2007 07:41:10 -0800 (PST) Received: from sauron.lan.box (unknown [200.203.29.31]) by spunkymail-a19.dreamhost.com (Postfix) with ESMTP id EA9C710CDA; Wed, 17 Jan 2007 07:41:05 -0800 (PST) Date: Wed, 17 Jan 2007 13:41:00 -0200 From: Ricardo Nabinger Sanchez To: Bruce Evans Message-Id: <20070117134100.94bb6137.rnsanchez@wait4.org> In-Reply-To: <20070117134022.V18339@besplex.bde.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <20070117134022.V18339@besplex.bde.org> Organization: SYS_WAIT4 X-Mailer: Sylpheed 2.3.0+svn (GTK+ 2.10.6; i386-unknown-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-current@FreeBSD.org, Ivan Voras , freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 15:58:34 -0000 On Wed, 17 Jan 2007 15:50:41 +1100 (EST) Bruce Evans wrote: > AXP: (my 5 year old system with a newer CPU): movq through MMX is 60% > faster than movsl for cached moves, but movdqa through XMM is only 4% > faster. movnt with block prefetch is 155% faster than movsl with no > prefetch, and 73% faster with no prefetch for both. > A64 in 32-bit mode: in between P4 and AXP (closer to AXP). movsl doesn't > lose by so much, and prefetchnta actually works so block prefetch is > not needed and there is a better chance of prefetching helping more > than benchmarks. This PDF is somewhat dated, but perhaps some of it still applies today: http://cdrom.amd.com/devconn/events/AMD_block_prefetch_paper.pdf -- Ricardo Nabinger Sanchez Powered by FreeBSD "Left to themselves, things tend to go from bad to worse." From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 20:09:24 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8D05516A415; Wed, 17 Jan 2007 20:09:24 +0000 (UTC) (envelope-from ivoras@fer.hr) Received: from ls405.t-com.hr (ls405.t-com.hr [195.29.150.135]) by mx1.freebsd.org (Postfix) with ESMTP id 0F9C413C428; Wed, 17 Jan 2007 20:09:23 +0000 (UTC) (envelope-from ivoras@fer.hr) Received: from ls242.t-com.hr (ls242.t-com.hr [195.29.150.134]) by ls405.t-com.hr (Postfix) with ESMTP id 0730C143C02; Wed, 17 Jan 2007 20:37:52 +0100 (CET) Received: from ls242.t-com.hr (localhost.localdomain [127.0.0.1]) by ls242.t-com.hr (Qmlai) with ESMTP id E272410F805F; Wed, 17 Jan 2007 20:37:51 +0100 (CET) Received: from ls242.t-com.hr (localhost.localdomain [127.0.0.1]) by ls242.t-com.hr (Qmlai) with ESMTP id C847310F803E; Wed, 17 Jan 2007 20:37:51 +0100 (CET) X-Envelope-Sender-Info: qNMqIwJL6pMxrLsv6Bv6Q4WPdw3S56b2lZWSpOFx2GhM9iIFzLoPeE8q5w/auABt X-Envelope-Sender: ivoras@fer.hr Received: from [10.0.0.101] (89-172-39-6.adsl.net.t-com.hr [89.172.39.6]) by ls242.t-com.hr (Qmali) with ESMTP id 7F25B6C0036; Wed, 17 Jan 2007 20:37:51 +0100 (CET) Message-ID: <45AE7BF8.10703@fer.hr> Date: Wed, 17 Jan 2007 20:41:44 +0100 From: Ivan Voras User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: Bruce Evans References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607251004wf94e238xb5ea7a31c973817f@mail.gmail.com> <3bbf2fe10607261127p3f01a6c3w80027754f7d4e594@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <20070117134022.V18339@besplex.bde.org> <20070117224812.Q23194@besplex.bde.org> In-Reply-To: <20070117224812.Q23194@besplex.bde.org> X-Enigmail-Version: 0.94.1.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig238686BF5CDEB8B50F4252DA" Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Optimized copy&move (was: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 20:09:24 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig238686BF5CDEB8B50F4252DA Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Bruce Evans wrote: > And MMX/XMM registers ar not needed to get movnt on machines with SSE2,= > since movnti is part of SSE2. This reduces the advantages of using MMX= /XMM > registers on P4's and A64's in 32-bit mode to the non-nt parts of the > above (fully cached case), which I think are less important than the nt= > parts. Hmm, I'm looking at i386/i386/support.s and there are several versions of bcopy and bmove functions, including some that optimize by using FPU registers (large_i586_bcopy_loop), and a version that uses movnti (sse2_pagezero), but I can't find the bit of magic which glues them to bzero() call. Also, as as I can tell by the comments, the FPU version works by manually saving context... why is this possible (i.e. won't something preempt it?) --------------enig238686BF5CDEB8B50F4252DA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFrnv5ldnAQVacBcgRAiR4AKCDQ6LnV8f1easSMto1WjFvD74GPgCfdciE 08MOyCrxIQhroC5tgvhqJo0= =7h/R -----END PGP SIGNATURE----- --------------enig238686BF5CDEB8B50F4252DA-- From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 20:29:55 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BB71516A51B; Wed, 17 Jan 2007 20:29:55 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from relay.talkpoint.com (pobox.talkpoint.com [204.141.15.158]) by mx1.freebsd.org (Postfix) with ESMTP id 8146A13C467; Wed, 17 Jan 2007 20:29:55 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from ASSP-nospam ([127.0.0.1]) by relay.talkpoint.com with Microsoft SMTPSVC(5.0.2195.6713); Wed, 17 Jan 2007 15:29:54 -0500 Received: from 204.141.15.136 ([204.141.15.136] helo=postal.talkpoint.com) by ASSP-nospam ; 17 Jan 07 20:29:54 -0000 Received: from pleiades.nextvenue.com ([204.141.15.194]) by postal.talkpoint.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id ZN3V7MJS; Wed, 17 Jan 2007 15:29:50 -0500 Date: Wed, 17 Jan 2007 15:29:50 -0500 From: Nick Evans To: Ivan Voras Message-ID: <20070117152950.6c372f24@pleiades.nextvenue.com> In-Reply-To: <45AE7BF8.10703@fer.hr> References: <45AE7BF8.10703@fer.hr> X-Mailer: Sylpheed-Claws 2.4.0 (GTK+ 2.8.20; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 17 Jan 2007 20:29:54.0310 (UTC) FILETIME=[3F221A60:01C73A76] Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Optimized copy&move (was: Re: [PATCH] Mantaining turnstile aligne d to 128 bytes in i386 CPUs) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 20:29:55 -0000 On Wed, 17 Jan 2007 14:41:44 -0500 Ivan Voras wrote: > Bruce Evans wrote: > > > And MMX/XMM registers ar not needed to get movnt on machines with > SSE2, > > since movnti is part of SSE2. This reduces the advantages of using > MMX/XMM > > registers on P4's and A64's in 32-bit mode to the non-nt parts of the > > above (fully cached case), which I think are less important than the > nt > > parts. > > Hmm, I'm looking at i386/i386/support.s and there are several versions > of bcopy and bmove functions, including some that optimize by using FPU > registers (large_i586_bcopy_loop), and a version that uses movnti > (sse2_pagezero), but I can't find the bit of magic which glues them to > bzero() call. > > Also, as as I can tell by the comments, the FPU version works by > manually saving context... why is this possible (i.e. won't something > preempt it?) > Potentially stupid question but, is it not possible to benchmark these variations at build or boot time and use the most appropriate method? Or at least the one most appropriate 90% of the time? Nick From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 20:47:42 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1B34A16A412; Wed, 17 Jan 2007 20:47:42 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 7926613C457; Wed, 17 Jan 2007 20:47:41 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.13.7/8.13.7) with ESMTP id l0HKMYFV053838; Wed, 17 Jan 2007 12:25:34 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.13.7/8.13.4/Submit) id l0HKMYV8053837; Wed, 17 Jan 2007 12:22:34 -0800 (PST) Date: Wed, 17 Jan 2007 12:22:34 -0800 (PST) From: Matthew Dillon Message-Id: <200701172022.l0HKMYV8053837@apollo.backplane.com> To: "Attilio Rao" References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> Cc: Maxim Sobolev , freebsd-current@freebsd.org, Ivan Voras , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 20:47:42 -0000 The cost of using the FPU can simply be thought of in terms of how many bytes you have to have to copy for it to become worth using the FPU over a far less complex integer copy loop. This is really easy to find out, and it is also fairly easy to instrument a sysctl to set the value used in the comparison and run benchmarks to determine at what point using the FP unit becomes the better choice. * Saving the FP state. The kernel doesn't have to save or restore anything if userland was not using the floating point unit. In fact, the kernel doesn't even need to FNINIT! All the kernel needs to do is CLTS and FNCLEX to make the FP unit usable for media copy instructions, then set CR0_TS when it is finished. Gee, that's nice! But if on the otherhand userland is using the floating point unit inbetween every system call then having the kernel try to use it does require calling fxsave and clearing npxthread == serious inefficiencies if userland is using the FP unit heavily. Or, alternatively, it can fxsave AND restore the state when it is done at a total cost of around 70ns plus write bandwidth cruft. In fact, I would say that if userland is not using the FP unit, that is npxthread == NULL or npxthread != curthread, you should *DEFINITELY* use the FP unit. Hands down, no question about it. * First, raw memory bandwidth is governed by RAS cycles. The fewer RAS cycles you have, the higher the bandwidth. This means that the more data you can load into the cpu on the 'read' side of the copy before transitioning to the 'write' side, the better. With XMM you can load 128 *BYTES* a shot (8 128 bit registers). For large copies, nothing beats it. * Modern cpu hardware uses a 128 bit data path for 128 bit media instructions and can optimize the 128 bit operation all the way through to a cache line or to main memory. It can't be beat. Alignment is critical. If the data is not aligned, don't bother. 128 bits means 16 byte alignment. * No extranious memory writes, no uncached extranious memory reads. If you do any writes to memory other then to the copy destination in your copy loop you screw up the cpu's write fifo and destroy performance. Systems are so sensitive to this that it is even better to spend the time linearly mapping large copy spaces into KVM and do a single block copy then to have an inner per-PAGE loop. * Use of prefetch or use of movntdq instead of movdqa is highly problematic. It is possible to use these to optimize very particular cases but the problem is they tend to nerf all OTHER cases. I've given up trying to use either mechanism. Instead, I prefer copying as large a block as possible to remove these variables from the cpu pipeline entirely. The cpu has a write fifo anyway, you don't need prefetch instructions if you can use instructions to write to memory faster then available L2 cache bandwidth. On some cpus this mandates the use of 64 or 128 bit media instructions or the cpu can't keep the write FIFO full and starts interleaving reads and writes on the wrong boundaries (creating more RAS cycles, which is what kills copy bandwidth). * RAS transitions also have to be aligned or you get boundary cases when the memory address transitions a RAS line. This again mandates maximal alignment (even more then 16 bytes, frankly, which is why being able to do 128 byte blocks with XMM registers is so nice). Even though reads and writes are reblocked to the cache line size by the cpu, your inner loop can still transition a RAS boundary in the middle of a large block read if it isn't aligned. But at this point the alignment requirements start to get kinda silly. 128 byte alignment requirement? I don't think so. I do a 16-byte alignment check in DragonFly as a pre-req for using XMM and that's it. But, as I said in the beginning... all you need is just one variable. Copying data below that threshold is faster without the FP unit, copying data above that threshold is faster with the FP unit. Implement it, test it, and see how you fare. If you are paranoid about having to save the FP state, then only use the FP unit when npxthread == NULL (no save required) or npxthread != curthread (save on behalf of a different thread required, which is ok)... It's that simple. Pinning is an issue with FreeBSD, one whos effect I cannot comment on. I don't know about AMD64. You only have 64 bit general registers in 64 bit mode so you may not be able to keep the write pipeline full. But you do have 8 of them so you are roughly equivalent to MMX (but not XMM's 8 128 bit registers). -Matt From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 20:58:11 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D432716A55C; Wed, 17 Jan 2007 20:58:11 +0000 (UTC) (envelope-from ivoras@fer.hr) Received: from ls405.t-com.hr (ls405.t-com.hr [195.29.150.135]) by mx1.freebsd.org (Postfix) with ESMTP id 8827C13C45B; Wed, 17 Jan 2007 20:58:11 +0000 (UTC) (envelope-from ivoras@fer.hr) Received: from ls422.t-com.hr (ls422.t-com.hr [195.29.150.237]) by ls405.t-com.hr (Postfix) with ESMTP id 4B01F146443; Wed, 17 Jan 2007 21:58:10 +0100 (CET) Received: from ls422.t-com.hr (localhost.localdomain [127.0.0.1]) by ls422.t-com.hr (Qmlai) with ESMTP id 336E5C90053; Wed, 17 Jan 2007 21:58:10 +0100 (CET) X-Envelope-Sender-Info: KDHLkYIFXCRHG7zIR7FVXvx6MUdRkOwZZsfH4/buQTKKesY80THoNSngionnLwUP X-Envelope-Sender: ivoras@fer.hr Received: from [10.0.0.101] (89-172-39-6.adsl.net.t-com.hr [89.172.39.6])by ls422.t-com.hr (Qmlai) with ESMTP id AFA961308056; Wed, 17 Jan 2007 21:58:05 +0100 (CET) Message-ID: <45AE8DDC.8030402@fer.hr> Date: Wed, 17 Jan 2007 21:58:04 +0100 From: Ivan Voras User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: Matthew Dillon References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bb f2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe107011608 51r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleia des.nextvenue.com> <3bbf2fe10701161525j6ad929 2y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe1 0701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> In-Reply-To: <200701172022.l0HKMYV8053837@apollo.backplane.com> X-Enigmail-Version: 0.94.1.2 Content-Type: multipart/mixed; boundary=------------enig5EF4019AFDFB80A298524055 X-imss-version: 2.045 X-imss-result: Passed X-imss-scores: Clean:0.00000 C:100 M:100 S:100 R:100 X-imss-settings: Baseline:1 C:1 M:1 S:1 R:1 (0.0000 0.0000) Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 20:58:11 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig5EF4019AFDFB80A298524055 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Matthew Dillon wrote: > * Saving the FP state. The kernel doesn't have to save or restore > anything if userland was not using the floating point unit. In > fact, the kernel doesn't even need to FNINIT! All the kernel nee= ds > to do is CLTS and FNCLEX to make the FP unit usable for media cop= y > instructions, then set CR0_TS when it is finished. Does the same hold true with kernel threads in FreeBSD (e.g. two threads using FPU)? --------------enig5EF4019AFDFB80A298524055-- From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 21:15:09 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ACAAD16A407 for ; Wed, 17 Jan 2007 21:15:09 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.187]) by mx1.freebsd.org (Postfix) with ESMTP id 316F813C45A for ; Wed, 17 Jan 2007 21:15:09 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so299357nfc for ; Wed, 17 Jan 2007 13:15:08 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=eRHA2G6sBSGuU/pxc8QwbYPRDp+e254PBnXyNIbj6NWRafkXrTOCX/187SbMoh5DzEJ9GTJmsKzUPrUoWtQ6WiS2Bx4QC5QaAEzpsIPA4DIDz9BreSVKo2hrgQ0qXpWQqUHzqWzYhB9TK/rdDuVF00apNH41aZlrEjwVlXuEtTs= Received: by 10.48.48.13 with SMTP id v13mr11083nfv.1169068508050; Wed, 17 Jan 2007 13:15:08 -0800 (PST) Received: by 10.48.238.9 with HTTP; Wed, 17 Jan 2007 13:15:07 -0800 (PST) Message-ID: <3bbf2fe10701171315g696bca4fi3bf676b62c06f4d@mail.gmail.com> Date: Wed, 17 Jan 2007 22:15:07 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Ivan Voras" In-Reply-To: <45AE7BF8.10703@fer.hr> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <20070117134022.V18339@besplex.bde.org> <20070117224812.Q23194@besplex.bde.org> <45AE7BF8.10703@fer.hr> X-Google-Sender-Auth: 540a52cd6fbe718e Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Optimized copy&move (was: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 21:15:09 -0000 2007/1/17, Ivan Voras : > Bruce Evans wrote: > > > And MMX/XMM registers ar not needed to get movnt on machines with SSE2, > > since movnti is part of SSE2. This reduces the advantages of using MMX/XMM > > registers on P4's and A64's in 32-bit mode to the non-nt parts of the > > above (fully cached case), which I think are less important than the nt > > parts. > > Hmm, I'm looking at i386/i386/support.s and there are several versions > of bcopy and bmove functions, including some that optimize by using FPU > registers (large_i586_bcopy_loop), and a version that uses movnti > (sse2_pagezero), but I can't find the bit of magic which glues them to > bzero() call. > > Also, as as I can tell by the comments, the FPU version works by > manually saving context... why is this possible (i.e. won't something > preempt it?) They are just broken. My implementation, which follows DragonFlyBSD patterns, just use a bts (which is atomic) in order to set a "lock" and avoid thread migration with scheduler pinning. This is enough to solve concurrency problems. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Wed Jan 17 22:52:21 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0089816A40F; Wed, 17 Jan 2007 22:52:21 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id CD8B413C45D; Wed, 17 Jan 2007 22:52:20 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.13.7/8.13.7) with ESMTP id l0HMhEx9054400; Wed, 17 Jan 2007 14:46:14 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.13.7/8.13.4/Submit) id l0HMhECY054399; Wed, 17 Jan 2007 14:43:14 -0800 (PST) Date: Wed, 17 Jan 2007 14:43:14 -0800 (PST) From: Matthew Dillon Message-Id: <200701172243.l0HMhECY054399@apollo.backplane.com> To: Ivan Voras References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bb f2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe107011608 51r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleia des.nextvenue.com> <3bbf2fe10701161525j6ad929 2y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe1 0701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <45AE8DDC.8030402@fer.hr> Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 22:52:21 -0000 :Does the same hold true with kernel threads in FreeBSD (e.g. two threads :using FPU)? Preemption and pinning make the issue a bit more difficult for FreeBSD, but the basic idea remains valid. From the point of view of NPXTHREAD the situation is very simple: * NPXTHREAD = NULL nobody owns the FP, nobody is using the FP. If the kernel wants to use the FP it just FNCLEX + CLTS and sets NPXTHREAD = curthread. When it is finished, it undoes that sequence (NPXTHREAD = NULL, set CR0_TS again). PLUSES: FP state does not need to be saved or restored ISSUES: due to cpu migration and preemption the setup and teardown sequence must be done with the cpu pinned, inside a critical section. But the actual use of the FP does not need to occur inside a critical section or with the cpu pinned. * NPXTHREAD = other_thread Some other thread owns the FP, but it isn't our thread so we can safely save the FP state for the other thread without worrying about creating a situation where we thrash the T_DNA exception. Save FP state, FNCLEX, CLTS, set NPXTHREAD = curthread. When finished, NPXTHREAD = NULL, set CR0_TS, do *not* restore the 'other' thread's FP state. PLUSES: The FP state probably had to be saved anyway, it's no big deal or at least it is not as big a deal as the NPXTHREAD = curthread case. ISSUES: Same as above. * NPXTHREAD = curthread The current thread (either userland or a pushed kernel FP context) is using the FP. If the kernel decides it needs the FP it must save the FP state, FNCLEX, CLTS, do its thing. When finished it can decide to set NPXTHREAD = NULL and set CR0_TS, or it can restore the previously saved state and leave NPXTHREAD = curthread. PLUSES: Very few ISSUES: Same as above, but here the kernel must decide whether it is worth stealing the FP or not, because it might get into a thrashing situation with the T_DNA exception under certain userland loads. Note that there are many cases where userland may use the FP unit very occassionally. In such cases you *DO* want to be able to steal it, so perhaps some heuristic is needed to determine the cost of stealing the FP unit dynamically. It is possible to abstract it even more... for example, one can set CR0_TS when going from userland to the kernel and completely abstract out the kernel's use of the FP unit at the cost of a higher entrance fee to get in and out of the kernel. I decided NOT to do this in DragonFly. If the DragonFly kernel wants to use the FP it has to check and adjust the NPXTHREAD state. But, to be absolutely clear here, it costs virtually *nothing* to use the FP in the kernel for non-FP media instructions (i.e. movdq and friends) if userland has not used the FP recently. You push a temporary save area, set NPXTHREAD, FNCLEX, CLTS, use the FP, then pop the save area pointer, set NPXTHREAD to NULL, and set CR0_TS, and that's it. It may seem like a lot of steps but those are all very fast instructions verses having to actually save and restore the 512 byte FP state. The biggest overhead would actually be the critical section and cpu pinning required to properly transition the NPXTHREAD state. -Matt Matthew Dillon From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 00:16:24 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1F1B816A40F; Thu, 18 Jan 2007 00:16:24 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2-3.pacific.net.au [61.8.2.226]) by mx1.freebsd.org (Postfix) with ESMTP id B036F13C428; Thu, 18 Jan 2007 00:16:23 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout2.pacific.net.au (Postfix) with ESMTP id 260B06E29F; Thu, 18 Jan 2007 11:16:20 +1100 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id 82F4727406; Thu, 18 Jan 2007 11:16:20 +1100 (EST) Date: Thu, 18 Jan 2007 11:16:19 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Attilio Rao In-Reply-To: <3bbf2fe10701171315g696bca4fi3bf676b62c06f4d@mail.gmail.com> Message-ID: <20070118094808.F11834@delplex.bde.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <20070117134022.V18339@besplex.bde.org> <20070117224812.Q23194@besplex.bde.org> <45AE7BF8.10703@fer.hr> <3bbf2fe10701171315g696bca4fi3bf676b62c06f4d@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org, Ivan Voras , freebsd-arch@freebsd.org Subject: Re: Optimized copy&move (was: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 00:16:24 -0000 On Wed, 17 Jan 2007, Attilio Rao wrote: > 2007/1/17, Ivan Voras : >> Bruce Evans wrote: >> >> > And MMX/XMM registers ar not needed to get movnt on machines with SSE2, >> > since movnti is part of SSE2. This reduces the advantages of using >> MMX/XMM >> > registers on P4's and A64's in 32-bit mode to the non-nt parts of the >> > above (fully cached case), which I think are less important than the nt >> > parts. One more point on movnt*: - The i386 (32-bit) kernel already uses movnti in the one place where it is certain to be an optimization (for zeroing pages but not for copying anything) if and only if movnti is known for sure to be available (esentially, if the CPU supports SSE2). See i386/pmap.c and i386/support.s. >> Hmm, I'm looking at i386/i386/support.s and there are several versions >> of bcopy and bmove functions, including some that optimize by using FPU >> registers (large_i586_bcopy_loop), and a version that uses movnti >> (sse2_pagezero), but I can't find the bit of magic which glues them to >> bzero() call. sse2_pagezero() is the SSE2 optimization mentioned above. It is glued in pmap.c. The i586 bcopy functions are my old mistakes which I'm trying to bury :-). They are glued in isa/npx.c. There is a runtime test for their efficiency there, and config-time configuration by npx flags (see NOTES). All use of the FPU routines is disabled in all released versions of FreeBSD later than FreeBSD-4 (the config-time configuration is ignored and the dynamic test and the glueing are not done; only the actual routines are compiled, so in theory you could enable them by glueing them in a kmod). >> Also, as as I can tell by the comments, the FPU version works by >> manually saving context... why is this possible (i.e. won't something >> preempt it?) In RELENG_4, the kernel is not preemptible so preemption isn't a problem. In later versions, preemption is a problem so the FPU routines are disabled. RELENG_4 has a limited amount of preemption for interrupt handlers, and the FPU routines have a limited amount of recursive saving of contexts to support this. No bugs are known in this under RELENG_4. Under later versions, the recursive saving doesn't quite work. RELENG_4 has a limited amount of support for SMP, and the FPU routines have a limited amount of locking to support this. No bugs are known in this under RELENG_4, but I wouldn't trust it without testing. It has probably been tested enough under RELENG_4 by now, but it might never have been tested before 2001 when the FPU routines were turned off in -current, because there were no machines with the critical combination of features until relatively recently: - SMP machines with Pentium-1's were rare and are now rarer - the FPU routines are slower on P2-P4, K5 and K6 so the dynamic configuration should prevent them being used on machines with P2-P4, K5 and K6. - the FPU routines are faster on Athlons (XP and 64 at least), but these didn't exist until 2001. The introduction of these CPUs may have been the trigger for turning off the FPU routines in -current in 2001. Until then problems were limited to Pentium-1's since the dynamic configuration prevented the routines being used on all other machines. > They are just broken. > My implementation, which follows DragonFlyBSD patterns, just use a bts > (which is atomic) in order to set a "lock" and avoid thread migration > with scheduler pinning. This is enough to solve concurrency problems. There is a bit more to it than that :-). The old implementation uses a sar instruction for the same purpose. Neither bts nor sar is atomic, but both can be made atomic using a lock prefix. The old implementation neglects to do this, so the instruction is only atomic with respect to interrupts. If it works at all for the SMP case under RELENG_4, then it is because Giant locking prevents all types of preemption. Giant locking certainly prevents process preemption, but it is less clear that it prevents interrupt handlers running on other CPUs from getting far enough to clobber the lock. I think it does. The unlocked sar just doesn't work under -current, especially starting much later than 2001 when the kernel became fully preemptible. (I like to use sar instead of bts/cmpxchg/ whatever since it is more portable.) Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 03:21:10 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5873E16A407 for ; Thu, 18 Jan 2007 03:21:10 +0000 (UTC) (envelope-from sbenabas@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.175]) by mx1.freebsd.org (Postfix) with ESMTP id D2EA513C45B for ; Thu, 18 Jan 2007 03:21:09 +0000 (UTC) (envelope-from sbenabas@gmail.com) Received: by ug-out-1314.google.com with SMTP id o2so58520uge for ; Wed, 17 Jan 2007 19:21:08 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type; b=fIu3/sA6tJVSN4zUBPA3ttKGbbwMtx4J2g05TkIum2hpWPHTSdzV0/KeaEOauQ5fKvR2gPp4Gp3Vmva72jdx7hcADP+p+zi4Bd92u5otW43goakn6khAsMkJBQ7nnte4inyhVSTzsDdLU9LqhEJ6OSPVuGjwfHIavHktOswZCek= Received: by 10.66.221.6 with SMTP id t6mr537814ugg.1169088977716; Wed, 17 Jan 2007 18:56:17 -0800 (PST) Received: by 10.67.101.20 with HTTP; Wed, 17 Jan 2007 18:56:17 -0800 (PST) Message-ID: <32d8477c0701171856i6f7c4cdy384ad2e10086a65f@mail.gmail.com> Date: Wed, 17 Jan 2007 21:56:17 -0500 From: "Siavosh Benabbas" To: freebsd-arch@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: siginfo_t.si_code macro constants X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 03:21:10 -0000 Hi, I am trying to compile parted on freebsd and I came across this. It seems that many macro constants of POSIX that one should be able to check the value of siginfo_t.si_code in a 3 parameter signal handler (set by sigaction) are missing. These in particular include SEGV_MAPPER, SEGV_ACCERR, and many ILL_* constants the man page of sigaction mentions: "The code argument of the BSD-style handler and the si_code member of the info argument to a SA_SIGINFO handler contain a numeric code explaining the cause of the signal, usually one of the SI_... values from or codes specific to a signal, i.e., one of the FPE_... values for SIGFPE." But it doesn't specifically list the macro's available. According to the POSIX standard there should be some SEGV_* and ILL_* constants too. I searched in the archives and it seems that this was pointed out around a year ago. The mail in the archive also mentioned that siginfo_t.si_code actually gets populated but there is no constant to check this against. The following is from (a distribution of) linux's man page for sigaction describing some of the constants available on Linux: +-------------------------------------+ | SIGILL | +-----------+-------------------------+ |ILL_ILLOPC | illegal opcode | +-----------+-------------------------+ |ILL_ILLOPN | illegal operand | +-----------+-------------------------+ |ILL_ILLADR | illegal addressing mode | +-----------+-------------------------+ |ILL_ILLTRP | illegal trap | +-----------+-------------------------+ |ILL_PRVOPC | privileged opcode | +-----------+-------------------------+ |ILL_PRVREG | privileged register | +-----------+-------------------------+ |ILL_COPROC | coprocessor error | +-----------+-------------------------+ |ILL_BADSTK | internal stack error | +-----------+-------------------------+ .... +----------------------------------------------------+ | SIGSEGV | +------------+---------------------------------------+ |SEGV_MAPERR | address not mapped to object | +------------+---------------------------------------+ |SEGV_ACCERR | invalid permissions for mapped object | +------------+---------------------------------------+ .... Unfortunately I lack the expertise to implement any of these but I wanted to ask if anybody has a similar problem and/or is working on this. Thanks, Siavosh Benabbas From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 03:41:37 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from [127.0.0.1] (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 101E616A412; Thu, 18 Jan 2007 03:41:36 +0000 (UTC) (envelope-from davidxu@freebsd.org) Message-ID: <45AEEC79.3050901@freebsd.org> Date: Thu, 18 Jan 2007 11:41:45 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.13) Gecko/20061204 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Siavosh Benabbas References: <32d8477c0701171856i6f7c4cdy384ad2e10086a65f@mail.gmail.com> In-Reply-To: <32d8477c0701171856i6f7c4cdy384ad2e10086a65f@mail.gmail.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: siginfo_t.si_code macro constants X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 03:41:37 -0000 Siavosh Benabbas wrote: > Hi, > I am trying to compile parted on freebsd and I came across this. It seems > that many macro constants of POSIX that one should be able to check the > value of siginfo_t.si_code in a 3 parameter signal handler (set by > sigaction) are missing. These in particular include SEGV_MAPPER, > SEGV_ACCERR, and many ILL_* constants the man page of sigaction mentions: > > "The code argument of the BSD-style handler and the si_code member of > the > info argument to a SA_SIGINFO handler contain a numeric code explaining > the cause of the signal, usually one of the SI_... values from > or codes specific to a signal, i.e., one of the FPE_... > values for SIGFPE." > > But it doesn't specifically list the macro's available. According to the > POSIX standard there should be some SEGV_* and ILL_* constants too. I > searched in the archives and it seems that this was pointed out around a > year ago. The mail in the archive also mentioned that siginfo_t.si_code > actually gets populated but there is no constant to check this against. The > following is from (a distribution of) linux's man page for sigaction > describing some of the constants available on Linux: > > +-------------------------------------+ > | SIGILL | > +-----------+-------------------------+ > |ILL_ILLOPC | illegal opcode | > +-----------+-------------------------+ > |ILL_ILLOPN | illegal operand | > +-----------+-------------------------+ > |ILL_ILLADR | illegal addressing mode | > +-----------+-------------------------+ > |ILL_ILLTRP | illegal trap | > +-----------+-------------------------+ > |ILL_PRVOPC | privileged opcode | > +-----------+-------------------------+ > |ILL_PRVREG | privileged register | > +-----------+-------------------------+ > |ILL_COPROC | coprocessor error | > +-----------+-------------------------+ > |ILL_BADSTK | internal stack error | > +-----------+-------------------------+ > .... > +----------------------------------------------------+ > | SIGSEGV | > +------------+---------------------------------------+ > |SEGV_MAPERR | address not mapped to object | > +------------+---------------------------------------+ > |SEGV_ACCERR | invalid permissions for mapped object | > +------------+---------------------------------------+ > .... > > Unfortunately I lack the expertise to implement any of these but I > wanted to > ask if anybody has a similar problem and/or is working on this. > > Thanks, > Siavosh Benabbas These codes are defined in -HEAD branch, RELENG_6 does not have them, I had added them when I was implementing signal queue. Regards, David Xu From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 04:20:49 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F3FED16A519 for ; Thu, 18 Jan 2007 04:20:48 +0000 (UTC) (envelope-from sbenabas@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.169]) by mx1.freebsd.org (Postfix) with ESMTP id 7742213C465 for ; Thu, 18 Jan 2007 04:20:48 +0000 (UTC) (envelope-from sbenabas@gmail.com) Received: by ug-out-1314.google.com with SMTP id o2so68014uge for ; Wed, 17 Jan 2007 20:20:47 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=CdpElUZCAgmntqy0LV6K7EF+WoBwnKZbnbw70TnZz4GvLIRkXrlMj0SoQn+7PHurI935nmmobmlePN/SgWtsrqHoOh1+V5WuLmG3Apm7CKPOzsI1gXSEzsuqo7rzsPI/GVsGUcp8NOWVBb6coXh/YiCX8yzy3PgdsQNdrXeRwWs= Received: by 10.66.242.20 with SMTP id p20mr639533ugh.1169094047388; Wed, 17 Jan 2007 20:20:47 -0800 (PST) Received: by 10.67.101.20 with HTTP; Wed, 17 Jan 2007 20:20:42 -0800 (PST) Message-ID: <32d8477c0701172020v5fff119dxce8d1a20d775339d@mail.gmail.com> Date: Wed, 17 Jan 2007 23:20:42 -0500 From: "Siavosh Benabbas" To: "David Xu" In-Reply-To: <45AEEC79.3050901@freebsd.org> MIME-Version: 1.0 References: <32d8477c0701171856i6f7c4cdy384ad2e10086a65f@mail.gmail.com> <45AEEC79.3050901@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-arch@freebsd.org Subject: Re: siginfo_t.si_code macro constants X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 04:20:49 -0000 Hi, Thanks. Are there going to be backported? -- Siavosh Benabbas On 1/17/07, David Xu wrote: > > Siavosh Benabbas wrote: > > Hi, > > I am trying to compile parted on freebsd and I came across this. It > seems > > that many macro constants of POSIX that one should be able to check the > > value of siginfo_t.si_code in a 3 parameter signal handler (set by > > sigaction) are missing. These in particular include SEGV_MAPPER, > > SEGV_ACCERR, and many ILL_* constants the man page of sigaction > mentions: > > > > "The code argument of the BSD-style handler and the si_code member > of > > the > > info argument to a SA_SIGINFO handler contain a numeric code > explaining > > the cause of the signal, usually one of the SI_... values from > > or codes specific to a signal, i.e., one of the > FPE_... > > values for SIGFPE." > > > > But it doesn't specifically list the macro's available. According to the > > POSIX standard there should be some SEGV_* and ILL_* constants too. I > > searched in the archives and it seems that this was pointed out around a > > year ago. The mail in the archive also mentioned that siginfo_t.si_code > > actually gets populated but there is no constant to check this against. > The > > following is from (a distribution of) linux's man page for sigaction > > describing some of the constants available on Linux: > > > > +-------------------------------------+ > > | SIGILL | > > +-----------+-------------------------+ > > |ILL_ILLOPC | illegal opcode | > > +-----------+-------------------------+ > > |ILL_ILLOPN | illegal operand | > > +-----------+-------------------------+ > > |ILL_ILLADR | illegal addressing mode | > > +-----------+-------------------------+ > > |ILL_ILLTRP | illegal trap | > > +-----------+-------------------------+ > > |ILL_PRVOPC | privileged opcode | > > +-----------+-------------------------+ > > |ILL_PRVREG | privileged register | > > +-----------+-------------------------+ > > |ILL_COPROC | coprocessor error | > > +-----------+-------------------------+ > > |ILL_BADSTK | internal stack error | > > +-----------+-------------------------+ > > .... > > +----------------------------------------------------+ > > | SIGSEGV | > > +------------+---------------------------------------+ > > |SEGV_MAPERR | address not mapped to object | > > +------------+---------------------------------------+ > > |SEGV_ACCERR | invalid permissions for mapped object | > > +------------+---------------------------------------+ > > .... > > > > Unfortunately I lack the expertise to implement any of these but I > > wanted to > > ask if anybody has a similar problem and/or is working on this. > > > > Thanks, > > Siavosh Benabbas > > These codes are defined in -HEAD branch, RELENG_6 does not have > them, I had added them when I was implementing signal queue. > > Regards, > David Xu > > > From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 07:03:39 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5706216A415; Thu, 18 Jan 2007 07:03:39 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1-3.pacific.net.au [61.8.2.210]) by mx1.freebsd.org (Postfix) with ESMTP id 84E2D13C459; Thu, 18 Jan 2007 07:03:38 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.2.162]) by mailout1.pacific.net.au (Postfix) with ESMTP id B39865A7C93; Thu, 18 Jan 2007 18:03:30 +1100 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy1.pacific.net.au (Postfix) with ESMTP id 638EF8C22; Thu, 18 Jan 2007 18:03:26 +1100 (EST) Date: Thu, 18 Jan 2007 18:03:20 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Matthew Dillon In-Reply-To: <200701172022.l0HKMYV8053837@apollo.backplane.com> Message-ID: <20070118113831.A11834@delplex.bde.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Attilio Rao , Maxim Sobolev , freebsd-current@freebsd.org, Ivan Voras , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 07:03:39 -0000 On Wed, 17 Jan 2007, Matthew Dillon wrote: > The cost of using the FPU can simply be thought of in terms of how > many bytes you have to have to copy for it to become worth using > the FPU over a far less complex integer copy loop. It's not that simple... > ... > In fact, I would say that if userland is not using the FP unit, > that is npxthread == NULL or npxthread != curthread, you should > *DEFINITELY* use the FP unit. Hands down, no question about it. Only if the raw FPU is actually faster. > * First, raw memory bandwidth is governed by RAS cycles. The fewer RAS > cycles you have, the higher the bandwidth. > > This means that the more data you can load into the cpu on the 'read' > side of the copy before transitioning to the 'write' side, the better. > > With XMM you can load 128 *BYTES* a shot (8 128 bit registers). For > large copies, nothing beats it. No, this is very machine-dependent. Read-write transitioning is slow, but it is almost a non-problem since it can be avoided by prefetchiung the read side. Dirtying the cache with the read side is unavoidable on current i386-amd64 CPUs. Prefetching makes this a feature: prefetch N bytes, preferably using a working prefetchnta (1) write N bytes, possibly to the cache (2), possibly many less than just one XMM register's worth at a time (3) Just make N much larger than 128 for this to easily beat simply loading and storing 128 bytes at a time (4) (5). (1) If the source is already in the cache, the extra instructions for the prefetch are a bit wasteful. Using prefetchnta limits the waste, perhaps to 0 depending on whether prefetchnta can be executed in an otherwise unised pipeline. However, a working prefetchnta is required for the prefetch to actually occur in the case that the source isn't already in the cache. A64's have a working prefetchnta, but AXP's don't. (2) Reads and writes through the L1 cache give much the same benefits as reads and writes through many registers. Reads must go through the L1 cache. (3) Write combining of sequential writes makes the number of memory accesses independent of the instruction operand size, at least for cached writes. If writes are direct, then the issues are different. I've never seen 128-bit nontemporal writes go faster than 64-bit ones so maybe they are combined too. (I may have seen this on AXP while writing this.) (4) Using only registers 128 bytes at a time may beat prefetch in cases where the prefetch is too wasteful. However, I think these cases are limited ones where the copy size is small so you wouldn't want to use prefetch or nontemporal writes and any benefits from using XMM registers are marginal (because of the context switching overheads). Such cases are handled well by caching. If the cache line size is 128, then read/write through the cache gives RAS cycles that are independent of the instruction operand sizes. Using 8*16 bytes from registers can only win if the cache line size is 64. (5) A64's have 16 not-very general integer 64-bit integer registers and 16 XMM registers, so you can get close to 128 bytes using integer registers (16*8) and can get 256 bytes using XMM registers (16*16). Caching and/or write buffering turns writes of the "small" 64-bit registers or even 32-bit registers into much larger writes, so levels of the memory system below L1 don't notice if we write tiny amounts at a time provided there is enough instruction bandwidth to keep up. > * Modern cpu hardware uses a 128 bit data path for 128 bit media > instructions and can optimize the 128 bit operation all the way through > to a cache line or to main memory. It can't be beat. Actually, integer instructions beat it easily on A64: Fully cached case: % copy0: 16907105855 B/s ( 242265 us) (movsq) % ... % copyQ: 13658478027 B/s ( 299887 us) (movdqa) % ... % copyT: 16456408196 B/s ( 248900 us) (movq (i)) This doesn't contradict your claim since main memory is not really involved. Everything should be limited by instruction and cache bandwidth, but for some reason movdqa is quite slow despite or because of my attempts to schedule it perfectly. Fully uncached case: % copy0: 829571300 B/s ( 493749 us) (movsq) % ... % copyQ: 835822845 B/s ( 490056 us) (movdqa) % copyR: 908423544 B/s ( 450891 us) (movdqa with prefetchnta) % copyS: 919026496 B/s ( 445689 us) (movdqa with block prefetch) % copyT: 828993732 B/s ( 494093 us) (movq (i)) % copyU: 905141362 B/s ( 452526 us) (movq (i) with prefetchnta) % copyV: 910626945 B/s ( 449800 us) (movq (i) with block prefetch) % copyW: 1350397932 B/s ( 303318 us) (movnti) % copyX: 1662594069 B/s ( 246362 us) (movnti with prefetchnta) % copyY: 1608204355 B/s ( 254694 us) (movnti with block prefetch) Now movdqa beats movsq by a bit, at least with prefetch, but has the same speed as integer moves of half the size (or MMX moves of half the size (benchmark data not shown)). It is necessary to use movnt to get anywhere near memory bandwidth, and any form of movnt can be used for this (I test mainly movntps in addition to the above, since it is most portable). There is apparently some magic to combine 8-byte writes if necessary for efficient main memory accesses. The above was run on an old A64 overclocked a lot to 2204 MHz, with mediocre DDR-1 PC3200 memory overclocked a bit and tuned a lot. The maximum bandwidth achieved is almost exactly PC3200 but the theoretical maximum is more like PC3400. > > Alignment is critical. If the data is not aligned, don't bother. 128 > bits means 16 byte alignment. The above benchmark output is for aligned data :-). I don't try hard to optimize or benchmark misaligned cases. > * No extranious memory writes, no uncached extranious memory reads. > If you do any writes to memory other then to the copy destination > in your copy loop you screw up the cpu's write fifo and destroy > performance. > > Systems are so sensitive to this that it is even better to spend the > time linearly mapping large copy spaces into KVM and do a single > block copy then to have an inner per-PAGE loop. I haven't tried this, but have seen and partly worked sensitivity to linear KVA maps not being physically (non)linear enough. Some CPUs and/or memory systems are remarkably sensitive to bank interleave. FreeBSD's page coloring doesn't know anything about banks, and accidentally starts up with perfect miscoloring for banks. This can make a difference of 30% for bzero bandwidth in benchmarks (not so much for bcopy bandwidth, and an insignificant amount for normal use). After the system warms up, the coloring becomes random with respect to banks, and random coloring works much better than perfect miscoloring. > > * Use of prefetch or use of movntdq instead of movdqa is highly > problematic. It is possible to use these to optimize very particular > cases but the problem is they tend to nerf all OTHER cases. This is very machine-dependent: - prefetchnta seems to work fine on A64 but not on AXP or P4 (see above and other postings. I haven't tested it much for small copy sizes. I never saw older prefetch instructions that give more control doing anything useful. - movntps may be free (once you use XMM and if ps instead of dq is OK) on AXP, since it runs at cache speed if the target is already cached, It is not free on A64 since it always runs at memory speed. I think ps instead of dq is not OK since ps is slow if there are denormals in the data, but don't remember if benchmarks support this. Unfortunately, the only movnt instruction in AXP is movntps. - movnt* is a clear win in at least one case: zeroing pages in idle. FreeBSD on i386 uses movnti for all zeroing of pages in the SSE2 case. This might not be best for non-idle zeroing. Maybe it should be used only for multiple pages. For single pages, it is probably for a pagefault and you want at least the data near the fault address in the cache. FreeBSD on amd64 also uses movnti for all copying of pages. I think SunOS on amd64 uses movnt* whenever the copy size is >= 128. This seems too small. > I've given up trying to use either mechanism. Instead, I prefer > copying as large a block as possible to remove these variables from > the cpu pipeline entirely. The cpu has a write fifo anyway, you > don't need prefetch instructions if you can use instructions to write > to memory faster then available L2 cache bandwidth. On some cpus > this mandates the use of 64 or 128 bit media instructions or the > cpu can't keep the write FIFO full and starts interleaving reads > and writes on the wrong boundaries (creating more RAS cycles, which > is what kills copy bandwidth). Which modern CPUs can't keep up with L2? On amd64 for the fully-L2-cached case: % copy0: 4382675165 B/s ( 934589 us) (movsq) % ... % copyN: 3432351420 B/s (1193351 us) (movntq) % copyO: 1777008820 B/s (2304997 us) (movntq with prefetchnta) % copyP: 3278961491 B/s (1249176 us) (movntq with block prefetch) % copyQ: 4369052686 B/s ( 937503 us) (movdqa) % copyR: 2785886655 B/s (1470268 us) (movdqa with prefetchnta) % copyS: 4553271385 B/s ( 899573 us) (movdqa with block prefetch) % copyT: 4401466152 B/s ( 930599 us) (movq (i)) % copyU: 2817507895 B/s (1453767 us) (movq (i) with prefetchnta) % copyV: 4523562615 B/s ( 905481 us) (movq (i) with block prefetch) % copyW: 3432429080 B/s (1193324 us) (movnti) % copyX: 1776507852 B/s (2305647 us) (movnti with prefetchnta) % copyY: 3136767185 B/s (1305803 us) (movnti with block prefetch) So here is a case where prefetchnta is bad. OTOH, movnti is quite fast provided prefetchnta is not used with it. movnti is even more competitive with L2 if main memory is DDR-2. > * RAS transitions also have to be aligned or you get boundary cases > when the memory address transitions a RAS line. This again mandates > maximal alignment (even more then 16 bytes, frankly, which is why > being able to do 128 byte blocks with XMM registers is so nice). > Even though reads and writes are reblocked to the cache line size > by the cpu, your inner loop can still transition a RAS boundary in > the middle of a large block read if it isn't aligned. > > But at this point the alignment requirements start to get kinda > silly. 128 byte alignment requirement? I don't think so. I > do a 16-byte alignment check in DragonFly as a pre-req for using > XMM and that's it. :-). There are just too many combinations to even think about. I prefer to reduce the number of cases by depending on the prefetching working reasonably well. > I don't know about AMD64. You only have 64 bit general registers in 64 > bit mode so you may not be able to keep the write pipeline full. But > you do have 8 of them so you are roughly equivalent to MMX (but not 16, almost > XMM's 8 128 bit registers). On A64 and AXP, most things are limited by the load/store unit (LSU), the 3 integer pipelines, and SSE latency (but latency is not a problem here). Load/store-unit: On A64, the LSU can do the following number of interesting L1 cache accesses per cycle: 2 64-bit loads 2 64-bit stores 1 64-bit load and 1 64-bit store all to the L1 cache. When you load or store a 128-bit XMM register, it takes the same load/store-unit resources as 2 64-bit integer loads or stores of the same type. Thus XMM and integer operations can both achieve the maximum L1 cache bandwidth, but no more of course, so XMM vs integer is irrelevant for bandwidth considerations for the L1 level, and since lower levels need less bandwidth, it is irrelevant for all levels except possibly the non-cache-level given by nontemporal stores. On AXP, the LSU can do the following number of interesting L1 cache accesses per cycle: 2 64-bit loads 2 32-bit store 1 64-bit store 1 64-bit load and 1 64-bit store Since there are no 64-bit integer registers, you have to use MMX or XMM to get the 64-bit accesses. It doesn't matter whether you use 2*MMX or 1*XMM to get to 128 bits -- both become 2*64 bits. 32-bit integer stores can only use half of the L1 cache bandwidth. 32-bit integer loads are also limited to 2 per cycle since they apparently aren't combined at this level. Combining also doesn't happen for movsl (this is apparently implemented as a 32-bit load/store). On A64 in 32-bit mode, the LSU can do 2 64-bit stores/cycle, but this only helps using XMM registers -- 32-bit accesses apparently never get combined at this level, so they always go at half speed. The asymmetric load/store capabilities on the AXP can't be good for using 128-bit XMM registers, especially using 8 loads followed by 8 stores in the instruction stream. They make 128 stores want to take twice as long as 2 64-bits ones, and it takes a large amount of out of order execution to reorder things to reach the "1 64-bit load and 1 64-bit store" per cycle pattern. Reordering is exactly what you don't want for controlling the interleaving of access to main memory. Even with a simple load-store- load-store with 64-bit accesses in the instruction, stream, the there will be some reordering -- the reads will get 1 ahead of the writes. This explains why my movdqa benchmark is slow on AXP but not why it is slow on A64. Benchmark output on AXP, fully cached case: % copy0: 5275909727 B/s ( 776359 us) (1417037051 tsc) (movsl) % ... % copyD: 6301722666 B/s ( 649981 us) (1175886987 tsc) (kernel bcopy (unroll 64 fp i-prefetch)) % copyH: 4336676894 B/s ( 944502 us) (1708211441 tsc) (movntps) % ... % copyK: 8750902650 B/s ( 468066 us) (846786401 tsc) (movq) % ... % copyN: 3029184746 B/s (1352179 us) (2445280543 tsc) (movntq) % ... % copyQ: 5487174266 B/s ( 746468 us) (1350423254 tsc) (movdqa) This shows: - movsl is quite slow - even the old kernel bcopy is barely worth using on AXP - movntps isn't too bad in the fully cached case (AXP implemenmtation detail) - movq (through 64-bit MMX) is quite fast, but not quite twice as fast as movsl. I thought that mainly instruction execution bandwidth limits prevented it being twice as fast, but maybe it is affected by the asymmetry. It does load-load-store-store repeated through 4 pairs of different MMX registers. I think it was written before I knoew anything about the CPU resource issues (I keep some mistakes for comparison). Now I think using different registers is unnecessary, and the limit of 8 or 16 MMX or 16 registers need not be hit. load-store repeated to the same register, or load-store-load-store repeated actually uses a large machine-dependent number of registers via register renaming. Even old AXP's have this. I think the register renaming works across loop boundaries and gives you many more than the 8 registers that you might think you are using. This goes with out of order execution and might be controllable only in the same way, by stalling at page boundaries. - movntq is much slower than movntps. movntps does 4 128-bit loads followed by 4 128-bit stores. movntq does 8 64-bit loads followed by 8 64-bit stores. This seems to show that 128-bit accesses are faster. - movdqa is quite slow. It uses load-store... of a single 128-bit register. Perhaps it is too optimized away from using multiple registers. In fact, rewriting it to use 8 loads followed by 8 stores shows one case where it beats movnti (64 bits) -- in the fullyL2-cached case, this speeds it up from 4.37GB/S to 4.77GB/S, making it about 0.4GB/S faster than both the other version of itself and movnti. But in the fully-L1-cached case, this speeds it down from 13.7GB/S to 9.2GB/S, making it much slower than itself and almost twice as slow as movsq. This is hard to explain, especially for the L1 case. In the L2 case, it might be explained by the 8 stores followed by 8 loads not being reordered very much so that they work almost like you want. Integer pipeline resources: Both AXP and A64 have 3 integer pipelines (they are remarkably similer in this and many other respects). It's easy to do a load/store in parallel using 2 of the pipelines. This saturates the LSU (loads must be at least 1 in advance of stores). The load/store can also be from FPU/SSE pipelines, but not independently, and on AXP the integer pipelines lose LSU bandwidth since they can only do 32-bit accesses. Other integer operations can run in the 3rd pipeline, but there are many restrictions which I don't quite understand or remember. These result in it being hard to keep the 3rd pipeline full. The above movq (i) benchmark for A64 indicates that it may be being used to hide all the loop control overhead -- I only unrolled the loop to 64 bytes, and that gives almost full speed. The loop must be unrolled a bit since a single load-store runs too fast for loop control. For the movnti benchmarks, the load-store runs slower than the loop control so unrolling is not needed. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 07:59:07 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from [127.0.0.1] (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id BD18A16A407; Thu, 18 Jan 2007 07:59:01 +0000 (UTC) (envelope-from davidxu@freebsd.org) Message-ID: <45AF28CE.4000101@freebsd.org> Date: Thu, 18 Jan 2007 15:59:10 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.13) Gecko/20061204 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Siavosh Benabbas References: <32d8477c0701171856i6f7c4cdy384ad2e10086a65f@mail.gmail.com> <45AEEC79.3050901@freebsd.org> <32d8477c0701172020v5fff119dxce8d1a20d775339d@mail.gmail.com> In-Reply-To: <32d8477c0701172020v5fff119dxce8d1a20d775339d@mail.gmail.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: siginfo_t.si_code macro constants X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 07:59:07 -0000 Siavosh Benabbas wrote: > Hi, > Thanks. > Are there going to be backported? > -- > Siavosh Benabbas No. ;-) Regards, David Xu From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 08:04:28 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DEFDA16A412; Thu, 18 Jan 2007 08:04:28 +0000 (UTC) (envelope-from ssouhlal@FreeBSD.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id C30D113C45A; Thu, 18 Jan 2007 08:04:28 +0000 (UTC) (envelope-from ssouhlal@FreeBSD.org) Received: from [192.168.0.100] (c-67-188-127-3.hsd1.ca.comcast.net [67.188.127.3]) by elvis.mu.org (Postfix) with ESMTP id EA1D71A3C19; Wed, 17 Jan 2007 23:39:10 -0800 (PST) Message-ID: <45AF23DE.6030801@FreeBSD.org> Date: Wed, 17 Jan 2007 23:38:06 -0800 From: Suleiman Souhlal User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051204) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bruce Evans References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> In-Reply-To: <20070118113831.A11834@delplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Maxim Sobolev , Ivan Voras , Attilio Rao , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 08:04:29 -0000 Bruce Evans wrote: > On Wed, 17 Jan 2007, Matthew Dillon wrote: >> * No extranious memory writes, no uncached extranious memory reads. >> If you do any writes to memory other then to the copy destination >> in your copy loop you screw up the cpu's write fifo and destroy >> performance. >> >> Systems are so sensitive to this that it is even better to spend the >> time linearly mapping large copy spaces into KVM and do a single >> block copy then to have an inner per-PAGE loop. > > > I haven't tried this, but have seen and partly worked sensitivity to > linear KVA maps not being physically (non)linear enough. Some CPUs > and/or memory systems are remarkably sensitive to bank interleave. > FreeBSD's page coloring doesn't know anything about banks, and > accidentally starts up with perfect miscoloring for banks. This can > make a difference of 30% for bzero bandwidth in benchmarks (not so > much for bcopy bandwidth, and an insignificant amount for normal use). > After the system warms up, the coloring becomes random with respect > to banks, and random coloring works much better than perfect miscoloring. About page coloring: Don't amd64 CPUs have virtually indexed, physically tagged caches? If so, wouldn't it make sense to turn off page coloring, since it's useless for virtually indexed caches (and probably hurts things a bit)? -- Suleiman From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 10:01:46 2007 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1378816A494; Thu, 18 Jan 2007 10:01:46 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1-3.pacific.net.au [61.8.2.210]) by mx1.freebsd.org (Postfix) with ESMTP id C5B3913C4BD; Thu, 18 Jan 2007 10:01:45 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.2.162]) by mailout1.pacific.net.au (Postfix) with ESMTP id 6A13C5AFC73; Thu, 18 Jan 2007 21:01:44 +1100 (EST) Received: from besplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailproxy1.pacific.net.au (Postfix) with ESMTP id CFF998C0B; Thu, 18 Jan 2007 21:01:42 +1100 (EST) Date: Thu, 18 Jan 2007 21:01:41 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Suleiman Souhlal In-Reply-To: <45AF23DE.6030801@FreeBSD.org> Message-ID: <20070118204641.M3613@besplex.bde.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> <45AF23DE.6030801@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Maxim Sobolev , Ivan Voras , Attilio Rao , freebsd-current@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 10:01:46 -0000 On Wed, 17 Jan 2007, Suleiman Souhlal wrote: > Bruce Evans wrote: >> I haven't tried this, but have seen and partly worked sensitivity to >> linear KVA maps not being physically (non)linear enough. Some CPUs >> and/or memory systems are remarkably sensitive to bank interleave. >> FreeBSD's page coloring doesn't know anything about banks, ... > > About page coloring: Don't amd64 CPUs have virtually indexed, physically > tagged caches? If so, wouldn't it make sense to turn off page coloring, > since it's useless for virtually indexed caches (and probably hurts things > a bit)? I think superpages will soon turn off coloring. Not sure if this is right. How is copying and zeroing a whole superpage at a time avoided? IIRC, my benchmarks (and LMbench) show some small positive benefits for page coloring on amd64. I don't remember exactly what they are, just that they are smaller than on AthlonXP. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 14:01:21 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7D31C16A40F for ; Thu, 18 Jan 2007 14:01:21 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.152]) by mx1.freebsd.org (Postfix) with ESMTP id 645D813C467 for ; Thu, 18 Jan 2007 14:01:21 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from c-66-31-35-94.hsd1.ma.comcast.net ([66.31.35.94]) by comcast.net (rwcrmhc12) with ESMTP id <20070118134936m12002gpk9e>; Thu, 18 Jan 2007 13:49:37 +0000 Received: from c-66-31-35-94.hsd1.ma.comcast.net (localhost.crodrigues.org [127.0.0.1]) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8) with ESMTP id l0IDnald007431; Thu, 18 Jan 2007 08:49:36 -0500 (EST) (envelope-from rodrigc@c-66-31-35-94.hsd1.ma.comcast.net) Received: (from rodrigc@localhost) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8/Submit) id l0IDna8w007430; Thu, 18 Jan 2007 08:49:36 -0500 (EST) (envelope-from rodrigc) Date: Thu, 18 Jan 2007 08:49:36 -0500 From: Craig Rodrigues To: freebsd-current@freebsd.org Message-ID: <20070118134936.GA7391@crodrigues.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline User-Agent: Mutt/1.4.2.1i Cc: freebsd-arch@freebsd.org Subject: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 14:01:21 -0000 --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, This is a different version of the patch I posted here: http://lists.freebsd.org/pipermail/freebsd-arch/2006-July/005439.html One of the pet peeves I have with FreeBSD is that if I have a device with a local filesystem that I want to mount, I need to explicitly know what type of filesystem is on the device in order to mount it from the command-line. For example, mount -t cd9660 mount -t udf mount -t ext2fs mount -t msdosfs Where this is particularly annoying is if I have multiple USB thumb drives with different filesystems on them. What I usually end up doing is something like: file - < /dev/ad0s4 to figure out the filesystem type, and then mount -t [whatever] to mount it. What I would like to do is: mount /dev/ad0s4 /mnt and if I do not specify a filesystem type with -t, the mount program should "magically" figure out how to mount the disk. This is closer to how the mount program behaves on Linux for example. In this patch, I only modified the userland mount program. If the user does not specify "-t vfstype" to mount, the mount program gets a list of local filesystems from the vfs.conflist sysctl. It then tries to mount the filesystem, always starting with "ufs", and then iterating through the list if the nmount() fails with EINVAL. Using this patch, I have been able to do: mount /dev/blah /mnt and mount a UFS, cd9660, or FAT filesystem, depending on what filesystem is on the /dev/blah device. Comments? -- Craig Rodrigues rodrigc@crodrigues.org --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mount_patch.txt" Index: mount.c =================================================================== RCS file: /home/ncvs/src/sbin/mount/mount.c,v retrieving revision 1.92 diff -u -u -r1.92 mount.c --- mount.c 14 Nov 2006 01:07:42 -0000 1.92 +++ mount.c 18 Jan 2007 13:41:53 -0000 @@ -139,6 +139,9 @@ NULL }; + if (vfstype == NULL) + return (0); + for (i = 0; fs[i] != NULL; ++i) { if (strcmp(vfstype, fs[i]) == 0) return (1); @@ -219,7 +222,7 @@ ro = 0; options = NULL; vfslist = NULL; - vfstype = "ufs"; + vfstype = NULL; while ((ch = getopt(argc, argv, "adF:flo:prt:uvw")) != -1) switch (ch) { case 'a': @@ -516,7 +519,7 @@ optbuf = catopt(optbuf, "update"); /* Compatibility glue. */ - if (strcmp(vfstype, "msdos") == 0) + if (vfstype != NULL && strcmp(vfstype, "msdos") == 0) vfstype = "msdosfs"; /* Construct the name of the appropriate mount command */ @@ -532,8 +535,11 @@ if (debug) { if (use_mountprog(vfstype)) printf("exec: mount_%s", vfstype); - else - printf("mount -t %s", vfstype); + else { + printf("mount "); + if (vfstype != NULL) + printf("-t %s", vfstype); + }; for (i = 1; i < argc; i++) (void)printf(" %s", argv[i]); (void)printf("\n"); Index: mount_fs.c =================================================================== RCS file: /home/ncvs/src/sbin/mount/mount_fs.c,v retrieving revision 1.3 diff -u -u -r1.3 mount_fs.c --- mount_fs.c 7 Dec 2006 03:24:43 -0000 1.3 +++ mount_fs.c 18 Jan 2007 13:41:53 -0000 @@ -50,8 +50,10 @@ #include #include +#include #include +#include #include #include #include @@ -62,6 +64,8 @@ #include "extern.h" #include "mntopts.h" +extern int debug, verbose; + struct mntopt mopts[] = { MOPT_STDOPTS, MOPT_END @@ -75,8 +79,9 @@ exit(1); } -int -mount_fs(const char *vfstype, int argc, char *argv[]) +static int +do_mount_fs(const char *vfstype, int argc, char *argv[], int *nmount_errno, + char *msg, size_t msg_len) { struct iovec *iov; int iovlen; @@ -131,8 +136,83 @@ build_iovec(&iov, &iovlen, "errmsg", errmsg, sizeof(errmsg)); ret = nmount(iov, iovlen, mntflags); + if (nmount_errno != NULL) + *nmount_errno = errno; if (ret < 0) - err(1, "%s %s", dev, errmsg); + snprintf(msg, msg_len, "%s : %s : %s", dev, errmsg, + strerror(errno)); + + return (ret); +} + +static int +mount_fs_try(int argc, char *argv[]) +{ + struct xvfsconf *vfsp; + char msg[512]; + unsigned int cnt, i; + int ret, j, nmount_errno; + size_t buflen; + + /* First, see if mounting mounting with vfstype "ufs" works. */ + ret = do_mount_fs("ufs", argc, argv, &nmount_errno, msg, sizeof(msg)); + if (ret == 0) + return (0); + + /* Get a list of registered vfs types */ + if (sysctlbyname("vfs.conflist", NULL, &buflen, NULL, 0) < 0) + err(1, "sysctl(vfs.conflist)"); + vfsp = malloc(buflen); + if (vfsp == NULL) + errx(1, "malloc failed"); + if (sysctlbyname("vfs.conflist", vfsp, &buflen, NULL, 0) < 0) + err(1, "sysctl(vfs.conflist)"); + cnt = buflen / sizeof(struct xvfsconf); + + for (i = 0; i < cnt; i++) { + /* We already tried "ufs" above, so skip it. */ + if (strcmp("ufs", vfsp[i].vfc_name) == 0) + continue; + /* + * Only try local filesystems. Skip network, synthetic, + * and loopback filesystems. + */ + if (vfsp[i].vfc_flags & + (VFCF_NETWORK | VFCF_SYNTHETIC | VFCF_LOOPBACK)) + continue; + + if (debug || verbose) { + printf("mount -t %s ", vfsp[i].vfc_name); + for (j = 1; j < argc; j++) + (void)printf(" %s", argv[j]); + (void)printf("\n"); + } + ret = do_mount_fs(vfsp[i].vfc_name, argc, argv, &nmount_errno, + msg, sizeof(msg)); + if (ret == 0 || nmount_errno != EINVAL) + break; + } + free(vfsp); + + if (ret < 0) + printf("%s", msg); + return (ret); +} + +int +mount_fs(const char *vfstype, int argc, char *argv[]) +{ + char msg[512]; + int ret, nmount_errno; + + if (vfstype != NULL) { + ret = do_mount_fs(vfstype, argc, argv, &nmount_errno, msg, + sizeof(msg)); + if (ret < 0) + printf("%s", msg); + } + else + ret = mount_fs_try(argc, argv); return (ret); } --W/nzBZO5zC0uMSeA-- From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 14:40:05 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0E73016A49E for ; Thu, 18 Jan 2007 14:40:04 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.234]) by mx1.freebsd.org (Postfix) with ESMTP id 8951713C469 for ; Thu, 18 Jan 2007 14:40:04 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: by wr-out-0506.google.com with SMTP id 68so81072wri for ; Thu, 18 Jan 2007 06:40:03 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=CM1ZIaHEYDletkS0zFuIe+I8fm0u9eDLBCxKg8jYWWaOvSKfhUt7k9ZGxhs6swPi3MEndBljtQoGf32EPB0e7dD1g3Om439HGVLSfb/vpujdjDgzOdVBLzM6vemZx3Wj+dUp8uyNSdT7ch1w+0oFUNDkkst99dzd7jawPH1UXrA= Received: by 10.78.123.4 with SMTP id v4mr839939huc.1169129689470; Thu, 18 Jan 2007 06:14:49 -0800 (PST) Received: by 10.78.164.20 with HTTP; Thu, 18 Jan 2007 06:14:49 -0800 (PST) Message-ID: Date: Thu, 18 Jan 2007 17:14:49 +0300 From: "Andrew Pantyukhin" Sender: infofarmer@gmail.com To: "Craig Rodrigues" In-Reply-To: <20070118134936.GA7391@crodrigues.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070118134936.GA7391@crodrigues.org> X-Google-Sender-Auth: 5fe0465356a9b5cb Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 14:40:05 -0000 On 1/18/07, Craig Rodrigues wrote: > Hi, > > This is a different version of the patch I posted here: > http://lists.freebsd.org/pipermail/freebsd-arch/2006-July/005439.html > > One of the pet peeves I have with FreeBSD is that > if I have a device with a local filesystem that I want to mount, > I need to explicitly know what type of filesystem is on the > device in order to mount it from the command-line. > > For example, > > mount -t cd9660 > mount -t udf > mount -t ext2fs > mount -t msdosfs > > Where this is particularly annoying is if I have multiple > USB thumb drives with different filesystems on them. > > What I usually end up doing is something like: > file - < /dev/ad0s4 > > to figure out the filesystem type, and then mount -t [whatever] to mount it. > > What I would like to do is: > > mount /dev/ad0s4 /mnt > > and if I do not specify a filesystem type with -t, the mount > program should "magically" figure out how to mount the disk. > This is closer to how the mount program behaves on Linux for example. > > In this patch, I only modified the userland mount program. > If the user does not specify "-t vfstype" to mount, > the mount program gets a list of local filesystems from the vfs.conflist > sysctl. It then tries to mount the filesystem, always > starting with "ufs", and then iterating through the list if > the nmount() fails with EINVAL. > > Using this patch, I have been able to do: > mount /dev/blah /mnt > > and mount a UFS, cd9660, or FAT filesystem, depending on what filesystem > is on the /dev/blah device. > > Comments? I would love to have this usability enhancement around! 1. Are there any (what are the) security implications? 2. "mount -t auto" might be closer to POLA I might have other comments later :) Thanks Craig! From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 15:14:32 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3A22216A407; Thu, 18 Jan 2007 15:14:32 +0000 (UTC) (envelope-from allbery@ece.cmu.edu) Received: from bache.ece.cmu.edu (BACHE.ECE.CMU.EDU [128.2.129.23]) by mx1.freebsd.org (Postfix) with ESMTP id 1100313C457; Thu, 18 Jan 2007 15:14:32 +0000 (UTC) (envelope-from allbery@ece.cmu.edu) Received: by bache.ece.cmu.edu (Postfix, from userid 953) id 4608C91; Thu, 18 Jan 2007 09:49:22 -0500 (EST) X-Spam-Checker-Version: SpamAssassin 3.1.4 (2006-07-25) on filt2.ece.cmu.edu X-Spam-Level: X-Spam-Status: No, score=0.0 required=6.0 tests=BAYES_50 autolearn=no version=3.1.4 Received: from [10.9.204.128] (dsl093-061-215.pit1.dsl.speakeasy.net [66.93.61.215]) by bache.ece.cmu.edu (Postfix) with ESMTP id 4F71B8E; Thu, 18 Jan 2007 09:49:21 -0500 (EST) In-Reply-To: References: <20070118134936.GA7391@crodrigues.org> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: "Brandon S. Allbery KF8NH" Date: Thu, 18 Jan 2007 09:49:19 -0500 To: "Andrew Pantyukhin" X-Mailer: Apple Mail (2.752.2) Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 15:14:32 -0000 On Jan 18, 2007, at 9:14 , Andrew Pantyukhin wrote: > On 1/18/07, Craig Rodrigues wrote: >> In this patch, I only modified the userland mount program. >> If the user does not specify "-t vfstype" to mount, >> the mount program gets a list of local filesystems from the >> vfs.conflist >> sysctl. It then tries to mount the filesystem, always >> starting with "ufs", and then iterating through the list if >> the nmount() fails with EINVAL. > > I would love to have this usability enhancement around! > > 1. Are there any (what are the) security implications? > 2. "mount -t auto" might be closer to POLA ISTR from back when Linux added this that the bigger problem is that the msdos filesystem may sometimes accept filesystems it shouldn't, because the only way to catch an invalid filesystem is heuristics. -- brandon s. allbery [linux,solaris,freebsd,perl] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 15:17:46 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8FF4716A416 for ; Thu, 18 Jan 2007 15:17:46 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.freebsd.org (Postfix) with ESMTP id 7AA6A13C442 for ; Thu, 18 Jan 2007 15:17:46 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.13.6) with ESMTP id l0IEnCAD039877; Thu, 18 Jan 2007 06:49:13 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id l0IEnCDM039876; Thu, 18 Jan 2007 06:49:12 -0800 (PST) (envelope-from rizzo) Date: Thu, 18 Jan 2007 06:49:12 -0800 From: Luigi Rizzo To: Andrew Pantyukhin Message-ID: <20070118064912.A39777@xorpc.icir.org> References: <20070118134936.GA7391@crodrigues.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from infofarmer@freebsd.org on Thu, Jan 18, 2007 at 05:14:49PM +0300 Cc: Craig Rodrigues , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 15:17:46 -0000 On Thu, Jan 18, 2007 at 05:14:49PM +0300, Andrew Pantyukhin wrote: > On 1/18/07, Craig Rodrigues wrote: ... > > One of the pet peeves I have with FreeBSD is that > > if I have a device with a local filesystem that I want to mount, > > I need to explicitly know what type of filesystem is on the > > device in order to mount it from the command-line. ... > > Where this is particularly annoying is if I have multiple > > USB thumb drives with different filesystems on them. ... > > What I would like to do is: > > > > mount /dev/ad0s4 /mnt > > > > and if I do not specify a filesystem type with -t, the mount > > program should "magically" figure out how to mount the disk. > > This is closer to how the mount program behaves on Linux for example. > > > > In this patch, I only modified the userland mount program. ... > 2. "mount -t auto" might be closer to POLA great feature. I probably agree that "mount -t auto" might be a safer way to implement it, but other than that i'd love to have it too. cheers luigi From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 15:48:35 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 170AC16A417; Thu, 18 Jan 2007 15:48:35 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id CC1A513C442; Thu, 18 Jan 2007 15:48:34 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id AD56A2089; Thu, 18 Jan 2007 16:13:44 +0100 (CET) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on tim.des.no Received: from dwp.des.no (des.no [80.203.243.180]) by tim.des.no (Postfix) with ESMTP id 3205B2087; Thu, 18 Jan 2007 16:13:44 +0100 (CET) Received: by dwp.des.no (Postfix, from userid 1001) id 9C047B86E; Thu, 18 Jan 2007 16:15:00 +0100 (CET) From: des@des.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) To: Craig Rodrigues References: <20070118134936.GA7391@crodrigues.org> Date: Thu, 18 Jan 2007 16:15:00 +0100 In-Reply-To: <20070118134936.GA7391@crodrigues.org> (Craig Rodrigues's message of "Thu, 18 Jan 2007 08:49:36 -0500") Message-ID: <86lkk02wp7.fsf@dwp.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 15:48:35 -0000 Craig Rodrigues writes: > This is a different version of the patch I posted here: > http://lists.freebsd.org/pipermail/freebsd-arch/2006-July/005439.html Didn't somebody mention ordering issues back then? Your revised patch still doesn't even try to address those. If we ever get ext3 support, for instance, mounting an ext2 file system as ext3 might cause it to be silently upgraded to ext3. Not to mention the slew of error messages this will result in as you walk the list of filesystems trying to find the right one... A better way would be to have a userland utility attempt to identify the file system, and if necessary load the appropriate module, before mounting it. DES --=20 Dag-Erling Sm=F8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 20:06:51 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B8B3C16A492; Thu, 18 Jan 2007 20:06:51 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 6824113C478; Thu, 18 Jan 2007 20:06:51 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.13.7/8.13.7) with ESMTP id l0IJmebi061674; Thu, 18 Jan 2007 11:51:41 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.13.7/8.13.4/Submit) id l0IJmdfn061671; Thu, 18 Jan 2007 11:48:39 -0800 (PST) Date: Thu, 18 Jan 2007 11:48:39 -0800 (PST) From: Matthew Dillon Message-Id: <200701181948.l0IJmdfn061671@apollo.backplane.com> To: Bruce Evans References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> Cc: Attilio Rao , Maxim Sobolev , freebsd-current@freebsd.org, Ivan Voras , freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 20:06:51 -0000 :Fully cached case: :% copy0: 16907105855 B/s ( 242265 us) (movsq) :% ... :% copyQ: 13658478027 B/s ( 299887 us) (movdqa) :% ... :% copyT: 16456408196 B/s ( 248900 us) (movq (i)) : :This doesn't contradict your claim since main memory is not really involved. :Everything should be limited by instruction and cache bandwidth, but for :some reason movdqa is quite slow despite or because of my attempts to :schedule it perfectly. Well, that's one of the problems. Actually two of the problems. First, prefetching tends to add more confusion then it solves when used in a copy loop, because for all intents and purposes you are already keeping the memory pipeline full simply by the fact that the writes are buffered. By the time the loop gets back around to the read it has to stall anyway because the writes are still being pushed out to memory (read-before-write ordering doesn't help because all that accomplishes is to force the write buffer to stay perpetually full (possibly in a non-optimal fashion), and the cpu stalls anyway. I've tried every combination of prefetching and non-temporal instructions imagineable and it has reduced performance more often then it has improved it. I can write an optimal copy loop for particular situations but the problem is that the same loop becomes extremely inefficient under any *OTHER* conditions. The behavior of any given copy algorithm is radically different if either the source or target are present in the L1 or L2 caches, verses if they aren't. The algorithm one uses to do tiny mostly in-cache copies is totally different from the algorithm one uses to do large bulk copies. The algorithm one uses where the source data is partially or fully cached is totally different from the one used if the target data is partially or fully cached, or if neither is cached, or if both are cached. :Fully uncached case: :% copy0: 829571300 B/s ( 493749 us) (movsq) :% ... :% copyQ: 835822845 B/s ( 490056 us) (movdqa) :% copyR: 908423544 B/s ( 450891 us) (movdqa with prefetchnta) :% copyS: 919026496 B/s ( 445689 us) (movdqa with block prefetch) :% copyT: 828993732 B/s ( 494093 us) (movq (i)) :% copyU: 905141362 B/s ( 452526 us) (movq (i) with prefetchnta) :% copyV: 910626945 B/s ( 449800 us) (movq (i) with block prefetch) :% copyW: 1350397932 B/s ( 303318 us) (movnti) :% copyX: 1662594069 B/s ( 246362 us) (movnti with prefetchnta) :% copyY: 1608204355 B/s ( 254694 us) (movnti with block prefetch) : :Now movdqa beats movsq by a bit, at least with prefetch, but has the :same speed as integer moves of half the size (or MMX moves of half the :size (benchmark data not shown)). It is necessary to use movnt to get :anywhere near memory bandwidth, and any form of movnt can be used for :this (I test mainly movntps in addition to the above, since it is most :portable). There is apparently some magic to combine 8-byte writes :if necessary for efficient main memory accesses. The above was run :on an old A64 overclocked a lot to 2204 MHz, with mediocre DDR-1 PC3200 :memory overclocked a bit and tuned a lot. The maximum bandwidth :achieved is almost exactly PC3200 but the theoretical maximum is more :like PC3400. But you can't safely use *ANY* non-temporal instruction unless you know, specifically, that the data is uncached. If you use a non-temporal instruction in a situation where the data is cached or can be easily cached, you destroy the performance for that case by causing the cpu not to cache data that it really should cache. Use of non-temporal instructions results in a non-self-healing algorithm... it is possible to get into a situation the is permanently non-optimal. That's why I don't use those instructions. Sometimes I wish the non-temporal instructions had a random component. That is, that they randomly operates as normal moves instead of nt moves for a small percentage of executions (~1-25%) in order to create a self-healing (performance wise) algorithm. :> :> Alignment is critical. If the data is not aligned, don't bother. 128 :> bits means 16 byte alignment. : :The above benchmark output is for aligned data :-). I don't try hard to :optimize or benchmark misaligned cases. Neither do I. I expect if the program wants to copy a large amount of data that the programmer is smart enough to align it. :... :- movnt* is a clear win in at least one case: zeroing pages in idle. : FreeBSD on i386 uses movnti for all zeroing of pages in the SSE2 case. : This might not be best for non-idle zeroing. Maybe it should be used : only for multiple pages. For single pages, it is probably for a : pagefault and you want at least the data near the fault address in : the cache. FreeBSD on amd64 also uses movnti for all copying of pages. Use of non-temporal instructions for non performance-critical operations is a clear win when executed from the idle loop, I agree. Use of NT instructions in any other case is difficult because it is virtually impossible to predict the conditions under which other cases occur. :I think SunOS on amd64 uses movnt* whenever the copy size is >= 128. This :seems too small. : :> I've given up trying to use either mechanism. Instead, I prefer :> copying as large a block as possible to remove these variables from :> the cpu pipeline entirely. The cpu has a write fifo anyway, you :> don't need prefetch instructions if you can use instructions to write :> to memory faster then available L2 cache bandwidth. On some cpus :> this mandates the use of 64 or 128 bit media instructions or the :> cpu can't keep the write FIFO full and starts interleaving reads :> and writes on the wrong boundaries (creating more RAS cycles, which :> is what kills copy bandwidth). : :Which modern CPUs can't keep up with L2? The instruction pipeline is the problem, not the L2 cache bandwidth. Things get a bit weird when every single instruction is reading or writing memory. I try to design algorithms that work reasonably well across a large swath of cpus. It turns out that algorithms which do block loads followed by block stores tend to maintain consistent and good performance across a large swath of cpus. I outline the reasons why later on. :On amd64 for the fully-L2-cached case: :% copy0: 4382675165 B/s ( 934589 us) (movsq) :% ... :% copyN: 3432351420 B/s (1193351 us) (movntq) :% copyO: 1777008820 B/s (2304997 us) (movntq with prefetchnta) :% copyP: 3278961491 B/s (1249176 us) (movntq with block prefetch) :% copyQ: 4369052686 B/s ( 937503 us) (movdqa) :% copyR: 2785886655 B/s (1470268 us) (movdqa with prefetchnta) :% copyS: 4553271385 B/s ( 899573 us) (movdqa with block prefetch) :% copyT: 4401466152 B/s ( 930599 us) (movq (i)) :% copyU: 2817507895 B/s (1453767 us) (movq (i) with prefetchnta) :% copyV: 4523562615 B/s ( 905481 us) (movq (i) with block prefetch) :% copyW: 3432429080 B/s (1193324 us) (movnti) :% copyX: 1776507852 B/s (2305647 us) (movnti with prefetchnta) :% copyY: 3136767185 B/s (1305803 us) (movnti with block prefetch) : :So here is a case where prefetchnta is bad. OTOH, movnti is quite fast :provided prefetchnta is not used with it. movnti is even more competitive :with L2 if main memory is DDR-2. It also depends *WHERE* in the loop you put the prefetchnta. I was able to get fairly close to block prefetching speeds using prefetchnta by carefully locating the instruction and adjusting the read-ahead point. For example, by locating the instruction before the write side and indexing the prefetched memory address 128 bytes ahead of the curve. The placement of the instruction is sensitive down to the instruction slot... moving it one forward or one back can result in crazy differences in performance (due to forced RAS cycles I believe, in particular with prefetchnta). So two variables.. how far ahead you try to prefetch (which depends heavily on the memory architecture, banking, instruction pipeline size, etc), and where you locate the instruction. Two impossible variables. I could optimize a particular case, but wound up nerfing every *OTHER* case. I finally gave up. :... :> But at this point the alignment requirements start to get kinda :> silly. 128 byte alignment requirement? I don't think so. I :> do a 16-byte alignment check in DragonFly as a pre-req for using :> XMM and that's it. : ::-). There are just too many combinations to even think about. I prefer :to reduce the number of cases by depending on the prefetching working :reasonably well. Yup. :> I don't know about AMD64. You only have 64 bit general registers in 64 :> bit mode so you may not be able to keep the write pipeline full. But :> you do have 8 of them so you are roughly equivalent to MMX (but not : 16, almost :> XMM's 8 128 bit registers). : :On A64 and AXP, most things are limited by the load/store unit (LSU), :the 3 integer pipelines, and SSE latency (but latency is not a problem :here). : :Load/store-unit: : : (lots of good information about load-store units) Well, but these are for very specific flavor-of-the-day hardware tweaks. The designers change these parameters literally every other month. All that can really be said here is: Avoid 32 bit accesses if you can, 64 bit accesses seem to be the most dependable, and 128 bit accesses are the up-and-coming thing. In particular I remember reading that AMD had been spending a lot of time optimizing XMM register loads and stores. I presume that Intel is doing the same thing since both are going face forward into gaming. I'm not surprised at all about movs* performance. That instruction was originally microcoded. AMD tried harder to optimize it then Intel, but it still doesn't work very well simply due to instruction pipeline constraints. The hardware goes nuts trying to optimize movs*. :The asymmetric load/store capabilities on the AXP can't be good for using :128-bit XMM registers, especially using 8 loads followed by 8 stores in :the instruction stream. They make 128 stores want to take twice as long :as 2 64-bits ones, and it takes a large amount of out of order execution :to reorder things to reach the "1 64-bit load and 1 64-bit store" per :cycle pattern. Reordering is exactly what you don't want for controlling :the interleaving of access to main memory. Even with a simple load-store- :load-store with 64-bit accesses in the instruction, stream, the there :will be some reordering -- the reads will get 1 ahead of the writes. :This explains why my movdqa benchmark is slow on AXP but not why it is :slow on A64. The advantages of doing N loads followed by N stores (where N >= 4 and operates on 64 or 128 bit entities) are as follows: * There is no instruction or register reordering (or very little) because the data loaded by any given instruction is not used until e.g. 8 instructions later. * The write buffer is ordered (either through to the L2 cache or to main memory, it doesn't matter). This enforces a degree of resynchronization of the instruction stream. That is, it creates a self-healing situation that makes it possible to write code that works very well under all sorts of conditions. * The read instructions should theoretically tend not to be reordered simply because the memory pipeline imposes a stall due to handling previously buffered writes and because of the burst nature of linear reads from main memory. Even if the cpu DID reorder the reads, it can only actually retire instructions as the data actually comes in from main memory. Such data will always be fairly well ordered in either 0-1-2-3 or 3-2-1-0 bursts, or at the minimum cache-line bursts. Some reordering might occur if some of the read instructions access cached data, but even in that case I just don't think it happens. There is simply nothing there for the cpu to reorder. * I don't think the cpu could reorder reads or reorder reads and writes under these conditions even if it wanted to, and we don't want it to. * Write combining is overrated. It serves an important purpose, but it is a bad idea to actually depend on it unless your algorithm is self-healing from the point of view of resynchronizing the pipeline and/or memory read and write ordering. Instruction pipelines and cache pipelines are quite evenly matched these days (usually no more then 1:2 performance) so depending on write combining will actually cause chain stalls in the cache pipeline (not using a cache cycle that was potentially available to use) when the instruction pipeline stalls on a memory read, because the cpu wanted to try to combine the writes. These are boundary conditions for sure. It is best to remember that the instruction pipeline is stalling several times on every loop due to main memory accesses. You can't avoid it, but you can try to manage it. I am not familiar enough with the architectures to know whether the cpu does burst reads from main memory into the cache... e.g. 4 cache line bursts at a time. My assumption has always been that the cpu does do burst reads, because it's so easy to do. Any sort of burst reading into the cache by the cpu only creates an advantage for this architecture because there are no dependancies between instructions for 8 instruction slots and because it enforces an ordering to both read and write instructions that tends to self-heal the algorithm from a performance standpoint. So, in summary... the reason I use block loads and stores (sequences of 8 MMX or XMM loads followed by 8 MMX or XMM stores) is that these sorts of inner loops appear to generate very good results under all operating conditions. When I have attempted to use prefetcha, prefetchnta, or non-temporal moves (movnt* instead of movdq*) the result has been better performance only under certain very carefully controlled circumstances, and terrible performance otherwise. IMHO, I do believe that 64 bit loads and stores is plenty good enough for today's cpus, but it is also clear that 128 bit loads and stores will be better for tomorrow's cpus and works at least as well on today's cpus as 64 bit loads and stores, at least insofar as 128 bit registers can be made available to the kernel. -- It should be possible to use MMX/XMM registers in the kernel simply as call-saved registers by issuing a FWAIT or equivalent at the user-kernel boundary to take care of any exceptions (instead of saving the state and calling clts ; fnclex). Presumably by the time the cpu actually gets the integer and cpu state context pushed any FP operations will have long sinced completed. But I haven't timed it to find out whether that actually works. The nice thing about most MMX/XMM media instructions is that they don't actually care about what state the FP unit is in because they aren't actually doing any FP math. It is just a matter of synchronizing the free FP registers from some prior user FP operation which may not have posted results yet. If this IS possible then it removes most of the NPXTHREAD junk from the kernel bcopy path for the case where NPXTHREAD != NULL, giving you the ability to use the FP optimally whether NPXTHREAD == NULL or NPXTHREAD != NULL. And, in such a case, it might be more beneficial for small copies to only save two or four XMM registers instead of all eight. What do you think? -Matt Matthew Dillon :Bruce : From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 20:42:31 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 86F6216A492 for ; Thu, 18 Jan 2007 20:42:31 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.189]) by mx1.freebsd.org (Postfix) with ESMTP id 20FEF13C465 for ; Thu, 18 Jan 2007 20:42:30 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so289021nfc for ; Thu, 18 Jan 2007 12:42:14 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer:mime-version:content-type:content-transfer-encoding; b=RmlnNMHa1WCIQJeEUA6JIYbkVhcT7oX+OAMbUn8sxW+1ky/hrWF7SWnVZs6U4XQ2jd+q3MnThcnZF9FJc36w09E6lT6/6RRkz70YUhSDUAnx6YdCdLwgsK/SyUptqty3Cx8XGkkk+WdASnSR8kdexeUriKKdxggcmom0vC+oWd4= Received: by 10.49.21.8 with SMTP id y8mr1300082nfi.1169151474631; Thu, 18 Jan 2007 12:17:54 -0800 (PST) Received: from saturn.lan ( [80.179.18.46]) by mx.google.com with ESMTP id k23sm3174124nfc.2007.01.18.12.17.53; Thu, 18 Jan 2007 12:17:54 -0800 (PST) Date: Thu, 18 Jan 2007 22:17:49 +0200 From: Rostislav Krasny To: Craig Rodrigues Message-Id: <20070118221749.3be589d9.rosti.bsd@gmail.com> In-Reply-To: <20070118134936.GA7391@crodrigues.org> References: <20070118134936.GA7391@crodrigues.org> X-Mailer: Sylpheed version 2.2.10 (GTK+ 2.10.6; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 20:42:31 -0000 On Thu, 18 Jan 2007 08:49:36 -0500 Craig Rodrigues wrote: > Using this patch, I have been able to do: > mount /dev/blah /mnt > > and mount a UFS, cd9660, or FAT filesystem, depending on what filesystem > is on the /dev/blah device. > > Comments? OpenBSD already has such a functionality. It uses readlabelfs(3) for this. What disadvantages or advantages does it have beside your implementation? From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 22:01:46 2007 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 09B8616A492; Thu, 18 Jan 2007 22:01:46 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk.360sip.com [72.236.70.226]) by mx1.freebsd.org (Postfix) with ESMTP id A907F13C441; Thu, 18 Jan 2007 22:01:45 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from [192.168.1.47] ([204.244.149.125]) (authenticated bits=0) by sippysoft.com (8.13.8/8.13.6) with ESMTP id l0ILwFPe064875 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 18 Jan 2007 13:58:17 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <45AFED63.7020009@FreeBSD.org> Date: Thu, 18 Jan 2007 13:57:55 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: Matthew Dillon References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> <200701181948.l0IJmdfn061671@apollo.backplane.com> In-Reply-To: <200701181948.l0IJmdfn061671@apollo.backplane.com> Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 7bit Cc: Attilio Rao , freebsd-current@FreeBSD.org, Ivan Voras , freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 22:01:46 -0000 Heh, it's so complex, so machine-dependent.... That makes me wonder why we still don't have 3 simple to use instructions that do equivalent of memmove(), memcpy() and memset() all in hardware in the best possible way with the respect of block size, alignment, caches, chipset, you-name-it? Virtually every program would benefit from such instructions. -Maxim From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 22:28:40 2007 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A17DC16A412; Thu, 18 Jan 2007 22:28:40 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk.360sip.com [72.236.70.226]) by mx1.freebsd.org (Postfix) with ESMTP id 4E84513C455; Thu, 18 Jan 2007 22:28:40 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from [192.168.1.47] ([204.244.149.125]) (authenticated bits=0) by sippysoft.com (8.13.8/8.13.6) with ESMTP id l0IMSYJd065671 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 18 Jan 2007 14:28:37 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <45AFF47E.3020905@FreeBSD.org> Date: Thu, 18 Jan 2007 14:28:14 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: Chuck Swiger References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> <200701181948.l0IJmdfn061671@apollo.backplane.com> <45AFED63.7020009@FreeBSD.org> <25EB3FED-71A9-4AE1-9A38-5D2DC27D0DF7@mac.com> In-Reply-To: <25EB3FED-71A9-4AE1-9A38-5D2DC27D0DF7@mac.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 22:28:40 -0000 Chuck Swiger wrote: > Unfortunately, there are simply different tradeoffs between mechanisms > for copying depending on whether you want to use or avoid > using/thrashing the L1/L2 caches, whether the data is cache-aligned, and > so forth; the CPU can't infer what you want to occur-- you have to tell > it. I find it interesting that some of the architectures (PA-RISC, Well, of course there are some special cases, but in general there should be some baseline suitable for most of uses. That's why we (and most other operating systems) only provide single version for the mem*(3) APIs. -Maxim From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 22:47:57 2007 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5460616A415; Thu, 18 Jan 2007 22:47:57 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from mail-out4.apple.com (mail-out4.apple.com [17.254.13.23]) by mx1.freebsd.org (Postfix) with ESMTP id 30DDA13C44B; Thu, 18 Jan 2007 22:47:57 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from relay5.apple.com (relay5.apple.com [17.128.113.35]) by mail-out4.apple.com (8.13.8/8.13.8) with ESMTP id l0IMluNl014484; Thu, 18 Jan 2007 14:47:56 -0800 (PST) Received: from relay5.apple.com (unknown [127.0.0.1]) by relay5.apple.com (Symantec Mail Security) with ESMTP id 7B0EA29C002; Thu, 18 Jan 2007 14:47:56 -0800 (PST) X-AuditID: 11807123-a3b5bbb0000039f2-94-45aff91c36f4 Received: from [17.214.13.96] (unknown [17.214.13.96]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by relay5.apple.com (Apple SCV relay) with ESMTP id 6322730400B; Thu, 18 Jan 2007 14:47:56 -0800 (PST) In-Reply-To: <45AFF47E.3020905@FreeBSD.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> <200701181948.l0IJmdfn061671@apollo.backplane.com> <45AFED63.7020009@FreeBSD.org> <25EB3FED-71A9-4AE1-9A38-5D2DC27D0DF7@mac.com> <45AFF47E.3020905@FreeBSD.org> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <0B6D259B-618B-466C-844E-3F79FDE272BB@mac.com> Content-Transfer-Encoding: 7bit From: Chuck Swiger Date: Thu, 18 Jan 2007 14:47:55 -0800 To: Maxim Sobolev X-Mailer: Apple Mail (2.752.2) X-Brightmail-Tracker: AAAAAA== Cc: freebsd-current@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 22:47:57 -0000 On Jan 18, 2007, at 2:28 PM, Maxim Sobolev wrote: >> Unfortunately, there are simply different tradeoffs between >> mechanisms for copying depending on whether you want to use or >> avoid using/thrashing the L1/L2 caches, whether the data is cache- >> aligned, and so forth; the CPU can't infer what you want to >> occur-- you have to tell it. I find it interesting that some of >> the architectures (PA-RISC, > > Well, of course there are some special cases, but in general there > should be some baseline suitable for most of uses. That's why we > (and most other operating systems) only provide single version for > the mem*(3) APIs. Well, a truly generic version in is lib/libc/string/bcopy.c; it's architecture-neutral (ie, it's pure C code) and it handles all kinds of things like overlapping source and destination addresses, non- aligned access, and so forth. The downside is that it's slower than using movl/movsl, much less some of the fancier variants that Bruce and Matt have been discussing (in considerable, interesting detail) earlier: http://now.cs.berkeley.edu/Td/bcopy.html If you're only moving, say, 5 bytes, the overhead of fancy loop unrolling and prefetching and so forth isn't going to help compared with a simple movb/movl combination, so it really depends. -- -Chuck From owner-freebsd-arch@FreeBSD.ORG Thu Jan 18 22:52:07 2007 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6426D16A412; Thu, 18 Jan 2007 22:52:07 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from mail-out3.apple.com (mail-out3.apple.com [17.254.13.22]) by mx1.freebsd.org (Postfix) with ESMTP id 2B12713C471; Thu, 18 Jan 2007 22:52:07 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from relay7.apple.com (relay7.apple.com [17.128.113.37]) by mail-out3.apple.com (8.13.8/8.13.8) with ESMTP id l0IMHw6Q026931; Thu, 18 Jan 2007 14:17:58 -0800 (PST) Received: from relay7.apple.com (unknown [127.0.0.1]) by relay7.apple.com (Symantec Mail Security) with ESMTP id 8E07330020; Thu, 18 Jan 2007 14:17:58 -0800 (PST) X-AuditID: 11807125-a3250bb000006e4c-93-45aff2160105 Received: from [17.214.13.96] (unknown [17.214.13.96]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by relay7.apple.com (Apple SCV relay) with ESMTP id 798A230002; Thu, 18 Jan 2007 14:17:58 -0800 (PST) In-Reply-To: <45AFED63.7020009@FreeBSD.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> <200701181948.l0IJmdfn061671@apollo.backplane.com> <45AFED63.7020009@FreeBSD.org> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <25EB3FED-71A9-4AE1-9A38-5D2DC27D0DF7@mac.com> Content-Transfer-Encoding: 7bit From: Chuck Swiger Date: Thu, 18 Jan 2007 14:17:57 -0800 To: Maxim Sobolev X-Mailer: Apple Mail (2.752.2) X-Brightmail-Tracker: AAAAAA== Cc: freebsd-current@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 22:52:07 -0000 On Jan 18, 2007, at 1:57 PM, Maxim Sobolev wrote: > Heh, it's so complex, so machine-dependent.... Well, yes. :) > That makes me wonder why we still don't have 3 simple to use > instructions that do equivalent of memmove(), memcpy() and memset() > all in hardware in the best possible way with the respect of block > size, alignment, caches, chipset, you-name-it? Virtually every > program would benefit from such instructions. Unfortunately, there are simply different tradeoffs between mechanisms for copying depending on whether you want to use or avoid using/thrashing the L1/L2 caches, whether the data is cache-aligned, and so forth; the CPU can't infer what you want to occur-- you have to tell it. I find it interesting that some of the architectures (PA- RISC, SPARC) do allow for simple instructions with cache-control hinting: http://gcc.gnu.org/projects/prefetch.html -- -Chuck From owner-freebsd-arch@FreeBSD.ORG Fri Jan 19 02:37:12 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 857E416A415; Fri, 19 Jan 2007 02:37:12 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.152]) by mx1.freebsd.org (Postfix) with ESMTP id 6D0CB13C44C; Fri, 19 Jan 2007 02:37:12 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from c-66-31-35-94.hsd1.ma.comcast.net ([66.31.35.94]) by comcast.net (rwcrmhc12) with ESMTP id <20070119023711m12002l9rce>; Fri, 19 Jan 2007 02:37:11 +0000 Received: from c-66-31-35-94.hsd1.ma.comcast.net (localhost.crodrigues.org [127.0.0.1]) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8) with ESMTP id l0J2bA2o001761; Thu, 18 Jan 2007 21:37:10 -0500 (EST) (envelope-from rodrigc@c-66-31-35-94.hsd1.ma.comcast.net) Received: (from rodrigc@localhost) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8/Submit) id l0J2b9Hi001760; Thu, 18 Jan 2007 21:37:09 -0500 (EST) (envelope-from rodrigc) Date: Thu, 18 Jan 2007 21:37:09 -0500 From: Craig Rodrigues To: freebsd-current@freebsd.org Message-ID: <20070119023709.GA1524@crodrigues.org> References: <20070118134936.GA7391@crodrigues.org> <20070118221749.3be589d9.rosti.bsd@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070118221749.3be589d9.rosti.bsd@gmail.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-arch@freebsd.org Subject: Re: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2007 02:37:12 -0000 On Thu, Jan 18, 2007 at 10:17:49PM +0200, Rostislav Krasny wrote: > OpenBSD already has such a functionality. It uses readlabelfs(3) for > this. What disadvantages or advantages does it have beside your > implementation? Thanks for the pointer, I did not know about readlabelfs on OpenBSD! From what I understand from reading the code: http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libutil/readlabel.c http://www.openbsd.org/cgi-bin/cvsweb/src/sys/sys/disklabel.h the readlabelfs() function tries to open the device and do a DIOCGDINFO ioctl() on the device to read the disklabel. Once the disklabel is read, (i.e. DIOCDINFO returns a 'struct disklabel'), the p_fstype member of 'struct partition' which is inside the 'struct disklabel' is converted to a string name based on the fstypesnames array in . This approach relies heavily on disklabels, and correct information being stored in disklabels. For FreeBSD, I do not like the idea of introducing a new dependency between mount(8) and BSD disklabels. The existing FreeBSD mount(8) program just tries to open a device, does an nmount() with a "fstype" parameter, and then relies on the underlying file system code (i.e. "ufs", "msdosfs", etc.) to read the superblock from the device, figure out if it can mount it, and return an error if it cannot. That's the behavior I tried to preserve with my patch. It's kind of simplistic, but it does work. -- Craig Rodrigues rodrigc@crodrigues.org From owner-freebsd-arch@FreeBSD.ORG Fri Jan 19 02:55:23 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E150916A415; Fri, 19 Jan 2007 02:55:23 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from alnrmhc14.comcast.net (alnrmhc14.comcast.net [204.127.225.94]) by mx1.freebsd.org (Postfix) with ESMTP id AB68413C441; Fri, 19 Jan 2007 02:55:23 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from c-66-31-35-94.hsd1.ma.comcast.net ([66.31.35.94]) by comcast.net (alnrmhc14) with ESMTP id <20070119025522b1400rmue9e>; Fri, 19 Jan 2007 02:55:22 +0000 Received: from c-66-31-35-94.hsd1.ma.comcast.net (localhost.crodrigues.org [127.0.0.1]) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8) with ESMTP id l0J2tLTl001935; Thu, 18 Jan 2007 21:55:21 -0500 (EST) (envelope-from rodrigc@c-66-31-35-94.hsd1.ma.comcast.net) Received: (from rodrigc@localhost) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8/Submit) id l0J2tL3X001934; Thu, 18 Jan 2007 21:55:21 -0500 (EST) (envelope-from rodrigc) Date: Thu, 18 Jan 2007 21:55:20 -0500 From: Craig Rodrigues To: Luigi Rizzo Message-ID: <20070119025520.GA1891@crodrigues.org> References: <20070118134936.GA7391@crodrigues.org> <20070118064912.A39777@xorpc.icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070118064912.A39777@xorpc.icir.org> User-Agent: Mutt/1.4.2.1i Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2007 02:55:24 -0000 On Thu, Jan 18, 2007 at 06:49:12AM -0800, Luigi Rizzo wrote: > great feature. I probably agree that "mount -t auto" might > be a safer way to implement it, but other than that i'd > love to have it too. I understand the motivation for "mount -t auto"....I wanted to get away from hijacking the "-t" parameter with values other than an "fstype". Also, if someone implements the autofs file system again for FreeBSD, there could be confusion. My tack for dealing with POLA was to always start with UFS when iterating through the vfs.conflist sysctl. I thought that having it as the default behavior if "-t" is not specified would be nice, especially for people coming from Linux. When I moved from Linux to FreeBSD, I found it a bit weird that I needed to specify a "-t" parameter to mount my CD-ROM...and figuring out that it was -t cd9660, which is not the same as Linux was odd too. :) -- Craig Rodrigues rodrigc@crodrigues.org From owner-freebsd-arch@FreeBSD.ORG Fri Jan 19 03:24:25 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8F63216A412; Fri, 19 Jan 2007 03:24:25 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from alnrmhc12.comcast.net (alnrmhc12.comcast.net [206.18.177.52]) by mx1.freebsd.org (Postfix) with ESMTP id 59A7F13C478; Fri, 19 Jan 2007 03:24:25 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from c-66-31-35-94.hsd1.ma.comcast.net ([66.31.35.94]) by comcast.net (alnrmhc12) with ESMTP id <20070119032423b1200jtatde>; Fri, 19 Jan 2007 03:24:24 +0000 Received: from c-66-31-35-94.hsd1.ma.comcast.net (localhost.crodrigues.org [127.0.0.1]) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8) with ESMTP id l0J3OMMr002121; Thu, 18 Jan 2007 22:24:22 -0500 (EST) (envelope-from rodrigc@c-66-31-35-94.hsd1.ma.comcast.net) Received: (from rodrigc@localhost) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8/Submit) id l0J3OMQE002120; Thu, 18 Jan 2007 22:24:22 -0500 (EST) (envelope-from rodrigc) Date: Thu, 18 Jan 2007 22:24:22 -0500 From: Craig Rodrigues To: Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= Message-ID: <20070119032422.GB1891@crodrigues.org> References: <20070118134936.GA7391@crodrigues.org> <86lkk02wp7.fsf@dwp.des.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86lkk02wp7.fsf@dwp.des.no> User-Agent: Mutt/1.4.2.1i Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2007 03:24:25 -0000 On Thu, Jan 18, 2007 at 04:15:00PM +0100, Dag-Erling Smrgrav wrote: > Didn't somebody mention ordering issues back then? Your revised patch > still doesn't even try to address those. If we ever get ext3 support, > for instance, mounting an ext2 file system as ext3 might cause it to > be silently upgraded to ext3. Since FreeBSD doesn't have any ext3 support right now, I think a more concrete example of ordering issues is to look at cd9660 vs. udf, which FreeBSD currently supports. In my patch, the mount_fs_try() function does deal with one ordering issue.....no matter what is in the vfs.conflist sysctl, it always tries to mount "ufs" first. This approximates the existing mount(8) behavior. It would not be hard to modify mount_fs_try() to deal with ordering of ext3 vs. ext2, udf vs. cd9660, or whatever. Not the most elegant solution, but workable for the limited subset of file systems that FreeBSD supports. > Not to mention the slew of error messages this will result in as you > walk the list of filesystems trying to find the right one... This is true, but in my tests, the "slew of error messages" is not too bad. Also, with my patch, if you specify the "-t" parameter, mount_fs_try() is not used, so things like mounting via /etc/fstab, where the fstype is specified, will behave the same way as now, and not iterate through a list of file systems. > A better way would be to have a userland utility attempt to identify > the file system, and if necessary load the appropriate module, before > mounting it. I disagree. To go down this road, instead of a separate utility, I think the best approach would be to put all this logic inside mount(8), the way Linux does, as Christoph Hellwig mentioned here: http://lists.freebsd.org/pipermail/freebsd-arch/2006-July/005447.html I've looked at the Linux mount utility source.....it maintains a table of magic numbers and offsets and tries them in a well defined order. The logic is quite involved, but it does work. The Linux code is under GPL though.... The other suggestion that someone had was to use libmagic in mount(8). libmagic is good, but using it in mount(8) would introduce link dependencies on libmagic and libz, and also introduce dependencies on files in /usr/share/misc/magic. -- Craig Rodrigues rodrigc@crodrigues.org From owner-freebsd-arch@FreeBSD.ORG Fri Jan 19 05:48:38 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1398416A415; Fri, 19 Jan 2007 05:48:38 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (vc4-2-0-87.dsl.netrack.net [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id C05C213C457; Fri, 19 Jan 2007 05:48:37 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.13.4/8.13.4) with ESMTP id l0J5jnQ2004773; Thu, 18 Jan 2007 22:45:50 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Thu, 18 Jan 2007 22:46:09 -0700 (MST) Message-Id: <20070118.224609.709403207.imp@bsdimp.com> To: rodrigc@crodrigues.org From: "M. Warner Losh" In-Reply-To: <20070119023709.GA1524@crodrigues.org> References: <20070118134936.GA7391@crodrigues.org> <20070118221749.3be589d9.rosti.bsd@gmail.com> <20070119023709.GA1524@crodrigues.org> X-Mailer: Mew version 4.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0 (harmony.bsdimp.com [127.0.0.1]); Thu, 18 Jan 2007 22:45:50 -0700 (MST) Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] mount(8) can figure out fstype X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2007 05:48:38 -0000 In message: <20070119023709.GA1524@crodrigues.org> Craig Rodrigues writes: : This approach relies heavily on disklabels, and correct information : being stored in disklabels. : : For FreeBSD, I do not like the idea of introducing a new dependency : between mount(8) and BSD disklabels. It's worse than you might think. On OpenBSD, the line between partition and slice that we have in FreeBSD doesn't exist. Everything is a partition, and they create the necessary labeling when the label is read. Since FreeBSD lacks that feature, for the most part, you are going to have to implement some kind of 'file'-like magic number recognition for filesystems if you want to use a similar approach. Warner From owner-freebsd-arch@FreeBSD.ORG Fri Jan 19 05:59:53 2007 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 89BA516A4D2; Fri, 19 Jan 2007 05:59:53 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2-3.pacific.net.au [61.8.2.226]) by mx1.freebsd.org (Postfix) with ESMTP id BD13C13C469; Fri, 19 Jan 2007 05:59:52 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.2.162]) by mailout2.pacific.net.au (Postfix) with ESMTP id 42E2811108E; Fri, 19 Jan 2007 16:59:49 +1100 (EST) Received: from besplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailproxy1.pacific.net.au (Postfix) with ESMTP id 91A388C18; Fri, 19 Jan 2007 16:59:47 +1100 (EST) Date: Fri, 19 Jan 2007 16:59:46 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Matthew Dillon In-Reply-To: <200701181948.l0IJmdfn061671@apollo.backplane.com> Message-ID: <20070119165802.Q1583@besplex.bde.org> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bbf2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe10701160851r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> <200701181948.l0IJmdfn061671@apollo.backplane.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Attilio Rao , Maxim Sobolev , freebsd-current@FreeBSD.org, Ivan Voras , freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2007 05:59:53 -0000 I'll try to keep this shorter :-). On Thu, 18 Jan 2007, Matthew Dillon wrote: > > :Fully cached case: > :% copy0: 16907105855 B/s ( 242265 us) (movsq) > :% ... > :% copyQ: 13658478027 B/s ( 299887 us) (movdqa) > :% ... > :% copyT: 16456408196 B/s ( 248900 us) (movq (i)) > : > :This doesn't contradict your claim since main memory is not really involved. > :Everything should be limited by instruction and cache bandwidth, but for > :some reason movdqa is quite slow despite or because of my attempts to > :schedule it perfectly. > > Well, that's one of the problems. Actually two of the problems. > First, prefetching tends to add more confusion then it solves when > used in a copy loop, because for all intents and purposes you are > already keeping the memory pipeline full simply by the fact that > the writes are buffered. The above is for the non-prefetched case (except for the target of nontemporal writes). Handling and interpreting prefetching is certainly confusing in other cases. I'd better describe what my benchmark does in a bit more detail: to get reasonably accuracy, it is necessary to iterate many times, but that gives the same not-very-real-world cache state for all iterations except the first. I just run it with different block sizes. 4K gives the "fully (L1) cached case", somewhere between 40K and 400K gives the "fully (L2) cached case", and 4M-200M gives the "fully uncached case". I don't test lower levels except by accidentally starting swapping. There is an option to pre-cache the target before starting the iterations. This can only make a difference if the block size is small enough for the target to remain cached (or there number of iterations is too small), and the copying used nontemporal stores so that the copying itself doesn't pre-cache the target. I didn't use this option in the benchmark results quoted in this thread. > By the time the loop gets back around to > the read it has to stall anyway because the writes are still being > pushed out to memory (read-before-write ordering doesn't help because > all that accomplishes is to force the write buffer to stay perpetually > full (possibly in a non-optimal fashion), and the cpu stalls anyway. I had mostly forgotten about read/write ordering. It is going to limit execution reordering in a MD way. I only have good/handy documentation of it for A64. On A64, write ordering is fairly strict, but almost any order is possible for a combination of reads and writes provided the writes can be held in the write buffer and the reads and writes are to different locations. Read ordering is fairly non-strict. Reads can be reordered ahead of writes. This doesn't quite agree with what you said. It allows enough reordering to prevent stalls provided the CPU is clever enough and the copy is not overlapped. The CPU can take almost any ordering of sequential reads and writes and turn it into: setup for (;;) { read next cache line write previous cache line from write buffer (write buffer size must be >= cache line size) } I don't know which CPUs are clever enough to do this or exactly which static instruction order makes it easiest for them, but have noticed reasonable orders which give mysterious slowdowns that are probably related to this. > I've tried every combination of prefetching and non-temporal > instructions imagineable and it has reduced performance more often > then it has improved it. I can write an optimal copy loop for > particular situations but the problem is that the same loop becomes > extremely inefficient under any *OTHER* conditions. Same here, except I wouldn't call most of the non-optimal cases _extremely_ inefficient. Do you have real-world tests? > The behavior of any given copy algorithm is radically different > if either the source or target are present in the L1 or L2 caches, > verses if they aren't. The algorithm one uses to do tiny mostly > in-cache copies is totally different from the algorithm one uses > to do large bulk copies. The algorithm one uses where the > source data is partially or fully cached is totally different from the > one used if the target data is partially or fully cached, or if neither > is cached, or if both are cached. And it's hard to know whether/where the data is cached. The only simplification is that if an FPU switch is required, then very small copy sizes need not be considered. > :Fully uncached case: > :% copy0: 829571300 B/s ( 493749 us) (movsq) > :% ... > :% copyQ: 835822845 B/s ( 490056 us) (movdqa) > : ... > :% copyW: 1350397932 B/s ( 303318 us) (movnti) > : ... > :Now movdqa beats movsq by a bit, at least with prefetch, but has the > :same speed as integer moves of half the size (or MMX moves of half the > :size (benchmark data not shown)). It is necessary to use movnt to get > :anywhere near memory bandwidth, and any form of movnt can be used for > : ... > > But you can't safely use *ANY* non-temporal instruction unless you > know, specifically, that the data is uncached. If you use a non-temporal I think you mean "optimally", not "safely". sfence is supposed to give coherent data. > instruction in a situation where the data is cached or can be easily > cached, you destroy the performance for that case by causing the cpu > not to cache data that it really should cache. Use of non-temporal > instructions results in a non-self-healing algorithm... it is possible > to get into a situation the is permanently non-optimal. That's why > I don't use those instructions. Not caching the target is part of the point of using movnt. It can win not only in copy speed but also by not thrashing useful things out of the cache. Unfortunately we don't really know when this happens (except for idle pagezero). You may be right that it happens so rarely that never doing it is best. > Sometimes I wish the non-temporal instructions had a random component. > That is, that they randomly operates as normal moves instead of nt > moves for a small percentage of executions (~1-25%) in order to create > a self-healing (performance wise) algorithm. Wouldn't normal (non-copy) accesses cache the data reasonably if it is actually used? Multiple copying of the same data doesn't seem to be an important case. I just remembered on case where nontemporal should win: for writes of data that will be DMAed to disks but not read soon by anything except the DMA. Even if disk drivers read it, the disk writes should be delayed by a few seconds on average so the data wouldn't remain in caches except on idle systems. > Use of non-temporal instructions for non performance-critical operations > is a clear win when executed from the idle loop, I agree. Use of NT > instructions in any other case is difficult because it is virtually > impossible to predict the conditions under which other cases occur. Do you use it for copying pages? > :Which modern CPUs can't keep up with L2? > > The instruction pipeline is the problem, not the L2 cache bandwidth. > Things get a bit weird when every single instruction is reading or > writing memory. I try to design algorithms that work reasonably well > across a large swath of cpus. It turns out that algorithms which do > block loads followed by block stores tend to maintain consistent and > good performance across a large swath of cpus. I outline the reasons > why later on. I think it's not mainly the instruction pipeline throughput (except on very old CPUs), but latency issues caused by interactions with the memory accesses. > :On amd64 for the fully-L2-cached case: > :% ... > :% copyQ: 4369052686 B/s ( 937503 us) (movdqa) > :% copyR: 2785886655 B/s (1470268 us) (movdqa with prefetchnta) > :% copyS: 4553271385 B/s ( 899573 us) (movdqa with block prefetch) > :% ... > : > :So here is a case where prefetchnta is bad. OTOH, movnti is quite fast > :provided prefetchnta is not used with it. movnti is even more competitive > :with L2 if main memory is DDR-2. > > It also depends *WHERE* in the loop you put the prefetchnta. I was > able to get fairly close to block prefetching speeds using > prefetchnta by carefully locating the instruction and adjusting > the read-ahead point. For example, by locating the instruction > before the write side and indexing the prefetched memory address > 128 bytes ahead of the curve. The placement of the instruction is > sensitive down to the instruction slot... moving it one forward or > one back can result in crazy differences in performance (due to > forced RAS cycles I believe, in particular with prefetchnta). I've only tried that for block prefetch. It was too MD to maintain. I hoped prefetchnta was easier to use. I tried several things, and just noticed that I missed a point in the AMD docs -- block prefetch should be in reverse order to confuse the hardware into not competing with the explicit prefetch. > :Load/store-unit: > : > : (lots of good information about load-store units) > > Well, but these are for very specific flavor-of-the-day hardware tweaks. > The designers change these parameters literally every other month. > > All that can really be said here is: Avoid 32 bit accesses if you can, > 64 bit accesses seem to be the most dependable, and 128 bit accesses > are the up-and-coming thing. I think the using the FPU and 128-bit accesses are also flavor-of-the-day (yesterday for the FPU). Let the cache and write buffer handle combining of accesses. > In particular I remember reading that AMD had been spending a lot > of time optimizing XMM register loads and stores. I presume that > Intel is doing the same thing since both are going face forward > into gaming. Bah, I wish the would spend more time optimizing important things like FPU latency and wider integer registers :-). > :The asymmetric load/store capabilities on the AXP can't be good for using > :128-bit XMM registers, especially using 8 loads followed by 8 stores in > :the instruction stream. They make 128 stores want to take twice as long > ... > > The advantages of doing N loads followed by N stores (where N >= 4 > and operates on 64 or 128 bit entities) are as follows: > > * There is no instruction or register reordering (or very little) > because the data loaded by any given instruction is not used until > e.g. 8 instructions later. But I think I want to maximize reordering possibilities. > * The write buffer is ordered (either through to the L2 cache or to > main memory, it doesn't matter). This enforces a degree of > resynchronization of the instruction stream. That is, it creates > a self-healing situation that makes it possible to write code > that works very well under all sorts of conditions. Ordered relative to reads? BTW, do you order the instruction stream so that the reader is 1 cache line ahead of the writer? I only do this for old methods optimized for P1's. It saves about 1 read access time per loop unless this time can be hidden by reordering or loop overhead. The cache line size is MD so it is better for this to happen automatically, but perhaps less confusing to program it explicitly for a preferred CPU. > * The read instructions should theoretically tend not to be reordered > simply because the memory pipeline imposes a stall due to handling > previously buffered writes and because of the burst nature of linear > reads from main memory. Even if the cpu DID reorder the reads, > ... > * I don't think the cpu could reorder reads or reorder reads and writes > under these conditions even if it wanted to, and we don't want it > to. > > * Write combining is overrated. It serves an important purpose, but > it is a bad idea to actually depend on it unless your algorithm > is self-healing from the point of view of resynchronizing the > pipeline and/or memory read and write ordering. Instruction Er, don't you depend on it even with 128-bit registers? 16 bytes isn't very large. Even an AthlonXP has a 64-byte write buffer to combine 4 of these. > ... > -- > > It should be possible to use MMX/XMM registers in the kernel simply as > call-saved registers by issuing a FWAIT or equivalent at the user-kernel > boundary to take care of any exceptions (instead of saving the state > and calling clts ; fnclex). Presumably by the time the cpu actually > gets the integer and cpu state context pushed any FP operations will > have long sinced completed. But I haven't timed it to find out whether > that actually works. I thought about making some XMM registers call-used (switched on every boundary crossing). Copying only needs 1 and switching just 1 doesn't have much overhead. Call-saved has different tradeoffs. Both mehods should drop the semi-lazy FPU context switching and do a full switch at context switch time so that DNA traps never occur and the FPU/XMM can just be used. This also hopefully doesn't have much additional overhead, since the FPU instructions run in an otherwise-idle pipeline and the extra memory traffic is either small enough of can be scheduled better. MMX would have complications for the tag word. I think new code shouldn't support MMX. The fnclex is unnecessary (see below). I don't quite understand what your fwait does or how call-saved XMM context switching could work well for preemptible kernels. Doesn't it give all the old problems with the state saved where the general context switcher can't see it? Call-used XMM context switching works using normal stack discipline. > The nice thing about most MMX/XMM media instructions is that they don't > actually care about what state the FP unit is in because they aren't > actually doing any FP math. It is just a matter of synchronizing the > free FP registers from some prior user FP operation which may not have > posted results yet. XMM doesn't even need the synchronization. It runs independently of the i387 state, and if it is not used for FP math then it also runs independently of mxcsr. Thus fnclex and fwait have no effect on it, but if it is used for FP math then something would have to be done to ignore (user) exceptions and mode bits in mxcsr. > If this IS possible then it removes most of the NPXTHREAD junk from the > kernel bcopy path for the case where NPXTHREAD != NULL, giving you the > ability to use the FP optimally whether NPXTHREAD == NULL or > NPXTHREAD != NULL. And, in such a case, it might be more beneficial > for small copies to only save two or four XMM registers instead of > all eight. > > What do you think? One XMM register is enough for even large copies :-). Sorry, didn't succeed in keeping this shorter. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri Jan 19 06:02:02 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3385816A412; Fri, 19 Jan 2007 06:02:02 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from turion.vk2pj.dyndns.org (c220-239-3-125.belrs4.nsw.optusnet.com.au [220.239.3.125]) by mx1.freebsd.org (Postfix) with ESMTP id A996D13C428; Fri, 19 Jan 2007 06:02:01 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from turion.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by turion.vk2pj.dyndns.org (8.13.8/8.13.8) with ESMTP id l0IIkpr9003184; Fri, 19 Jan 2007 05:46:51 +1100 (EST) (envelope-from peter@turion.vk2pj.dyndns.org) Received: (from peter@localhost) by turion.vk2pj.dyndns.org (8.13.8/8.13.8/Submit) id l0IIkojM003183; Fri, 19 Jan 2007 05:46:50 +1100 (EST) (envelope-from peter) Date: Fri, 19 Jan 2007 05:46:50 +1100 From: Peter Jeremy To: Bruce Evans Message-ID: <20070118184650.GB845@turion.vk2pj.dyndns.org> References: <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="RASg3xLB4tUQ4RcS" Content-Disposition: inline In-Reply-To: <20070118113831.A11834@delplex.bde.org> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.13 (2006-08-11) Cc: Maxim Sobolev , Ivan Voras , Attilio Rao , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2007 06:02:02 -0000 --RASg3xLB4tUQ4RcS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, 2007-Jan-18 18:03:20 +1100, Bruce Evans wrote: >On Wed, 17 Jan 2007, Matthew Dillon wrote: >> Alignment is critical. If the data is not aligned, don't bother. 1= 28 >> bits means 16 byte alignment. > >The above benchmark output is for aligned data :-). I don't try hard to >optimize or benchmark misaligned cases. How realistic is this? Has anyone collected statistics on the size and alignment of bzero/bcopy calls? How much of the time is the size known at compile time? --=20 Peter Jeremy --RASg3xLB4tUQ4RcS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFr8Ca/opHv/APuIcRAqZsAJsHlCSLkdmwl1x/AD/7FV3TJ+K7pgCghDLv lEQ/MSu+HFYDfsQNmk9KIYg= =mPQs -----END PGP SIGNATURE----- --RASg3xLB4tUQ4RcS-- From owner-freebsd-arch@FreeBSD.ORG Fri Jan 19 07:14:26 2007 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9D1A116A60D; Fri, 19 Jan 2007 07:14:26 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1-3.pacific.net.au [61.8.2.210]) by mx1.freebsd.org (Postfix) with ESMTP id 36A8713C45E; Fri, 19 Jan 2007 07:14:26 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout1.pacific.net.au (Postfix) with ESMTP id 1709F5A7CAA; Fri, 19 Jan 2007 18:14:24 +1100 (EST) Received: from besplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id 3DF8227423; Fri, 19 Jan 2007 18:14:22 +1100 (EST) Date: Fri, 19 Jan 2007 18:14:21 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Peter Jeremy In-Reply-To: <20070118184650.GB845@turion.vk2pj.dyndns.org> Message-ID: <20070119172335.O2216@besplex.bde.org> References: <20070116154258.568e1aaf@pleiades.nextvenue.com> <3bbf2fe10701161525j6ad9292y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe10701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <20070118113831.A11834@delplex.bde.org> <20070118184650.GB845@turion.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Maxim Sobolev , Ivan Voras , Attilio Rao , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2007 07:14:26 -0000 On Fri, 19 Jan 2007, Peter Jeremy wrote: > On Thu, 2007-Jan-18 18:03:20 +1100, Bruce Evans wrote: >> On Wed, 17 Jan 2007, Matthew Dillon wrote: >>> Alignment is critical. If the data is not aligned, don't bother. 128 >>> bits means 16 byte alignment. >> >> The above benchmark output is for aligned data :-). I don't try hard to >> optimize or benchmark misaligned cases. > > How realistic is this? Has anyone collected statistics on the size and > alignment of bzero/bcopy calls? How much of the time is the size known > at compile time? I think perfect alignment is very realistic. If not, it is an application bug :), just like for misaligned integer accesses on arches that allow this. In the kernel, other parts of the kernel are the application and it is reasonable to require perfect alignment. I recently did a dynamic search for misaligned (but only 32-bit non-aligned) bxx's (maybe only bzeros) in low-level network code and found only a couple. For the original i586 FPU optimizations, I gatherer statistics for bcopy/bzero. IIRC, alignment (64-bit?) was normal, at least for the large copies of interest, and large bcopys were so uncommon that it was a complete waste of time to optimize them (at least for my applications). Large bzeros/copyins/copyouts are more common. FreeBSD has some optimizations in low-level networking code for bcopys with a small size that is known at compile time (just use gcc's builtin_memcpy). These were lost to -ffreestanding and/or gcc's aggressive optimization of things like printf using the builtin printf. (-ffreestanding implies -fno-builtin, and no one cared enough about the loss to turn builtins back on. If you turn them back on, then they should be turned on individually as recommended in gcc.info to avoid conflicts. This is easy enough for the memcpy builtin but messy if you want all the old builtins starting with strlen.) I looked at these lost optimizations again while trying to optimize the low- level networking code for packets-per-second. The difficulty of implementing memcpy/bcopy perfectly is shown by gcc's builtin not being very close to getting it right for small fixed sizes even with -march=... I lost interest in this for now when I found that optimizations were impossible to measure because the packet rate depends mysteriously on the layout of the text section. My changes may have given +10%, but unrelated changes gave +-30%. The most mysterious one was -20% when cvs updated added ~500 bytes of object code that is never executed. Using builtin memcpy didn't have a noticeable effect here. Bruce