From owner-freebsd-arch@FreeBSD.ORG Sun Jul 20 09:33:12 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 454261065680 for ; Sun, 20 Jul 2008 09:33:12 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.178]) by mx1.freebsd.org (Postfix) with ESMTP id 246A18FC12 for ; Sun, 20 Jul 2008 09:33:12 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by wa-out-1112.google.com with SMTP id j4so477646wah.3 for ; Sun, 20 Jul 2008 02:33:11 -0700 (PDT) Received: by 10.114.25.3 with SMTP id 3mr1767112way.22.1216546391599; Sun, 20 Jul 2008 02:33:11 -0700 (PDT) Received: from ?10.0.1.199? ( [24.94.72.120]) by mx.google.com with ESMTPS id m40sm6176263waf.46.2008.07.20.02.33.09 (version=SSLv3 cipher=RC4-MD5); Sun, 20 Jul 2008 02:33:10 -0700 (PDT) Date: Sat, 19 Jul 2008 23:33:02 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Attilio Rao In-Reply-To: <3bbf2fe10807190525y65facf80uad2a974619198186@mail.gmail.com> Message-ID: <20080719233219.O954@desktop> References: <20080718163231.B954@desktop> <3bbf2fe10807190525y65facf80uad2a974619198186@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org, ivmaykov@gmail.com Subject: Re: witness performance improvements X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2008 09:33:12 -0000 On Sat, 19 Jul 2008, Attilio Rao wrote: > 2008/7/19, Jeff Roberson : >> Hello, >> >> I have a patch that improves witness performance available at: >> >> http://people.freebsd.org/~jeff/witness.diff >> >> This improvement comes at the cost of some significant space overhead. It >> changes the witness graph from a linked tree to a matrix based approach. >> Relationships can be quickly resolved with a table lookup. The table size >> is WITNESS_COUNT^2, or 1MB with the current count of 1024. >> >> This patch also makes struct witness objects persistent even after the last >> lock using this name has been removed. This is helpful for short lived >> objects which may be created frequently. >> >> To reduce lock contention on SMP witness_checkorder() now runs without the >> w_mtx when there are no lock violations. I also cache a lock_list_entry in >> each thread as allocating these requires the w_mtx. The entry is disposed >> of at thread_exit(). >> >> There is also a new sysctl that produces dot output which graphs lock order >> relationships with the graphviz program. > > As I alredy said, I don't like this. > I mostly prefer the current approach (comma separated stuff) that one > can shape as its need. > If you also think there are some informations the current sysctl > doesn't export and it should we could fix it, but IMHO we should axe > this part of the patch (I have still to look at this patch, but I > remind in the Isilon's version it was a good amount of structures and > modifies just to handle that part). Can you estimate how much effort it would take to port your previous graph solution to the current witness code? > >> Most of this work was done by Ilya Maykov while he was at Isilon systems. >> The locking work and some cleanup/porting/refinement was done by me on >> behalf of Nokia. >> >> The performance improvement can be significant. It is only on the order of >> 10-20% for buildkernel but on a packet forwarding test at nokia it sped >> things up by 5x putting a witness enabled kernel within about 50% of the >> performance of a kernel without. I believe buildworld isn't helped as much >> because forking and exiting a lot would then contend on the witness lock. >> >> I'm mostly interested in hearing what people have to say about the space >> bloat. I believe it is in a commit ready state. > > This should not be a big problem, it is a debugging kernel after all > if it has WITNESS. > > I hope I will have more time for a detailed revision in the day. I would appreciate that. Thanks, Jeff > > Thanks, > Attilio > > > -- > Peace can only be achieved by understanding - A. Einstein > From owner-freebsd-arch@FreeBSD.ORG Sun Jul 20 12:34:57 2008 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CC3B106566B; Sun, 20 Jul 2008 12:34:57 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from palm.hoeg.nl (mx0.hoeg.nl [IPv6:2001:610:652::211]) by mx1.freebsd.org (Postfix) with ESMTP id 60DB28FC1B; Sun, 20 Jul 2008 12:34:57 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: by palm.hoeg.nl (Postfix, from userid 1000) id 869D61CF83; Sun, 20 Jul 2008 14:32:56 +0200 (CEST) Date: Sun, 20 Jul 2008 14:32:56 +0200 From: Ed Schouten To: FreeBSD Current , FreeBSD Arch Message-ID: <20080720123256.GE21188@hoeg.nl> References: <20080702190901.GS14567@hoeg.nl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lkTb+7nhmha7W+c3" Content-Disposition: inline In-Reply-To: <20080702190901.GS14567@hoeg.nl> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: Subject: Re: MPSAFE TTY schedule X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2008 12:34:57 -0000 --lkTb+7nhmha7W+c3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello everyone, Today is July 20, which means I'm supposed to send you a message: * Ed Schouten wrote: > July 20 2008: > Send another heads-up to the lists about the new TTY layer. > Kindly ask people to test the patchset, port more drivers, etc. As usual, the latest mpsafetty patchset can be found here. I would really appreciate it if I could get more reviews on the code. Thanks! http://www.il.fontys.nl/~ed/projects/mpsafetty/patches/ The following drivers have not been ported to the new TTY layer yet: cy(4), digi(4), ng_h4(4), ng_tty(4), nmdm(4), rc(4), rp(4), si(4), sio(4), snp(4), ubser(4). I've been working on nmdm(4). I'll probably get it working in time. If not it will be fixed not long after the integration next month. The line disciplines like snp(4), ng_tty(4) and ng_h4(4) can only be fixed after the import, because the hooks layer will be written after the import. In the other news: kris@ reported a possible performance regression to me. He discovered `make -C /usr/ports index' consumed more system time on his hardware when the mpsafetty patches were applied. For some reason, I'm not capable of reproducing them. I even experience a performance gain when running mpsafetty, which is quite plausible, because I've also made some small improvements to `struct session' locking, but we also don't pick up Giant in kern_proc.c anymore. Because kris@ committed a patch to improve `make index' performance yesterday, I re-ran my tests today, showing the performance difference is now nihil. Here are the raw numbers: http://80386.nl/files/mpsafetty-stats.txt Maybe someone is interested in performing more thorough benchmarks? Yours, --=20 Ed Schouten WWW: http://80386.nl/ --lkTb+7nhmha7W+c3 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iEYEARECAAYFAkiDMHgACgkQ52SDGA2eCwXIaACePBQ+9QntL3faYFCyck8VOlOp C6QAn3E0Pd1dm3xBTa77pRrAbAvlPO5j =+NBc -----END PGP SIGNATURE----- --lkTb+7nhmha7W+c3-- From owner-freebsd-arch@FreeBSD.ORG Sun Jul 20 18:46:25 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CC48106571C for ; Sun, 20 Jul 2008 18:46:25 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id 0B5068FC17 for ; Sun, 20 Jul 2008 18:46:24 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from kobe.laptop (adsl10-74.kln.forthnet.gr [77.49.137.74]) (authenticated bits=128) by igloo.linux.gr (8.14.3/8.14.3/Debian-4) with ESMTP id m6KIXTTJ007280 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 20 Jul 2008 21:33:35 +0300 Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.2/8.14.2) with ESMTP id m6KIXTh6009813; Sun, 20 Jul 2008 21:33:29 +0300 (EEST) (envelope-from keramida@ceid.upatras.gr) Received: (from keramida@localhost) by kobe.laptop (8.14.2/8.14.2/Submit) id m6KIXTwJ009812; Sun, 20 Jul 2008 21:33:29 +0300 (EEST) (envelope-from keramida@ceid.upatras.gr) From: Giorgos Keramidas To: Ed Schouten References: <20080702190901.GS14567@hoeg.nl> <20080720123256.GE21188@hoeg.nl> Date: Sun, 20 Jul 2008 21:33:29 +0300 In-Reply-To: <20080720123256.GE21188@hoeg.nl> (Ed Schouten's message of "Sun, 20 Jul 2008 14:32:56 +0200") Message-ID: <87wsjg2yo6.fsf@kobe.laptop> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-MailScanner-ID: m6KIXTTJ007280 X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-3.79, required 5, autolearn=not spam, ALL_TRUSTED -1.80, AWL 0.61, BAYES_00 -2.60) X-Hellug-MailScanner-From: keramida@ceid.upatras.gr X-Spam-Status: No Cc: FreeBSD Arch , FreeBSD Current Subject: Re: MPSAFE TTY schedule X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2008 18:46:25 -0000 On Sun, 20 Jul 2008 14:32:56 +0200, Ed Schouten wrote: > Hello everyone, > > Today is July 20, which means I'm supposed to send you a message: > > * Ed Schouten wrote: >> July 20 2008: >> Send another heads-up to the lists about the new TTY layer. >> Kindly ask people to test the patchset, port more drivers, etc. > > As usual, the latest mpsafetty patchset can be found here. I would > really appreciate it if I could get more reviews on the code. Thanks! > > http://www.il.fontys.nl/~ed/projects/mpsafetty/patches/ Hi Ed, I see the latest patch at: http://www.il.fontys.nl/~ed/projects/mpsafetty/patches/mpsafetty-20080720.diff.gz Kris has mentioned that it breaks tcsh's autodetection for ptys (we've seen that before when /dev/pts kern.pts.enable=1 was added), so I'd like to build a test kernel+world with the patch to check this. Anything I should be careful about? From owner-freebsd-arch@FreeBSD.ORG Mon Jul 21 11:06:52 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7C821065670 for ; Mon, 21 Jul 2008 11:06:52 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A8FED8FC1F for ; Mon, 21 Jul 2008 11:06:52 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m6LB6qro031816 for ; Mon, 21 Jul 2008 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m6LB6qts031812 for freebsd-arch@FreeBSD.org; Mon, 21 Jul 2008 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 21 Jul 2008 11:06:52 GMT Message-Id: <200807211106.m6LB6qts031812@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2008 11:06:53 -0000 Current FreeBSD problem reports Critical problems Serious problems Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Mon Jul 21 20:55:49 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A8E61065683 for ; Mon, 21 Jul 2008 20:55:49 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id A72598FC19 for ; Mon, 21 Jul 2008 20:55:48 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.2/8.14.2) with ESMTP id m6LKtZI4084378; Mon, 21 Jul 2008 16:55:41 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-arch@freebsd.org Date: Mon, 21 Jul 2008 10:49:47 -0400 User-Agent: KMail/1.9.7 References: <34889018-8358-46AC-897E-32767FB84E14@mac.com> In-Reply-To: <34889018-8358-46AC-897E-32767FB84E14@mac.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200807211049.47579.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Mon, 21 Jul 2008 16:55:42 -0400 (EDT) X-Virus-Scanned: ClamAV 0.93.1/7770/Mon Jul 21 15:30:47 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.1 required=4.2 tests=AWL,BAYES_00, DATE_IN_PAST_06_12,NO_RELAYS autolearn=no version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Marcel Moolenaar Subject: Re: RFC: cross-libkvm/libthread_db/proc_service X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2008 20:55:49 -0000 On Saturday 19 July 2008 01:59:29 pm Marcel Moolenaar wrote: > All, > > We have a couple of interfaces/APIs that can't be used cross-platform. > > Take for example libkvm. On a 32-bit platform, we can't typically use > libkvm on a 64-kernel, because the libkvm interface uses u_long for > the target address, which on 32-bit platforms is 32 bits wide. > > Likewise, libthread_db and proc_service are designed for native use > only and need API tweaks to work in a cross-environment. Both use > psaddr_t to represent a target address, which is defined as a void* > in . > > I'd like to change those interfaces/APIs to allow them to be used in > a cross-platform debugging environment. Basically, this means that a > target address will have to be defined as a uint64_t. Other datatypes > may also need to be retyped. > > For libkvm in particular I don't want to redefine struct kinfo_proc, > struct nlist, etc. While it could be useful in a hybrid 32/64-bit > environment, the effect of such changes have too high a chance to > trickle down various other components/interfaces. Thus, for libkvm > the focus is on kvm_read() and kvm_write(). > > Suggested plan of attack: > o add kvm_xread() and kvm_xwrite() to the libkvm API to minimize > the overall impact. The new functions operate on a 64-bit target > address (psaddr_t). > o change psaddr_t from a void* to a 64-bit integral (sys/procfs.h) > This affects proc_service and libthread_db. And consequently our > threading support in GDB. > > Comments/thoughts? I think this is ok. However, can't you just make newer (1.1) versions of kvm_read/write instead of adding a new API? -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Jul 21 20:56:12 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBDFF106566B; Mon, 21 Jul 2008 20:56:12 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 31B068FC15; Mon, 21 Jul 2008 20:56:12 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.2/8.14.2) with ESMTP id m6LKtZI8084378; Mon, 21 Jul 2008 16:56:06 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: Jeff Roberson Date: Mon, 21 Jul 2008 11:41:09 -0400 User-Agent: KMail/1.9.7 References: <20080718163231.B954@desktop> In-Reply-To: <20080718163231.B954@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200807211141.09387.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Mon, 21 Jul 2008 16:56:06 -0400 (EDT) X-Virus-Scanned: ClamAV 0.93.1/7770/Mon Jul 21 15:30:47 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.3 required=4.2 tests=AWL,BAYES_00, DATE_IN_PAST_03_06,NO_RELAYS autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: attilio@freebsd.org, arch@freebsd.org, ivmaykov@gmail.com Subject: Re: witness performance improvements X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2008 20:56:12 -0000 On Friday 18 July 2008 10:41:58 pm Jeff Roberson wrote: > Hello, > > I have a patch that improves witness performance available at: > > http://people.freebsd.org/~jeff/witness.diff > > This improvement comes at the cost of some significant space overhead. It > changes the witness graph from a linked tree to a matrix based approach. > Relationships can be quickly resolved with a table lookup. The table size > is WITNESS_COUNT^2, or 1MB with the current count of 1024. Woo! Thanks for polishing this. > This patch also makes struct witness objects persistent even after the > last lock using this name has been removed. This is helpful for short > lived objects which may be created frequently. Originally, the idea was that if one had a LOR bug in a driver, one could kldunload the driver and have WITNESS forget about any orders for the driver's lock, fix the bug, and try again, but the short-lived names problem is much more common in practice, and trying to remove info about a specific lock class from the graph is a bit tenuous, so I think this is the better approach going forward. > To reduce lock contention on SMP witness_checkorder() now runs without the > w_mtx when there are no lock violations. I also cache a lock_list_entry > in each thread as allocating these requires the w_mtx. The entry is > disposed of at thread_exit(). Neat. > I'm mostly interested in hearing what people have to say about the space > bloat. I believe it is in a commit ready state. I think the space usage is perfectly fine. Also, now that you malloc the actual witness objects instead of putting them in the BSS (something that should have been done anyway I think), I would make the number of witness objects a loader tunable. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Jul 21 21:11:36 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A39B8106567B; Mon, 21 Jul 2008 21:11:36 +0000 (UTC) (envelope-from eischen@vigrid.com) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 5544E8FC16; Mon, 21 Jul 2008 21:11:36 +0000 (UTC) (envelope-from eischen@vigrid.com) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.3/8.14.3/NETPLEX) with ESMTP id m6LL0KXF018450; Mon, 21 Jul 2008 17:00:20 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Mon, 21 Jul 2008 17:00:20 -0400 (EDT) Date: Mon, 21 Jul 2008 17:00:19 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: John Baldwin In-Reply-To: <200807211049.47579.jhb@freebsd.org> Message-ID: References: <34889018-8358-46AC-897E-32767FB84E14@mac.com> <200807211049.47579.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Marcel Moolenaar , freebsd-arch@freebsd.org Subject: Re: RFC: cross-libkvm/libthread_db/proc_service X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2008 21:11:36 -0000 On Mon, 21 Jul 2008, John Baldwin wrote: > On Saturday 19 July 2008 01:59:29 pm Marcel Moolenaar wrote: >> All, >> >> We have a couple of interfaces/APIs that can't be used cross-platform. >> >> Take for example libkvm. On a 32-bit platform, we can't typically use >> libkvm on a 64-kernel, because the libkvm interface uses u_long for >> the target address, which on 32-bit platforms is 32 bits wide. >> >> Likewise, libthread_db and proc_service are designed for native use >> only and need API tweaks to work in a cross-environment. Both use >> psaddr_t to represent a target address, which is defined as a void* >> in . >> >> I'd like to change those interfaces/APIs to allow them to be used in >> a cross-platform debugging environment. Basically, this means that a >> target address will have to be defined as a uint64_t. Other datatypes >> may also need to be retyped. >> >> For libkvm in particular I don't want to redefine struct kinfo_proc, >> struct nlist, etc. While it could be useful in a hybrid 32/64-bit >> environment, the effect of such changes have too high a chance to >> trickle down various other components/interfaces. Thus, for libkvm >> the focus is on kvm_read() and kvm_write(). >> >> Suggested plan of attack: >> o add kvm_xread() and kvm_xwrite() to the libkvm API to minimize >> the overall impact. The new functions operate on a 64-bit target >> address (psaddr_t). >> o change psaddr_t from a void* to a 64-bit integral (sys/procfs.h) >> This affects proc_service and libthread_db. And consequently our >> threading support in GDB. >> >> Comments/thoughts? > > I think this is ok. However, can't you just make newer (1.1) versions of > kvm_read/write instead of adding a new API? You mean, "how about symbol versioning it"? -- DE From owner-freebsd-arch@FreeBSD.ORG Mon Jul 21 22:00:50 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A75A11065676 for ; Mon, 21 Jul 2008 22:00:50 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 8A7BC8FC08 for ; Mon, 21 Jul 2008 22:00:50 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 0E4721A4DA2; Mon, 21 Jul 2008 14:41:04 -0700 (PDT) Date: Mon, 21 Jul 2008 14:41:04 -0700 From: Alfred Perlstein To: Daniel Eischen Message-ID: <20080721214104.GF76659@elvis.mu.org> References: <34889018-8358-46AC-897E-32767FB84E14@mac.com> <200807211049.47579.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: Marcel Moolenaar , freebsd-arch@freebsd.org Subject: Re: RFC: cross-libkvm/libthread_db/proc_service X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2008 22:00:50 -0000 * Daniel Eischen [080721 14:11] wrote: > On Mon, 21 Jul 2008, John Baldwin wrote: > > >On Saturday 19 July 2008 01:59:29 pm Marcel Moolenaar wrote: > >>All, > >> > >>We have a couple of interfaces/APIs that can't be used cross-platform. > >> > >>Take for example libkvm. On a 32-bit platform, we can't typically use > >>libkvm on a 64-kernel, because the libkvm interface uses u_long for > >>the target address, which on 32-bit platforms is 32 bits wide. > >> > >>Likewise, libthread_db and proc_service are designed for native use > >>only and need API tweaks to work in a cross-environment. Both use > >>psaddr_t to represent a target address, which is defined as a void* > >>in . > >> > >>I'd like to change those interfaces/APIs to allow them to be used in > >>a cross-platform debugging environment. Basically, this means that a > >>target address will have to be defined as a uint64_t. Other datatypes > >>may also need to be retyped. > >> > >>For libkvm in particular I don't want to redefine struct kinfo_proc, > >>struct nlist, etc. While it could be useful in a hybrid 32/64-bit > >>environment, the effect of such changes have too high a chance to > >>trickle down various other components/interfaces. Thus, for libkvm > >>the focus is on kvm_read() and kvm_write(). > >> > >>Suggested plan of attack: > >>o add kvm_xread() and kvm_xwrite() to the libkvm API to minimize > >> the overall impact. The new functions operate on a 64-bit target > >> address (psaddr_t). > >>o change psaddr_t from a void* to a 64-bit integral (sys/procfs.h) > >> This affects proc_service and libthread_db. And consequently our > >> threading support in GDB. > >> > >>Comments/thoughts? > > > >I think this is ok. However, can't you just make newer (1.1) versions of > >kvm_read/write instead of adding a new API? > > You mean, "how about symbol versioning it"? Isn't it a bit strange to export 64bit pointers to 32 bit userspace? Once this "hurdle" is breached, everything else will be horked as well, all pointers to internal structures will be hosed as well and need magic to read them. It seems like a real uphill battle to pound 64bit pegs into 32bit holes. I would like to see the "next steps" as in, what is expected to work after this is added and what will need additional shoe-horning to work. It seems to make a lot more sense to allow for 32bit compat sysctls than this. -- - Alfred Perlstein From owner-freebsd-arch@FreeBSD.ORG Mon Jul 21 22:14:47 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2EA6B1065684 for ; Mon, 21 Jul 2008 22:14:47 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout018.mac.com (asmtpout018.mac.com [17.148.16.93]) by mx1.freebsd.org (Postfix) with ESMTP id 12A8F8FC14 for ; Mon, 21 Jul 2008 22:14:46 +0000 (UTC) (envelope-from xcllnt@mac.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Received: from macbook-pro.jnpr.net (natint3.juniper.net [66.129.224.36]) by asmtp018.mac.com (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) with ESMTPSA id <0K4D00A8TKC9IR50@asmtp018.mac.com>; Mon, 21 Jul 2008 14:14:34 -0700 (PDT) Sender: xcllnt@mac.com Message-id: <0AD2E758-D7E1-41B0-BA31-6FBDEE3470A9@mac.com> From: Marcel Moolenaar To: John Baldwin In-reply-to: <200807211049.47579.jhb@freebsd.org> Date: Mon, 21 Jul 2008 14:14:32 -0700 References: <34889018-8358-46AC-897E-32767FB84E14@mac.com> <200807211049.47579.jhb@freebsd.org> X-Mailer: Apple Mail (2.928.1) Cc: freebsd-arch@freebsd.org Subject: Re: RFC: cross-libkvm/libthread_db/proc_service X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2008 22:14:47 -0000 On Jul 21, 2008, at 7:49 AM, John Baldwin wrote: > On Saturday 19 July 2008 01:59:29 pm Marcel Moolenaar wrote: >> All, >> >> We have a couple of interfaces/APIs that can't be used cross- >> platform. >> >> Take for example libkvm. On a 32-bit platform, we can't typically use >> libkvm on a 64-kernel, because the libkvm interface uses u_long for >> the target address, which on 32-bit platforms is 32 bits wide. >> >> Likewise, libthread_db and proc_service are designed for native use >> only and need API tweaks to work in a cross-environment. Both use >> psaddr_t to represent a target address, which is defined as a void* >> in . >> >> I'd like to change those interfaces/APIs to allow them to be used in >> a cross-platform debugging environment. Basically, this means that a >> target address will have to be defined as a uint64_t. Other datatypes >> may also need to be retyped. >> >> For libkvm in particular I don't want to redefine struct kinfo_proc, >> struct nlist, etc. While it could be useful in a hybrid 32/64-bit >> environment, the effect of such changes have too high a chance to >> trickle down various other components/interfaces. Thus, for libkvm >> the focus is on kvm_read() and kvm_write(). >> >> Suggested plan of attack: >> o add kvm_xread() and kvm_xwrite() to the libkvm API to minimize >> the overall impact. The new functions operate on a 64-bit target >> address (psaddr_t). >> o change psaddr_t from a void* to a 64-bit integral (sys/procfs.h) >> This affects proc_service and libthread_db. And consequently our >> threading support in GDB. >> >> Comments/thoughts? > > I think this is ok. However, can't you just make newer (1.1) > versions of > kvm_read/write instead of adding a new API? The impact will be bigger, though. Any reference to kvm_read/kvm_write needs to be changed then. Think about stock GDB and various ports for example. Are we willing to differentiate from other OSes, provided we are willing to follow it through all the way? -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Wed Jul 23 05:12:11 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11ED6106567D for ; Wed, 23 Jul 2008 05:12:11 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx08.syd.optusnet.com.au (fallbackmx08.syd.optusnet.com.au [211.29.132.10]) by mx1.freebsd.org (Postfix) with ESMTP id 2A4BF8FC15 for ; Wed, 23 Jul 2008 05:12:06 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail02.syd.optusnet.com.au (mail02.syd.optusnet.com.au [211.29.132.183]) by fallbackmx08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m6MHIQE2025652 for ; Wed, 23 Jul 2008 03:18:27 +1000 Received: from c220-239-252-11.carlnfd3.nsw.optusnet.com.au (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail02.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m6MHIKrK025636 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 23 Jul 2008 03:18:22 +1000 Date: Wed, 23 Jul 2008 03:18:20 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Alfred Perlstein In-Reply-To: <20080721214104.GF76659@elvis.mu.org> Message-ID: <20080723025519.F18257@delplex.bde.org> References: <34889018-8358-46AC-897E-32767FB84E14@mac.com> <200807211049.47579.jhb@freebsd.org> <20080721214104.GF76659@elvis.mu.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Marcel Moolenaar , freebsd-arch@FreeBSD.org Subject: Re: RFC: cross-libkvm/libthread_db/proc_service X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2008 05:12:11 -0000 On Mon, 21 Jul 2008, Alfred Perlstein wrote: > Isn't it a bit strange to export 64bit pointers to 32 bit userspace? Only for pointers in kernel objects, and I think the proposed change doesn't touch that. kvm_read() doesn't use pointers for kernel addresses. It uses unsigned longs. But even uintmax_t is not enough in general, since the application uintmax_t might be too small to represent a kernel pointer. The type used shouldn't be fixed-width, but typedefed in an MD way like vm_offset_t. vm_offset_t gives the correct integral type to use for (mapped) kernel addresses and related compat_fewer_bit[s] type[s] are needed in userland. It would probably be too hard to support the general case which requires the compat types to be arrays or structs. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Jul 23 11:11:26 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C86E106566B for ; Wed, 23 Jul 2008 11:11:26 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx10.syd.optusnet.com.au (fallbackmx10.syd.optusnet.com.au [211.29.132.251]) by mx1.freebsd.org (Postfix) with ESMTP id 6AABB8FC15 for ; Wed, 23 Jul 2008 11:11:25 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail05.syd.optusnet.com.au (mail05.syd.optusnet.com.au [211.29.132.186]) by fallbackmx10.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m6MHRNu0005863 for ; Wed, 23 Jul 2008 03:27:23 +1000 Received: from c220-239-252-11.carlnfd3.nsw.optusnet.com.au (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail05.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m6MHRJRH032634 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 23 Jul 2008 03:27:20 +1000 Date: Wed, 23 Jul 2008 03:27:19 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Bruce Evans In-Reply-To: <20080723025519.F18257@delplex.bde.org> Message-ID: <20080723032109.W18257@delplex.bde.org> References: <34889018-8358-46AC-897E-32767FB84E14@mac.com> <200807211049.47579.jhb@freebsd.org> <20080721214104.GF76659@elvis.mu.org> <20080723025519.F18257@delplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Alfred Perlstein , Marcel Moolenaar , freebsd-arch@FreeBSD.org Subject: Re: RFC: cross-libkvm/libthread_db/proc_service X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2008 11:11:26 -0000 On Wed, 23 Jul 2008, Bruce Evans wrote: > On Mon, 21 Jul 2008, Alfred Perlstein wrote: > >> Isn't it a bit strange to export 64bit pointers to 32 bit userspace? > > Only for pointers in kernel objects, and I think the proposed change > doesn't touch that. > > kvm_read() doesn't use pointers for kernel addresses. It uses unsigned > longs. But even uintmax_t is not enough in general, since the application > uintmax_t might be too small to represent a kernel pointer. The type > used shouldn't be fixed-width, but typedefed in an MD way like vm_offset_t. > vm_offset_t gives the correct integral type to use for (mapped) kernel > addresses and related compat_fewer_bit[s] type[s] are needed in userland. > It would probably be too hard to support the general case which requires > the compat types to be arrays or structs. Bah, I forgot the original mail which already says to use an integral type named psaddr_t, and that, unfortunately, this seems to need being 64 bits even on pure 32-bit systems in case you want to run an (not quite pure) 32-bit application in compat32 mode on 64-bit system without recompiling. If psaddr_t is 32-bits on i386 but 64-bits on amd64, then pure 32-bit i386 applications won't run in compat32 mode on amd64, though (not quite pure) 32-bit applications compiled on amd64 will. I don't like putting 64-bit knowledge in 32-bit applications but I often compile on i386 and run on amd64. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Jul 23 17:14:32 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3FA5106566B; Wed, 23 Jul 2008 17:14:32 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout022.mac.com (asmtpout022.mac.com [17.148.16.97]) by mx1.freebsd.org (Postfix) with ESMTP id AA51F8FC13; Wed, 23 Jul 2008 17:14:32 +0000 (UTC) (envelope-from xcllnt@mac.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Received: from macbook-pro.jnpr.net (natint3.juniper.net [66.129.224.36]) by asmtp022.mac.com (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) with ESMTPSA id <0K4G002JQVT8LF90@asmtp022.mac.com>; Wed, 23 Jul 2008 09:15:09 -0700 (PDT) Sender: xcllnt@mac.com Message-id: <71C01B9B-1E42-4D65-A3D7-F1DA14123524@mac.com> From: Marcel Moolenaar To: Bruce Evans In-reply-to: <20080723032109.W18257@delplex.bde.org> Date: Wed, 23 Jul 2008 09:15:06 -0700 References: <34889018-8358-46AC-897E-32767FB84E14@mac.com> <200807211049.47579.jhb@freebsd.org> <20080721214104.GF76659@elvis.mu.org> <20080723025519.F18257@delplex.bde.org> <20080723032109.W18257@delplex.bde.org> X-Mailer: Apple Mail (2.928.1) Cc: Alfred Perlstein , freebsd-arch@FreeBSD.org Subject: Re: RFC: cross-libkvm/libthread_db/proc_service X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2008 17:14:32 -0000 On Jul 22, 2008, at 10:27 AM, Bruce Evans wrote: > On Wed, 23 Jul 2008, Bruce Evans wrote: > >> On Mon, 21 Jul 2008, Alfred Perlstein wrote: >> >>> Isn't it a bit strange to export 64bit pointers to 32 bit userspace? >> >> Only for pointers in kernel objects, and I think the proposed change >> doesn't touch that. >> >> kvm_read() doesn't use pointers for kernel addresses. It uses >> unsigned >> longs. But even uintmax_t is not enough in general, since the >> application >> uintmax_t might be too small to represent a kernel pointer. The type >> used shouldn't be fixed-width, but typedefed in an MD way like >> vm_offset_t. >> vm_offset_t gives the correct integral type to use for (mapped) >> kernel >> addresses and related compat_fewer_bit[s] type[s] are needed in >> userland. >> It would probably be too hard to support the general case which >> requires >> the compat types to be arrays or structs. > > Bah, I forgot the original mail which already says to use an integral > type named psaddr_t, and that, unfortunately, this seems to need being > 64 bits even on pure 32-bit systems in case you want to run an (not > quite pure) 32-bit application in compat32 mode on 64-bit system > without > recompiling. Actually, the intend is more generic (or more limited, depending on how you look at it): the ability to cross-debug any (say ia64) kernel on any other (say powerpc) machine. The integral needs to be constant and equal in width across all platforms. This means that an uint<#>_t is the best option. The largest we support is 64-bit. We can already build a cross gdb from the source tree, but without threading support (libthread_db and proc_service limitation). We can't build a cross kgdb from the source tree because of libkvm. Both I'd like to be able to do. FYI, -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Fri Jul 25 02:30:21 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D3221065672 for ; Fri, 25 Jul 2008 02:30:21 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.176]) by mx1.freebsd.org (Postfix) with ESMTP id 578F28FC14 for ; Fri, 25 Jul 2008 02:30:21 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by wa-out-1112.google.com with SMTP id j4so1706064wah.3 for ; Thu, 24 Jul 2008 19:30:20 -0700 (PDT) Received: by 10.114.89.1 with SMTP id m1mr1387048wab.146.1216953020653; Thu, 24 Jul 2008 19:30:20 -0700 (PDT) Received: from ?10.0.1.199? ( [24.94.72.120]) by mx.google.com with ESMTPS id n37sm15385400wag.43.2008.07.24.19.30.17 (version=SSLv3 cipher=RC4-MD5); Thu, 24 Jul 2008 19:30:19 -0700 (PDT) Date: Thu, 24 Jul 2008 16:30:36 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: John Baldwin In-Reply-To: <200807211141.09387.jhb@freebsd.org> Message-ID: <20080724162733.B954@desktop> References: <20080718163231.B954@desktop> <200807211141.09387.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: attilio@freebsd.org, arch@freebsd.org, ivmaykov@gmail.com Subject: Re: witness performance improvements X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2008 02:30:21 -0000 On Mon, 21 Jul 2008, John Baldwin wrote: > On Friday 18 July 2008 10:41:58 pm Jeff Roberson wrote: >> Hello, >> >> I have a patch that improves witness performance available at: >> >> http://people.freebsd.org/~jeff/witness.diff >> >> This improvement comes at the cost of some significant space overhead. It >> changes the witness graph from a linked tree to a matrix based approach. >> Relationships can be quickly resolved with a table lookup. The table size >> is WITNESS_COUNT^2, or 1MB with the current count of 1024. > > Woo! Thanks for polishing this. > >> This patch also makes struct witness objects persistent even after the >> last lock using this name has been removed. This is helpful for short >> lived objects which may be created frequently. > > Originally, the idea was that if one had a LOR bug in a driver, one could > kldunload the driver and have WITNESS forget about any orders for the > driver's lock, fix the bug, and try again, but the short-lived names problem > is much more common in practice, and trying to remove info about a specific > lock class from the graph is a bit tenuous, so I think this is the better > approach going forward. > >> To reduce lock contention on SMP witness_checkorder() now runs without the >> w_mtx when there are no lock violations. I also cache a lock_list_entry >> in each thread as allocating these requires the w_mtx. The entry is >> disposed of at thread_exit(). > > Neat. > >> I'm mostly interested in hearing what people have to say about the space >> bloat. I believe it is in a commit ready state. > > I think the space usage is perfectly fine. Also, now that you malloc the > actual witness objects instead of putting them in the BSS (something that > should have been done anyway I think), I would make the number of witness > objects a loader tunable. Well, I'm glad there is a consensus that this is the right way forward. The state of the code is that there may be a bug in the dot output but I've not had any problems with the regular witness operation. There are still some style bugs in it. Attilio has expressed some interest in a full review and style clean-up. I'd like to get what I have now, minus the dot output, into svn. And then do a set of follow on commits to add back dot or Attilio's comma separated graph output that can be parsed to dot. Any objections to commiting this knowing it has some style bugs and a little work left? I'd like to get people testing the core functionality more. We've sat on this patch for a couple of years now as well. Thanks, Jeff > > -- > John Baldwin > From owner-freebsd-arch@FreeBSD.ORG Fri Jul 25 16:27:36 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B955510656A7; Fri, 25 Jul 2008 16:27:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 2D4638FC14; Fri, 25 Jul 2008 16:27:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.2/8.14.2) with ESMTP id m6PGRDEi071433; Fri, 25 Jul 2008 12:27:29 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: Jeff Roberson Date: Fri, 25 Jul 2008 11:46:18 -0400 User-Agent: KMail/1.9.7 References: <20080718163231.B954@desktop> <200807211141.09387.jhb@freebsd.org> <20080724162733.B954@desktop> In-Reply-To: <20080724162733.B954@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200807251146.19058.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Fri, 25 Jul 2008 12:27:29 -0400 (EDT) X-Virus-Scanned: ClamAV 0.93.1/7826/Fri Jul 25 08:51:06 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,NO_RELAYS autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: attilio@freebsd.org, arch@freebsd.org, ivmaykov@gmail.com Subject: Re: witness performance improvements X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2008 16:27:36 -0000 On Thursday 24 July 2008 10:30:36 pm Jeff Roberson wrote: > On Mon, 21 Jul 2008, John Baldwin wrote: > > > On Friday 18 July 2008 10:41:58 pm Jeff Roberson wrote: > >> Hello, > >> > >> I have a patch that improves witness performance available at: > >> > >> http://people.freebsd.org/~jeff/witness.diff > >> > >> This improvement comes at the cost of some significant space overhead. It > >> changes the witness graph from a linked tree to a matrix based approach. > >> Relationships can be quickly resolved with a table lookup. The table size > >> is WITNESS_COUNT^2, or 1MB with the current count of 1024. > > > > Woo! Thanks for polishing this. > > > >> This patch also makes struct witness objects persistent even after the > >> last lock using this name has been removed. This is helpful for short > >> lived objects which may be created frequently. > > > > Originally, the idea was that if one had a LOR bug in a driver, one could > > kldunload the driver and have WITNESS forget about any orders for the > > driver's lock, fix the bug, and try again, but the short-lived names problem > > is much more common in practice, and trying to remove info about a specific > > lock class from the graph is a bit tenuous, so I think this is the better > > approach going forward. > > > >> To reduce lock contention on SMP witness_checkorder() now runs without the > >> w_mtx when there are no lock violations. I also cache a lock_list_entry > >> in each thread as allocating these requires the w_mtx. The entry is > >> disposed of at thread_exit(). > > > > Neat. > > > >> I'm mostly interested in hearing what people have to say about the space > >> bloat. I believe it is in a commit ready state. > > > > I think the space usage is perfectly fine. Also, now that you malloc the > > actual witness objects instead of putting them in the BSS (something that > > should have been done anyway I think), I would make the number of witness > > objects a loader tunable. > > Well, I'm glad there is a consensus that this is the right way forward. > The state of the code is that there may be a bug in the dot output but > I've not had any problems with the regular witness operation. > > There are still some style bugs in it. Attilio has expressed some > interest in a full review and style clean-up. I'd like to get what I have > now, minus the dot output, into svn. And then do a set of follow on > commits to add back dot or Attilio's comma separated graph output that can > be parsed to dot. > > Any objections to commiting this knowing it has some style bugs and a > little work left? I'd like to get people testing the core functionality > more. We've sat on this patch for a couple of years now as well. I think you can commit and work on the other stuff afterwards. I had noticed a few style things too but don't want those to hold this up. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Jul 25 22:17:41 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D59251065680 for ; Fri, 25 Jul 2008 22:17:41 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.154]) by mx1.freebsd.org (Postfix) with ESMTP id 5F68A8FC0C for ; Fri, 25 Jul 2008 22:17:40 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so2788743fgb.35 for ; Fri, 25 Jul 2008 15:17:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=EeNVVKWUD956mOQ1k2CH2jzHhyr1W7imaA8K2BIZcFs=; b=biMOP4cC+1jO5KzUCsacrF4pLerovKPjWc4J45zjsQh9ySn0M0wR79XMUj0YaAa9ji 9daCgjFIiGWOlA/rNnJRcs8ApHp8Mzzv9VJG4sKx9km0gnafx0Nw+n4rLsdolyV0s9/q VoKZiBINnklzYrdhgxVWY424viZ9GOKHV/q+s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=hv0+//QdqY0yuocAyG5hRhMXbJpvLPwLJ9DKCxObUqauK535KdiwZrABsrHsSEy++C LuuATfT5rl3jFvshQHzLPb4GlE1F2F2OATVkQn3cBqVU8DoWtgPJQRtMxOST1VmsHkjh LHilEYwokSJO6YcL/IHzBj+AjShpxQHukJVcE= Received: by 10.86.98.14 with SMTP id v14mr379263fgb.74.1217024259643; Fri, 25 Jul 2008 15:17:39 -0700 (PDT) Received: by 10.86.2.18 with HTTP; Fri, 25 Jul 2008 15:17:39 -0700 (PDT) Message-ID: <3bbf2fe10807251517v73447626j90458ebcd5345eaf@mail.gmail.com> Date: Sat, 26 Jul 2008 00:17:39 +0200 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Jeff Roberson" In-Reply-To: <20080724162733.B954@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20080718163231.B954@desktop> <200807211141.09387.jhb@freebsd.org> <20080724162733.B954@desktop> X-Google-Sender-Auth: 548be71ed4e18716 Cc: arch@freebsd.org, ivmaykov@gmail.com Subject: Re: witness performance improvements X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2008 22:17:41 -0000 2008/7/25, Jeff Roberson : > On Mon, 21 Jul 2008, John Baldwin wrote: > > > On Friday 18 July 2008 10:41:58 pm Jeff Roberson wrote: > > > > > Hello, > > > > > > I have a patch that improves witness performance available at: > > > > > > http://people.freebsd.org/~jeff/witness.diff > > > > > > This improvement comes at the cost of some significant space overhead. > It > > > changes the witness graph from a linked tree to a matrix based approach. > > > Relationships can be quickly resolved with a table lookup. The table > size > > > is WITNESS_COUNT^2, or 1MB with the current count of 1024. > > > > > > > Woo! Thanks for polishing this. > > > > > > > This patch also makes struct witness objects persistent even after the > > > last lock using this name has been removed. This is helpful for short > > > lived objects which may be created frequently. > > > > > > > Originally, the idea was that if one had a LOR bug in a driver, one could > > kldunload the driver and have WITNESS forget about any orders for the > > driver's lock, fix the bug, and try again, but the short-lived names > problem > > is much more common in practice, and trying to remove info about a > specific > > lock class from the graph is a bit tenuous, so I think this is the better > > approach going forward. > > > > > > > To reduce lock contention on SMP witness_checkorder() now runs without > the > > > w_mtx when there are no lock violations. I also cache a lock_list_entry > > > in each thread as allocating these requires the w_mtx. The entry is > > > disposed of at thread_exit(). > > > > > > > Neat. > > > > > > > I'm mostly interested in hearing what people have to say about the space > > > bloat. I believe it is in a commit ready state. > > > > > > > I think the space usage is perfectly fine. Also, now that you malloc the > > actual witness objects instead of putting them in the BSS (something that > > should have been done anyway I think), I would make the number of witness > > objects a loader tunable. > > > > Well, I'm glad there is a consensus that this is the right way forward. The > state of the code is that there may be a bug in the dot output but I've not > had any problems with the regular witness operation. > > There are still some style bugs in it. Attilio has expressed some interest > in a full review and style clean-up. I'd like to get what I have now, minus > the dot output, into svn. And then do a set of follow on commits to add > back dot or Attilio's comma separated graph output that can be parsed to > dot. > > Any objections to commiting this knowing it has some style bugs and a little > work left? I'd like to get people testing the core functionality more. > We've sat on this patch for a couple of years now as well. Please go on and commit the code, delaying any further improvement. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein