Date: Fri, 14 Nov 2014 12:22:09 +1100 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Navdeep Parhar <np@freebsd.org> Cc: "Grier, James" <Grier@netapp.com>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: missing DTrace FBT return probes Message-ID: <20141114115632.J1286@besplex.bde.org> In-Reply-To: <5464FBFF.6020200@FreeBSD.org> References: <AD83D10D-C58C-4A6F-93B7-62592C973919@netapp.com> <5464FBFF.6020200@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 13 Nov 2014, Navdeep Parhar wrote: > On 11/13/14 08:33, Grier, James wrote: >> On Wed, Jun 5, 2013 at 5:50 PM, Navdeep Parhar <np at freebsd.org> wrote= : >=20 >> A large number of kernel functions have an FBT entry probe but no return >>> probe. I believe this is due to tail call optimization by the compiler. >>> Should we disable this optimization for kernel configs that have DTrace >>> support? The missing return probes make it very difficult to write >>> DTrace scripts that want to set flags etc. at function entry and then >>> clean them up on return. >>>=20 >>> A quick sample from a recent HEAD shows ~4000 out of ~27000 functions >>> are missing return probes. See the list of functions in these files >>> (the ones listed in entry-only.txt do not have return probes). >>>=20 >>> http://people.freebsd.org/~np/entry-only.txt >>> http://people.freebsd.org/~np/entry.txt >>> http://people.freebsd.org/~np/return.txt >>=20 >> By trial and error, I discovered that it=E2=80=99s =E2=80=98-funit-at-a-= time=E2=80=99 that=20 >> does the dirty work, so >> using =E2=80=98-fno-unit-at-a-time=E2=80=99 with -O2 we get all the retu= rn fbt probes.=20 >> I haven=E2=80=99t determined >> the performance impact of this. > > Hmm, I thought it was -fno-optimize-sibling-calls that controlled this=20 > particular optimization. Perhaps it is all of these, plus -fno-inline-functions-called-once. Inlining functions when not requested to do so gives undebuggable code. -funit-at-a-time exposes even static functions that are defined after they are used to inlining. > These days I compile my debug kernels with -O0. There is a very signific= ant=20 > performance impact but it makes kgdb and DTrace very happy. But -O0 means that you are not testing what will be used on production systems. gcc used to promise that -g not disturb anything and that -O not make debugging impossible. Lots of flags that were under -O2 in gcc-3 crept into -O in gcc-4. clang is worse -- its -O is more agressive than gcc-4's -O2, and its -O2 is little different from its -O. Apart from kgdb and DTrace, this breaks: - profiling - kernel stack traces from ddb - ddb generally. With no debugging symbol support, it is even harder to find where args and local variables are if they are optimized away. (In userland, they often cannot be found even with debugging symbol support.) ddb's heuristic for finding args never worked for amd64 and is now under #if !1 with a comment that dwarf2 is needed. Thus, stack traces never showed any args correctly on amd64. I recently noticed a case where gcc without many optimizations on i386 messed up the arg printing because the function was static and gcc "optimized" it by passing an arg in %eax instead of on the stack. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri Nov 14 01:55:30 2014 Return-Path: <owner-freebsd-arch@FreeBSD.ORG> Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB146812; Fri, 14 Nov 2014 01:55:30 +0000 (UTC) Received: from mail-ie0-x229.google.com (mail-ie0-x229.google.com [IPv6:2607:f8b0:4001:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8DE6094B; Fri, 14 Nov 2014 01:55:30 +0000 (UTC) Received: by mail-ie0-f169.google.com with SMTP id y20so179475ier.0 for <multiple recipients>; Thu, 13 Nov 2014 17:55:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=OAuWdS4dmVFXMZ0gdjxpYWjaqa36yqtAKFjmj8tcR8M=; b=MWa8Whoz4AT4F/0UEoNbcS7rLL+yj5U/JzeLZYv6cEkvmemGxxlNEIIg2+UNu6U6qf 16QMvWPXznvNyDqYEArmdDiRn8l50fwYffpRkuIl1N2k+rL7ubiwsskbhuzYsgeKyC0O 7GnXe1/O6XP+4+aCzjvIbj9GGP68/CC+qrrAh68JVfL3d/tphwSySXhh7ZqrRLv0PILL znwcc6QV4PBg8BruIvcy/Q1SOtD4ynlJ39i+gJ6ES588xnHNVciTEQDgUMMUMj9loGay Yi/UdxEuGVfabGengUBUbzCiQCaWB9jGFPA9VXUfRX9HB/dOflk0dmwVOzFID5VJH4xu CPCw== MIME-Version: 1.0 X-Received: by 10.107.13.6 with SMTP id 6mr6562099ion.68.1415930129975; Thu, 13 Nov 2014 17:55:29 -0800 (PST) Received: by 10.50.235.49 with HTTP; Thu, 13 Nov 2014 17:55:29 -0800 (PST) In-Reply-To: <CAJ-VmokfyLTfx-HVkKV5bLY3S07AeQgO0kBaw7JkGdgDzscuTg@mail.gmail.com> References: <CAJ-VmomrauhCMoF_dZfMWWhZp0EgwfE9RmxL5Pc37PhLSzZ6Qg@mail.gmail.com> <54647D1E.9010904@freebsd.org> <CAJ-VmonbEfxz9Bgw9O9f-5+b=UM1b1nzPK9zfAAnmYKVumOKkQ@mail.gmail.com> <201411130948.23785.jhb@freebsd.org> <CAJ-Vmo=ZW0i0upfPkf2fx8GgnKn943dZr96_PsGnw5NgcbC7NQ@mail.gmail.com> <ECB3576A-CD92-4F19-9EA3-CFF430165D0B@gmail.com> <CAJ-VmokfyLTfx-HVkKV5bLY3S07AeQgO0kBaw7JkGdgDzscuTg@mail.gmail.com> Date: Thu, 13 Nov 2014 17:55:29 -0800 Message-ID: <CAGHfRMCrZud2nUTYj=MMFnfdh9ke1M6O_6BgZw_zmi8ia4=iEw@mail.gmail.com> Subject: Re: Questions about locking; turnstiles and sleeping threads From: NGie Cooper <yaneurabeya@gmail.com> To: Adrian Chadd <adrian@freebsd.org> Content-Type: text/plain; charset=UTF-8 Cc: Hans Petter Selasky <hps@selasky.org>, Alfred Perlstein <alfred@freebsd.org>, Steve Kargl <sgk@troutmask.apl.washington.edu>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture <freebsd-arch.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-arch>, <mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch/>; List-Post: <mailto:freebsd-arch@freebsd.org> List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>, <mailto:freebsd-arch-request@freebsd.org?subject=subscribe> X-List-Received-Date: Fri, 14 Nov 2014 01:55:30 -0000 On Thu, Nov 13, 2014 at 3:01 PM, Adrian Chadd <adrian@freebsd.org> wrote: > I'd like to try and add WITNESS_WARN() where appropriate. > > How's this look? > > Index: kern_timeout.c > =================================================================== > --- kern_timeout.c (revision 274304) > +++ kern_timeout.c (working copy) > @@ -1096,7 +1096,22 @@ > struct lock_class *class; > int direct, sq_locked, use_lock; > > + /* XXX GIANTOK? c_lock can be NULL? */ > + > /* > + * If safe is clear then we're not going to drop into the > + * sleepq wait routines. So, we don't have to check > + * for the witness warning here. > + * > + * If safe is set then we may drop into the sleep routines > + * so do the check. > + */ > + if (safe) { > + WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, c->c_lock, > + "calling %s", __func__); > + } > + > + /* > * Some old subsystems don't hold Giant while running a callout_stop(), > * so just discard this check for the moment. > */ - This should have WITNESS #ifdef guards (for clarity... it would probably be optimized out by the compiler if -O0 wasn't specified, but some folks seem to use -O0 lately, like np@). - Should this have __predict_{false,true} applied to the if (safe) check? Thanks!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141114115632.J1286>