From owner-freebsd-arch@freebsd.org Wed Dec 28 11:56:53 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7A7BC94DF4; Wed, 28 Dec 2016 11:56:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 58D7B1250; Wed, 28 Dec 2016 11:56:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBSBukm6036421 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 28 Dec 2016 13:56:46 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBSBukm6036421 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBSBuk7e036420; Wed, 28 Dec 2016 13:56:46 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 28 Dec 2016 13:56:46 +0200 From: Konstantin Belousov To: Andriy Gapon Cc: "freebsd-arch@freebsd.org" , freebsd-fs Subject: Re: INVARIANTS vs DIAGNOSTIC % lf_advlockasync Message-ID: <20161228115646.GU94325@kib.kiev.ua> References: <2225968a-7bce-b100-f3fa-a5e2eb8b9f47@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2225968a-7bce-b100-f3fa-a5e2eb8b9f47@FreeBSD.org> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2016 11:56:53 -0000 On Wed, Dec 28, 2016 at 11:28:54AM +0200, Andriy Gapon wrote: > > I wonder if there are any guidelines on when to use INVARIANTS vs DIAGNOSTIC vs > something else. Should the amount of output be taken into account? Or the > performance impact? Or just the common sense? > > What I really mean is that if some sanity check could be rather expensive (e.g. > it needs to iterate over a potentially long list), what option should be used to > enabled it? > > I ask this question with one particular case in mind. > lf_advlockasync() has a block of code under INVARIANTS with a loop over a list > that has a nested loop over the same list for pair-wise checks. > What's worse is that that code is executed with a lock held and that lock can > potentially be highly contended (ls_lock). > In our test environment we can observe the lock being held for as much as 125 > milliseconds resulting in a huge backlog on the lock. (Even though the > requested advisory locks are all shared locks and unlocks.) There are at least two blocks of code in kern_lockf.c that you might want to conditionally enable, one is lf_advlockasync(), another in the beginning of lf_add_edge(). > > So, we would like to reduce the performance hit in that code, but still have the > benefits of INVARIANTS enabled over all. > We have a precedent with DIAGNOSTIC enabling very heavy weight checks, see kern/subr_vmem.c enable_vmem_check sysctl.