From owner-freebsd-arch@FreeBSD.ORG Wed Aug 13 01:00:52 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D300EE72; Wed, 13 Aug 2014 01:00:52 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 452932066; Wed, 13 Aug 2014 01:00:52 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id f8so73937wiw.0 for ; Tue, 12 Aug 2014 18:00:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=APSIwi3AQXGXeivwOXOxKI01UsBuX07vUJZozdwLxjI=; b=ZJA/pAKuKxktIrV11bUuZFoWyGQUNxk826zt74hKHbIcHUN4lb1c8ARlcZGFDC4a+A HeGoOvSiqsMSW2svL3yrH6lTJMeUI/KUhMDv7IHTobIcqDpz1+wQqtt99sWpZSrnxUKw Hp9lbXPviQ3cIgwxD3YKV2k5OHQqKadX0gt79Y6hIjxeNyIPvXj8TKLaE5D5m6bALP34 L3jiUUNrL1BUV3dIDQv+TZhZmDxhO9FR8c86n2kzHUuRpmdum2LTIua6fGM0EDdvruHY kq8/tmMpStCXlkwSHDT8YSD7aPM/TkEtw6gB+8zqL2uAqt7vxy4ejQkUyIhQ8Cx8+vvW dDKg== X-Received: by 10.180.206.84 with SMTP id lm20mr35747330wic.9.1407891650529; Tue, 12 Aug 2014 18:00:50 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id lq15sm2512779wic.1.2014.08.12.18.00.49 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 12 Aug 2014 18:00:49 -0700 (PDT) Date: Wed, 13 Aug 2014 03:00:46 +0200 From: Mateusz Guzik To: Konstantin Belousov Subject: Re: Getting rid of atomic_load_acq_int(&fdp->fd_nfiles)) from fget_unlocked Message-ID: <20140813010046.GB17869@dft-labs.eu> References: <20140713035500.GC16884@dft-labs.eu> <20140713132521.GY93733@kib.kiev.ua> <20140713133421.GA93733@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140713133421.GA93733@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Aug 2014 01:00:52 -0000 On Sun, Jul 13, 2014 at 04:34:21PM +0300, Konstantin Belousov wrote: > On Sun, Jul 13, 2014 at 04:25:21PM +0300, Konstantin Belousov wrote: > > On Sun, Jul 13, 2014 at 05:55:00AM +0200, Mateusz Guzik wrote: > > > Currently: > > > /* > > > * Avoid reads reordering and then a first access to the > > > * fdp->fd_ofiles table which could result in OOB operation. > > > */ > > > if (fd < 0 || fd >= atomic_load_acq_int(&fdp->fd_nfiles)) > > > return (EBADF); > > > > > > However, if we put fd_nfiles and fd_otable into one atomically replaced > > > structure the only need to: > > > 1. make sure the pointer is read once > > > 2. issue a data dependency barrier - this is a noop on all supported > > > architectures and we don't even have approprate macro, so doing nothing > > > seems fine > > > > > > The motivation is to boost performance to amortize for seqlock cost, in > > > case it hits the tree. > > > > > > This has no impact on races with capability lookup. > > > > > > In a microbenchmark of 16 threads reading from the same pipe fd > > > immediately returning EAGAIN the numbers are: > > > x vanilla-readpipe-run-sum > > > + noacq-readpipe-run-sum > > > [..] > > > N Min Max Median Avg Stddev > > > x 20 13133671 14900364 13893331 13827075 471500.82 > > > + 20 59479718 59527286 59496714 59499504 13752.968 > > > Difference at 95.0% confidence > > > 4.56724e+07 +/- 213483 > > > 330.312% +/- 1.54395% > > > > > > There are 3 steps: > > > 1. tidy up capsicum to accept fde: > > > http://people.freebsd.org/~mjg/patches/single-fdtable-read-capsicum.patch > > > 2. add __READ_ONCE: > > > http://people.freebsd.org/~mjg/patches/read-once.patch > > > 3. put stuff into one structure: > > > http://people.freebsd.org/~mjg/patches/filedescenttable.patch > > > > > > Comments? > > > > We use 4-space indent for the continuation lines. Look at the malloc(9) > > call in the patch 3. > > > > The filedescenttable is really long name. Could it be, for instance, > > fdescenttbl ? > > > > Other than that, I think that the patches 2 and 3 are fine. I did not > > looked at the patch 1. > > > As an afterthought, you do not need __READ_ONCE(), the __DEVOLATILE() alone > would do what you need as well. Turns out patch 2 was quite bad. Reading http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf (pdf page 77) reveals: A cast of a value to a qualified type has no effect; the qualification (volatile, say) can have no effect on the access since it has occurred prior to the cast. If it is necessary to access a non-volatile object using volatile semantics, the technique is to cast the address of the object to the appropriate pointer-to-qualified type, then dereference that pointer. So how about we just follow the recomandation and also get the type automagically like linux folks do (added to sys/param.h): #define READ_ONCE(var) (*(volatile __typeof(var) *)&(var)) http://people.freebsd.org/~mjg/patches/read-once.patch I incorporated suggested changes have overwritten old patches. http://people.freebsd.org/~mjg/patches/filedescenttable.patch I would like to commit these changes this week with 2 weeks mfc. -- Mateusz Guzik