From owner-freebsd-arch@FreeBSD.ORG Tue Dec 15 19:39:04 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23B4C1065676; Tue, 15 Dec 2009 19:39:04 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id E6A7B8FC13; Tue, 15 Dec 2009 19:39:03 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 881EA46B39; Tue, 15 Dec 2009 14:39:03 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id BF5FB8A01B; Tue, 15 Dec 2009 14:39:02 -0500 (EST) From: John Baldwin To: Harti Brandt Date: Tue, 15 Dec 2009 13:13:28 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20091103; KDE/4.3.1; amd64; ; ) References: <20091215103759.P97203@beagle.kn.op.dlr.de> <200912150812.35521.jhb@freebsd.org> <20091215183859.S53283@beagle.kn.op.dlr.de> In-Reply-To: <20091215183859.S53283@beagle.kn.op.dlr.de> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200912151313.28326.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 15 Dec 2009 14:39:02 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-arch@freebsd.org Subject: Re: network statistics in SMP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Dec 2009 19:39:04 -0000 On Tuesday 15 December 2009 12:45:13 pm Harti Brandt wrote: > On Tue, 15 Dec 2009, John Baldwin wrote: > > JB>On Tuesday 15 December 2009 4:38:04 am Harti Brandt wrote: > JB>> Hi all, > JB>> > JB>> I'm working on our network statistics (in the context of SNMP) and wonder, > JB>> to what extend we want them to be correct. I've re-read part of the past > JB>> discussions about 64-bit counters on 32-bit archs and got the impression, > JB>> that there are users that would like to have almost correct statistics > JB>> (for accounting, for example). If this is the case I wonder whether the > JB>> way we do the statistics today is correct. > JB>> > JB>> Basically all statistics are incremented or added to simply by a += b oder > JB>> a++. As I understand, this worked fine in the old days, where you had > JB>> spl*() calls at the right places. Nowadays when everything is SMP > JB>> shouldn't we use at least atomic operations for this? Also I read that on > JB>> architectures where cache coherency is not implemented in hardware even > JB>> this does not help (I found a mail from jhb why for the mutex > JB>> implementation this is not a problem, but I don't understand what to do > JB>> for the += and ++ operations). I failed to find a way, though, to > JB>> influence the caching policy (is there a function one can call to > JB>> change the policy?). > JB> > JB>Atomic ops will always work for reliable statistics. However, I believe > JB>Robert is working on using per-CPU statistics for TCP, UDP, etc. similar to > JB>what we do now for many of the 'cnt' stats (context switches, etc.). For > JB>'cnt' each CPU has its own count of stats that are updated using non-atomic > JB>ops (since they are CPU local). sysctl handlers then sum up the various per- > JB>CPU counts to report global counts to userland. > > I see. I was also thinking along these lines, but was not sure whether it > is worth the trouble. I suppose this does not help to implement 64-bit > counters on 32-bit architectures, though, because you cannot read them > reliably without locking to sum them up, right? Either that or you just accept that you have a small race since it is only stats. :) -- John Baldwin