From owner-freebsd-arch@FreeBSD.ORG Tue Dec 15 16:43:03 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3BDC1065697; Tue, 15 Dec 2009 16:43:03 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id B606C8FC1A; Tue, 15 Dec 2009 16:43:03 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 5859B46B23; Tue, 15 Dec 2009 11:43:03 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id A40E58A01B; Tue, 15 Dec 2009 11:43:02 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org, Harti Brandt Date: Tue, 15 Dec 2009 08:12:35 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20091103; KDE/4.3.1; amd64; ; ) References: <20091215103759.P97203@beagle.kn.op.dlr.de> In-Reply-To: <20091215103759.P97203@beagle.kn.op.dlr.de> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200912150812.35521.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 15 Dec 2009 11:43:02 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00, DATE_IN_PAST_03_06,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Subject: Re: network statistics in SMP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Dec 2009 16:43:04 -0000 On Tuesday 15 December 2009 4:38:04 am Harti Brandt wrote: > Hi all, > > I'm working on our network statistics (in the context of SNMP) and wonder, > to what extend we want them to be correct. I've re-read part of the past > discussions about 64-bit counters on 32-bit archs and got the impression, > that there are users that would like to have almost correct statistics > (for accounting, for example). If this is the case I wonder whether the > way we do the statistics today is correct. > > Basically all statistics are incremented or added to simply by a += b oder > a++. As I understand, this worked fine in the old days, where you had > spl*() calls at the right places. Nowadays when everything is SMP > shouldn't we use at least atomic operations for this? Also I read that on > architectures where cache coherency is not implemented in hardware even > this does not help (I found a mail from jhb why for the mutex > implementation this is not a problem, but I don't understand what to do > for the += and ++ operations). I failed to find a way, though, to > influence the caching policy (is there a function one can call to > change the policy?). Atomic ops will always work for reliable statistics. However, I believe Robert is working on using per-CPU statistics for TCP, UDP, etc. similar to what we do now for many of the 'cnt' stats (context switches, etc.). For 'cnt' each CPU has its own count of stats that are updated using non-atomic ops (since they are CPU local). sysctl handlers then sum up the various per- CPU counts to report global counts to userland. -- John Baldwin