From owner-freebsd-arch@FreeBSD.ORG Tue Dec 15 17:07:56 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C523C106568F for ; Tue, 15 Dec 2009 17:07:56 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outB.internet-mail-service.net (outb.internet-mail-service.net [216.240.47.225]) by mx1.freebsd.org (Postfix) with ESMTP id A9F408FC12 for ; Tue, 15 Dec 2009 17:07:56 +0000 (UTC) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 1C08D44441; Tue, 15 Dec 2009 09:07:57 -0800 (PST) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id DB9182D6014; Tue, 15 Dec 2009 09:07:55 -0800 (PST) Message-ID: <4B27C279.8030402@elischer.org> Date: Tue, 15 Dec 2009 09:08:09 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: John Baldwin References: <20091215103759.P97203@beagle.kn.op.dlr.de> <200912150812.35521.jhb@freebsd.org> In-Reply-To: <200912150812.35521.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Harti Brandt , freebsd-arch@freebsd.org Subject: Re: network statistics in SMP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Dec 2009 17:07:56 -0000 John Baldwin wrote: > On Tuesday 15 December 2009 4:38:04 am Harti Brandt wrote: >> Hi all, >> >> I'm working on our network statistics (in the context of SNMP) and wonder, >> to what extend we want them to be correct. I've re-read part of the past >> discussions about 64-bit counters on 32-bit archs and got the impression, >> that there are users that would like to have almost correct statistics >> (for accounting, for example). If this is the case I wonder whether the >> way we do the statistics today is correct. >> >> Basically all statistics are incremented or added to simply by a += b oder >> a++. As I understand, this worked fine in the old days, where you had >> spl*() calls at the right places. Nowadays when everything is SMP >> shouldn't we use at least atomic operations for this? Also I read that on >> architectures where cache coherency is not implemented in hardware even >> this does not help (I found a mail from jhb why for the mutex >> implementation this is not a problem, but I don't understand what to do >> for the += and ++ operations). I failed to find a way, though, to >> influence the caching policy (is there a function one can call to >> change the policy?). > > Atomic ops will always work for reliable statistics. However, I believe > Robert is working on using per-CPU statistics for TCP, UDP, etc. similar to > what we do now for many of the 'cnt' stats (context switches, etc.). For > 'cnt' each CPU has its own count of stats that are updated using non-atomic > ops (since they are CPU local). sysctl handlers then sum up the various per- > CPU counts to report global counts to userland. the trouble is that PCPU and VNET collide. you then need to have Per-CPU, per VNET counters. which would be yet a different pool of linker set symbols.. >