From owner-freebsd-net@freebsd.org Fri Nov 6 17:23:08 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DED7A26B4E for ; Fri, 6 Nov 2015 17:23:08 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from erouter6.ore.mailhop.org (erouter6.ore.mailhop.org [54.187.213.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2CE7A1519 for ; Fri, 6 Nov 2015 17:23:07 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from ilsoft.org (unknown [73.34.117.227]) by outbound3.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA; Fri, 6 Nov 2015 17:22:04 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id tA6HN5r1051171; Fri, 6 Nov 2015 10:23:05 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1446830585.91534.435.camel@freebsd.org> Subject: Re: Timing issue with Dummynet on high kernel timer interrupt From: Ian Lepore To: Hans Petter Selasky , Luigi Rizzo Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" , Alexander Motin , FreeBSD Current Date: Fri, 06 Nov 2015 10:23:05 -0700 In-Reply-To: <563CDBF9.3090800@selasky.org> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> <563CC186.9000807@selasky.org> <563CD533.2000909@selasky.org> <1446828229.91534.417.camel@freebsd.org> <563CDA8F.5010901@selasky.org> <563CDBF9.3090800@selasky.org> Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.16.5 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 17:23:08 -0000 On Fri, 2015-11-06 at 17:57 +0100, Hans Petter Selasky wrote: > On 11/06/15 17:51, Hans Petter Selasky wrote: > > On 11/06/15 17:43, Ian Lepore wrote: > > > On Fri, 2015-11-06 at 17:28 +0100, Hans Petter Selasky wrote: > > > > Hi, > > > > > > > > Do the test II results change with this setting? > > > > > > sysctl kern.timecounter.alloweddeviation=0 > > > > > > > Yes, it looks much better: > > > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10012 -> 0 > > debug.total: 10012 -> 0 > > debug.total: 10012 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10012 -> 1 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10012 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > > > --HPS > > > > Thought I still see some unexpected dips, as the test runs for a > longer > amount of time. Not sure what the cause might be. > > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 9844 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10013 -> 0 > > debug.total: 10012 -> 0 > > --HPS Is it possible your machine is occasionally falling into a deeper sleep state (C3 or whatever)? I think it can take hundreds of microseconds to wake up from some of the deeper power-saving modes. If not power-saving, just plain old system-is-busy could lead to the reduced counts occasionally, since each callout is scheduled as a delta from the current actual wakeup time, not a delta from the current scheduled wakeup time (i.e., there is a phase shift in the firing of events over time). Raising the priority of the test thread into the realtime range might help with system-busy variation. It's also a bit iffy that you're using eventtimers to measure the performance of eventtimers (by using sleep 1 in the script). That explains the +12/13 count every second, not the reduced counts unless we speculate that sleep is returning early (which I have NEVER caught it doing). If you have a PPS source available, that can make a good way to sleep based on an off-box timing signal and avoid using a clock to measure itself. But I doubt it would make a big difference in this case. -- Ian