From owner-freebsd-bugs@FreeBSD.ORG Wed Oct 6 14:30:02 2010 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 773FD106566C for ; Wed, 6 Oct 2010 14:30:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 32B738FC21 for ; Wed, 6 Oct 2010 14:30:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o96EU2rf062947 for ; Wed, 6 Oct 2010 14:30:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o96EU2YL062946; Wed, 6 Oct 2010 14:30:02 GMT (envelope-from gnats) Resent-Date: Wed, 6 Oct 2010 14:30:02 GMT Resent-Message-Id: <201010061430.o96EU2YL062946@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Jeremy Chadwick Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE3F81065674 for ; Wed, 6 Oct 2010 14:25:38 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.westchester.pa.mail.comcast.net (qmta01.westchester.pa.mail.comcast.net [76.96.62.16]) by mx1.freebsd.org (Postfix) with ESMTP id 98F188FC1F for ; Wed, 6 Oct 2010 14:25:38 +0000 (UTC) Received: from omta05.westchester.pa.mail.comcast.net ([76.96.62.43]) by qmta01.westchester.pa.mail.comcast.net with comcast id FNh41f0020vyq2s51SReU5; Wed, 06 Oct 2010 14:25:38 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta05.westchester.pa.mail.comcast.net with comcast id FSRd1f00H3LrwQ23RSReEE; Wed, 06 Oct 2010 14:25:38 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 4798C9B427; Wed, 6 Oct 2010 07:25:36 -0700 (PDT) Message-Id: <20101006142536.4798C9B427@icarus.home.lan> Date: Wed, 6 Oct 2010 07:25:36 -0700 (PDT) From: Jeremy Chadwick To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: syrinx@FreeBSD.org, philip@FreeBSD.org Subject: bin/151264: bsnmpd(1): pf counters aren't updated on some SNMP queries X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Jeremy Chadwick List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 14:30:02 -0000 >Number: 151264 >Category: bin >Synopsis: bsnmpd(1): pf counters aren't updated on some SNMP queries >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Oct 06 14:30:01 UTC 2010 >Closed-Date: >Last-Modified: >Originator: Jeremy Chadwick >Release: FreeBSD 8.1-STABLE amd64 >Organization: >Environment: System: FreeBSD icarus.home.lan 8.1-STABLE FreeBSD 8.1-STABLE #0: Wed Sep 15 14:59:46 PDT 2010 root@icarus.home.lan:/usr/obj/usr/src/sys/X7SBA_RELENG_8_amd64 amd64 >Description: This is an interesting problem and one I've chatted with philip@ about in the past on IRC. Basically, bsnmpd(1), when requested for some OIDs "in bulk", will return cached data -- meaning, what the OIDs return doesn't match what "pfctl -s info" shows. If a manual SNMP GET (ex. snmpget) is executed for a single OID that would get handled by snmp_pf.ko, the counters/stats for all OIDs within that module (snmp_pf.so) are updated and returned. But all subsequent "bulk" requests continue to return the data that the last GET showed. I do have tcpdumps available that confirm this problem, and can also provide user-level access to production machines if a developer wants to do some analysis. I can provide SNMP community names and so on privately. I've only tested/confirmed this using SNMP protocol 2c due to the need for 64-bit counters; I cannot test version 1. Because of the nature of this problem, I've marked it serious/medium given how important SNMP monitoring is on production *IX machines. >How-To-Repeat: 1. Run bsnmpd(1), making sure that pf is in use, and snmp_pf.so is loaded as defined by snmpd.config. 2. Install something like net-mgmt/rrdbot and set up SNMP monitoring of OIDs such as the following, polled at a 30 second intervals: pfCounterMatch = begemot.200.1.2.1.0 pfCounterBadOffset = begemot.200.1.2.2.0 pfCounterFragment = begemot.200.1.2.3.0 pfCounterShort = begemot.200.1.2.4.0 pfCounterNormalize = begemot.200.1.2.5.0 pfCounterMemDrop = begemot.200.1.2.6.0 pfStateTableCount = begemot.200.1.3.1.0 pfStateTableSearches = begemot.200.1.3.2.0 pfStateTableInserts = begemot.200.1.3.3.0 pfStateTableRemovals = begemot.200.1.3.4.0 Note: "begemot" OID name/label expands to 1.3.6.1.4.1.12325.1 3. Launch a tcpdump that will capture the SNMP data between the machine running bsnmpd and the machine running rrdbotd. They can be the same host (in which case, packet capture from lo0). 4. Run rrdbotd and watch the SNMP results which are returned. They'll repetitively be the same value. 5. Execute rrdbot-get against one of the above OIDs. 6. Watch for the next SNMP query/response in tcpdump; you'll see that the values returned are all up-to-date. 7. Watch for the next SNMP query/response; you'll see that the values returned continue to be those from step #6. >Fix: None known. >Release-Note: >Audit-Trail: >Unformatted: