From owner-freebsd-net@FreeBSD.ORG Wed Jul 6 19:49:31 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1580E1065673 for ; Wed, 6 Jul 2011 19:49:31 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id C6BD28FC17 for ; Wed, 6 Jul 2011 19:49:30 +0000 (UTC) Received: by yxl31 with SMTP id 31so161469yxl.13 for ; Wed, 06 Jul 2011 12:49:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=U2CsmY0wMYStuQmhi2ATHyx67LQJKYbKo/pEsJrQqXw=; b=ur/U1uxFzh5PBN2SmrBEg8kCLgivwsO9akgVWIj5dU41GDI43sZ9FICa5doEs+6KG4 1ZmJmQCqQsEzYvUvaeUtkRbb94oVH7PGXdgiHkeKRQI/eNJRXzmDgd6bAFOBWvwPw2T8 AXVxsbTzu97Ymq/dHrMkJAbU2PQsNwUDco3ks= MIME-Version: 1.0 Received: by 10.150.251.9 with SMTP id y9mr181176ybh.371.1309980465402; Wed, 06 Jul 2011 12:27:45 -0700 (PDT) Received: by 10.151.111.7 with HTTP; Wed, 6 Jul 2011 12:27:45 -0700 (PDT) Received: by 10.151.111.7 with HTTP; Wed, 6 Jul 2011 12:27:45 -0700 (PDT) In-Reply-To: References: Date: Wed, 6 Jul 2011 09:27:45 -1000 Message-ID: From: Kevin Oberman To: Chuck Swiger Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, Charles Sprickman Subject: Re: bce packet loss X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 19:49:31 -0000 On Jul 5, 2011 4:49 PM, "Chuck Swiger" wrote: > > On Jul 4, 2011, at 6:32 PM, Charles Sprickman wrote: > > We're running a few 8.1-R servers with Broadcom bce interfaces (Dell R510) and I'm seeing occasional packet loss on them (enough that it trips nagios now and then). Cabling seems fine as neither the switch nor the sysctl info for the device show any errors/collisions/etc, however there is one odd one, which is "dev.bce.1.stat_IfHCInBadOctets: 539369". See [1] below for full sysctl output. The switch shows no errors but for "Dropped packets 683868". > > A quick check of the #'s is showing an error rate of 8.6e-6, or about one byte out of 100,000. That's much better than what TCP/IP and ethernet were designed for. At a guess, your box or the switch occasionally gets busy enough that it drops a few packets now and again. > > Regards, > -- > -Chuck > 1 in 10**6? That is totally excessive. The Ethernet spec requires no worse than 10**13 and that is far worse than should ever be seen in the real world. At one in a million, any remotely high volume transfer will crawl, especially over a long path. If dropped packets ate being reported, the most common cause is fan-in. If two input ports are both trying to talk a line rate to a single output port, the buffer will fill an packets will be dropped. Most switches do tail drop, so queue management is terrible, compounding the effects. R. Kevin Oberman, Network Engineer Retired kob6558@gmail.com