From owner-freebsd-net@FreeBSD.ORG Thu Apr 26 18:07:41 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C147D106566B; Thu, 26 Apr 2012 18:07:41 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id 7A93E8FC0A; Thu, 26 Apr 2012 18:07:41 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q3QI7NVf055615; Thu, 26 Apr 2012 11:07:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1335463644; bh=nRhzhOfLbaXPsy0y7PvjssMCCY4rJUgnc9ccPeMABqw=; h=Subject:From:To:Cc:Content-Type:Date:Message-ID:Mime-Version: Content-Transfer-Encoding; b=cLlhCeFSjxsFXZlVD/8Q8mytrC0Kpx0Fr/z0siFfsLa81YyMNnB+p5YJt/B0DYBKp BHinwLXviMaBMozv3JTtYwZsHs+1GJp77hZSQGdoIBLtxPEbOFE3luC7vqLndX3aTB 8P/rALxCxPKhlErIAq5pvAmw6OijS7//qm9axhl4= From: Sean Bruno To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" Date: Thu, 26 Apr 2012 11:07:23 -0700 Message-ID: <1335463643.2727.10.camel@powernoodle-l7.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 463643002 Cc: Jack Vogel , John Baldwin Subject: igb(4) at peak in big purple X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2012 18:07:41 -0000 8 core box with 2 igb(4) interfaces serving internet traffic in/out over here in Yahoo land. This is configuring igb(4) allow 32k TXD/RXD descriptors(but only configuring for 8k), 4 queues per interface and changing the logic of the call to bus_bind_intr() such that it will iterate over all cpus in the system: http://people.freebsd.org/~sbruno/if_igb.c.txt http://people.freebsd.org/~sbruno/igb_stats.txt We're seeing >100MB/s which is very nice. Thanks! I note form top that igb0 queue 0 is always "more busy" than any other queue, there appears to be a second kernel igb0 "queue" process/thread that appears to be moderately busy and 3 kernel igb0 "queue" processes/threads that appear to be doing nothing in particular. There are not causing issues ... I'm just curious if this is meaningful to anyone or indicative of interesting things. Sean