From owner-svn-src-head@FreeBSD.ORG  Thu May 10 00:00:29 2012
Return-Path: <owner-svn-src-head@FreeBSD.ORG>
Delivered-To: svn-src-head@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 4E4E71065672;
	Thu, 10 May 2012 00:00:29 +0000 (UTC)
	(envelope-from sbruno@FreeBSD.org)
Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c])
	by mx1.freebsd.org (Postfix) with ESMTP id 391078FC15;
	Thu, 10 May 2012 00:00:29 +0000 (UTC)
Received: from svn.freebsd.org (localhost [127.0.0.1])
	by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4A00TIe008379;
	Thu, 10 May 2012 00:00:29 GMT (envelope-from sbruno@svn.freebsd.org)
Received: (from sbruno@localhost)
	by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4A00T5m008377;
	Thu, 10 May 2012 00:00:29 GMT (envelope-from sbruno@svn.freebsd.org)
Message-Id: <201205100000.q4A00T5m008377@svn.freebsd.org>
From: Sean Bruno <sbruno@FreeBSD.org>
Date: Thu, 10 May 2012 00:00:29 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
	svn-src-head@freebsd.org
X-SVN-Group: head
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Cc: 
Subject: svn commit: r235210 - head/sys/dev/e1000
X-BeenThere: svn-src-head@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: SVN commit messages for the src tree for head/-current
	<svn-src-head.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-head>,
	<mailto:svn-src-head-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-head>
List-Post: <mailto:svn-src-head@freebsd.org>
List-Help: <mailto:svn-src-head-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-head>,
	<mailto:svn-src-head-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 10 May 2012 00:00:29 -0000

Author: sbruno
Date: Thu May 10 00:00:28 2012
New Revision: 235210
URL: http://svn.freebsd.org/changeset/base/235210

Log:
  Modify the binding of queues to attach to as many CPUs
  as possible when using more than one igb(4) adapter.  This
  means that queues will not be bound to the same CPUs if
  there are more CPUs availble.
  
  This is only applicable to a system that has multiple interfaces.
  
  Obtained from:	Yahoo! Inc.
  MFC after:	3 days

Modified:
  head/sys/dev/e1000/if_igb.c

Modified: head/sys/dev/e1000/if_igb.c
==============================================================================
--- head/sys/dev/e1000/if_igb.c	Wed May  9 22:13:56 2012	(r235209)
+++ head/sys/dev/e1000/if_igb.c	Thu May 10 00:00:28 2012	(r235210)
@@ -364,6 +364,13 @@ TUNABLE_INT("hw.igb.num_queues", &igb_nu
 SYSCTL_INT(_hw_igb, OID_AUTO, num_queues, CTLFLAG_RDTUN, &igb_num_queues, 0,
     "Number of queues to configure, 0 indicates autoconfigure");
 
+/*
+** Global variable to store last used CPU when binding queues
+** to CPUs in igb_allocate_msix.  Starts at CPU_FIRST and increments when a
+** queue is bound to a cpu.
+*/
+static int igb_last_bind_cpu = -1;
+
 /* How many packets rxeof tries to clean at a time */
 static int igb_rx_process_limit = 100;
 TUNABLE_INT("hw.igb.rx_process_limit", &igb_rx_process_limit);
@@ -2493,8 +2500,16 @@ igb_allocate_msix(struct adapter *adapte
 		** Bind the msix vector, and thus the
 		** rings to the corresponding cpu.
 		*/
-		if (adapter->num_queues > 1)
-			bus_bind_intr(dev, que->res, i);
+		if (adapter->num_queues > 1) {
+			if (igb_last_bind_cpu < 0)
+				igb_last_bind_cpu = CPU_FIRST();
+			bus_bind_intr(dev, que->res, igb_last_bind_cpu);
+			device_printf(dev,
+				"Bound queue %d to cpu %d\n",
+				i,igb_last_bind_cpu);
+			igb_last_bind_cpu = CPU_NEXT(igb_last_bind_cpu);
+			igb_last_bind_cpu = igb_last_bind_cpu % mp_ncpus;
+		}
 #if __FreeBSD_version >= 800000
 		TASK_INIT(&que->txr->txq_task, 0, igb_deferred_mq_start,
 		    que->txr);