From owner-freebsd-current@FreeBSD.ORG  Wed Jul 25 14:54:07 2012
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: current@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id DD463106566C;
	Wed, 25 Jul 2012 14:54:07 +0000 (UTC)
	(envelope-from luigi@onelab2.iet.unipi.it)
Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238])
	by mx1.freebsd.org (Postfix) with ESMTP id 996918FC12;
	Wed, 25 Jul 2012 14:54:07 +0000 (UTC)
Received: by onelab2.iet.unipi.it (Postfix, from userid 275)
	id F02D57300A; Wed, 25 Jul 2012 17:14:03 +0200 (CEST)
Date: Wed, 25 Jul 2012 17:14:03 +0200
From: Luigi Rizzo <rizzo@iet.unipi.it>
To: Adrian Chadd <adrian@freebsd.org>
Message-ID: <20120725151403.GA33640@onelab2.iet.unipi.it>
References: <20120724202019.GA22927@onelab2.iet.unipi.it>
	<CAJ-VmokG-+kjaOC2g2uvVX5z4eBtry_-L8nMFaOPBan9SSzyYQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAJ-VmokG-+kjaOC2g2uvVX5z4eBtry_-L8nMFaOPBan9SSzyYQ@mail.gmail.com>
User-Agent: Mutt/1.4.2.3i
Cc: current@freebsd.org
Subject: Re: RFC: use EM_LEGACY_IRQ in if_lem.c ?
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
	<freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
	<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 25 Jul 2012 14:54:07 -0000

On Wed, Jul 25, 2012 at 07:48:57AM -0700, Adrian Chadd wrote:
> On 24 July 2012 13:20, Luigi Rizzo <rizzo@iet.unipi.it> wrote:
> > if_lem.c ("lem", one of the e1000 drivers) has 2 possible interrupt modes:
> > EM_LEGACY_IRQ uses the standard dispatch mechanism, whereas
> > FAST_INTR has a custom handler that signals a taskqueue to do the job.
> >
> > I have no idea which actual hardware uses it (all of my Intel 1G
> > cards use either "em" or "igb"), but "lem" is the driver used in
> > qemu, and there the EM_LEGACY_IRQ gives approx 10% higher packet
> > rates than the other.
> >
> > Any objections if i change the default to EM_LEGACY_IRQ ?
> 
> I suggest doing some digging to understand why. I bet we all know the
> answer, but it would be nice to have it documented and investigated. I
> bet em(4) isn't the only device that would benefit from this?

I am not so sure i know the answer on bare iron (and my take is that the
difference is more or less irrelevant there), but in the virtualized case
the improvement is almost surely because the code used in FAST_INTR
has a couple of MMIO accesses to disable/enable interrupts on the
card while the taskqueue runs.  These are expensive in a VM
(such accesses cost ~10K cycles each, even with hw support)

cheers
luigi