Date: Wed, 16 Jan 2013 06:19:01 -0800 (PST) From: Barney Cordoba <barney_cordoba@yahoo.com> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: freebsd-net@freebsd.org Subject: Re: two problems in dev/e1000/if_lem.c::lem_handle_rxtx() Message-ID: <1358345941.38671.YahooMailClassic@web121604.mail.ne1.yahoo.com> In-Reply-To: <CA%2BhQ2%2BjEETBeN%2BHx=5aThr3pzcuK2L-%2BRd3cd1qDnYR67WSyxA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
=0A=0A--- On Tue, 1/15/13, Luigi Rizzo <rizzo@iet.unipi.it> wrote:=0A=0A> F= rom: Luigi Rizzo <rizzo@iet.unipi.it>=0A> Subject: two problems in dev/e100= 0/if_lem.c::lem_handle_rxtx()=0A> To: head@freebsd.org, "freebsd-net@freebs= d.org" <freebsd-net@freebsd.org>=0A> Cc: "Jack Vogel" <jfvogel@gmail.com>= =0A> Date: Tuesday, January 15, 2013, 8:23 PM=0A> Hi,=0A> i found a couple = of problems in=0A> =A0 =A0 =A0 =A0=0A> dev/e1000/if_lem.c::lem_handle_rxtx(= ) ,=0A> (compare with dev/e1000/if_em.c::em_handle_que() for better=0A> und= erstanding):=0A> =0A> 1. in if_em.c::em_handle_que(), when em_rxeof() excee= ds the=0A> =A0 rx_process_limit, the task is rescheduled so it can=0A> comp= lete the work.=0A> =A0 Conversely, in if_lem.c::lem_handle_rxtx() the=0A> l= em_rxeof() is=0A> =A0 only run once, and if there are more pending packets= =0A> the only=0A> =A0 chance to drain them is to receive (many) more=0A> in= terrupts.=0A> =0A> =A0 This is a relatively serious problem, because the=0A= > receiver has=0A> =A0 a hard time recovering.=0A> =0A> =A0 I'd like to com= mit a fix to this same as it is done=0A> in e1000.=0A> =0A> 2. in if_em.c::= em_handle_que(), interrupts are reenabled=0A> unconditionally,=0A> =A0=A0= =A0whereas lem_handle_rxtx() only enables=0A> them if IFF_DRV_RUNNING is se= t.=0A> =0A> =A0=A0=A0I cannot really tell what is the correct=0A> way here,= so I'd like=0A> =A0=A0=A0to put a comment there unless there is a=0A> good= suggestion on=0A> =A0=A0=A0what to do.=0A> =0A> =A0=A0=A0Accesses to the i= ntr register are=0A> race-prone anyways=0A> =A0=A0=A0(disabled in fastintr,= enabled in the rxtx=0A> task without=0A> =A0=A0=A0holding any lock, and ge= nerally accessed=0A> under EM_CORE_LOCK=0A> =A0=A0=A0in other places), and = presumably=0A> enabling/disabling the=0A> =A0=A0=A0interrupts around activa= tions of the taks=0A> is just an=0A> =A0=A0=A0optimization (and on a VM, it= is actually=0A> a pessimization=0A> =A0=A0=A0due to the huge cost of VM ex= its).=0A> =0A> cheers=0A> luigi=0A=0AThis is not really a big deal; this is= how things works for a million =0Ayears before we had task queues.=0A=0AIn= tel controllers have built in interrupt moderation (unless you're on an=0AI= SA bus or something), so interrupt storms aren't possible. Typical default= =0Ais 6K ints per second, so you can't get another interrupt for 1/6000th o= f=0Aa second whether there's more work to do or not.=0A=0AThe "work" parame= ter should be an indicator that something is happening too=0Aslow, which ca= n happen with a shaper that's taking a lot more time than=0Anormal to proce= ss packets. =0A=0ASystems should have a maximum pps engineered into its tun= ing depending on=0Athe cpu to avoid live-lock on legacy systems. the defaul= t work limit of=0A100 is too low on a gigabit system. =0A=0Aqueuing tasks a= ctually creates more overhead in the system, not less. The=0Aissue is wheth= er the process_limit * interrupt_moderation is set to a =0Apps that's suita= ble for your system. =0A=0ASetting low work limits isn't really a good idea= unless you have some other=0Atime sensitive kernel task. Usually networkin= g is a priority, so setting=0Aarbitrary work limits makes less sense than q= ueuing an additional task,=0Awhich defeats the purpose of interrupt moderat= ion.=0A=0ABC
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1358345941.38671.YahooMailClassic>