From owner-freebsd-net@FreeBSD.ORG Thu Nov 10 09:46:59 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CDB31065674; Thu, 10 Nov 2011 09:46:59 +0000 (UTC) (envelope-from hoomanfazaeli@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id ECA458FC19; Thu, 10 Nov 2011 09:46:57 +0000 (UTC) Received: by eyd10 with SMTP id 10so2800881eyd.13 for ; Thu, 10 Nov 2011 01:46:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=aqAQdJ5Yv6MTr6P+KhQCzks6D6lWsdAn6DoGUZ1X00c=; b=qvYZgSNKIaPCfuB2HNknfFAvII3lZQ0jd8HREImsgCvvnuYz+uKwCFrAzn1grUW1Yo OjpBwq3ETPzkcVb44ruiTSvVLTsCBj8eJG93pOnnTMC9TF3rC9KFT30mUZGpxAC11pJG +n7k07eyYzApwzxFNQ/BTwcLv5Kiy8nP2peW8= Received: by 10.14.7.78 with SMTP id 54mr457724eeo.164.1320918417262; Thu, 10 Nov 2011 01:46:57 -0800 (PST) Received: from [127.0.0.1] ([84.241.57.181]) by mx.google.com with ESMTPS id z58sm21017284eea.3.2011.11.10.01.46.53 (version=SSLv3 cipher=OTHER); Thu, 10 Nov 2011 01:46:56 -0800 (PST) Message-ID: <4EBB9D88.4030304@gmail.com> Date: Thu, 10 Nov 2011 13:16:48 +0330 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.23) Gecko/20110920 Thunderbird/3.1.15 MIME-Version: 1.0 To: Adrian Chadd References: <4EA7E203.3020306@sepehrs.com> <4EA80818.3030504@sentex.net> <4EA80F88.4000400@hotplug.ru> <4EA82715.2000404@gmail.com> <4EA8FA40.7010504@hotplug.ru> <4EA91836.2040508@gmail.com> <4EA959EE.2070806@hotplug.ru> <4EAD116A.8090006@gmail.com> <4EAE58A2.9040803@gmail.com> <4EB96511.50701@gmail.com> <4EBA3F22.2060204@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pyunyh@gmail.com, freebsd-net@freebsd.org, Jason Wolfe , Jack Vogel , Emil Muratov Subject: Re: Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2011 09:46:59 -0000 On 11/10/2011 3:39 AM, Adrian Chadd wrote: > There's no locking around the OACTIVE flag set/clear, right? > Is it possible that multiple TX threads are fiddling with OACTIVE and > then it's not being properly cleared and tx kicked? > > > Adrian sorry! I forgot to cleanup the the last message ... here is the correct one: If we check for OACTIVE periodically (for instance, in local_timer) and under transient resource shortage, the driver will finally end up with OACTIVE cleared. Under frequent resource shortages, the driver may remain OACTIVE longer than it is ~OACTIVE or it may constantly toggles but there is not much the driver can do about this and a simple locking around OACTIVE set/clear does not change the situation. The problem _is_ low resources and the only fix is to increase it. The problems we should focus on here are two things: 1- The driver _must_ be able to recover from OACTIVE after transient resource shortages. 2- It is desirable to do this as fast as possible. Doing recovery in local_timer accommodates the first need but it is very far from from the second. One possible solution for 2 would be to defer setting OACTIVE until N consecutive transmissions fail (i.e., N == 75% (if_snd.ifq_maxlen - if_snd.ifq_len)). The overhead is a little wasted cpu time consumed in longer OACTIVE states. We still need local_timer to recover from these states.