From owner-freebsd-fs@FreeBSD.ORG Fri Mar 28 22:44:43 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2E7083C0; Fri, 28 Mar 2014 22:44:43 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id BFC30EAB; Fri, 28 Mar 2014 22:44:42 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArwEAFH6NVODaFve/2dsb2JhbABZg0FXgwq4N4YXTVGBLnSCJQEBAQMBAQEBICsgCwUWGAICDRkCKQEJJg4HBAEcBIdQCA2vFaJ7F4EpiCOEXQEBGzQHgm+BSQSVd4QKkQKDTCExgQQ5 X-IronPort-AV: E=Sophos;i="4.97,753,1389762000"; d="scan'208";a="110110613" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 28 Mar 2014 18:44:36 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 5C9DBB3F23; Fri, 28 Mar 2014 18:44:36 -0400 (EDT) Date: Fri, 28 Mar 2014 18:44:36 -0400 (EDT) From: Rick Macklem To: araujo@FreeBSD.org Message-ID: <1377879526.2465097.1396046676367.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Filesystems , Alexander Motin X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 22:44:43 -0000 Marcelo Araujo wrote: > 2014-03-28 5:37 GMT+08:00 Rick Macklem : > > > Christopher Forgeron wrote: > > > I'm quite sure the problem is on 9.2-RELEASE, not 9.1-RELEASE or > > > earlier, > > > as a 9.2-STABLE from last year I have doesn't exhibit the > > > problem. > > > New > > > code in if.c at line 660 looks to be what is starting this, which > > > makes me > > > wonder how TSO was being handled before 9.2. > > > > > > I also like Rick's NFS patch for cluster size. I notice an > > > improvement, but > > > don't have solid numbers yet. I'm still stress testing it as we > > > speak. > > > > > Unfortunately, this causes problems for small i386 systems, so I > > am reluctant to commit it to head. Maybe a variant that is only > > enabled for amd64 systems with lots of memory would be ok? > > > > > Rick, > > Maybe you can create a SYSCTL to enable/disable it by the end user > will be > more reasonable. Also, of course, it is so far safe if only 64Bits > CPU can > enable this SYSCTL. Any other option seems not OK, will be hard to > judge > what is lots of memory and what is not, it will depends what is > running > onto the system. > I guess adding it so it can be optionally enabled via a sysctl isn't a bad idea. I think the largest risk here is "how do you tell people what the risk of enabling this is"? There are already a bunch of sysctls related to NFS that few people know how to use. (I recall that Alexander has argued that folk don't want worry about these tunables and I tend to agree.) If I do a variant of the patch that uses m_getjcl(..M_WAITOK..), then at least the "breakage" is thread(s) sleeping on "btallo", which is fairly easy to check for, althouggh rather obscure. (Btw, I've never reproduced this for a patch that changes the code to always use MJUMPAGESIZE mbuf clusters. I can only reproduce it intermittently when the patch mixes allocation of MCLBYTES clusters and MJUMPAGESIZE clusters.) I've been poking at it to try and figure out how to get m_getjcl(..M_NOWAIT..) to return NULL instead of looping when it runs out of boundary tags (to see if that can result in a stable implementation of the patch), but haven't had much luck yet. Bottom line: I just don't like committing a patch that can break the system in such an obscure way, even if it is enabled via a sysctl. Others have an opinion on this? Thanks, rick > The SYSCTL will be great, and in case you don't have time to do it, I > can > give you a hand. > > I'm gonna do more benchmarks today and will send another report, but > in our > product here, I'm inclined to use this patch, because 10~20% speed up > in > read for me is a lot. :-) > > Thank you so much and best regards, > -- > Marcelo Araujo > araujo@FreeBSD.org > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >