From owner-freebsd-current@FreeBSD.ORG Mon Jul 7 21:42:16 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8E1966B1; Mon, 7 Jul 2014 21:42:16 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3C92F2488; Mon, 7 Jul 2014 21:42:15 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYEAEMTu1ODaFve/2dsb2JhbABZg2Bagm+8CYZyUwGBLHWEAwEBAQQBAQEgBCcgCxsYAgINGQIpAQkmBggHBAEaAgSIIQ2vYJlgF4EsiDmEZgYBARs0B4J3gUwFmAqENIoEiECDXyE1fQgXIg X-IronPort-AV: E=Sophos;i="5.01,620,1400040000"; d="scan'208";a="138237862" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 07 Jul 2014 17:42:08 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B34C6B3F45; Mon, 7 Jul 2014 17:42:08 -0400 (EDT) Date: Mon, 7 Jul 2014 17:42:08 -0400 (EDT) From: Rick Macklem To: Hans Petter Selasky Message-ID: <1613382893.8316833.1404769328724.JavaMail.root@uoguelph.ca> In-Reply-To: <53BA5657.8010309@selasky.org> Subject: Re: [RFC] Allow m_dup() to use JUMBO clusters MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org, freebsd-current@FreeBSD.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 21:42:16 -0000 Hans Petter Selasky wrote: > Hi, > > I'm asking for some input on the attached m_dup() patch, so that > existing functionality or dependencies are not broken. The background > for the change is to allow m_dup() to defrag long mbuf chains that > doesn't fit into a specific hardware's scatter gather entries, > typically > when doing TSO. > > In my case the HW limit is 16 entries of length 4K for doing a > 64KByte > TSO packet. Currently m_dup() is at best producing 32 entries of each > 2K > for a 64Kbytes TSO packet. > > By allowing m_dup() to get JUMBO clusters when allocating mbufs, we > avoid creating a new function, specific to the hardware, to defrag > some > rare-occurring very long mbuf chains into a mbuf chain below 16 > entries. > > Any comments? > 1 - If you are using NFS with the default (64K) I/O size, then long mbuf chains of 35 entries aren't rare. They happen on every read reply/write request. 2 - When I changed NFS to use pagesize clusters for reads/writes I was able to get the system into a state where threads were persistently stuck on "btalloc". If I understand this correctly, the system was not able to allocate boundary tags because the kernel address space had been fragmented too much. --> As such, I never committed this patch to head and would caution against using pagesize clusters. I do not have a better solution at this point, but I do have an untested patch (I need to get access to some TSO enabled hardware to test it) that adds if_hw_tsomaxseg, which is a count of the maximum number of transmit segments (mbufs in chain) that a network device driver supports. I think that having the driver set if_hw_tsomaxseg == 16 is preferable to doing a copy of the data to pagesize clusters. (I'd also say that hardware that supports only 16 transmit segments for a TSO segment is not a good piece of hardware for FreeBSD.) rick > --HPS > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org"