From owner-freebsd-current@FreeBSD.ORG Tue Jul 8 04:31:46 2014 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3008626F; Tue, 8 Jul 2014 04:31:46 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E78702858; Tue, 8 Jul 2014 04:31:45 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 68E9D1FE028; Tue, 8 Jul 2014 06:31:43 +0200 (CEST) Message-ID: <53BB7433.2010306@selasky.org> Date: Tue, 08 Jul 2014 06:31:47 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: pyunyh@gmail.com Subject: Re: [RFC] Allow m_dup() to use JUMBO clusters References: <53BA5657.8010309@selasky.org> <20140708021439.GA3965@michelle.fasterthan.com> In-Reply-To: <20140708021439.GA3965@michelle.fasterthan.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@FreeBSD.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 04:31:46 -0000 On 07/08/14 04:14, Yonghyeon PYUN wrote: > On Mon, Jul 07, 2014 at 10:12:07AM +0200, Hans Petter Selasky wrote: >> Hi, >> >> I'm asking for some input on the attached m_dup() patch, so that >> existing functionality or dependencies are not broken. The background >> for the change is to allow m_dup() to defrag long mbuf chains that >> doesn't fit into a specific hardware's scatter gather entries, typically >> when doing TSO. >> >> In my case the HW limit is 16 entries of length 4K for doing a 64KByte > > I wonder how HW can handle a full-sized TSO packet(64KB + Ethernet > header + VLAN tag). > >> TSO packet. Currently m_dup() is at best producing 32 entries of each 2K >> for a 64Kbytes TSO packet. >> >> By allowing m_dup() to get JUMBO clusters when allocating mbufs, we >> avoid creating a new function, specific to the hardware, to defrag some >> rare-occurring very long mbuf chains into a mbuf chain below 16 entries. >> > > I think m_dup() was used to get a copy of writable mbuf chains. If > m_dup() starts to allocate jumbo mbufs it will eventually fail on > long running boxes. This will break firewall(ipfw divert, pf/ipf > dup-to) rules and several ethernet drivers. > > I don't know how many TSO requests could be queued by HW but if the > number is very small, the driver may be able to pre-allocate that > number of buffers (N * (64KB + Ethernet header + VLAN tag)) in > driver. Upper stack will almost always generate more than 16 mbufs > for TSO packets. When driver knows the length of mbuf chain of TSO > packet is more than 16, you can copy the mbuf chain to the > pre-allocated buffer. > > I recall I didn't implement TSO on txp(4) because the firmware of > txp(4) controller does not support more than 16 fragment > descriptors. Hi, Would it be better if my patch used the PAGE_SIZE clusters instead of the 16K ones? Then it should not be affected by memory defragmentation. Thanks for shedding some light into this area? --HPS