From owner-freebsd-arch@FreeBSD.ORG Mon Oct 27 06:50:57 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7B7F516A4B3 for ; Mon, 27 Oct 2003 06:50:57 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1981D43F85 for ; Mon, 27 Oct 2003 06:50:56 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9p2/8.12.9) with ESMTP id h9REndMg099603; Mon, 27 Oct 2003 09:49:39 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h9REncXc099599; Mon, 27 Oct 2003 09:49:39 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Mon, 27 Oct 2003 09:49:38 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: "Saye Balasubramaniam.S" In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-arch@freebsd.org Subject: Re: Adding a new field into mbuf pkthdr structure X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Oct 2003 14:50:57 -0000 On Mon, 27 Oct 2003, Saye Balasubramaniam.S wrote: > We are using FreeBsd stack in our new product. > I am planning to add a new field (integer)into the pkthdr struct > used in mbuf. I am worried about the side-effects of this. > Will there be any side-effects? Please give your > valuable feedback. The primary issue is binary compatibility, which may not be an issue at all for your product. By changing the size of struct mbuf, all kernel components using mbufs typically need to be recompiled, as the offset of data in an M_PKTHDR mbuf is located using the size of struct pkthdr. If you don't care about binary compatibility: i.e., you don't plan to use third party binary-only drivers, then this is not an issue. In our first pass at adding MAC labels to mbuf headers, we stored them directly in "struct pkthdr". The consequence was a slight reduction in the size of the data that could be held in an mbuf + M_PKTHDR without using a cluster. In practice this wasn't measurable in any normal benchmarks, although it you carefully sized test packets above and below that threshold size, it was noticeable. In addition, because "struct mac" is fairly large, you could measure an additional cost to zero the field even in systems without MAC enabled--again, fairly small, but measurable. The final problem we had to address in adding fields to the mbuf header was making sure that the value was properly managed when mbufs were copied. We found that, in earlier versions of FreeBSD, mbuf headers were relatively frequently allocated to hold copies of existing mbufs, but that the mbuf header fields (such as size, flags) were manually copied by calling code. In more recent versions of FreeBSD, header-copying and moving primitives, such as M_MOVE_PKTHDR() and m_dup_pkthdr() are used, which make it easier to maintain header values due to their central implementation. I would not be surprised if there are a couple of lurking places where packet header routines still need to be used to make sure that header entries are properly propagated, but most of them are handled. For example, I believe we may still need to modify the IP fragmentation code for outgoing packets to properly copy additional header fields and tags. One of the pieces of infrastructure introduced to support IPsec better was m_tags, a list of "tags" hung off of the mbuf header. This allows you to extend the contents of a header in a binary-compatible manner, although at a slight increase in cost in infrastructure due to additional memory allocation and pointer values. Tags are automatically maintained and copied over various mbuf operations, so you might want to give them a spin and see how the performance compares for your application. Hope this is helpful, Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories