From owner-freebsd-arch@FreeBSD.ORG Wed Feb 5 04:55:40 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10DE1B48 for ; Wed, 5 Feb 2014 04:55:40 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C0EB31C98 for ; Wed, 5 Feb 2014 04:55:39 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WAuWP-000ILd-V7 for freebsd-arch@FreeBSD.org; Wed, 05 Feb 2014 04:55:38 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s154tYc2065449 for ; Tue, 4 Feb 2014 21:55:35 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1942q61qUETa3TjK1kKAMIO Subject: Malloc alignment in libstand / loader(8) From: Ian Lepore To: freebsd-arch Content-Type: multipart/mixed; boundary="=-xc5zpUDqlEDhTKoyb6bT" Date: Tue, 04 Feb 2014 21:55:34 -0700 Message-ID: <1391576134.1196.21.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2014 04:55:40 -0000 --=-xc5zpUDqlEDhTKoyb6bT Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On newer ARM chips, the device drivers used by loader(8) require that I/O buffers be aligned on cache line sized boundaries. The drivers are part of u-boot which serves as a sort of load-time bios. Attached is a patch that sets the malloc alignment in libstand to 64 bytes when compiled on ARM, and leaves it at 16 bytes for all other platforms. If there are no objections I'd like to commit this soon. I've tested this on ARM, but have no way to test it on other platforms. The changes should be a no-op on other platforms. -- Ian --=-xc5zpUDqlEDhTKoyb6bT Content-Disposition: inline; filename="libstand_mallocalign.diff" Content-Type: text/x-patch; name="libstand_mallocalign.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: lib/libstand/zalloc.c =================================================================== --- lib/libstand/zalloc.c (revision 261446) +++ lib/libstand/zalloc.c (working copy) @@ -71,6 +71,15 @@ __FBSDID("$FreeBSD$"); #include "zalloc_defs.h" /* + * Objects in the pool must be aligned to at least the size of struct MemNode. + * They must also be aligned to MALLOCALIGN, which should normally be larger + * than the struct, so assert that to be so at compile time. + */ +typedef char assert_align[(sizeof(struct MemNode) <= MALLOCALIGN) ? 1 : -1]; + +#define MEMNODE_SIZE_MASK MALLOCALIGN_MASK + +/* * znalloc() - allocate memory (without zeroing) from pool. Call reclaim * and retry if appropriate, return NULL if unable to allocate * memory. Index: lib/libstand/zalloc_mem.h =================================================================== --- lib/libstand/zalloc_mem.h (revision 261446) +++ lib/libstand/zalloc_mem.h (working copy) @@ -48,8 +48,6 @@ typedef struct MemPool { uintptr_t mp_Used; } MemPool; -#define MEMNODE_SIZE_MASK ((sizeof(MemNode) <= 8) ? 7 : 15) - #define ZNOTE_FREE 0 #define ZNOTE_REUSE 1 Index: lib/libstand/sbrk.c =================================================================== --- lib/libstand/sbrk.c (revision 261446) +++ lib/libstand/sbrk.c (working copy) @@ -33,6 +33,7 @@ __FBSDID("$FreeBSD$"); #include #include "stand.h" +#include "zalloc_defs.h" static size_t maxheap, heapsize = 0; static void *heapbase; @@ -40,8 +41,9 @@ static void *heapbase; void setheap(void *base, void *top) { - /* Align start address to 16 bytes for the malloc code. Sigh. */ - heapbase = (void *)(((uintptr_t)base + 15) & ~15); + /* Align start address for the malloc code. Sigh. */ + heapbase = (void *)(((uintptr_t)base + MALLOCALIGN_MASK) & + ~MALLOCALIGN_MASK); maxheap = (char *)top - (char *)heapbase; } Index: lib/libstand/zalloc_defs.h =================================================================== --- lib/libstand/zalloc_defs.h (revision 261446) +++ lib/libstand/zalloc_defs.h (working copy) @@ -52,18 +52,26 @@ #define BLKEXTENDMASK (BLKEXTEND - 1) /* - * required malloc alignment. Just hardwire to 16. + * Required malloc alignment. * - * Note: if we implement a more sophisticated realloc, we should ensure that - * MALLOCALIGN is at least as large as MemNode. + * ARM platforms using the u-boot API drivers require that all I/O buffers be on + * a cache line sized boundary. The worst case size for that is 64 bytes. For + * all other platforms, 16 bytes works fine. The size also must be at least + * sizeof(struct MemNode); this is asserted in zalloc.c. */ +#ifdef __arm__ +#define MALLOCALIGN 64 +#else +#define MALLOCALIGN 16 +#endif +#define MALLOCALIGN_MASK (MALLOCALIGN - 1) + typedef struct Guard { size_t ga_Bytes; size_t ga_Magic; /* must be at least 32 bits */ } Guard; -#define MALLOCALIGN 16 #define GAMAGIC 0x55FF44FD #define GAFREE 0x5F54F4DF --=-xc5zpUDqlEDhTKoyb6bT-- From owner-freebsd-arch@FreeBSD.ORG Wed Feb 5 19:28:31 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B46FE4AD; Wed, 5 Feb 2014 19:28:31 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8C2511E2C; Wed, 5 Feb 2014 19:28:31 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 941F2B9C6; Wed, 5 Feb 2014 14:28:30 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Malloc alignment in libstand / loader(8) Date: Wed, 05 Feb 2014 08:29:19 -0500 Message-ID: <2882663.sRzekugaiD@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <1391576134.1196.21.camel@revolution.hippie.lan> References: <1391576134.1196.21.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 05 Feb 2014 14:28:30 -0500 (EST) Cc: Ian Lepore X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2014 19:28:31 -0000 On Tuesday, February 04, 2014 09:55:34 PM Ian Lepore wrote: > On newer ARM chips, the device drivers used by loader(8) require that > I/O buffers be aligned on cache line sized boundaries. The drivers are > part of u-boot which serves as a sort of load-time bios. > > Attached is a patch that sets the malloc alignment in libstand to 64 > bytes when compiled on ARM, and leaves it at 16 bytes for all other > platforms. If there are no objections I'd like to commit this soon. > > I've tested this on ARM, but have no way to test it on other platforms. > The changes should be a no-op on other platforms. I think this looks fine, but perhaps use CTASSERT() instead of rolling your own? (I would say to use _Static_assert(), but I don't think that works with our old GCC) -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Feb 5 20:07:19 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D56E336E; Wed, 5 Feb 2014 20:07:19 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A77711346; Wed, 5 Feb 2014 20:07:19 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WB8kb-000PKe-0J; Wed, 05 Feb 2014 20:07:13 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s15K7928066286; Wed, 5 Feb 2014 13:07:09 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/XgrM3KQlgFaYgSGKBbufn Subject: Re: Malloc alignment in libstand / loader(8) From: Ian Lepore To: John Baldwin In-Reply-To: <2882663.sRzekugaiD@ralph.baldwin.cx> References: <1391576134.1196.21.camel@revolution.hippie.lan> <2882663.sRzekugaiD@ralph.baldwin.cx> Content-Type: text/plain; charset="us-ascii" Date: Wed, 05 Feb 2014 13:07:09 -0700 Message-ID: <1391630829.1196.31.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2014 20:07:19 -0000 On Wed, 2014-02-05 at 08:29 -0500, John Baldwin wrote: > On Tuesday, February 04, 2014 09:55:34 PM Ian Lepore wrote: > > On newer ARM chips, the device drivers used by loader(8) require that > > I/O buffers be aligned on cache line sized boundaries. The drivers are > > part of u-boot which serves as a sort of load-time bios. > > > > Attached is a patch that sets the malloc alignment in libstand to 64 > > bytes when compiled on ARM, and leaves it at 16 bytes for all other > > platforms. If there are no objections I'd like to commit this soon. > > > > I've tested this on ARM, but have no way to test it on other platforms. > > The changes should be a no-op on other platforms. > > I think this looks fine, but perhaps use CTASSERT() instead of rolling your > own? (I would say to use _Static_assert(), but I don't think that works with > our old GCC) > I wasn't sure if it's a good idea to #include in the libstand environment, since it has declarations for a lot of stuff. I looked at what's in systm.h and then hand-replicated what seemed like the most generic of the choices. -- Ian From owner-freebsd-arch@FreeBSD.ORG Wed Feb 5 20:52:18 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 623EAAA6 for ; Wed, 5 Feb 2014 20:52:18 +0000 (UTC) Received: from mail-ig0-f175.google.com (mail-ig0-f175.google.com [209.85.213.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2B2A5184D for ; Wed, 5 Feb 2014 20:52:17 +0000 (UTC) Received: by mail-ig0-f175.google.com with SMTP id uq10so14259240igb.2 for ; Wed, 05 Feb 2014 12:52:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=h9wYSb2mVSkbhe6vT3iWgYgRGuzOciUjQI9c/0+ROBk=; b=MN+3ErKKkg6uDCrd5YAi2DDnKjCHS3EFXEF/NdRgVnRdy3N5sI50hymoqYGG3+AnPj l4Uc4+YFHpkW5Hb/XpSaroOS4BIk2IegHgIjWg4FhvDI9I2RZdShOl70ifQvAZ+7GBKE ssJnVkJE0cIjPcFHOvTLg0GVXQME0mps279on1od24PcG3rREc1XNx/z4ZeGiMBR2S+K QLnl7IoSDyqr/NaJRe+EP/jmT0lU1LDHnPei9r7a0jMF8jABJo8miubiOEUGJJhPwl0p ZRa56Zf1pjURR1EUIeGWoPrDMfJsjIBR50DS4dAgHwSE7aRcBQoVCpAWbPTbIxdY/0S1 DkMw== X-Gm-Message-State: ALoCoQmVSdZQ6lI9xgLsyFCVZW56fO6ombEqbIIe5UmwPcmhreRC3Gr8A1NoZTXss5VqvRcB/Xp/ X-Received: by 10.43.16.2 with SMTP id pw2mr3329836icb.56.1391633526322; Wed, 05 Feb 2014 12:52:06 -0800 (PST) Received: from fusionlt2834a.int.fusionio.com ([209.117.142.2]) by mx.google.com with ESMTPSA id ml2sm51832255igb.10.2014.02.05.12.52.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 05 Feb 2014 12:52:05 -0800 (PST) Sender: Warner Losh Subject: Re: Malloc alignment in libstand / loader(8) Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <2882663.sRzekugaiD@ralph.baldwin.cx> Date: Wed, 5 Feb 2014 13:51:56 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <0AB67017-D232-40CF-A8F2-382F5A5A92B1@bsdimp.com> References: <1391576134.1196.21.camel@revolution.hippie.lan> <2882663.sRzekugaiD@ralph.baldwin.cx> To: John Baldwin X-Mailer: Apple Mail (2.1085) Cc: Ian Lepore , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2014 20:52:18 -0000 On Feb 5, 2014, at 6:29 AM, John Baldwin wrote: > On Tuesday, February 04, 2014 09:55:34 PM Ian Lepore wrote: >> On newer ARM chips, the device drivers used by loader(8) require that >> I/O buffers be aligned on cache line sized boundaries. The drivers = are >> part of u-boot which serves as a sort of load-time bios. >>=20 >> Attached is a patch that sets the malloc alignment in libstand to 64 >> bytes when compiled on ARM, and leaves it at 16 bytes for all other >> platforms. If there are no objections I'd like to commit this soon. >>=20 >> I've tested this on ARM, but have no way to test it on other = platforms. >> The changes should be a no-op on other platforms. >=20 > I think this looks fine, but perhaps use CTASSERT() instead of rolling = your=20 > own? (I would say to use _Static_assert(), but I don't think that = works with=20 > our old GCC) I'd enable it for mips and powerpc, the other two platforms that have = uboot callback support. Other than that, the change looks good to me. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Feb 5 22:13:06 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 042C3955; Wed, 5 Feb 2014 22:13:06 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CBFCF10BD; Wed, 5 Feb 2014 22:13:05 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C0BADB99A; Wed, 5 Feb 2014 17:13:04 -0500 (EST) From: John Baldwin To: Ian Lepore Subject: Re: Malloc alignment in libstand / loader(8) Date: Wed, 05 Feb 2014 15:38:34 -0500 Message-ID: <2263843.rxdRK42Wk4@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <1391630829.1196.31.camel@revolution.hippie.lan> References: <1391576134.1196.21.camel@revolution.hippie.lan> <2882663.sRzekugaiD@ralph.baldwin.cx> <1391630829.1196.31.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 05 Feb 2014 17:13:04 -0500 (EST) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2014 22:13:06 -0000 On Wednesday, February 05, 2014 01:07:09 PM Ian Lepore wrote: > On Wed, 2014-02-05 at 08:29 -0500, John Baldwin wrote: > > On Tuesday, February 04, 2014 09:55:34 PM Ian Lepore wrote: > > > On newer ARM chips, the device drivers used by loader(8) require that > > > I/O buffers be aligned on cache line sized boundaries. The drivers are > > > part of u-boot which serves as a sort of load-time bios. > > > > > > Attached is a patch that sets the malloc alignment in libstand to 64 > > > bytes when compiled on ARM, and leaves it at 16 bytes for all other > > > platforms. If there are no objections I'd like to commit this soon. > > > > > > I've tested this on ARM, but have no way to test it on other platforms. > > > The changes should be a no-op on other platforms. > > > > I think this looks fine, but perhaps use CTASSERT() instead of rolling > > your > > own? (I would say to use _Static_assert(), but I don't think that works > > with our old GCC) > > I wasn't sure if it's a good idea to #include in the libstand > environment, since it has declarations for a lot of stuff. I looked at > what's in systm.h and then hand-replicated what seemed like the most > generic of the choices. That's fair. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Feb 7 12:51:29 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 52EBCB13 for ; Fri, 7 Feb 2014 12:51:29 +0000 (UTC) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E26011875 for ; Fri, 7 Feb 2014 12:51:28 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id hm4so800776wib.7 for ; Fri, 07 Feb 2014 04:51:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=XE9VSP2WNgKuRt7DQbCtxfh1XUZURdhjmAkTuD8gKZo=; b=pLyQ+yqMLMU9JlPh5RpvJ5WIphNG6ABGtrS6jNu+6l2phMFlFA80YS1xnwwjndiaea PIo6eweD923/jo1pxEyIHfL1TpOzgHfFExYFGv4WsN2cH7XMQ7kCN9SjrnlSIl/EOkP9 +Tw1B8gAoSbIc51TEgm1WSbQDQ9osA5JxCmlfRpbm9RpQHUBSdJih36c+f4nQFXqqvnM pdJLu4jIip9aFNrsCSgc91jHVWY8C8qv1c7pYFMRFzvw4TV2od8BL5+Akof8+QFONadw 913P2aGo2o8w6Hm4nBVoDbO/qYpdAJHSm1i9Wcd/JZNHB5H9+b6U3jBz3BnKq3RJMe1i b8Ug== MIME-Version: 1.0 X-Received: by 10.180.13.33 with SMTP id e1mr3801843wic.38.1391777487255; Fri, 07 Feb 2014 04:51:27 -0800 (PST) Sender: fluca1978@gmail.com Received: by 10.194.79.136 with HTTP; Fri, 7 Feb 2014 04:51:27 -0800 (PST) Date: Fri, 7 Feb 2014 13:51:27 +0100 X-Google-Sender-Auth: AAEIsT2gIGRGrNeclB3sAyB3v_8 Message-ID: Subject: mbuf and uio From: Luca Ferrari To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Feb 2014 12:51:29 -0000 Hi all, I'm just wondering why mbufs seems to be much more important from an administrator point of view than uio. I mean, both structures are used to move data thru a stack (network or i/o), but the mbufs get accounted by serveral tools (like netstat and so on) while uio does not. Am I totally wrong on this? Luca From owner-freebsd-arch@FreeBSD.ORG Fri Feb 7 16:08:34 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 105A12CF for ; Fri, 7 Feb 2014 16:08:34 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id ED8891CAF for ; Fri, 7 Feb 2014 16:08:33 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 2BFF21A3C19 for ; Fri, 7 Feb 2014 08:08:30 -0800 (PST) Message-ID: <52F50501.8080708@mu.org> Date: Fri, 07 Feb 2014 08:08:33 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: mbuf and uio References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Feb 2014 16:08:34 -0000 On 2/7/14 4:51 AM, Luca Ferrari wrote: > Hi all, > I'm just wondering why mbufs seems to be much more important from an > administrator point of view than uio. I mean, both structures are used > to move data thru a stack (network or i/o), but the mbufs get > accounted by serveral tools (like netstat and so on) while uio does > not. > Am I totally wrong on this? uios are transient structures that can be allocated on the stack at any time. In general you'll have at most 1 uio per process/thread active. tracking them would not really gain us much. -- Alfred Perlstein From owner-freebsd-arch@FreeBSD.ORG Fri Feb 7 16:32:32 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 803A51E5 for ; Fri, 7 Feb 2014 16:32:32 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5053B107E for ; Fri, 7 Feb 2014 16:32:32 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 24104B926; Fri, 7 Feb 2014 11:32:31 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: mbuf and uio Date: Fri, 7 Feb 2014 11:23:38 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <52F50501.8080708@mu.org> In-Reply-To: <52F50501.8080708@mu.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402071123.38516.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 07 Feb 2014 11:32:31 -0500 (EST) Cc: Alfred Perlstein X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Feb 2014 16:32:32 -0000 On Friday, February 07, 2014 11:08:33 am Alfred Perlstein wrote: > On 2/7/14 4:51 AM, Luca Ferrari wrote: > > Hi all, > > I'm just wondering why mbufs seems to be much more important from an > > administrator point of view than uio. I mean, both structures are used > > to move data thru a stack (network or i/o), but the mbufs get > > accounted by serveral tools (like netstat and so on) while uio does > > not. > > Am I totally wrong on this? > uios are transient structures that can be allocated on the stack at any > time. In general you'll have at most 1 uio per process/thread active. > > tracking them would not really gain us much. The other thing is that uios do not allocate their own storage for I/O buffers. They merely serve as metadata so that data can be copied between userland buffers and other buffers in the kernel (such as the VM pages that back a file). The actual data used for I/O does not belong to the uio. mbufs, on the other hand, store packet data. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Sat Feb 8 00:12:57 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC4C7522; Sat, 8 Feb 2014 00:12:57 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9B557185F; Sat, 8 Feb 2014 00:12:57 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id x13so7251336qcv.34 for ; Fri, 07 Feb 2014 16:12:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=290YDpzC24+MbRtM0BeTpWV9PQF57zhBvZhb6GDqtZg=; b=oDJ85mloMepNIkphWUVLHQIwYe63m5k5+yNmyG2atgfvUnevtOdwVk50qxARPsTa44 iye8SIP8K3GhYly2CYQrnsX1Whv8erQOeTObIBfs6v5sFYZv+bM6VAzNo9wNaeQ6NJWV RobXfsyFrAsswtcyO9kuKibBFtfvoQkJLBYLIOAzidZ+jP0FgjHUBf6HyOdqTMoBRtbj Tz2ZsfLiuoaNtbQM2gTESBLPDdvcndlGacFTUTl/82SA8xnLAWDNBsdc0OeK4PR3TfFU dB5Z6Zf+Ygz62QWjdXiz/LHwDnSfuvLuz6rbZCzhpx54ibJ4pYsjiZYagYhl4fi0xLQC zAzQ== MIME-Version: 1.0 X-Received: by 10.140.50.235 with SMTP id s98mr3479529qga.12.1391818376799; Fri, 07 Feb 2014 16:12:56 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.52.8 with HTTP; Fri, 7 Feb 2014 16:12:56 -0800 (PST) Date: Fri, 7 Feb 2014 16:12:56 -0800 X-Google-Sender-Auth: Zrp8Eb97FVS4GLRmXRn0pUIHxw4 Message-ID: Subject: flowtable, collisions, locking and CPU affinity From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2014 00:12:58 -0000 Hi, I've been knee deep in the flowtable code looking at some of the less .. predictable ways it behaves. One of them is the collisions that do pop up from time to time. I dug into it in quite some depth and found out what's going on. This assumes it's a per-CPU flowtable. * A flowtable lookup is performed, on say CPU #0 * the flowtable lookup fails, so it goes to do a flowtable insert * .. but since in between the two, the flowtable "lock" is released so it can do a route/adjacency lookup, and that grabs a lock * .. then the flowtable insert is done on a totally different CPU * .. which happens to _have_ the flowtable entry already, so it fails as a collision which already has a matching entry. Now, the reason for this is primarily because there's no CPU pinning in the lookup path and if there's contention during the route lookup phase, the scheduler may decide to schedule the kernel thread on a totally different CPU to the one that was running the code when the lock was entered. Now, Gleb's recent changes seem to have made the instances of this drop, but he didn't set out to fix it. So there's something about his changes that has changed the locking/contention profile that I was using to easily reproduce it. In any case - the reason it's happening above is because there's no actual lock held over the whole lookup/insert path. It's a per-CPU critical enter/exit path, so the only way to guarantee consistency is to use sched_pin() for the entirety of the function. I'll go and test that out in a moment and see if it quietens the collisions that I see in lab testing. Has anyone already debugged/diagnosed this? Can anyone think of an alternate (better) way to fix this? Thanks, -a From owner-freebsd-arch@FreeBSD.ORG Sat Feb 8 01:05:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE638CF3; Sat, 8 Feb 2014 01:05:58 +0000 (UTC) Received: from mail-oa0-x22e.google.com (mail-oa0-x22e.google.com [IPv6:2607:f8b0:4003:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7DD521BD7; Sat, 8 Feb 2014 01:05:58 +0000 (UTC) Received: by mail-oa0-f46.google.com with SMTP id n16so5145615oag.33 for ; Fri, 07 Feb 2014 17:05:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=X/E1R8j29/YJMQ0lt+NFFkfYeCo4xQyRTjCfcZ1UOKo=; b=0sBtERLIcX6RnMwD6GItOfhud47wuXL0TRTBQIqao8Cy48z+sdfyYKP9YQY2Nz5tP9 2A6EulqE6I6iyC4S/LdxeJFzB8u/k18yzAYBrvYbkpnuQkGR3nZ1x7CwZr/UU2ZXeaRq hiOkIpGiPEtYyyuoZO7nnUW+Ucedae7ZjKJV6zOJkn6J+VbsHRGuQoT3dTXWsHljcpJQ tUNeKz2IW7TIsW0GliOQXKWQ12oVrxLXj5IAcxrNCaHvI/76Vyg4xIJNLlwT86DjpkNS R7Zd2yMCLQ/RuD+THPBJ2aJwXs8p8psINaalTx4z+8o+06IYtfuFCKzn9L1v9r07M1xB gexw== MIME-Version: 1.0 X-Received: by 10.182.225.137 with SMTP id rk9mr14688254obc.51.1391821557788; Fri, 07 Feb 2014 17:05:57 -0800 (PST) Received: by 10.76.130.196 with HTTP; Fri, 7 Feb 2014 17:05:57 -0800 (PST) In-Reply-To: References: Date: Fri, 7 Feb 2014 20:05:57 -0500 Message-ID: Subject: Re: flowtable, collisions, locking and CPU affinity From: Ryan Stone To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2014 01:05:58 -0000 On Fri, Feb 7, 2014 at 7:12 PM, Adrian Chadd wrote: > In any case - the reason it's happening above is because there's no > actual lock held over the whole lookup/insert path. It's a per-CPU > critical enter/exit path, so the only way to guarantee consistency is > to use sched_pin() for the entirety of the function. sched_pin seems like a very heavy hammer for what has to be a very rare and mostly harmless race. Why not redo the lookup after you have reacquired the lock, and if you don't have to do the insert anymore then don't and move on? From owner-freebsd-arch@FreeBSD.ORG Sat Feb 8 01:13:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 864CFF4A; Sat, 8 Feb 2014 01:13:02 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5D6A41C60; Sat, 8 Feb 2014 01:13:02 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s181CugE082530 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 7 Feb 2014 17:12:56 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s181Cubw082529; Fri, 7 Feb 2014 17:12:56 -0800 (PST) (envelope-from jmg) Date: Fri, 7 Feb 2014 17:12:56 -0800 From: John-Mark Gurney To: Ryan Stone Subject: Re: flowtable, collisions, locking and CPU affinity Message-ID: <20140208011256.GH89104@funkthat.com> Mail-Followup-To: Ryan Stone , Adrian Chadd , FreeBSD Net , "freebsd-arch@freebsd.org" References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 07 Feb 2014 17:12:56 -0800 (PST) Cc: FreeBSD Net , Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2014 01:13:02 -0000 Ryan Stone wrote this message on Fri, Feb 07, 2014 at 20:05 -0500: > On Fri, Feb 7, 2014 at 7:12 PM, Adrian Chadd wrote: > > In any case - the reason it's happening above is because there's no > > actual lock held over the whole lookup/insert path. It's a per-CPU > > critical enter/exit path, so the only way to guarantee consistency is > > to use sched_pin() for the entirety of the function. > > sched_pin seems like a very heavy hammer for what has to be a very > rare and mostly harmless race. Why not redo the lookup after you have > reacquired the lock, and if you don't have to do the insert anymore > then don't and move on? Why not drop the work since the current CPU has the results? It sucks to throw away work, but the other option is to remeber what cpu you were on, and put it there, but that would also be expensive... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Sat Feb 8 01:56:31 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C84E9E5; Sat, 8 Feb 2014 01:56:31 +0000 (UTC) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D09101FE4; Sat, 8 Feb 2014 01:56:30 +0000 (UTC) Received: by mail-pa0-f42.google.com with SMTP id kl14so3956256pab.1 for ; Fri, 07 Feb 2014 17:56:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=AYYGh5i5fu5dRqBRMnnDew3pMyPWxg2dWO2b7l4RjkU=; b=W4gvYc/GkDY/RyvL8+eBNSSf7gE3bFUetfEMopoAZbDMxwC/l+fKuKebyY/kat4PBV apiQNhD5AmYndoN8tebL/y0vfwE+vmZnmKTK+VtpxLnGg04aLHBlHRLt3Ctpa1t6TdS/ rXHIjRn+GOWXVccR8Lgk20nM5JtuHuLkXxKf2/6M5GGVjk0pM1YbL6piBfBnoW/8L+y/ Aoa9DTVFg+mpfm5qjMhuS9tuPI3ExRnr4pjZGk5+iIltvWZieleryXicjEqhWO3+U4cC thkBka3uaqmv3OJYa44ACV0PIFy1onNTCIP6hze9oyTPmYWNMx30mqUUfl83vGhXVek9 EM7w== MIME-Version: 1.0 X-Received: by 10.66.144.102 with SMTP id sl6mr11367016pab.96.1391824590415; Fri, 07 Feb 2014 17:56:30 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.70.103.174 with HTTP; Fri, 7 Feb 2014 17:56:29 -0800 (PST) In-Reply-To: References: Date: Fri, 7 Feb 2014 17:56:29 -0800 X-Google-Sender-Auth: DlLOcpG-6OiTJIEw5vOsRxf6oOc Message-ID: Subject: Re: flowtable, collisions, locking and CPU affinity From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2014 01:56:31 -0000 On 7 February 2014 17:05, Ryan Stone wrote: > On Fri, Feb 7, 2014 at 7:12 PM, Adrian Chadd wrote: >> In any case - the reason it's happening above is because there's no >> actual lock held over the whole lookup/insert path. It's a per-CPU >> critical enter/exit path, so the only way to guarantee consistency is >> to use sched_pin() for the entirety of the function. > > sched_pin seems like a very heavy hammer for what has to be a very > rare and mostly harmless race. Why not redo the lookup after you have > reacquired the lock, and if you don't have to do the insert anymore > then don't and move on? You say "rare and harmless race"; I can trick the system to do this: flowtable for IPv4: 57448 lookups 39543 hits 17905 misses 17820 collisions 382 free checks 82 frees .. and it gets stuck in a loop of never quite correctly updating/using the flowtable entries. So, it is mostly harmless, except exactly when it bites you in the ass. Yes, we could just reacquire the lock and insert if required, but I still have to be absolutely sure that the thread isn't preempted and migrated to another core. Otherwie we'd have teh same issue again. I'll keep tickling it. Thanks, -a From owner-freebsd-arch@FreeBSD.ORG Sat Feb 8 08:05:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54366829; Sat, 8 Feb 2014 08:05:28 +0000 (UTC) Received: from mail-we0-x235.google.com (mail-we0-x235.google.com [IPv6:2a00:1450:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B9E6314FA; Sat, 8 Feb 2014 08:05:27 +0000 (UTC) Received: by mail-we0-f181.google.com with SMTP id w61so2958009wes.12 for ; Sat, 08 Feb 2014 00:05:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=oqQ9duU3JBy4bGQUs9XAiaQryg9aTxyH31tgpsY4NV4=; b=gD4BN/FTbU0f6yyjyjBsXGJEVe69Lgt/NnUqlNNMG3k5aIs1qUDQ4C1A1ugFcZNO/L u7pSoQG5PL43Gg/e9ikGAosADC1qicKey3M6hwN85nCceWv4aQfZzwV5Y0FE1cfFbUDs LnwGNbCaR3LRp+Pq0KOcvb/v0kzHEmlNGj/PyuWQz8172I5ZHdd5l8xRC0trCoz+QgHh 9+4XjqLle1wEI79ncrsEBA/mIL36l2auXgvMrWMnJkQmoz3w+CFSZRke6h2A9vrcZwod uF3oP5wKtp/oEjTSCPjyig10xEs3Y+sqJQ8yUlSXVx+s2HoL5MyV7YBNjYm2ypamBDm4 MWTw== MIME-Version: 1.0 X-Received: by 10.180.25.46 with SMTP id z14mr2762074wif.49.1391846726208; Sat, 08 Feb 2014 00:05:26 -0800 (PST) Sender: fluca1978@gmail.com Received: by 10.194.79.136 with HTTP; Sat, 8 Feb 2014 00:05:26 -0800 (PST) In-Reply-To: <201402071123.38516.jhb@freebsd.org> References: <52F50501.8080708@mu.org> <201402071123.38516.jhb@freebsd.org> Date: Sat, 8 Feb 2014 09:05:26 +0100 X-Google-Sender-Auth: C9d7ZUanwHeyZZ_lEpNXVbZj8OE Message-ID: Subject: Re: mbuf and uio From: Luca Ferrari To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: Alfred Perlstein , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2014 08:05:28 -0000 On Fri, Feb 7, 2014 at 5:23 PM, John Baldwin wrote: > On Friday, February 07, 2014 11:08:33 am Alfred Perlstein wrote: > The actual data used for I/O does not belong to the uio. > mbufs, on the other hand, store packet data. Thanks, it is very clear now. Luca