From owner-freebsd-arch@FreeBSD.ORG Mon Jan 13 06:04:40 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 69748D4A for ; Mon, 13 Jan 2014 06:04:40 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 46F7D1632 for ; Mon, 13 Jan 2014 06:04:36 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s0D64ais021220 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sun, 12 Jan 2014 22:04:36 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s0D64aTA021219 for arch@FreeBSD.org; Sun, 12 Jan 2014 22:04:36 -0800 (PST) (envelope-from jmg) Date: Sun, 12 Jan 2014 22:04:36 -0800 From: John-Mark Gurney To: arch@FreeBSD.org Subject: casting td_retval to off_t... Message-ID: <20140113060436.GC2982@funkthat.com> Mail-Followup-To: arch@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sun, 12 Jan 2014 22:04:36 -0800 (PST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jan 2014 06:04:40 -0000 So, I've been working on bringing FreeBSD back up on an AVILA board and have run into an issue where we cast td_retval to an off_t to write back a value (see the end of kern/vfs_vnops.c). The issue is that td_retval is defined as: register_t td_retval[2]; and on arm register_t is an int32_t so in struct thread, it doesn't get aligned to an 8 byte boundary... Recently attilo added an int before td_retval that caused td_retval to be no longer be aligned to 8 bytes. What is the best way to fix this? One simple & quick fix was to add __aligned(sizeof(off_t)) to the td_retval, but if we really want to fix this cleanly (make it alias clean), we should change td_retval to be a union, and I guess have a define for td_retval to access it through the union, something like: union { register_t tdu_retval[2]; off_t tdu_off; } td_uretoff; #define td_retval td_uretoff.tdu_retval Comments? Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Mon Jan 13 20:19:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8C5B36DD; Mon, 13 Jan 2014 20:19:28 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 626CB1364; Mon, 13 Jan 2014 20:19:28 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 55ECAB995; Mon, 13 Jan 2014 15:19:27 -0500 (EST) From: John Baldwin To: Adrian Chadd Subject: Re: Acquiring a lock on the same CPU that holds it - what can be done? Date: Mon, 13 Jan 2014 14:43:52 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <9508909.MMfryVDtI5@ralph.baldwin.cx> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201401131443.52550.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 13 Jan 2014 15:19:27 -0500 (EST) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jan 2014 20:19:28 -0000 On Thursday, January 09, 2014 1:44:51 pm Adrian Chadd wrote: > On 9 January 2014 10:31, John Baldwin wrote: > > On Friday, January 03, 2014 04:55:48 PM Adrian Chadd wrote: > >> Hi, > >> > >> So here's a fun one. > >> > >> When doing TCP traffic + socket affinity + thread pinning experiments, > >> I seem to hit this very annoying scenario that caps my performance and > >> scalability. > >> > >> Assume I've lined up everything relating to a socket to run on the > >> same CPU (ie, TX, RX, TCP timers, userland thread): > > > > Are you sure this is really the best setup? Especially if you have free CPUs > > in the system the time you lose in context switches fighting over the one > > assigned CPU for a flow when you have idle CPUs is quite wasteful. I know > > that tying all of the work for a given flow to a single CPU is all the rage > > right now, but I wonder if you had considered assigning a pair of CPUs to a > > flow, one CPU to do the top-half (TX and userland thread) and one CPU to > > do the bottom-half (RX and timers). This would remove the context switches > > you see and replace it with spinning in the times when the two cores actually > > contend. It may also be fairly well suited to SMT (which I suspect you might > > have turned off currently). If you do have SMT turned off, then you can get > > a pair of CPUs for each queue without having to reduce the number of queues > > you are using. I'm not sure this would work better than creating one queue > > for every CPU, but I think it is probably something worth trying for your use > > case at least. > > > > BTW, the problem with just slapping critical enter into mutexes is you will > > run afoul of assertions the first time you contend on a mutex and have to > > block. It may be that only the assertions would break and nothing else, but > > I'm not certain there aren't other assumptions about critical sections and > > not ever context switching for any reason, voluntary or otherwise. > > It's the rage because it turns out it bounds the system behaviour rather nicely. Yes, but are you willing to try the suggestion? This doesn't restrict to you a single queue-pair. It might net you 1 per core (instead of 1 per thread), but that's still more than 1. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Jan 14 05:08:45 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EEE4E68C; Tue, 14 Jan 2014 05:08:44 +0000 (UTC) Received: from mail-qe0-x22b.google.com (mail-qe0-x22b.google.com [IPv6:2607:f8b0:400d:c02::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9B0D71E6B; Tue, 14 Jan 2014 05:08:44 +0000 (UTC) Received: by mail-qe0-f43.google.com with SMTP id nc12so1196561qeb.16 for ; Mon, 13 Jan 2014 21:08:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Bk9AfpGuJOF6kUFQrof4uasT0sDq3k4RzLvTjP4lkFI=; b=lITVFquLDEzRw10NmziOQ9UeKYGI8Ns4JyZTYVSu/Z3DMhOoAdqqWtuoNLXyHnnLBo d96nBYi5rdJzUuoc5U8UrLSZ104yElBBZewepvvJhz9xgtmH1U6+QGHqDpJIy3Ntkodi wSFSfbYI9+t+vRGPoiK7UGmZyIFaYV5oJ/7nG0b0IoJ056HwoFZsvC6P6TWFXdBnD1By ZGzM5Azs/5QWx/nWd9gzpOvcNFB99FcZJVoDpFZkK0hVEzMH4C+q0AWhR40n85OaTcwu jsIV6OVB+9R8QO3sKm5apg9FuixHtpPaThhURh4nrGYaVTZUxi8kE9L5unh4wXHZztn+ m3sw== MIME-Version: 1.0 X-Received: by 10.49.24.211 with SMTP id w19mr47495914qef.9.1389676123711; Mon, 13 Jan 2014 21:08:43 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.52.8 with HTTP; Mon, 13 Jan 2014 21:08:43 -0800 (PST) In-Reply-To: <201401131443.52550.jhb@freebsd.org> References: <9508909.MMfryVDtI5@ralph.baldwin.cx> <201401131443.52550.jhb@freebsd.org> Date: Mon, 13 Jan 2014 21:08:43 -0800 X-Google-Sender-Auth: ypmVWlLHCVMA1YlNLPCEBO2b2T0 Message-ID: Subject: Re: Acquiring a lock on the same CPU that holds it - what can be done? From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jan 2014 05:08:45 -0000 On 13 January 2014 11:43, John Baldwin wrote: > On Thursday, January 09, 2014 1:44:51 pm Adrian Chadd wrote: >> On 9 January 2014 10:31, John Baldwin wrote: >> > On Friday, January 03, 2014 04:55:48 PM Adrian Chadd wrote: >> >> Hi, >> >> >> >> So here's a fun one. >> >> >> >> When doing TCP traffic + socket affinity + thread pinning experiments, >> >> I seem to hit this very annoying scenario that caps my performance and >> >> scalability. >> >> >> >> Assume I've lined up everything relating to a socket to run on the >> >> same CPU (ie, TX, RX, TCP timers, userland thread): >> > >> > Are you sure this is really the best setup? Especially if you have free CPUs >> > in the system the time you lose in context switches fighting over the one >> > assigned CPU for a flow when you have idle CPUs is quite wasteful. I know >> > that tying all of the work for a given flow to a single CPU is all the rage >> > right now, but I wonder if you had considered assigning a pair of CPUs to a >> > flow, one CPU to do the top-half (TX and userland thread) and one CPU to >> > do the bottom-half (RX and timers). This would remove the context switches >> > you see and replace it with spinning in the times when the two cores actually >> > contend. It may also be fairly well suited to SMT (which I suspect you might >> > have turned off currently). If you do have SMT turned off, then you can get >> > a pair of CPUs for each queue without having to reduce the number of queues >> > you are using. I'm not sure this would work better than creating one queue >> > for every CPU, but I think it is probably something worth trying for your use >> > case at least. >> > >> > BTW, the problem with just slapping critical enter into mutexes is you will >> > run afoul of assertions the first time you contend on a mutex and have to >> > block. It may be that only the assertions would break and nothing else, but >> > I'm not certain there aren't other assumptions about critical sections and >> > not ever context switching for any reason, voluntary or otherwise. >> >> It's the rage because it turns out it bounds the system behaviour rather nicely. > > Yes, but are you willing to try the suggestion? This doesn't restrict to you > a single queue-pair. It might net you 1 per core (instead of 1 per thread), > but that's still more than 1. Sure. I can also try your suggestion of binding them to SMT pairs and see if that has any effect. But I'm specifically looking to _avoid_ contention at all in the main data path, not try to occasionally have the cores spin. -a From owner-freebsd-arch@FreeBSD.ORG Wed Jan 15 17:55:34 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00D52473 for ; Wed, 15 Jan 2014 17:55:33 +0000 (UTC) Received: from mail-pa0-x24e.google.com (mail-pa0-x24e.google.com [IPv6:2607:f8b0:400e:c03::24e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CFD9811A5 for ; Wed, 15 Jan 2014 17:55:33 +0000 (UTC) Received: by mail-pa0-f78.google.com with SMTP id rd3so14982pab.9 for ; Wed, 15 Jan 2014 09:55:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:message-id:date:subject:from:to:content-type; bh=vey4mGXbJufQSIYTt/pW9JSw0N2EW597py22iVnjT/0=; b=BMrbSepbrQ/Km8bwooISBF6nK9flkKreYsbT9SyjnCQ7KyQ7vg1OztOpYsdoyvodWA t+PyCgZa5gg2pLoPoJ7YxzxF4SR5DtvGhOK7Uy0N/V5dxwxvue9cQ0RBw/AsQwz/FdVd 040JfN79WPeoqzASnO37Zv4FnGNtEP9vWiOf699tKS+6/lk7O5bMrfTUvt8VqPggzk5y QQ9tgYqj5trnhvy2fwsP6z0I1j0jWzh4t7FdFtiMC8ofbOJ7LNt863IsS7q8TdgNcKXt O/33LEpvFUlSf05s2xBo2Daxim69YizR/+w066Gt/YeMd7RqbGeJnlPWGEyEsrXFDG+W dOuA== MIME-Version: 1.0 X-Received: by 10.66.190.197 with SMTP id gs5mr1304577pac.14.1389808533458; Wed, 15 Jan 2014 09:55:33 -0800 (PST) Message-ID: <047d7bf0ee6cf1551404f0060303@google.com> Date: Wed, 15 Jan 2014 17:55:33 +0000 Subject: www.freebsd.org From: Ava Hobbs To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=windows-1252; format=flowed; delsp=yes Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Jan 2014 17:55:34 -0000 DQoNCkhpLA0KDQpJIGp1c3Qgd2FudGVkIHRvIHNlbmQgeW91IGEgcXVpY2sgbm90ZS4gV2l0aCBh IGZldyBzaW1wbGUgY2hhbmdlcyB0byBtYWtlDQp5b3VyIHNpdGUgbW9yZSBTRU8tZnJpZW5kbHkg SZJtIHN1cmUgeW91IGNhbiBjb252ZXJ0IG1vcmUgdmlzaXRvcnMgaW50bw0KbGVhZHMgYW5kIGdl dCBpdCBwbGFjZWQgaGlnaGVyIGluIHRoZSBvcmdhbmljIHNlYXJjaCByZXN1bHRzLCBmb3Iga2V5 d29yZHMNCnRoYXQgbWF0dGVyIHRvIHlvdSB0aGUgbW9zdC4NCg0KV2UgYXJlIFVTQSBiYXNlZCBj b21wYW55IHdpdGggYSBncmVhdCBpbi1ob3VzZSB0ZWNobmljYWwgdGVhbSB3aG8gcmVhbGx5DQpr bm93IHRoZWlyIHN0dWZmIGFib3V0IHNlYXJjaCBlbmdpbmUgb3B0aW1pemF0aW9uLg0KDQpXb3Vs ZCB5b3UgbGlrZSBhIGJpdCBtb3JlIGluZm9ybWF0aW9uIGFib3V0IGhvdyB0byBnaXZlIHlvdXIg d2Vic2l0ZSBhDQpib29zdCB3aXRoIGJldHRlciBTRU8/DQoNCkJlc3QgcmVnYXJkcywNCg0KQXZh IEhvYmJzDQpTRU8vV0VCIFNwZWNpYWxpc3QNCg0KW2ltYWdlOiBMaW5rZWRJbl0gW2ltYWdlOiBG YWNlYm9va10gW2ltYWdlOiBUd2l0dGVyXSBbaW1hZ2U6IFNreXBlXQ0KICAgICAgICAgICAgIFMg ICBFICBPICAgICAgICAgICAgKlNlYXJjaCBFbmdpbmUgT3B0aW1pemF0aW9uKg0KDQpXZSByZXNw ZWN0IHlvdXIgcHJpdmFjeSBhbmQgd2FudCB0byBtYWtlIHN1cmUgeW91IGFyZSBhd2FyZSBvZiBh IGZldw0KdGhpbmdzLiBCeSByZXBseWluZyB0byB0aGlzIGVtYWlsLCB5b3UgYXV0aG9yaXplIG91 ciBVU0EgYWZmaWxpYXRlcyB0aGF0DQpjYW4gaGVscCB3aXRoIHlvdXIgcHJvamVjdCB0byBjYWxs IHlvdSBhdCB0aGUgbnVtYmVyIHlvdSBwcm92aWRlZCwgYW5kIHlvdQ0KdW5kZXJzdGFuZCB0aGF0 IHRoZXkgbWF5IHVzZSBhdXRvbWF0ZWQgcGhvbmUgdGVjaG5vbG9neSB0byBjYWxsIHlvdS4gQXQg bm8NCnRpbWUgYXJlIHlvdSByZXF1aXJlZCB0byBtYWtlIGEgcHVyY2hhc2UuDQo= From owner-freebsd-arch@FreeBSD.ORG Thu Jan 16 16:24:25 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F307D51; Thu, 16 Jan 2014 16:24:25 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0C5721CCE; Thu, 16 Jan 2014 16:24:20 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA07312; Thu, 16 Jan 2014 18:24:12 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1W3pjo-000K5J-3i; Thu, 16 Jan 2014 18:24:12 +0200 Message-ID: <52D80773.4060300@FreeBSD.org> Date: Thu, 16 Jan 2014 18:23:15 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-stable List , freebsd-arch@FreeBSD.org Subject: MFC-ability of SDT change X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: freebsd-dtrace@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jan 2014 16:24:25 -0000 I would like to ask your opinion on the following change http://svnweb.freebsd.org/base?view=revision&revision=258622 Can this change be MFC-ed to the stable branches? It does not actually change anything in ABI, but there is a semantic change that may result in incorrect names for SDT probes in third-party kernel code. Thank you! -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Thu Jan 16 17:43:36 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 454DDB81; Thu, 16 Jan 2014 17:43:36 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 093F114A1; Thu, 16 Jan 2014 17:43:35 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s0GHhY6h088351 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 16 Jan 2014 09:43:34 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s0GHhYSR088350; Thu, 16 Jan 2014 09:43:34 -0800 (PST) (envelope-from jmg) Date: Thu, 16 Jan 2014 09:43:34 -0800 From: John-Mark Gurney To: arch@FreeBSD.org Subject: CFR: removing off_t casts of td_retval Message-ID: <20140116174334.GC75135@funkthat.com> Mail-Followup-To: arch@FreeBSD.org, Bruce Evans Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 16 Jan 2014 09:43:35 -0800 (PST) Cc: Bruce Evans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jan 2014 17:43:36 -0000 In attempting to bring up an AVILA board, I found that td_retval has became unaligned to an 8 byte boundary on 32bit platforms. As we cast td_retval (of struct thread) to off_t, this results in an alignment fault. After consulting w/ bde, we decided on making a union of td_retval and a new off_t member. This will now align things properly on arches that require it (arm). Keep the 32bit alignment on arches that doesn't need it (verified on i386), and should keep things unchanged on 64bit arches (since there register_t, type of td_retval provided 64bit alignment). This also has the added benefit of making our code more correct as the code will be alias safe as we are not casting pointers between types. There never was a problem with the code as in all cases I saw, as we would only ever read/write from/to these members in a function once. This also eliminates a bad example. I have tested this patch on amd64 and armeb (as far as I can due to other bugs). tinderbox builds are running, and assuming they complete cleanly, I'll commit. I may have missed some locations as I only did a: grep td_retval | grep off_t Comments? Index: compat/freebsd32/freebsd32_misc.c =================================================================== --- compat/freebsd32/freebsd32_misc.c (revision 260499) +++ compat/freebsd32/freebsd32_misc.c (working copy) @@ -1504,7 +1504,7 @@ ap.whence = uap->whence; error = sys_lseek(td, &ap); /* Expand the quad return into two parts for eax and edx */ - pos = *(off_t *)(td->td_retval); + pos = td->td_uretoff.tdu_off; td->td_retval[RETVAL_LO] = pos & 0xffffffff; /* %eax */ td->td_retval[RETVAL_HI] = pos >> 32; /* %edx */ return error; Index: kern/uipc_shm.c =================================================================== --- kern/uipc_shm.c (revision 260499) +++ kern/uipc_shm.c (working copy) @@ -270,7 +270,7 @@ if (offset < 0 || offset > shmfd->shm_size) error = EINVAL; else - *(off_t *)(td->td_retval) = offset; + td->td_uretoff.tdu_off = offset; } foffset_unlock(fp, offset, error != 0 ? FOF_NOUPDATE : 0); return (error); Index: kern/vfs_vnops.c =================================================================== --- kern/vfs_vnops.c (revision 260499) +++ kern/vfs_vnops.c (working copy) @@ -2080,7 +2080,7 @@ if (error != 0) goto drop; VFS_KNOTE_UNLOCKED(vp, 0); - *(off_t *)(td->td_retval) = offset; + td->td_uretoff.tdu_off = offset; drop: foffset_unlock(fp, offset, error != 0 ? FOF_NOUPDATE : 0); return (error); Index: sys/proc.h =================================================================== --- sys/proc.h (revision 260499) +++ sys/proc.h (working copy) @@ -300,7 +300,11 @@ TDS_RUNQ, TDS_RUNNING } td_state; /* (t) thread state */ - register_t td_retval[2]; /* (k) Syscall aux returns. */ + union { + register_t tdu_retval[2]; + off_t tdu_off; + } td_uretoff; /* (k) Syscall aux returns. */ +#define td_retval td_uretoff.tdu_retval struct callout td_slpcallout; /* (h) Callout for sleep. */ struct trapframe *td_frame; /* (k) */ struct vm_object *td_kstack_obj;/* (a) Kstack object. */ -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Thu Jan 16 20:28:39 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBB44728; Thu, 16 Jan 2014 20:28:39 +0000 (UTC) Received: from mail-qe0-x229.google.com (mail-qe0-x229.google.com [IPv6:2607:f8b0:400d:c02::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6C1A515AC; Thu, 16 Jan 2014 20:28:39 +0000 (UTC) Received: by mail-qe0-f41.google.com with SMTP id i11so3099659qej.28 for ; Thu, 16 Jan 2014 12:28:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=kqVwmanGwcd1AeA3L+3k/XPOonHxQuZpAx7hSNBYflg=; b=ob/9V3uagq1CACMtMUH7eU9sLDrdE1zyL8q8jEFLnkheWKsHDJXYZft5BvIXvU8rr4 zlK1unik4HbVP0zN5zippgaZ18q6HrL6g88hFQ8UHefNlLoqkrmG9FZRsfiTBiWQxYuJ a6l5GWOOX4G1Y1t1iUeyF2kL/W03bo4O9ePe4eCzpTUGEgfnMVZTZH1uMDGVufCktYop 3HY0m7PkNRiPp4NO1tLimQBSyR2PX9SFt0ehTYmEJbqQBEyo+4LfpvfOo095F0jDFBXA fW1RpsCP/iTrPb63nFwDB0pIRKrt+vMYiwlnz0f5456JxU+FxnxdUtwI1zVMSAyHcTQR WmKg== MIME-Version: 1.0 X-Received: by 10.229.122.195 with SMTP id m3mr19467158qcr.7.1389904118630; Thu, 16 Jan 2014 12:28:38 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.52.8 with HTTP; Thu, 16 Jan 2014 12:28:38 -0800 (PST) Date: Thu, 16 Jan 2014 12:28:38 -0800 X-Google-Sender-Auth: l6Xtus_xKpTZTe7V0K7MzCHciIc Message-ID: Subject: [rfc] set inp_flowid on initial TCP connection From: Adrian Chadd To: FreeBSD Net , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jan 2014 20:28:39 -0000 Hi, This patch sets the inp_flowid on incoming connections. Without this, the initial connection has no flowid, so things like the per-CPU TCP callwheel stuff would map to a different CPU on the initial incoming setup. -a Index: sys/netinet/tcp_syncache.c =================================================================== --- sys/netinet/tcp_syncache.c (revision 260499) +++ sys/netinet/tcp_syncache.c (working copy) @@ -722,6 +722,16 @@ #endif /* + * If there's an mbuf and it has a flowid, then let's initialise the + * inp with that particular flowid. + */ + if (m != NULL && m->m_flags & M_FLOWID) { + inp->inp_flags |= INP_HW_FLOWID; + inp->inp_flags &= ~INP_SW_FLOWID; + inp->inp_flowid = m->m_pkthdr.flowid; + } + + /* * Install in the reservation hash table for now, but don't yet * install a connection group since the full 4-tuple isn't yet * configured. From owner-freebsd-arch@FreeBSD.ORG Fri Jan 17 05:08:12 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5653368C for ; Fri, 17 Jan 2014 05:08:12 +0000 (UTC) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0F2331E2C for ; Fri, 17 Jan 2014 05:08:11 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id n7so3216729qcx.16 for ; Thu, 16 Jan 2014 21:08:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=mlHC+O2Ur74FCh8Gub7eqgmfN9q9YqTgYwP3hddKXkc=; b=S/34RBxMJ1HLV1jwSz5Lxmcn3abRoWH3G8hHoHtv05MW+XNVf8uBgqrO+QTkgBZYew E8QYpIAYzY5VxtoFyTgtWEGyzKEzdz6EMVJizxgrHAn77S8uMnOvU7/3YtOIcoelulOz AiayuWG0kx1sFOTo6J9+GeKQNU+LBnJVHZEb8Gswi+NgrW10x38oQoQU3r2aoD4N2rlI 1ymi1ED/SGKJBn+fmVKCGjDjzFQA7S4x6IT8JlsgK9sV+sE+KMDr3/G9BV/UxevsFXuk B35P6837anz80n+ksU/wEcoS2qkVjbXonEKDynnnK74upBypbdVIiBSwc/Bii7WJjAys JZLg== MIME-Version: 1.0 X-Received: by 10.229.56.200 with SMTP id z8mr22650666qcg.1.1389935291275; Thu, 16 Jan 2014 21:08:11 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.52.8 with HTTP; Thu, 16 Jan 2014 21:08:11 -0800 (PST) In-Reply-To: <20140111123050.Q940@besplex.bde.org> References: <9C1291B5-215B-440E-B8B0-6308840F755C@bsdimp.com> <5821DA91-E9C7-4D1A-A108-63E74CFF1FC5@bsdimp.com> <20140108152632.L969@besplex.bde.org> <20140109092602.M957@besplex.bde.org> <20140110222640.F2338@besplex.bde.org> <20140111123050.Q940@besplex.bde.org> Date: Thu, 16 Jan 2014 21:08:11 -0800 X-Google-Sender-Auth: PCn-n9yFUDvwpTqiQ49tEqJSaWE Message-ID: Subject: Re: Using sys/types.h types in sys/socket.h From: Adrian Chadd To: Bruce Evans Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jan 2014 05:08:12 -0000 [snip] Ok. Thanks for your feedback thus far. I'm going to commit the header as-is so I can make a final push to get these changes into -HEAD. I'd like to go through and properly fix up the sys/socket.h changes before I MFC the lot to stable/10 so I don't create more work later on. Thanks! -a